Print Page - Searching for all unused global elements in the project

Using the Lazarus IDE => General => Topic started by: furious programming on January 12, 2019, 03:06:27 pm

Title: Searching for all unused global elements in the project
Post by: furious programming on January 12, 2019, 03:06:27 pm

Hi everyone.

I need to find all unused elements in my project (such as global constants, types, variables and subroutines) and remove them from the source code. I am talking about elements that are declared in the units of the current (opened) project.

How can I do that? I don't see any option in the environment.

There are hundreds of constants in the project (not to mention the rest elements), so manual checking (eg using the "Find or Rename Identifier" or "Find in Files" tools) would take a very long time and it would drive me crazy.

Is there a way to automate this task?

Title: Re: Searching for all unused elements in the project
Post by: howardpc on January 12, 2019, 03:26:57 pm

Let the compiler point these things out.
In your project Options dialog there is a Compiler Options node.
Under this node make sure that on the Verbosity page you have checked at least:
Show Warnings
Show Hints
Show Notes
On the Messages page, ensure all messages are checked, especially those with "unused" in them.

Right-click on the Messages window that reports compilation progress, and in the context menu under
Filter non-urgent Messages
choose Filter None

You may be overwhelmed... depending on how much clutter you have allowed to accumulate.

Title: Re: Searching for all unused elements in the project
Post by: dsiders on January 12, 2019, 04:56:44 pm

Quote from: furious programming on January 12, 2019, 03:06:27 pm

Hi everyone.

I need to find all unused elements in my project (such as global constants, types, variables and subroutines) and remove them from the source code. I am talking about elements that are declared in the units of the current (opened) project.

How can I do that? I don't see any option in the environment.

There are hundreds of constants in the project (not to mention the rest elements), so manual checking (eg using the "Find or Rename Identifier" or "Find in Files" tools) would take a very long time and it would drive me crazy.

Is there a way to automate this task?

Every time I rebuild the IDE, I get a list of hints and messages in the Messages window. Make sure Hints are enabled (and not ignored) in Project Options. They can also be hidden using directives in the source code (which IMO is a bad idea). Right click on the Messages window and select one of the Copy > Copy All options.

Title: Re: Searching for all unused elements in the project
Post by: furious programming on January 12, 2019, 06:14:20 pm

Thanks for replies.

All options in the branch Compiler Options\Messages in the project options window are enabled. In the source code I have no {$HINTS OFF} directives (I never use these directives). And the compiler does not inform me about unused global elements.

I think the problem is that these elements are global (in the interface sections). For local elements, the apropriate hints appears in the messages window. But not for global—even if I choose the filter named Filter non urgent messages\Filter none from the context menu in the messages window, still I don't get such hints.

I ask differently—how to list all unused elements that are declared in the interface sections in all units of current (opened) project? Probably this question is more precise.

BTW: is the compiler supposed to inform about it at all? Because something seems to me that it does not.

Title: Re: Searching for all unused global elements in the project
Post by: howardpc on January 13, 2019, 01:55:44 pm

The only way I know for the compiler to pick up unused unit-declared global variables is to declare them in the main .lpr of the project.
If your project has many units, each with var declarations, simply move the entire var declaration(s) in each unit:

Code: Pascal [Select][+]

var  Form1: TForm1; 
  etc.

into the main .lpr file.
Of course, you may have to rename one or two such global variables if you get a clash. And you will have to ensure that the .lpr uses clause names all the units in your project (which it does unless you have changed the default Lazarus settings).

However, the compiler will then report any of the .lpr variables as "unused local variable ..." if they are not used either in the.lpr nor in any of the units from which you had extracted them.
The disadvantage to this approach is that any global variables that are used in a unit will then give compilation errors of "identifier not found" etc.
You then have to move such used variables one by one back to the units where they are used.
But you are left with a list of unused variables from the .lpr (if there are any) when your project finally compiles again.

I think one moral of this is that if you really, really need a global variable (and honestly you rarely do -- good OOP practice should have you encapsulating all needed variables locally), then declare it in the .lpr in the first place, where it is truly and obviously global, or in a special Globals.pas unit which collects all the dangerous global variables in one place where it is easier to ensure that each is really needed.
I know this does not cover every possible scenario, but such an approach may help you clean up your code, and refactor it so that very little -- i.e only things that actually must be global -- is in fact truly global.

Title: Re: Searching for all unused global elements in the project
Post by: Bart on January 13, 2019, 03:52:08 pm

Compile with -B.
This will rebuild all your units (and the LCL etc. if you use those in a Lazarus program) and the compiler will issue hints for evrerything it finds.

Bart

Title: Re: Searching for all unused global elements in the project
Post by: Martin_fr on January 13, 2019, 04:23:52 pm

For unused units, the IDE has an entry under refactor. But you have to check for side effects, such of initialization/finalization blocks in that unit (or any unit used by that unit)

For global variables.
Nothing I know, but...

Try WPO (whole program optimization) [google]

Look through the options you can pass to the linker. Maybe you can get the linker (on Win, may need external linker?) to generate useful info. [google + try and error]

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 13, 2019, 07:51:59 pm

Much has been said, so I will write a little about the specifics of this project.

I am currently working on a platformer game (or rather a tech demo), created without using any library to build games (just for fun). For this purpose, I use the TForm class and various utils from the standard library. No hardware acceleration. More about I described in this old post (http://forum.lazarus.freepascal.org/index.php/topic,39495.msg293070.html#msg293070). The code of the game is practically ready, now only need is to create levels and I can publish the project. That's why I'd like to remove what is unused from the code.

Currently, the main project consists of 35 units (with .lpr unit), slightly over 12,000 LoC. Each unit has a separate logic wrapped in a class or several classes, using inheritance or not (depending on the needs). If only one instance of a given class is used throughout the entire game, it is simply contained in a global variable, declared in the same unit as the class declaration. It is convenient (although no one likes global variables, but they do not bother me).

For example, the unit Platformer.Levels.pp contains a global variable named Level and it is used by other classes from other units (because always only one instance of the TLevel class is used at the moment). It looks the same as with global variables with form instances in a classic application. Each global variable is used to store an instance of a class. Instances of classes for global variables are created and released in the main game class (ie in one place, not using the initialization and finalization sections—these are not used at all).

I decided to play with the code and in addition to global variables, also use other less common techniques. For example, multi-level nesting of class declarations, custom helpers for various types, class variables, some syntactic sugar and so on.

For now, I'm still using the Debug mode generated with the Build modes tool in the project options window. All message settings are enabled (default for this mode). The compilation is clean—no hints or warnings. The only exception is the Symbol "ScanLine" is not portable warnings, but that's not a problem.

In Release mode (generated, also with default options), I have 30 warnings, only about ScanLine and uninitialized local variables and Result in methods (all by using case of statement, but this is also not a problem, because the control flow is always correct). Example:

Code: Pascal [Select][+]

function THeroRenderer.GetArea(AHero: THero): TRect;
var
  CropSize: Integer;
begin
  Result := AHero.Location.Area;
 
  if AHero.Dozing or AHero.Walking then
  begin
    if AHero.Dozing then
      CropSize := AHero.DozePhase
    else
    case AHero.WalkPhase of
      0: Exit;
      1: CropSize := 1;
      2: CropSize := 2;
      3: CropSize := 1;
    end;
 
    case AHero.Orientation of
      voNormal:     Result.Inflate(0, -CropSize, 0, 0);
      voUpsideDown: Result.Inflate(0, 0, 0, -CropSize);
    end;
  end
  else
    if AHero.Falling and not AHero.Rising then
      Result.Inflate(0, Abs(AHero.VerticalSpeed));
end;

Variable CropSize is uninitialized for the compiler, but it will never be used without initialization. If I turn off the optimizations (or set the -O1 or -O2), the compilation is clean. So the warnings about uninitialized variables is shown only if I use more aggressive optimizations.

Well, I wrote unnecessarily. Since there is no way to get a list of such elements (also for the debug mode), then I will have to check each one separately... Thanks for the answers.

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 15, 2019, 04:31:39 pm

Hmm… I think that this functionality should be supported by the environment. This option can be added to the Source/Refactoring submenu in the main menu and Refactoring submenu in the code editor context menu. The name Unused identifiers... should be proper.

Do you think this functionality is useful, or do I only see the need for it? I'm asking seriously.

Title: Re: Searching for all unused global elements in the project
Post by: CCRDude on January 15, 2019, 04:42:02 pm

Think about re-usable code.

Just because one instance using a certain unit doesn't use all the constants defined in there does not mean that no other instance using this unit would not require them either.

How would any IDE functionality for this be able to determine if a global const is there for use by other projects, or if it is unrequired at all?

One usage case: I often use interfaces to third party DLLs. There's often consts for dozens of parameters and return codes. If my program uses just a tiny portion of such a header translation, should all the others would be reported as unused? That might make me miss the really important messages in the output.

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 15, 2019, 05:54:27 pm

Of course, I realize that such functionality would be certain for a "closed" project (eg console or window application). In the case of a library or other type of project that publishes an API (classes, constants, routines, etc.), this functionality would not apply.

The programmer should make a choice, knowing what type of project he creates and what can actually be removed. Therefore, the output list of all unused identifiers in the project would be only a suggestion, just like in the case of unused units.

Title: Re: Searching for all unused global elements in the project
Post by: sash on January 15, 2019, 08:51:38 pm

Obviously, if "elements" are global, i.e. located in such scope, where it could be accessed by anyone (units not listed in project, units in other projects), the term "unused" is not applicable.

Normally, one should have a reason for identifier to be in global public scope. And if you have a lot of identifiers that became "unused", you should reconsider your code structure. Not just "clean unused".

Title: Re: Searching for all unused global elements in the project
Post by: garlar27 on January 15, 2019, 09:10:47 pm

Quote from: sash on January 15, 2019, 08:51:38 pm

Obviously, if "elements" are global, i.e. located in such scope, where it could be accessed by anyone (units not listed in project, units in other projects), the term "unused" is not applicable.

Normally, one should have a reason for identifier to be in global public scope. And if you have a lot of identifiers that became "unused", you should reconsider your code structure. Not just "clean unused".

Yes, but I think it would be good to have a tool to show you unused element in the interface if you want to know them.

But with the things we have now, the only workaround I can think of is to move all constants, types and vars to the implementation section. Thus what is inside the unit will not generate a compiler error, but if it is used outside the unit then the compiler will complain. Once you know something uses that identifier/type/whatsoever do a search for it to see if it worth keeping it or move it to other unit or if it's better to erase it with anything that use it.

Going down and dirty is not my fav but sometimes is better do it now and not the next year when you will have lots more of everything %)

It will take you some time but it can help the project since it is not just "throw away the unused" it also might organize your resources.

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 15, 2019, 10:16:17 pm

Quote from: sash on January 15, 2019, 08:51:38 pm

Normally, one should have a reason for identifier to be in global public scope. And if you have a lot of identifiers that became "unused", you should reconsider your code structure. Not just "clean unused".

I do not understand why the need to find unused "identifiers" is considered by you as a sign of poor code quality. I have been developing this game for many months (about 10) and by adding new functionalities and modifying existing (and also fixing bugs) some identifiers were no longer needed and I could forget to remove them from the code.

First of all, we talk about constants. All constants are declared in the Platformer.Constants.pp unit, which is used by many other units (thats why are global, in the interface section). They are grouped and declared in the right order, about 300 pieces. This is not a library project, so all unused constants should be removed from this unit (other "elements" from other units too).

Is the existence of such a unit a bad idea? I don't think so.

Quote from: garlar27 on January 15, 2019, 09:10:47 pm

But with the things we have now, the only workaround I can think of is to move all constants, types and vars to the implementation section.

Yes, but it is a different way of manual search.

It is faster to check the identifiers with the Find in Files tool, which can be easily operated from the keyboard. If the list in the Search Results window contains only one item (declaration line), it means that the constant is not used anywhere else.

I intentionally write about the Find in Files tool, because the Find or Rename Identifier tool often does not find all instances. I often use the second mentioned tool and I also often have compilation errors after changing the identifier of the property or the method (because it didn't change all instances in all project units). Therefore, I prefer to use Find in Files for the search itself. However, if the identifiers to be checked is very much, then such a search is tiring.

I'm not going to push the creation of such a search tool, I'm just asking to sue your opinions on this subject.

Title: Re: Searching for all unused global elements in the project
Post by: sash on January 16, 2019, 01:08:17 am

Well, I said there should be a reason, and perhaps you have one, but I could hardly imagine 300 const identifiers in platformer game :).

The only kind of automation I'm thinking about is to use CodeTools' parser and write your own cleanup procedure (not easy).

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 16, 2019, 01:32:14 am

Quote from: sash on January 16, 2019, 01:08:17 am

Well, I said there should be a reason, and perhaps you have one, but I could hardly imagine 300 const identifiers in platformer game :).

Hmm… I thought it was not much. 8-)

Many of these constants are used only in one place in the project (outside the Platformer.Constants.pp unit) and are declared so as not to use hard-coded magic numbers and other literals (whose meanings I would forget after a few days).

I chose this way of gathering constants (in a separate unit), because thanks to this unit it becomes something like a control panel. While calibrating various elements of the game, I modify the content of only one unit.

Quote

The only kind of automation I'm thinking about is to use CodeTools' parser and write your own cleanup procedure (not easy).

If no one needs such a tool, I can write it myself, install into the IDE and use. But certainly not in the near future, due to the lack of free time and due to the lack of sufficient knowledge in the field of extending environment functionality.

Title: Re: Searching for all unused global elements in the project
Post by: garlar27 on January 16, 2019, 06:29:47 pm

Quote from: furious programming on January 15, 2019, 10:16:17 pm

Quote from: sash on January 15, 2019, 08:51:38 pm
Normally, one should have a reason for identifier to be in global public scope. And if you have a lot of identifiers that became "unused", you should reconsider your code structure. Not just "clean unused".

I do not understand why the need to find unused "identifiers" is considered by you as a sign of poor code quality.

It is in certain way.

It is a symptom of poor design (if there is one) or sloppy implementation. It's very common in projects born as a proof of concept or experiments which suddenly become serious without: stopping to look what was done and how could be improved. Then outline a basic diagram (anyone you think could help you), how you will allocate your resources and constants. Question any global variable, constant, resource, and type: When are needed? Which class or process needs it? Is it OK that could bee seen or used in places where is not needed? etc.

I can tell that because of different reasons I usually end up in the same situation as you are now. And always try to throw everything to the garbage and start over again but with a plan or design. :-[ :'(

Now every time I have to do a proof of concept or an experiment I treat it like a NEW PROJECT :(

Quote from: furious programming on January 15, 2019, 10:16:17 pm

Quote from: garlar27 on January 15, 2019, 09:10:47 pm
But with the things we have now, the only workaround I can think of is to move all constants, types and vars to the implementation section.

Yes, but it is a different way of manual search.

It is faster to check the identifiers with the Find in Files tool, which can be easily operated from the keyboard. If the list in the Search Results window contains only one item (declaration line), it means that the constant is not used anywhere else.

Believe me. Probably the project needs it. You will be surprised of what you'll find: methods, functions and procedures which are not used which in turn use constants or types not used in other places, things that should be in other unit, etc.

But I repeat

Quote from: garlar27 on January 15, 2019, 09:10:47 pm

Yes, but I think it would be good to have a tool to show you unused element in the interface if you want to know them.

A tool or the compiler

You should open a Ticket in the bug tracker as requirement for the compiler....

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 16, 2019, 07:33:24 pm

Quote from: garlar27 on January 16, 2019, 06:29:47 pm

It's very common in projects born as a proof of concept or experiments which suddenly become serious without: stopping to look what was done and how could be improved.

Yes, this project was made as PoC and it still is. It was never meant to be a full-fledged game, but it just happened that it almost became it. But that's the way it will be, because I'm not going to develop it in the future.

I have stopped many times and corrected the old code so that it would cooperate with future new functions, but I never thought about throwing away the project and starting again.

Quote

Then outline a basic diagram (anyone you think could help you), how you will allocate your resources and constants.

When they are needed variables are created and right after that they are released.

For example, if a given class is needed throughout the game, it is created during the creation of the main game class and released when it is released. If a class is needed, for example, only in one scene, it is created during the initialization of the scene and released when it is released.

I pay attention to this, so as not to leave unused resources unnecessary.

Quote

Question any global variable, constant, resource, and type: When are needed?

Practically throughout the entire process.

Quote

Which class or process needs it?

Many classes use the same constants, which is why they are global. If any constant is used only in one unit, it is local.

Quote

Is it OK that could bee seen or used in places where is not needed? etc.

No, it's not good, but there is no tragedy either. Sometimes there is no way to easily hide all the elements, that's why stays as global, without complicate the whole code.

Quote

Believe me. Probably the project needs it. You will be surprised of what you'll find: methods, functions and procedures which are not used which in turn use constants or types not used in other places, things that should be in other unit, etc.

Yesterday I took care of it and manually checked all constants and other identifiers. I found two unused constants and no unused class or routine. It cost me almost two hours, and with the help of a dedicated tool it would cost me a few seconds. :P

Quote

A tool or the compiler
[…]
You should open a Ticket in the bug tracker as requirement for the compiler....

I'm not sure. If the compiler will report such elements, then during the compilation of a regular library/package a lot of misleading suggestions will pop up. That's why I would prefer an IDE plugin and use it only when it makes sense.

But who knows, maybe some special option for the compiler (turned off by default) would be a better solution.

This project is quite specific. Certain constructions I used only to check how they will look in practice (because I have never used them before), and some exist only for a joke (e.g. one routine with goto's instead of a regular loops). It is a one big experiment.

I think that with the code evaluation we will wait until the publication of the whole project, because in this thread such discussions do not make sense. But thanks for replies!

Title: Re: Searching for all unused global elements in the project
Post by: garlar27 on January 16, 2019, 08:30:15 pm

Quote

some special option for the compiler (turned off by default) would be a better solution.

That will be the ideal. Though I don't know if it easy to implement or not. FPC core developers should know. With a ticket at least one of them will take it if it's possible or a no go.

Title: Re: Searching for all unused global elements in the project
Post by: argb32 on January 17, 2019, 08:51:07 pm

In I-Pascal (https://forum.lazarus.freepascal.org/index.php?topic=21914.0) there is an inspection which shows unused local identifiers.
The only reason why there is no such for global identifiers is because it's not needed for each or most projects.
If you are interested I can add the inspection (disabled by default) for global identifiers to I-Pascal.

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 18, 2019, 07:27:48 pm

Quote from: argb32 on January 17, 2019, 08:51:07 pm

The only reason why there is no such for global identifiers is because it's not needed for each or most projects.

I agree with this statement only if the majority of created projects are libraries, packages and everything else that is non-executable. I do not know the statistics, so all I can do is suppose. And according to my conjecture, the most often created projects are executable applications.

Quote

If you are interested I can add the inspection (disabled by default) for global identifiers to I-Pascal.

That's nice, but I do not want to change IDE. But if you also think that such a functionality is useful, then for the sake of your project you can add it.

Title: Re: Searching for all unused global elements in the project
Post by: garlar27 on January 18, 2019, 11:44:01 pm

Quote from: furious programming on January 18, 2019, 07:27:48 pm

I agree with this statement only if the majority of created projects are libraries, packages and everything else that is non-executable.

I don't think that's the logic behind the "No interface's element will be reported as not used" thing...

I think the logic is "what you put in the interface is something you want to share that element with other units so can be used at will" sort of thing. Thus it SHOULD not be tagged with a note. If so, I do not agree with that logic either. Sometimes unit's header grows bigger and bigger and you can lose track of an element you need to remove (or whatever the reason why an element stay in the code as garbage) and it stays there using resources and not being used...

Title: Re: Searching for all unused global elements in the project
Post by: furious programming on January 19, 2019, 12:23:28 am

Quote from: garlar27 on January 18, 2019, 11:44:01 pm

I don't think that's the logic behind the "No interface's element will be reported as not used" thing...

[…]

I think the logic is "what you put in the interface is something you want to share that element with other units so can be used at will" sort of thing.

Of course, however, this is due to the code structure design of the units.

A short example. We have a class declared in interface section, whose properties are indexed with enums. The enumeration type must be declared above the class to be visible to it and not to be global (becouse it is used internally only, there is no way to use it in other unit anyway).

We have a problem—or the enumeration type will be visible globally (declared outside a class, above it in the same section), or it must be declared (embedded) inside the class, for example in private section. The same applies to constants that are used to specify default property values. Does anyone in such a case declare such elements within the class to actually be internal?

Similar cases may be more.