Recent

Author Topic: Disable parsing of include files? [solved]  (Read 7652 times)

creaothceann

  • Full Member
  • ***
  • Posts: 117
Disable parsing of include files? [solved]
« on: November 29, 2018, 07:36:43 pm »
I use compiler directives to (un-)define symbols and to include files over several levels.

Code: [Select]
case i of
        00:  {$define Phase00}  {$i I_en}  {$undef Phase00};
        01:  {$define Phase01}  {$i I_en}  {$undef Phase01};
        02:  {$define Phase02}  {$i I_en}  {$undef Phase02};
        03:  {$define Phase03}  {$i I_en}  {$undef Phase03};
        04:  {$define Phase04}  {$i I_en}  {$undef Phase04};
        05:  {$define Phase05}  {$i I_en}  {$undef Phase05};
        06:  {$define Phase06}  {$i I_en}  {$undef Phase06};
        07:  {$define Phase07}  {$i I_en}  {$undef Phase07};
        08:  {$define Phase08}  {$i I_en}  {$undef Phase08};
        09:  {$define Phase09}  {$i I_en}  {$undef Phase09};
        10:  {$define Phase10}  {$i I_en}  {$undef Phase10};
        11:  {$define Phase11}  {$i I_en}  {$undef Phase11};
        12:  {$define Phase12}  {$i I_en}  {$undef Phase12};
        13:  {$define Phase13}  {$i I_en}  {$undef Phase13};
        14:  {$define Phase14}  {$i I_en}  {$undef Phase14};
        15:  {$define Phase15}  {$i I_en}  {$undef Phase15};
        16:  {$define Phase16}  {$i I_en}  {$undef Phase16};
        17:  {$define Phase17}  {$i I_en}  {$undef Phase17};
end;
Code: [Select]
case o of
        {$define n}  {$i I_o}  {$undef n}
        {$define e}  {$i I_o}  {$undef e}
end
Code: [Select]
{$ifdef n} $00 {$else} $0100 {$endif}:    begin  {$define i_BRK}  {$i am_Stack_BRK_COP}                 {$undef i_BRK}  end;
{$ifdef n} $01 {$else} $0101 {$endif}:    begin  {$define i_ORA}  {$i am_DirectIndexedIndirect}         {$undef i_ORA}  end;
{$ifdef n} $02 {$else} $0102 {$endif}:    begin  {$define i_COP}  {$i am_Stack_BRK_COP}                 {$undef i_COP}  end;
{...}
{$ifdef n} $FF {$else} $01FF {$endif}:    begin  {$define i_SBC}  {$i am_AbsoluteLongX}                 {$undef i_SBC}  end;


Naturally this makes compilation slow, but it also makes editing slow.

In my test program (see below), loading the project, editing the unit, and editing the include files (if they're loaded into the IDE) incurs pause(s) of several seconds. My guess is that it's for syntax highlighting.

Is that unavoidable (and I have to edit these files with another editor), or can I speed up working with these files?
« Last Edit: November 30, 2018, 07:47:00 pm by creaothceann »

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Disable parsing of include files?
« Reply #1 on: November 29, 2018, 08:13:11 pm »
But why do you need to include the same file 18 times?
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Re: Disable parsing of include files?
« Reply #2 on: November 29, 2018, 08:43:57 pm »
But why do you need to include the same file 18 times?

One may assume that what gets really included depends on the previous $DEFINE. But yes, it looks weird: I had to stop and think :)

ETA
 
Is that unavoidable (and I have to edit these files with another editor), or can I speed up working with these files?

That ... let's call it "style" ... of programming will inevitably slow down any "smart" editor like the Lazarus one. Notepad style editors won't care because they don't have to parse your source to try to help you.
« Last Edit: November 29, 2018, 09:01:55 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

wp

  • Hero Member
  • *****
  • Posts: 11858
Re: Disable parsing of include files?
« Reply #3 on: November 29, 2018, 08:46:05 pm »
A very strange programming style for Pascal. I get the impression you are confusing Pascal with an interpreted script. Why don't you use variables to control program flow?

creaothceann

  • Full Member
  • ***
  • Posts: 117
Re: Disable parsing of include files?
« Reply #4 on: November 29, 2018, 09:10:37 pm »
But why do you need to include the same file 18 times?
The included files contain different sections that are activated by the PhaseXX defines.

This is for an emulator (a bytecode interpreter). There are 256 opcodes, and each opcode can be 2..9 cycles. 1 cycle is 2 phases. Also, the CPU can be in 2 modes (indicated by n and e). So I have to differentiate between up to 18*2*256 = 9216 phase handlers; each of them is actually only a few lines of code (and many are empty).

I have most of that code already written, now I have to put it together somehow though and turn it into a working program.

A very strange programming style for Pascal.
Yeah, it's a bit closer to C etc. than to Pascal. Including compilation times.

I get the impression you are confusing Pascal with an interpreted script.
I'm deliberately using $ifdefs here, this is supposed to be resolved at compile time. (And no, inline doesn't always work - already tried that.)

Why don't you use variables to control program flow?
Because maintaining that state adds to the system requirements and we don't have 10GHz CPUs yet.

But this is actually not solely for running the program: this is for storing the code that is easily accessible. I don't know yet if my approach at the main loop is going to be the fastest one, so I put the code into include files to be able to try several designs without huge rewrites.

Now it turns out that each added or removed character slows down the editor to a crawl. Sigh.
« Last Edit: November 29, 2018, 09:13:10 pm by creaothceann »

creaothceann

  • Full Member
  • ***
  • Posts: 117
Re: Disable parsing of include files?
« Reply #5 on: November 29, 2018, 09:12:42 pm »
That style of programming will inevitably slow down any "smart" editor like the Lazarus one. Notepad style editors won't care because they don't have to parse your source to try to help you.
Yeah, that's what I've been thinking. But maybe there's a checkbox somewhere that disables it? I don't want to change and recompile Lazarus just for that.

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Re: Disable parsing of include files?
« Reply #6 on: November 29, 2018, 09:23:09 pm »
You'd have to disable well over half of what makes the editor distinctive: most of CodeTools, interface<->implementation jumps, etc.

I may be wrong (never found myself in a situation like this) but maybe it will go quicker if you have the include files opened in an editor window.( ??? )
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

creaothceann

  • Full Member
  • ***
  • Posts: 117
Re: Disable parsing of include files?
« Reply #7 on: November 29, 2018, 09:43:24 pm »
I may be wrong (never found myself in a situation like this) but maybe it will go quicker if you have the include files opened in an editor window. (???)
An external editor? Because right now I'm editing them directly in Lazarus.

wp

  • Hero Member
  • *****
  • Posts: 11858
Re: Disable parsing of include files?
« Reply #8 on: November 29, 2018, 09:44:46 pm »
I'm deliberately using $ifdefs here, this is supposed to be resolved at compile time.
But how are you going to resolve this at compile time, i is only known at runtime.
Code: Pascal  [Select][+][-]
  1. case i of
  2.         00:  {$define Phase00}  {$i I_en}  {$undef Phase00};
  3.         01:  {$define Phase01}  {$i I_en}  {$undef Phase01};
  4.    ...
  5.  

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Disable parsing of include files?
« Reply #9 on: November 29, 2018, 09:48:11 pm »
Because maintaining that state adds to the system requirements and we don't have 10GHz CPUs yet.

So your "optimization" is for the sake of elimination of 2-3 nested cases/ifs ?
But wait, you still use cases/ifs, which are evaluated at runtime.
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

creaothceann

  • Full Member
  • ***
  • Posts: 117
Re: Disable parsing of include files?
« Reply #10 on: November 29, 2018, 10:06:21 pm »
I'm deliberately using $ifdefs here, this is supposed to be resolved at compile time.
But how are you going to resolve this at compile time, i is only known at runtime.
Code: Pascal  [Select][+][-]
  1. case i of
  2.         00:  {$define Phase00}  {$i I_en}  {$undef Phase00};
  3.         01:  {$define Phase01}  {$i I_en}  {$undef Phase01};
  4.    ...
  5.  
I mean that these are compiler directives, not script directives.

So your "optimization" is for the sake of elimination of 2-3 nested cases/ifs ?
But wait, you still use cases/ifs, which are evaluated at runtime.
Yes, currently my approach is to use case-of on the current phase (will be in the local variable i), then use case-of on the current mode and opcode (stored in a Word as (Mode SHL 8) OR Opcode). So there will be one case-of selecting between 18 code paths, and 18 nested case-ofs each selecting between 2*256 code paths.

So while having many case-ofs isn't ideal, I think this is going to work the best with the CPU branch predictors. But I don't know yet because I can't run any benchmarks yet.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Disable parsing of include files?
« Reply #11 on: November 29, 2018, 10:38:54 pm »
My guess is that it's for syntax highlighting.

The main highlighting works per file, and single pass. It will not care about includes.

But codetools do. That includes the "lowlighting" of inactive ifdef blocks. That will be a lot in your case. But more important, it is used for codecompletion and all that. Not sure how well that works in your project.

What you could try....
Move the include files to a different folder. Add this path in a place where codetools does not find it (not sure, maybe fpc.cfg? or a wrapper script around fpc).
Then codetools should not be able to find the files. Even if open, they would be seen as independent files.
Dont know if it will help...

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Disable parsing of include files?
« Reply #12 on: November 29, 2018, 11:10:29 pm »
Quote
So while having many case-ofs isn't ideal, I think this is going to work the best with ...

My point that you are gaining nothing: you still need to traverse and evaluate your control-flow nodes (case-of) the same number of times as you would without your "include hell".

The only benefit you could probably have (what depends on amount of repetitive fragments) - a more compact source, what is irrelevant to runtime optimizations, because binary code is still generated for each {$include}, but adds drawbacks as poorly maintainable code (IMO) and IDE performance issues.
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Re: Disable parsing of include files?
« Reply #13 on: November 29, 2018, 11:32:25 pm »
With so many code paths your best bet, IMHO, is to use a jump table--or two--built on startup or at compile-time. That will probably be quicker than using case statements.

ETA: Sorry, didn't see this:
I may be wrong (never found myself in a situation like this) but maybe it will go quicker if you have the include files opened in an editor window. (???)
An external editor? Because right now I'm editing them directly in Lazarus.

No, I meant opened in the Lazarus editor; but if you're alredy doing that ... :-X
« Last Edit: November 29, 2018, 11:35:50 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

creaothceann

  • Full Member
  • ***
  • Posts: 117
Re: Disable parsing of include files?
« Reply #14 on: November 29, 2018, 11:42:17 pm »
What you could try....
Move the include files to a different folder. Add this path in a place where codetools does not find it (not sure, maybe fpc.cfg? or a wrapper script around fpc).
Then codetools should not be able to find the files. Even if open, they would be seen as independent files.
Don't know if it will help...
Alright.
I guess I'll just remove the directory from the "include files" path in the project settings, and add it to a batch file for compiling the project (via lazbuild?). Or something like that. This should allow me to still use the IDE for basic editing.

you still need to traverse and evaluate your control-flow nodes (case-of) the same number of times as you would without your "include hell".
Yes, but that's not the sole decisive factor. Not all CPU operations are equal. An ADD instruction that reads a random memory address is slower than dozens of correctly predicted if/case-of jumps.

In my case, the first case-of follows a simple pattern: most phases are executed in order. Phase 6 follows 5 follows 4 follows 3 etc. ... The second case-of would benefit from modern branch prediction that exploits correlations between the opcodes of the emulated program (e.g. instruction ADC is often preceded by instruction CLC).

The only benefit you could probably have (what depends on amount of repetitive fragments) - a more compact source, what is irrelevant to runtime optimizations, because binary code is still generated for each {$include}, but adds drawbacks as poorly maintainable code (IMO) and IDE performance issues.
A more compact source is very important to me if the alternative is writing thousands of lines by hand. Maybe I'll find a more maintainable way, too.

With so many code paths your best bet, IMHO, is to use a jump table--or two--built on startup or at compile-time. That will probably be quicker than using case statements.
Afaik case-of is already compiled into a jump table if it has more than a dozen cases or so.
I did dabble with computed GOTOs, but they need you to list every label and label address, and CPUs >= Haswell shouldn't need this these days anyway. Hopefully.
« Last Edit: November 29, 2018, 11:50:35 pm by creaothceann »

 

TinyPortal © 2005-2018