Recent

Author Topic: Laz 1.8.2 No memory left  (Read 32276 times)

Cyrax

  • Hero Member
  • *****
  • Posts: 836
Re: Laz 1.8.2 No memory left
« Reply #30 on: March 07, 2018, 10:03:15 am »
Does this bug occur if you are compiling Lazarus with 64-bit compiler? Note that you need native compiler, crosscompiler may have the same bug.

af0815

  • Hero Member
  • *****
  • Posts: 1289
Re: Laz 1.8.2 No memory left
« Reply #31 on: March 07, 2018, 04:51:39 pm »
Maybe an old Bug 0031517 ?!

One change is in the pp.pas aroud line 150. Now i patched the actual trunk 35551. Now fpc compiles the 'Big' Lazarus for me.

===============

{$ifdef win32}
  { 256 MB stack }
  { under windows the stack can't grow }
  {$MAXSTACKSIZE 256000000}
  {$setpeflags $20}
{$else win32}
  {$ifdef win64}
    { 512 MB stack }
    { under windows the stack can't grow }
    {$MAXSTACKSIZE 512000000}
  {$else win64}
    { 1 MB stack }
    {$MINSTACKSIZE 1000000}
  {$endif win64}
{$endif win32}

--------------------------
The PE must be set to compile Lazarus with more than 2GB at linktime.

Andreas
« Last Edit: March 07, 2018, 04:54:00 pm by af0815 »
regards
Andreas

snorkel

  • Hero Member
  • *****
  • Posts: 817
Re: Laz 1.8.2 No memory left
« Reply #32 on: March 07, 2018, 08:09:20 pm »
As I mentioned before the optimized profile seems to work around the issue whatever it really is.
In my case it just gave more room before it strikes again. My guess is that it will strike you too if you install another 10-15 packages. If you need bunch of packages for testing then you can use these: https://bitbucket.org/avra/ct4laz/downloads/ct4laz-preview.zip.

The only permanent work around so far is to use external linker (add -Xe in active build profile). I have so many packages installed that at the moment I remove -Xe and try to rebuild IDE, "No memory left" error shows every time. So yes, so far I can reproduce it whenever needed.

That could be as I only have a fairly small amount of packages I install.   I have been installing the same packages every time I update Lazarus ( or install new)  and never saw this problem before.
Even a couple of weeks again when I installed 1.8 on a new PC using same packages this error never popped up.  Is it possible some windows 10 update has messed with the stack size or something?
***Snorkel***
If I forget, I always use the latest stable 32bit version of Lazarus and FPC. At the time of this signature that is Laz 3.0RC2 and FPC 3.2.2
OS: Windows 10 64 bit

af0815

  • Hero Member
  • *****
  • Posts: 1289
Re: Laz 1.8.2 No memory left
« Reply #33 on: March 07, 2018, 09:58:47 pm »
from https://msdn.microsoft.com/en-us/library/aa384219(VS.85).aspx

Quote
WOW64 enables 32-bit applications to take advantage of the 64-bit kernel. Therefore, 32-bit applications can use a larger number of kernel handles and window handles. However, 32-bit applications may not be able to create as many threads under WOW64 as they can when running natively on x86-based systems because WOW64 allocates an additional 64-bit stack (usually 512 KB) for each thread. In addition, some amount of address space is reserved for WOW64 itself and the data structures it uses. The amount reserved depends on the processor; more is reserved on the Intel Itanium than on the x64 or ARM64 processors.

If the application has the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in the image header, each 32-bit application receives 4 GB of virtual address space in the WOW64 environment. If the IMAGE_FILE_LARGE_ADDRESS_AWARE flag is not set, each 32-bit application receives 2 GB of virtual address space in the WOW64 environment.

Maybe Lazarus is reaching the 2GB line at linktime of FPC. This is the reason for $setpeflag $20.
« Last Edit: March 07, 2018, 10:01:51 pm by af0815 »
regards
Andreas

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Re: Laz 1.8.2 No memory left
« Reply #34 on: March 08, 2018, 06:49:53 pm »
Maybe an old Bug 0031517 ?!
No, sorry. Adding {$setpeflags $20} does not help with "No memory left" error. I have even tried to set {$MAXSTACKSIZE 512000000} for win32 but that didn't help either. Thank you for your input.

The only thing that helps is -Xe switch to force external linker. Should I report this as a bug?
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

balazsszekely

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #35 on: March 08, 2018, 08:44:32 pm »
I installed almost every package from OPM and I did not see the "No memory left" error. Few notes:
1. The whole process took approximately 30 minutes
2. A few packages(3-4) failed to install for various reasons
3. I saw at least two "duplicate unit" warning
4. The lazarus.exe after install is ~0.8GB

As a conclusion the "No memory left" error is not a generic bug. Something specific to your system triggers the issue and I have no idea what is it.

PS: Lazarus Trunk/FPC 3.0.4, Win7(64 bit)
      CPU: i7-3770 3.40 GHz, RAM: 16 GB
« Last Edit: March 08, 2018, 09:01:43 pm by GetMem »

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Laz 1.8.2 No memory left
« Reply #36 on: March 08, 2018, 08:49:22 pm »
Maybe some security software enforces some memory limit ?

balazsszekely

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #37 on: March 08, 2018, 08:59:36 pm »
@marcov
Quote
Maybe some security software enforces some memory limit ?
That's a possibility. Personally I use the lightweight Microsoft Security Essentials, both FPC and Lazarus directories are white listed.
« Last Edit: March 08, 2018, 09:02:54 pm by GetMem »

zeljko

  • Hero Member
  • *****
  • Posts: 1594
    • http://wiki.lazarus.freepascal.org/User:Zeljan
Re: Laz 1.8.2 No memory left
« Reply #38 on: March 08, 2018, 09:21:01 pm »
Maybe some security software enforces some memory limit ?

Not sure, used 3.0.2 (or even it was 3.0.0) on Win10 virtual machine, when switched to fpc-3.0.4 got "No memory left" when rebuilding lazarus. My bigide contains extra > 25 packages.

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Laz 1.8.2 No memory left
« Reply #39 on: March 09, 2018, 01:14:40 am »
0.8G after a some packages being compiled tells me debug info is getting attached to the file.

That is an insane EXE file size, one that I only see if Debug info is on...

maybe if debug info is to be generated one could use an external file instead...

I have a huge program myself that I compile with gobs of large units and it fits nicely in
 6 Megs of EXE(32 bit mode), loaded with GUI stuff...
 
 BUT!, the debug file is HUGE and very slow to debug...

 I am glad for the new feature lazarus has now to RUN your app without debug! I was setting
up an ICON on the desktop to run it outside the debugger before.

« Last Edit: March 09, 2018, 01:17:12 am by jamie »
The only true wisdom is knowing you know nothing

sam707

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #40 on: March 09, 2018, 01:48:30 am »
@Jamie
debug infos have nothing to do with mem allocation issues.

1st of all, a debugger lives inside its own process, accessing its own mem mapping

2) if you run an application without a debuger, then the Header of the EXE is interpreted in such a way that the debug info are never loaded by the subsystem into memory

SO merging or not debug infos inside the app file have no impact at all in memory while not using debugers... you can forget your "INSANE" qualifier
« Last Edit: March 09, 2018, 01:52:07 am by sam707 »

sam707

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #41 on: March 09, 2018, 01:58:49 am »
keep going! folks, I can confirm there is a bug something Strange at mem allocating

laz 1.8.3

I made a loop (say 1..1000), allocating blocks, and a subloop writing bytes in the allocated blocks of 10MB (to ensure they are truely in allocation, not marked allocated by pointers)

when effective RAM ends (and OFTEN before, see below), it raises a SIGSEGV, it doesn't try to allocate virtual mems nor tell other applications running to swap to pagefile if possible!

AT the MOMENT I use cmem unit and it no more happens ;-)

I was expecting a swap attempt! as it goes on VC++ ... and longer time for allocating function to get mem before effective fail if swapfile is full. Nothing like that never happened...

the test is simple for you to reproduce.

I thought on 32 bits machines limit was 4GB allocation more or less... It happens around 2GB to HANG

I don't know on 64bits machines, because the app I actually develop doesn't need huge mem
« Last Edit: March 09, 2018, 02:12:31 am by sam707 »

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Laz 1.8.2 No memory left
« Reply #42 on: March 09, 2018, 02:53:37 am »
 Say what you want, but considering the linker needs to attach the debug file onto the
end of the EXE file, when large as it can get could cause an issue if the linker is trying to do this
in memory. I've switched using external debug files, it seems to work better for me.

 But on the other hand, it seems  you are the only one having this issue apparently.

 I know that most of us are happy with our results, sorry you can't get what you are looking for.

 Since you claimed it worked before with 1.8.0 and now it does not should tell you something..

 Have a good day.

The only true wisdom is knowing you know nothing

sam707

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #43 on: March 09, 2018, 04:02:19 am »
@Jamie
yeah have a good day AND explore PE or ELF headers you'll see I DO NOT say what I want BUT exactly WHAT SYSTEMS LOADERS DO (linux/windows) while loading LINKERED linked program

meaning => a linker CAN use specia/optional sections (segments some OSes) to write infos, resources and so on (called overlays in old good times, nowadays useless) THAT ARE NEVER LOADED EXCEPT for debugers purposes ON debugers DEMANDS

Have fun! come back when you have study a lil bit more around "insanity" hehehe

for the Strange behavior, do yourself a loop allocation test you'll see if i am the only one HAHAHAHAHAH if you go forward 2GB you'll hang like me myself all alone HAHAHAHAH
« Last Edit: March 09, 2018, 04:14:11 am by sam707 »

balazsszekely

  • Guest
Re: Laz 1.8.2 No memory left
« Reply #44 on: March 09, 2018, 06:14:29 am »
It looks like the issue is not just an isolated bug, triggered by some strange settings. @avra since you're able to reproduce with 100% accuracy, can you please run a few tests with FPC 3.0.2 and Lazarus Trunk?

 

TinyPortal © 2005-2018