Recent

Author Topic: [SOLVED] Output from TProcess breaks lines over 80 characters  (Read 10880 times)

guest62822

  • Guest
I have managed to use TProcess to run a command-line tool and output all messages in a component in my application.

The issue is, if a line exceeds 80 characters, it breaks into two lines. Is there any way to fix this?

This is not all the code, but shows how I run the process and get the output:

Code: Pascal  [Select][+][-]
  1. procedure TThreadRunProcess.UpdateTerminal;
  2. begin
  3.  
  4.   UseForm.MemoTerminal.Lines.Add(ReadBuffer);
  5.   UseForm.MemoTerminal.Update;
  6.  
  7. end;
  8.  
  9. procedure TThreadRunProcess.Execute;
  10. var
  11.   RunProcess  : TProcess;
  12. begin
  13.  
  14.   FreeOnTerminate:= true;
  15.  
  16.   { Start process }
  17.   RunProcess:= TProcess.Create(nil);
  18.  
  19.   try
  20.  
  21.     RunProcess.Executable:= ProcessExecutable;
  22.     if ProcessFlags <> '' then
  23.      RunProcess.Parameters.Add(ProcessFlags);
  24.     if ProcessFileName <> '' then
  25.      RunProcess.Parameters.Add(ProcessFileName);
  26.  
  27.     RunProcess.Options := [poUsePipes];
  28.  
  29.     RunProcess.Execute;
  30.  
  31.     { Get messages while the process is running }
  32.     while ProcessIsRunning do begin
  33.  
  34.       if (not Runprocess.Running) and (RunProcess.Output.NumBytesAvailable = 0)
  35.        then break;
  36.  
  37.       if RunProcess.Output.NumBytesAvailable > 0 then begin
  38.  
  39.         SetLength(ReadBuffer, RunProcess.Output.NumBytesAvailable);
  40.  
  41.         RunProcess.Output.Read(ReadBuffer[1],
  42.          RunProcess.Output.NumBytesAvailable);
  43.  
  44.         Synchronize(@UpdateTerminal);
  45.  
  46.       end;
  47.  
  48.     end;
  49.  
  50.   finally
  51.  
  52.     RunProcess.Free;
  53.  
  54.   end;
  55.  
  56.   { Complete thread }
  57.   Synchronize(@Complete);
  58.  
  59. end;
  60.  

ReadBuffer is a string in this case.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Output from TProcess breaks lines over 80 characters
« Reply #1 on: July 20, 2018, 11:20:44 am »
A little less than enough information, like who makes it exceed 80 characters? It can be both the executing and executed program. Personally I would do:
Code: Pascal  [Select][+][-]
  1. UseForm.MemoTerminal.Lines.Text := UseForm.MemoTerminal.Lines.Text + ReadBuffer;
  2. ReadBuffer := '';
  3. UseForm.MemoTerminal.Update;
  4.  
instead in UpdateTerminal.

There's a little dangerous point in the output reading code:
Code: Pascal  [Select][+][-]
  1. SetLength(ReadBuffer, RunProcess.Output.NumBytesAvailable); // (1)
  2.  
  3.         RunProcess.Output.Read(ReadBuffer[1],
  4.          RunProcess.Output.NumBytesAvailable); // (2)
  5.  
(2) may be larger than (1) and that can cause buffer overflow. Save it to a local variable before SetLength and use the same variable for Read, that's the safe way.

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11382
  • FPC developer.
Re: Output from TProcess breaks lines over 80 characters
« Reply #2 on: July 20, 2018, 11:40:31 am »
I'm not sure, but can't you simply enable scrollbars for your memo ? Then it won't break lines.

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #3 on: July 20, 2018, 12:06:22 pm »
A little less than enough information, like who makes it exceed 80 characters?

It's the command-line tool that output lines over 80 characters. It seems like the TProcess assumes the maximum length per line is 80 and automatically add a #13 + #10 (CR) to break the lines. I tried to filter this out, but this will also filter out the CRs that's supposed to be there and multiple lines will now be on one line.

I can't guarantee other command-line tools. but I guess a solution would be to change my own comand-line tool so it splits the messages into multiple lines.


I'm not sure, but can't you simply enable scrollbars for your memo ? Then it won't break lines.

It's not related to the memo component, I can type longer lines manually.

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11382
  • FPC developer.
Re: Output from TProcess breaks lines over 80 characters
« Reply #4 on: July 20, 2018, 12:19:27 pm »
Tprocess is binary, but doesn't return "whole" lines. So it might read in 80 bytes increments, but that is windows.

Probably the adding to a memo adds the linefeeds.

So you need to buffer what you read in the tprocess loop till you have a line  with a (cr)lf, and only then pass to memo.

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #5 on: July 20, 2018, 01:13:34 pm »
Tprocess is binary, but doesn't return "whole" lines. So it might read in 80 bytes increments, but that is windows.

Probably the adding to a memo adds the linefeeds.

So you need to buffer what you read in the tprocess loop till you have a line  with a (cr)lf, and only then pass to memo.

I did a test and noticed that the procedure UpdateTerminal was only called twice and the output is like 12 lines in total. So it grabs multiple lines at once and it seem like the output already contains all the (cr)lf. The problem is, I can't really guess which ones to remove.

I could assume if I read 80 characters and find a (cr)lf, then the line is broken into two lines, but what IF the line is exactly 80 characters long?

edit:
However, I had a quick look on the Internet and it's seems to be a good practice to keep lines under 80 characters, so I think the best solution would be to modify my command-line tool and make sure that no lines exceed 80 characters.

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Output from TProcess breaks lines over 80 characters
« Reply #6 on: July 23, 2018, 08:13:17 pm »
C's printf("string\r\n"); or Pascal's WriteLn('string'); already have (CR|LF) in the end.

MemoTerminal.Lines.Add(ReadBuffer); - And this code adds additional CR for each call.

Just echo everything as is: Leledumbo gave you a code that should work.

However, I had a quick look on the Internet and it's seems to be a good practice to keep lines under 80 characters

A good practice is to to respect actual console session's setting (MODE command for dos/windows), but I don't see how it matters, if you're piping output.

Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #7 on: July 23, 2018, 09:27:56 pm »
Quote
C's printf("string\r\n"); or Pascal's WriteLn('string'); already have (CR|LF) in the end.

MemoTerminal.Lines.Add(ReadBuffer); - And this code adds additional CR for each call.

Just echo everything as is: Leledumbo gave you a code that should work.

Yes, I have changed the code according to Leledumbo's suggestions.

It reduces a few extra linefeeds, but it doesn't solve my main issue - lines that's longer than 80 characters is still split into multiple lines - ReadBuffer already contains a (LF), even before adding them to the Memo component.

ReadBuffer =
#1: The first 80 characters(LF) <-- Added by TProcess
#2: The rest of the line(LF)

So, my question is: Is it possible to change the behavior of TProcess?

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Output from TProcess breaks lines over 80 characters
« Reply #8 on: July 23, 2018, 10:08:38 pm »
ReadBuffer =
#1: The first 80 characters(LF) <-- Added by TProcess
#2: The rest of the line(LF)

So, my question is: Is it possible to change the behavior of TProcess?
AFAIR, TProcess does *not* change the stream. It does not add LF. In fact it uses THandleStream.

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #9 on: July 23, 2018, 11:16:56 pm »
ReadBuffer =
#1: The first 80 characters(LF) <-- Added by TProcess
#2: The rest of the line(LF)

So, my question is: Is it possible to change the behavior of TProcess?
AFAIR, TProcess does *not* change the stream. It does not add LF. In fact it uses THandleStream.

Ah, I see.

Well, the only solution I have for now is to keep the output lines (from my command-line tool) shorter than 80 characters.

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Output from TProcess breaks lines over 80 characters
« Reply #10 on: July 23, 2018, 11:25:15 pm »
#1: The first 80 characters(LF) <-- Added by TProcess

Normally it should not, unless you set WindowColumns or similar properties (are you on Windows?)
Are you sure your child process (isn't it chained like with more?) does not chop lines by itself? To check, run it from console with output redirection like

Code: Bash  [Select][+][-]
  1. childproc -args > output.txt

Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #11 on: July 23, 2018, 11:47:50 pm »
#1: The first 80 characters(LF) <-- Added by TProcess

Normally it should not, unless you set WindowColumns or similar properties (are you on Windows?)
Are you sure your child process (isn't it chained like with more?) does not chop lines by itself? To check, run it from console with output redirection like

Code: Bash  [Select][+][-]
  1. childproc -args > output.txt

The command-line tool is written in Pascal (compiled with FPC) and use Writeln to output the messages.

I'm a Linux user, so no clue if it behave differently on Windows. I guess I can give it a try.

440bx

  • Hero Member
  • *****
  • Posts: 3944
Re: Output from TProcess breaks lines over 80 characters
« Reply #12 on: July 24, 2018, 12:49:31 am »
It reduces a few extra linefeeds, but it doesn't solve my main issue - lines that's longer than 80 characters is still split into multiple lines - ReadBuffer already contains a (LF), even before adding them to the Memo component.

I only know Windows and only the API (I don't use components), I don't know if what I am going to mention is applicable in the case you've described but... just in case. 

When you output to a console in Windows, the default width of the console is 80 characters.  Whenever a program writes 80 characters or more onto the console, the line automatically gets split into two (since it does not fit on one line.)

In Windows, you can programmatically control the width and height of the console.  In Windows, the solution would simply be to make the console wider, that would solve the problem you are experiencing.  I presume that this is possible in the OS you are using but, I have no idea how it's done but, I figured you might.  If you can and know how, simply make the console window wider.

I hope that comment might be useful to you.
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Output from TProcess breaks lines over 80 characters
« Reply #13 on: July 24, 2018, 02:09:10 am »
#1: The first 80 characters(LF) <-- Added by TProcess
Before outputting to a TMemo, try outputting to a console. I'm still not clear who wraps that 80 characters, it could be the TMemo itself. But I assure you TProcess doesn't mess with it at all.
AFAIR, TProcess does *not* change the stream. It does not add LF. In fact it uses THandleStream.
Correct.

guest62822

  • Guest
Re: Output from TProcess breaks lines over 80 characters
« Reply #14 on: July 24, 2018, 03:07:59 am »
#1: The first 80 characters(LF) <-- Added by TProcess
Before outputting to a TMemo, try outputting to a console. I'm still not clear who wraps that 80 characters, it could be the TMemo itself. But I assure you TProcess doesn't mess with it at all.
AFAIR, TProcess does *not* change the stream. It does not add LF. In fact it uses THandleStream.
Correct.

I did check the whole string (the content of ReadBuffer) by outputting character by character - I wrote '(cr)' for character code 13 and '(lf)' for character code 10. So, I can confirm that a LF is already added at this point (before the string is added to TMemo).

 

TinyPortal © 2005-2018