Lazarus

Programming => Networking and Web Programming => Topic started by: xinyiman on November 07, 2018, 09:39:39 am

Title: (Indy) Web application get page error
Post by: xinyiman on November 07, 2018, 09:39:39 am
Hi guys, I created a web app with integrated web server (indy 10) to replicate errors. I would like your opinion on this. What's wrong?
I have to read a json string through an http call and I tried 3 ways. The first works, the other two do not. But I would like to understand what is different. For me it is absurd. Who can help me? With indy, reading this string generates an error that is not always replicable:

HTTP / 1.1 200 OK Connection: close Content-Length: 397

While synapse the version that I expect to work does not work, while the one written by me. Tips? I enclose an example.

This is my browser result:
Code: Pascal  [Select][+][-]
  1. correct: { "token" : "ZVRoQWN0aFF6UExkdmQ4VTVlcmd4VGkxM1duQVdpUFFydnBESGJ4eUhQTUtTUUhoNldGTU9sWlFscjhna0JZdXlaeTBiWFVaWSs5a1dVa0RtbnJtN1NPVm1RT25sd1VuLzdINlBHcUl4NWtoVjl0bFJJWXZ6blFDYytrMEdseStjRitwNFlRY1U4WWJBdjc2U1pXbk11RnBzWFlGZ2hBWThJQThxaTBrLy9MQlBhU1loeTVCYzI1VzdWcTBFVGEyRGxSMkV3WkRqdWxlbmNRMWo2bWlVK1kzRi9sbnEwU2R6Q3BHV3JaR3gwTGYrblpwUmNEaVc1UEdDdC9mQTVUbGs2OUtOanloUlJWN1ZIR2JydlRYQnh0V3ljbz0=" }
  2. error 1: { "token" : "ZVRoQWN0aFF6UExkdmQ4VTVlcmd4VGkxM1duQVdpUFFydnBESGJ4eUhQTUtTUUhoNldGTU9sWlFscjhna0JZdXlaeTBiWFVaWSs5a1dVa0RtbnJtN1NPVm1RT25sd1VuLzdINlBHcUl4NWtoVjl0bFJJWXZ6blFDYytrMEdseStjRitwNFlRY1U4WWJBdjc2U1pXbk11RnBzWFlGZ2hBWThJQThxaTBrLy9MQlBhU1loeTVCYz
  3. error 2: HTTP/1.1 200 OK Connection: close Content-Length: 397 Server: NGIT Bootstrap Application Set-Cookie: IDHTTPSESSIONID=nh2ari4EVMF2sxBnh2ari4EVMF2sxBnh2ari4EVMF2sxBnh2ari4EVMF2sxBnh2ari4EVMF2sxBnh2ari4EVMF2sxB; path=/ { "token" : "ZVRoQWN0aFF6UExkdmQ4VTVlcmd4VGkxM1duQVdpUFFydnBESGJ4eUhQTUtTUUhoNldGTU9sWlFscjhna0JZdXlaeTBiWFVaWSs5a1dVa0RtbnJtN1NPVm1RT25sd1VuLzdINlBHcUl4NWtoVjl0bFJJWXZ6blFDY
  4.  
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 07, 2018, 12:01:59 pm
It was pointed out to me that I left dcpcrypt among the dependencies. You can remove it, it is not necessary for the use of the test program
Title: Re: (Indy) Web application get page error
Post by: Thaddy on November 07, 2018, 12:16:56 pm
The error is obviously a terminating zero because it expects a PChar. Will test after lunch.
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 07, 2018, 07:34:14 pm
Quote from: xinyiman on November 07, 2018, 09:39:39 am
Hi guys, I created a web app with integrated web server (indy 10) to replicate errors. I would like your opinion on this. What's wrong?

Some mistakes I see:

- your TIdHTTPServer.OnCommandGet event handler is catching and discarding all raised exceptions.  Don't do that!  Let TIdHTTPServer handle exceptions so it can send error messages back to the HTTP client.  If you are going to catch exceptions for your own logging, at least re-raise them when you are done using them.  Otherwise, when the OnCommandGet event handler exits cleanly, TIdHTTPServer will send whatever reply data is currently in the the TIdHTTPResponse object (if it has not already been sent manually by the OnCommandGet event handler), and that will default to sending an HTTP 200 reply.

- your TNGITBootstrapApplication constructor should be initializing its FHTMLDir and FTemplate members before activating the TIdHTTPServer, so they are ready for use before any clients can connect and potentially access them before they are ready.

- TNGITController.GetHTML() is not protecting itself from exceptions when version is 2.  You need to add try..finally blocks to free the TIdHTTP and TMemoryStream objects correctly.  TIdHTTP.Get() will raise an exception if it receives an error from the server (unless you disable that option in the TIdHTTP.HTTPOptions property).
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 08, 2018, 08:23:42 am
Thank you Remy Lebea. One question, your suggestion is like my attachment?

Thank you
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 08, 2018, 12:10:10 pm
Quote
(unless you disable that option in the TIdHTTP.HTTPOptions property)

How? I searched but only HTTPOptions value are:

Code: Pascal  [Select][+][-]
  1.   // Protocol options
  2.   TIdHTTPOption = (hoInProcessAuth, hoKeepOrigProtocol, hoForceEncodeParams);
  3.   TIdHTTPOptions = set of TIdHTTPOption;  
  4.  

Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 08, 2018, 08:19:07 pm
Quote from: xinyiman on November 08, 2018, 08:23:42 am
One question, your suggestion is like my attachment?

I don't see my suggested changes applied to that code.  Especially the OnCommandGet change in particular.  It is still catching and discarding all exceptions.

Quote from: xinyiman on November 08, 2018, 12:10:10 pm
How? I searched but only HTTPOptions value are:

Code: Pascal  [Select][+][-]
  1.   // Protocol options
  2.   TIdHTTPOption = (hoInProcessAuth, hoKeepOrigProtocol, hoForceEncodeParams);
  3.   TIdHTTPOptions = set of TIdHTTPOption;  
  4.  

You claim to be using Indy 10, but are you maybe using Indy 9 instead?  What you show above is what TIdHTTPOption looks like in Indy 9.  In the current Indy 10 release, TIdHTTPOption has MANY more options:

Code: [Select]
  TIdHTTPOption = (hoInProcessAuth, hoKeepOrigProtocol, hoForceEncodeParams,
    hoNonSSLProxyUseConnectVerb, hoNoParseMetaHTTPEquiv, hoWaitForUnexpectedData,
    hoTreat302Like303, hoNoProtocolErrorException, hoNoReadMultipartMIME,
    hoNoParseXmlCharset, hoWantProtocolErrorContent, hoNoReadChunked
    );

When I mentioned configuring TIdHTTP.Get() to not raise an exception on a server error, I was referring to the hoNoProtocolErrorException option.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 08, 2018, 11:45:09 pm
I see but lazarus have install indylaz v10.2.0.3 but i have only: hoInProcessAuth, hoKeepOrigProtocol, hoForceEncodeParams

where I find the version of indy 10 that contains all the options you say?!
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 09, 2018, 09:31:50 am
Ok, i find a correct version of indy. Now i change with your suggestion. But now my program closed when into browser page run fast. If you try to quickly reload the page without waiting for the page to load before the program closes without giving any kind of error. Why?
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 09, 2018, 10:27:53 pm
Quote from: xinyiman on November 08, 2018, 11:45:09 pm
I see but lazarus have install indylaz v10.2.0.3 but i have only: hoInProcessAuth, hoKeepOrigProtocol, hoForceEncodeParams

That is an extremely old version of Indy.  The current version is 10.6.2.5485.

Quote from: xinyiman on November 08, 2018, 11:45:09 pm
where I find the version of indy 10 that contains all the options you say?!

You can download Indy from its official download mirrors (http://www.indyproject.org/Sockets/Download/DevSnapshot.EN.aspx), which includes Lazarus's own Online Package Manager (http://wiki.freepascal.org/Online_Package_Manager) (though, I don't know which version of Indy is currently in OPM.  IIRC, it does sync up fairly often).

Quote from: xinyiman on November 09, 2018, 09:31:50 am
Ok, i find a correct version of indy. Now i change with your suggestion. But now my program closed when into browser page run fast. If you try to quickly reload the page without waiting for the page to load before the program closes without giving any kind of error. Why?

I have no idea what you are referring to.  Please clarify.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 09, 2018, 10:50:17 pm
Have you tried to download and run my last attached example? Have you tried to run it? You'll see that if you try to run my program and quickly load the page in the browser, the http server closes itself without giving an error.
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 09, 2018, 11:46:45 pm
Quote from: xinyiman on November 09, 2018, 10:50:17 pm
Have you tried to download and run my last attached example? Have you tried to run it?

No, because I'm not a FreePascal/Lazarus user, so I couldn't compile and run your project even if I wanted to.

Quote from: xinyiman on November 09, 2018, 10:50:17 pm
You'll see that if you try to run my program and quickly load the page in the browser, the http server closes itself without giving an error.

It is your responsibility to debug your own project.  I can't help you with that.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 10, 2018, 12:27:50 am
But Is not a Project problem, because whit old version of Indy not exists this problem. For me Is a library problem. You know a Lazarus developer from Indy team?!
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 10, 2018, 04:56:21 pm
I testes my project example on other O.S. and this is the result

1. Windows: no problem, my project it did not close unexpectedly

2. Linux: problem, my project it closed unexpectedly

3. Mac OSX: problem, my project it closed unexpectedly

Into unix like OS is problem
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 10, 2018, 05:58:23 pm
I found the place where the problem is generated, in the IdHTTP unit there is the function

Code: Pascal  [Select][+][-]
  1. function TIdCustomHTTP.Get(AURL: string
  2.   {$IFDEF STRING_IS_ANSI}; ADestEncoding: IIdTextEncoding = nil{$ENDIF}
  3.   ): string;
  4. begin
  5.   Result := Get(AURL, []{$IFDEF STRING_IS_ANSI}, ADestEncoding{$ENDIF});
  6. end;  
  7.  

It happens that if the server receives two simultaneous calls it closes. This is on linux and on mac osx. Only I do not understand how to solve. Some idea?
Title: Re: (Indy) Web application get page error
Post by: asdf121 on November 10, 2018, 06:45:03 pm
Did you define a TIdCompressorZLib for using gzip? I think there is an issue because my old code does not work anymore after updating Indy to latest version.
Maybe it's related to the change from Updating various DLL related functions to use THandle instead of HMODULE (https://github.com/graemeg/indy/commit/f1a0fd71844dbaa2a9b3cd10c373d43e38f48421)? If I remove the support for gzip from my TIdHTTP object, it works flawless.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 11, 2018, 01:10:11 am
how? if like this not found

idhttp1.Compressor:=nil;
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 11, 2018, 04:33:14 pm
Ok, I've done other tests and I understand that the problem is not the GET call, the problem is that the http server crashes when idhttpserver1.OnCommandGet is called the second time but has not finished running the first one yet. I modified the source, if you open it you will see that now there is a sleep (4000); in order to give you time to load the page in the browser the second time. Can someone tell me what's wrong? I do not understand. It should be managed multithreaded so it should not happen. Ideas?
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 12, 2018, 11:20:12 am
No suggestions? Has anyone at least managed to replicate my problem?
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 12, 2018, 01:34:22 pm
In the face of other tests I understood that the problem exists only if the requests are made by the same PC on which the http server runs. If I do it with another computer, or the smartphone or tablet, the server does not crash. Ideas?
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 12, 2018, 08:22:08 pm
Quote from: xinyiman on November 10, 2018, 12:27:50 am
You know a Lazarus developer from Indy team?!

I'm the only developer left on the team.  Other remaining members are admins.

Quote from: xinyiman on November 10, 2018, 05:58:23 pm
I found the place where the problem is generated, in the IdHTTP unit there is the function

Code: Pascal  [Select][+][-]
  1. function TIdCustomHTTP.Get(AURL: string
  2.   {$IFDEF STRING_IS_ANSI}; ADestEncoding: IIdTextEncoding = nil{$ENDIF}
  3.   ): string;
  4. begin
  5.   Result := Get(AURL, []{$IFDEF STRING_IS_ANSI}, ADestEncoding{$ENDIF});
  6. end;  
  7.  

It happens that if the server receives two simultaneous calls it closes. This is on linux and on mac osx. Only I do not understand how to solve. Some idea?

What 2 calls are you referring to?  There is only 1 GET request in that code.  Just one overloaded Get() calling another overloaded Get() with extra parameters.  But only 1 GET request is sent to the server.

Quote from: xinyiman on November 11, 2018, 04:33:14 pm
Ok, I've done other tests and I understand that the problem is not the GET call, the problem is that the http server crashes when idhttpserver1.OnCommandGet is called the second time but has not finished running the first one yet.

What kind of crash EXACTLY? What is your logging telling you?

TIdHTTPServer (like all of Indy's TCP servers) is a multi-threaded component.  The OnCommandGet event (and other events) is triggered in the context of a worker thread for a particular client (represented by the AContext parameter).  OnCommandGet can be called multiple times in parallel, but only when receiving requests from multiple clients.  For a given client, the events are serialized, and HTTP is a command/response protocol, so it is simply not possible for OnCommandGet to be called for a client while it is already running for that same client.  But, it can certainly be called for another client.  A browser can (and frequently does) make multiple TCP connections to a server.  That is perfectly normal.  It is your responsibility to make sure the code in your event handlers is thread-safe.

Quote from: xinyiman on November 11, 2018, 04:33:14 pm
I modified the source, if you open it you will see that now there is a sleep (4000); in order to give you time to load the page in the browser the second time.

That merely blocks a client from responding for 4 seconds.  If you ask your browser to make a new request while the previous request is still blocked, the browser will simply open a new TCP connection to the server (thus running a new thread in the server) to make that new request.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 13, 2018, 08:13:35 am
I meant two calls from the clients. If I open the browser from the same computer it is the server and very quickly launch two pages the program closes without daring errors. Can you explain why? On windows it does not happen, only on unix-like systems (I tried mac osx and ubuntu).
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 13, 2018, 08:51:46 pm
Quote from: xinyiman on November 13, 2018, 08:13:35 am
If I open the browser from the same computer it is the server and very quickly launch two pages the program closes without daring errors.

Then you are doing something wrong in your code.  You need to debug it to see how it reacts with multiple browser requests being made at the same time.
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 14, 2018, 08:11:40 am
My code is the one I have above. It comes from one of the examples released with the package. I do not doubt it is a mistake. I'm asking for help precisely because I do not understand where I'm wrong.
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 14, 2018, 08:58:48 pm
Quote from: xinyiman on November 14, 2018, 08:11:40 am
I do not doubt it is a mistake. I'm asking for help precisely because I do not understand where I'm wrong.

Well, you didn't answer my earlier questions yet:

Quote
What kind of crash EXACTLY? What is your logging telling you?

You are the one running the server.  How does it behave exactly?  HOW is it crashing?  WHAT is being reported to you when the crash is detected?
Title: Re: (Indy) Web application get page error
Post by: xinyiman on November 15, 2018, 08:24:38 am
As I have already told you, it does not report any errors, simply the program closes itself.
Title: Re: (Indy) Web application get page error
Post by: Remy Lebeau on November 19, 2018, 09:34:17 pm
Quote from: xinyiman on November 15, 2018, 08:24:38 am
As I have already told you, it does not report any errors, simply the program closes itself.

As I stated earlier, you need to step through your code with a debugger and find out exactly what is happening.  I can't help you with that.
TinyPortal © 2005-2018