Recent

Author Topic: [SOLVED] How to download HTML from secure https site  (Read 3457 times)

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 590
    • Double Dummy Solver - free download
[SOLVED] How to download HTML from secure https site
« on: October 20, 2018, 12:05:34 am »
I've been using the following:

Code: Pascal  [Select][+][-]
  1. function DownloadHTTP(URL, TargetFile: string): boolean;
  2. var
  3.   HTTPSender: THTTPSend;
  4. begin
  5.   Result := False;
  6.   HTTPSender := THTTPSend.Create;
  7.   try
  8.     HTTPSender.HTTPMethod('GET', URL);
  9.     if (HTTPSender.ResultCode >= 100) and (HTTPSender.ResultCode <= 299) then
  10.     begin
  11.       HTTPSender.Document.SaveToFile(TargetFile);
  12.       Result := True;
  13.     end;
  14.   finally
  15.     HTTPSender.Free;
  16.   end;
  17. end;  

to download the HTML from HTTP web site. Is there a similar function I can use to download from HTTPS sites?
« Last Edit: October 20, 2018, 06:14:54 pm by bobonwhidbey »
Lazarus 3.0RC2, FPC 3.2.2 x86_64-win64-win32/win64

CCRDude

  • Hero Member
  • *****
  • Posts: 596
Re: How to download HTML from secure https site
« Reply #1 on: October 20, 2018, 07:23:56 am »
Yes, the same, but you need to make sure you add a SSL library to the uses list... like ssl_openssl (also from synapse) :)

Thaddy

  • Hero Member
  • *****
  • Posts: 14201
  • Probably until I exterminate Putin.
Re: How to download HTML from secure https site
« Reply #2 on: October 20, 2018, 02:03:53 pm »
Yes, I recognize that code  :D O:-)
It is indeed sufficient to add ssl_openssl to the uses clause. Nothing more required.
Full program:
Code: Pascal  [Select][+][-]
  1. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
  2. uses sysutils,classes,httpsend, ssl_openssl;
  3.  
  4. function DownloadHTTP(URL, TargetFile: string): boolean;
  5. var
  6.   HTTPSender: THTTPSend;
  7. begin
  8.   Result := False;
  9.   HTTPSender := THTTPSend.Create;
  10.   try
  11.     HTTPSender.HTTPMethod('GET', URL);
  12.     if (HTTPSender.ResultCode >= 100) and (HTTPSender.ResultCode <= 299) then
  13.     begin
  14.       HTTPSender.Document.SaveToFile(TargetFile);
  15.       Result := True;
  16.     end;
  17.   finally
  18.     HTTPSender.Free;
  19.   end;
  20. end;  
  21. var
  22.   s:string ='testme.html';
  23.   l:Tstringlist;
  24. begin
  25.   if fileexists(s) then deletefile(s);
  26.   if Downloadhttp('https://www.freepascal.org/index.html',s) then
  27.   try
  28.     l:=Tstringlist.create;
  29.     l.LoadfromFile(s);
  30.     writeln(l.text);
  31.   finally
  32.     l.free;
  33.   end;
  34. end.

Or much simpler, like this:
Code: Pascal  [Select][+][-]
  1. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
  2. uses classes, httpsend, ssl_openssl;
  3. var
  4.   l:Tstrings;
  5. begin
  6.   l:=Tstringlist.create;
  7.   try
  8.     // this is a standard utility function from synapse
  9.     if HttpGetText('https://www.freepascal.org/index.html',L) then writeln(l.text); // or l.savetofile
  10.   finally
  11.     l.free;
  12.   end;
  13. end.

That's really all...



« Last Edit: October 20, 2018, 10:09:08 pm by Thaddy »
Specialize a type, not a var.

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 590
    • Double Dummy Solver - free download
Re: How to download HTML from secure https site
« Reply #3 on: October 20, 2018, 06:13:59 pm »
Thank you - works perfectly.
Lazarus 3.0RC2, FPC 3.2.2 x86_64-win64-win32/win64

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 590
    • Double Dummy Solver - free download
Re: [SOLVED] How to download HTML from secure https site
« Reply #4 on: October 22, 2018, 05:43:34 pm »
While MyApp seems to work perfectly, I am getting a compiler warning that I would like to know how to eliminate.

"Warning: other unit files search path (aka unit paty) of "MyApp" contains "C:\Lazarus\components\custom\Synapse\source\lib", which belongs to package "laz_synapse".

I have added the laz_synapse package to the Project Inspector.
Lazarus 3.0RC2, FPC 3.2.2 x86_64-win64-win32/win64

Thaddy

  • Hero Member
  • *****
  • Posts: 14201
  • Probably until I exterminate Putin.
Re: [SOLVED] How to download HTML from secure https site
« Reply #5 on: October 22, 2018, 06:37:41 pm »
The synapse component, not the synapse library, is utterly useless and you should uninstall it.
It is also the cause of this warning. It will go away when you remove the component.

Synapse (the original, real Synapse) is purposefully written to NOT be a component but a set of classes and routines to be used from source.

Synapse itself is world-class
The  component is a piece of cow dung. The writer -  not the writer of synapse - could have known it would cause your problem and never took the trouble to test it let alone fix it..

Again: >Synapse is designed to be used without components and the component (laz_synapse) is ...

« Last Edit: October 22, 2018, 06:46:50 pm by Thaddy »
Specialize a type, not a var.

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 590
    • Double Dummy Solver - free download
Re: [SOLVED] How to download HTML from secure https site
« Reply #6 on: October 22, 2018, 06:46:49 pm »
Thank you Thaddy. I removed the component and thus got rid of the warnings  ;D
Lazarus 3.0RC2, FPC 3.2.2 x86_64-win64-win32/win64

 

TinyPortal © 2005-2018