Recent

Author Topic: is Mojave blocking HTTP in my non-release non-signed debugging apps?  (Read 867 times)

MISV

  • Sr. Member
  • ****
  • Posts: 493
Both my native API implementation and Indy has stopped working loading even simple URLs like http://example.com and https://example.com

Now I am starting to wonder if something changed in Mojave which limits/sandboxes/similar newly compiled, non-signed, apps wih debug info?



Hansaplast

  • Sr. Member
  • ****
  • Posts: 484
  • Tweaking4All.com
    • Tweaking4All
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #1 on: December 01, 2018, 11:29:44 am »
I ran into the same issue when trying to use sandboxing.
When I just sign the application without sandboxing, then the native call (Phil's units) work just fine for https (Synapse did not work).
I have not tried http yet, but I do recall iOS for example only allowing http if I would add the following to the plist (source):
Code: XML  [Select]
  1. <key>NSExceptionAllowsInsecureHTTPLoads</key>
  2. <true/>
Maybe Mojave has gone that route as well for http?
As for not using the sandbox; using sandboxing came with other issues for my application as well, so I'm not sure if it was http/https specific.
I know it's not answering your question, but maybe it's helpful.

Hansaplast

  • Sr. Member
  • ****
  • Posts: 484
  • Tweaking4All.com
    • Tweaking4All
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #2 on: December 01, 2018, 11:31:42 am »
p.s. retrieving https with Phil's unit and Synapse did work just fine while testing the application in Lazarus. (under Mojave)

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #3 on: December 01, 2018, 02:48:01 pm »
Thank you your help. Just pulling my hair out how suddenly two separate methods for http/https stopped working at the same time (stable codebase on mac 3 years, windows 15 years) after upgrading lazarus + macOS :)

Maybe some kind of firewall in Mac? I will have to try investigate. But I appreciate all info and help I get - I will have to continue dig through all possible explanations :)

Will post if I find something interesting

Hansaplast

  • Sr. Member
  • ****
  • Posts: 484
  • Tweaking4All.com
    • Tweaking4All
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #4 on: December 01, 2018, 03:02:50 pm »
I know the feeling hahah ...
Programming used to be fun, but Apple (and Microsoft is going the same route) is most certainly trying to make it a pain.

Making my little projects used to be about "automating" an idea, but these days that's just half of the job.


For what it's worth; I use Little Snitch on my Mac as a firewall (with macOS firewall disabled).
It used to block all traffic from unsigned apps, or apps that have been signed with an unknown developer ID (eg. self made cert).
Each time I test (outside of Lazarus), I had to confirm and enable the exception - each and every time.
Not sure if the Mac Firewall has similar behavior? (isn't the firewall by default Off under Mojave? I think I read that somewhere)

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #5 on: December 01, 2018, 03:47:03 pm »
One thing I have noticed.


Indy seems to try use SSL for .Get http://example.com

And SSL fails then for some reason - stalls after loading crypto lib in LoadSSLCryptoLibray

Hansaplast

  • Sr. Member
  • ****
  • Posts: 484
  • Tweaking4All.com
    • Tweaking4All
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #6 on: December 01, 2018, 04:56:25 pm »
I don't think I've ever used Indy  :(  ...
I've used Synapse for as long as I can remember, and of course recently Phils unit (native) with the signing of apps drama  :D [size=78%].[/size]

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 515
    • Lebeau Software
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #7 on: December 01, 2018, 08:53:31 pm »
Indy seems to try use SSL for .Get http://example.com

And SSL fails then for some reason - stalls after loading crypto lib in LoadSSLCryptoLibray

TIdHTTP DOES NOT try to use SSL/TLS for non-HTTPS requests.  However, if you have assigned the TIdIOHandlerSocketOpenSSL component to the TIdHTTP.IOHandler property (or left the IOHandler property set to nil, and then requested an HTTPS url, which creates and assigns a default TIdIOHandlerSocketOpenSSL instance internally), then the IOHandler component will at least attempt to initialize the OpenSSL library when connecting to a server (for instance, if an HTTP url redirects to an HTTPS url).  But, if a non-HTTPS url is being requested, an OpenSSL load failure will not stop the request from progressing, Indy will simply catch and discard the load error.

But, if the library is actually stalling, blocking the calling thread from continuing, that is a different matter.  In what way is it stalling exactly?
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #8 on: December 01, 2018, 11:54:54 pm »
OK - stepping through code

inside HackLoad calls LoadLibVer('')
- where Filename becomes libcrypto.dylib
- and LoadLibrary(FileName) is called (which I apparently can not step into)
- Result is 1

I choose Step into twice more (at "end;") until I am last line of "HackLoad". Clicking "Step Into" stalls Lazarus/app/Indy - nothing happens. (And "Step into" and other similar are grayed/disabled)

Right now Lazarus appears hung/stalled

...

Waiting long enough I have earlier had it return with ERangeError (but using debugger is a bit flimsy so I don't have better info right now)
...

And that is where I am at currently.

...

looking at /usr/lib/libcrypto.dylib it points to /usr/lib/libcrypto.35.dylib (whi h I believe is the same a pre-Mojave)

...

Maybe something blocking read access?



Various bits of information I have Googled'. Take it for what it is worth - I am completely open for ideas and searching pretty broadly:

https://github.com/Homebrew/brew/issues/4870
"The answer to the problem is that Xcode's svn is built against an older LibreSSL version (libcrypto.35.dylib, shipping with Mojave and previous macOS releases). On Mojave, this older LibreSSL is built with a default CA bundle path other than /private/etc/ssl/cert.pem, thus pointing to a file that is missing from production installs."

« Last Edit: December 02, 2018, 12:04:55 am by MISV »

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #9 on: December 02, 2018, 01:18:49 am »
posting more soon about native API problem
« Last Edit: December 02, 2018, 02:17:09 am by MISV »

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 515
    • Lebeau Software
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #10 on: December 02, 2018, 02:59:55 am »
OK - stepping through code

inside HackLoad calls LoadLibVer('')
- where Filename becomes libcrypto.dylib
- and LoadLibrary(FileName) is called (which I apparently can not step into)
- Result is 1

I choose Step into twice more (at "end;") until I am last line of "HackLoad". Clicking "Step Into" stalls Lazarus/app/Indy - nothing happens. (And "Step into" and other similar are grayed/disabled)

Nothing in that code should be stalling, especially if LoadLibrary() exits.  Either way, not an Indy issue, sounds like a FreePascal/Lazarus issue.

looking at /usr/lib/libcrypto.dylib it points to /usr/lib/libcrypto.35.dylib (whi h I believe is the same a pre-Mojave)

IIRC, that is pointing to LibreSSL, not OpenSSL (see this discussion).
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #11 on: December 02, 2018, 11:22:31 pm »
After some fiddling with my IDE debugger settings I now have it to break on exception (somehow a ton were ignored, maybe something I did in a dumb moment, not sure)

Anyhow, the LoadLibrary call to the OpenSSL/LibreSSL is throwing ERangeError

So... There's that. I think that is the problem concerning Indy. Not sure yet if it work work for a signed app. (I will report back)

...

Regarding native API it works for https but not http sites. The odd thing is I have tried to add the plist file added earlier that should solve that. But maybe needs to be signed. (I will report back)

However - as my app defaults to use Indy as only uses native API as fallback for https - I may never have noticed this - not sure. Will need to check.
« Last Edit: December 02, 2018, 11:27:35 pm by MISV »

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #12 on: December 02, 2018, 11:34:14 pm »
Indy: Seems Remy released an update 22 hours ago which seems to solve my Indy problems - will report back if further testing shows differently

MISV

  • Sr. Member
  • ****
  • Posts: 493
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #13 on: December 06, 2018, 03:55:40 pm »
I have a Lazarus (trunk) application which worked fine for years, but after upgrading to Mojave this code (at least in debug builds) causes issues:

    // trying http://example.com

Code: Pascal  [Select]
  1.     urlConnection := NSURLConnection.alloc.init;
  2.     urlData := urlConnection.sendSynchronousRequest_returningResponse_error(
  3.       urlRequest,
  4.       @urlResponse,
  5.       @error
  6.     );
  7.  
error.code is -1022 ... NSURLErrorAppTransportSecurityRequiresSecureConnection

I have not tried with signing yet to see if it solves the issue, but I really need my debug builds to work as well :)

...

Lazarus updates the executable inside an .app file when compiling. I have tried adding this plist file which explicitly allows http:// for my debug builds - with no success:

Code: Pascal  [Select]
  1.     <?xml version="1.0" encoding="UTF-8"?>
  2.     <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  3.     <plist version="1.0">
  4.     <dict>
  5.             <key>CFBundleDevelopmentRegion</key>
  6.             <string>English</string>
  7.             <key>CFBundleExecutable</key>
  8.             <string>MyEN</string>
  9.             <key>CFBundleName</key>
  10.             <string>EN</string>
  11.             <key>LSApplicationCategoryType</key>
  12.             <string></string>
  13.             <key>CFBundleIdentifier</key>
  14.             <string>com.example.name</string>
  15.             <key>CFBundleInfoDictionaryVersion</key>
  16.             <string>6.0</string>
  17.             <key>CFBundlePackageType</key>
  18.             <string>APPL</string>
  19.             <key>CFBundleSignature</key>
  20.             <string>en</string>
  21.             <key>CFBundleShortVersionString</key>
  22.             <string>1.0</string>
  23.             <key>CFBundleVersion</key>
  24.             <string>1</string>
  25.             <key>CFBundleIconFile</key>
  26.             <string>ExampleName.icns</string>
  27.             <key>CSResourcesFileMapped</key>
  28.             <true/>
  29.             <key>NSAllowsArbitraryLoads</key>
  30.             <true/>
  31.             <key>CFBundleDocumentTypes</key>
  32.             <array>
  33.                     <dict>
  34.                             <key>CFBundleTypeRole</key>
  35.                             <string>Viewer</string>
  36.                             <key>CFBundleTypeExtensions</key>
  37.                             <array>
  38.                                     <string>*</string>
  39.                             </array>
  40.                             <key>CFBundleTypeOSTypes</key>
  41.                             <array>
  42.                                     <string>fold</string>
  43.                                     <string>disk</string>
  44.                                     <string>****</string>
  45.                             </array>
  46.                     </dict>
  47.             </array>
  48.             <key>NSHighResolutionCapable</key>
  49.             <true/>
  50.     </dict>
  51.     </plist>
  52.  
     





Hansaplast

  • Sr. Member
  • ****
  • Posts: 484
  • Tweaking4All.com
    • Tweaking4All
Re: is Mojave blocking HTTP in my non-release non-signed debugging apps?
« Reply #14 on: December 06, 2018, 04:03:39 pm »
I recall seeing that for iOS (link) - I have not tried it myself, but maybe this info helps.
I see you tried NSAllowsArbitraryLoads,


I did see a slightly different notation;


Code: XML  [Select]
  1. <key>NSAppTransportSecurity</key>  
  2.  <dict>  
  3.       <key>NSAllowsArbitraryLoads</key><true/>  
  4.  </dict>


but maybe this works better:


Code: XML  [Select]
  1. <key>NSAppTransportSecurity</key>
  2. <dict>
  3.     <key>NSExceptionDomains</key>
  4.     <dict>
  5.         <key>yourdomain.com</key>
  6.         <dict>
  7.             <!--Include to allow subdomains-->
  8.             <key>NSIncludesSubdomains</key>
  9.             <true/>
  10.             <!--Include to allow HTTP requests-->
  11.             <key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
  12.             <true/>
  13.             <!--Include to specify minimum TLS version-->
  14.             <key>NSTemporaryExceptionMinimumTLSVersion</key>
  15.             <string>TLSv1.1</string>
  16.         </dict>
  17.     </dict>
  18. </dict>