Lazarus

Programming => General => Topic started by: Ñuño_Martínez on February 13, 2018, 01:35:23 pm

Title: "How to program using UNICODE in Pascal"
Post by: Ñuño_Martínez on February 13, 2018, 01:35:23 pm
I looked for a "step by step" tutorial about how to correctly manage UNICODE texts with Free Pascal but I didn't found any in the whole Internet, I only find partial explanations about data types or how to convert text form this enconding to that other one, but not an actual simple example.  I mean, it shouldn't be hard to "the user inputs text, then I save that text in a UTF-8 text file in a filesystem that uses UTF-8 for filenames, while using LCL internationalization utils for labels and buttons" or something, but reading the wiki explanations I find it extremely hard. May be I'm doing the wrong question? (https://duckduckgo.com/?q=how+to+unicode+free+pascal&atb=v88-2_g&ia=web)

I understand how UNICODE works, that's not the problem.  I was using UTF-8 for a long time, but I used ANSISTRING to store the texts, and never used UTF-8 to name files (for example).  I know it is wrong but I still used it because it worked for me since I know how the text was encoded, but now and then I'm doing stuff that other people is using and they shouldn't expect to use it wrong, should they?
Title: Re: "How to program using UNICODE in Pascal"
Post by: JuhaManninen on February 13, 2018, 05:27:35 pm
I looked for a "step by step" tutorial about how to correctly manage UNICODE texts with Free Pascal but I didn't found any in the whole Internet, I only find partial explanations about data types or how to convert text form this enconding to that other one, but not an actual simple example.  I mean,

... it shouldn't be hard to "the user inputs text, then I save that text in a UTF-8 text file in a filesystem that uses UTF-8 for filenames, while using LCL internationalization utils for labels and buttons" or something, but reading the wiki explanations I find it extremely hard. May be I'm doing the wrong question? (https://duckduckgo.com/?q=how+to+unicode+free+pascal&atb=v88-2_g&ia=web)
I don't know if you are serious. The very first link from your search is:
 Unicode Support in Lazarus - Free Pascal wiki
Shortly put: everything just works with UTF-8, most of the time you don't need to think of it.
You can find examples here:
 http://wiki.freepascal.org/UTF8_strings_and_characters
See also this if you want to make 100% Delphi compatible code also when iterating codepoints and "user perceived characters":
 http://wiki.freepascal.org/Unicode_Support_in_Lazarus#CodePoint_functions_for_encoding_agnostic_code
Title: Re: "How to program using UNICODE in Pascal"
Post by: Ñuño_Martínez on February 14, 2018, 01:52:59 pm
The "UTF8 strings and characters" seems to be what I was looking for. Thank-you.

I don't know if you are serious. The very first link from your search is:
 Unicode Support in Lazarus - Free Pascal wiki
I actually find that "Unicode Support in Lazarus" page quite confusing (may be because I'm not an English native-speaker and I'm not used to it yet, or may be because there are some concepts about UNICODE that I don't have).  As far as I can understand, it enumerates how it works but I can't see how or why or when to use that stuff.
Title: Re: "How to program using UNICODE in Pascal"
Post by: wp on February 14, 2018, 02:01:44 pm
I looked for a "step by step" tutorial about how to correctly manage UNICODE texts with Free Pascal [...]
With "manage" do you mean: How to load and save text files with unicode encoding? What do you mean with "unicode", by the way? UTF-8 (like Lazarus) or UTF-16 (like Delphi)?
Title: Re: "How to program using UNICODE in Pascal"
Post by: Ñuño_Martínez on February 14, 2018, 02:10:38 pm
I looked for a "step by step" tutorial about how to correctly manage UNICODE texts with Free Pascal [...]
With "manage" do you mean: How to load and save text files with unicode encoding? What do you mean with "unicode", by the way? UTF-8 (like Lazarus) or UTF-16 (like Delphi)?
A bit of everything.  I need to control how text is encoded.

The actual problem is that I'm having some troubles with Allegro.pas.  It works correctly with Free Pascal but Delphi seems to have problems with texts (among others).  May be also because Allegro is written in plain C and its UNICODE support is the weirdest thing I ever saw.
Title: Re: "How to program using UNICODE in Pascal"
Post by: JuhaManninen on February 14, 2018, 04:40:24 pm
With "manage" do you mean: How to load and save text files with unicode encoding? What do you mean with "unicode", by the way? UTF-8 (like Lazarus) or UTF-16 (like Delphi)?
I understood he made clear he works with UTF-8:
 "the user inputs text, then I save that text in a UTF-8 text file in a filesystem that uses UTF-8 for filenames, while using LCL internationalization utils for labels and buttons"
Hence my earlier answer.

The actual problem is that I'm having some troubles with Allegro.pas.  It works correctly with Free Pascal but Delphi seems to have problems with texts (among others).  May be also because Allegro is written in plain C and its UNICODE support is the weirdest thing I ever saw.
I see from other posts that you have improved Allegro.pas. What are the actual problems? The Lazarus UTF-8 solution is amazingly compatible with Delphi at source code level.
That kind of problems exactly with Unicode are you experiencing?
Title: Re: "How to program using UNICODE in Pascal"
Post by: Ñuño_Martínez on February 14, 2018, 06:02:12 pm
Some of the examples renders gibberish instead of the actual texts.  Also Delphi shows a hint when converting from STRING to ANSISTRING (or vice-versa) saying that data may be lost in the operation.

I'm aware the origin of this problem may be different, but I'm investigating.
Title: Re: "How to program using UNICODE in Pascal"
Post by: rick2691 on February 14, 2018, 08:16:28 pm
Quote
Some of the examples renders gibberish instead of the actual texts.

That sounds like FONT BINDING, which Microsoft does with their infinite wisdom, to prevent you from publishing a document that they think is defective. They switch a font with yours, and it displays as gibberish in another language. They are so clever!

Rick
Title: Re: "How to program using UNICODE in Pascal"
Post by: Ñuño_Martínez on April 24, 2018, 02:00:31 pm
No, it is not FONT BINDING.  It was that I used the incorrect data type.  Delphi STRING is an alias of WIDESTRING wich is UTF-16 (AFAIK).  I've changed my definitions to use ANSISTRING instead and it works.  Now I have other problems (https://forum.lazarus.freepascal.org/index.php/topic,32568.msg283959.html#msg283959).
TinyPortal © 2005-2018