View Single Post
Old 8th December 2011, 13:34   #522  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Could you upload/attach the problematic CUE file please as-is?

LameXP handles Unicode characters properly, as long as the input is in the proper UTF-8 format. I suspect that this is not the case here

Currently LameXP will assume that the input is UTF-8, iff a BOM is found. Otherwise it will interpret the input with your local 8-Bit Codepage - whatever that may be.

I know this this isn't very reliable, but storing text files with some local 8-Bit Codepage is extremely error-prone anyway, because for the app reading the file later, it will be impossible to know the correct Codepage. Assuming that the "local" Codepage is the right one, is just a wild guess. But probably the best we can do.

Using Unicode with proper UTF-8 encoding is the right way to store text files that contain "foreign" (Non-ASCII) characters! And a BOM should be prepended to indicate that this is UTF-8. Still the BOM is optional. Assuming that all text without a BOM is not UTF-8, is another wild guess. But again I don't know a better way...

(BTW: As for the console, you won't see proper Unicode output there anyway, unless you change font to "Lucidia Console")


[Small Update]

I hacked together a quick workaround that may work for your case:
http://www.mediafire.com/file/pqfj5o....Build-803.exe

If LameXP assumes that the input is not UTF-8, it will now test whether decoding the input with the "local 8-Bit" Codepage results in any '�' (U+FFFD) characters. In that particular case we can assume that the "local 8-Bit" Codepage is not the suitable one and thus we will fall back to the "Latin-1" Codepage. There is absolutely no guarantee that the Latin-1 Codepage will work any better than the local one. It's just another try (and will succeed with Western European encodings).
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 8th December 2011 at 15:19.
LoRd_MuldeR is offline   Reply With Quote