On the same (code) page
by, 17-Dec-2009 at 10:00 AM (5112 Views)
International characters have always been very complicated to deal with in computer systems, and configuring your computer for your locale is often more difficult than it should be. Have you ever had the problem where extended ascii characters such as ends up being displayed in a dbForm or similar in your VDF application as instead? Or ends up as . Another common symptom is losing characters when typing and saving the record. Often it gets converted to some other character when saved. Usually this is due to a simple Windows configuration that is quite confusing and far too easy to miss.
Technically speaking, when this problem occurs in a Visual DataFlex application, it's an OEM code page configuration problem, where the data is stored in OEM codepage 850 for example, but accidentally read back and interpreted using another OEM codepage, such as 437. Or alternatively, while the particular extended ascii character is present in code page 850, it's not available in code page 437, and therefore disappears when you save the data using the wrong code page. There are also many other reasons for various types of international character issues, but if you haven't done anything special, and the characters usually display correctly, this is probably the most likely reason.
Ansi, OEM code pages are confusing enough as is, configuring Windows correctly for your locale is even more confusing. Because VDF relies on OEM/Ansi translations, it's important that Windows is configured correctly both for Current User and System locale. I won't go into detail about OEM/Ansi for now (that's a whole topic in itself for another time), this article will just focus on what happens when it's not configured correctly, a utility to detect misconfigured systems, and how to fix conflicting configurations.
You may not notice right away if Windows has conflicting settings, because your program will appear to run just fine. But you'll eventually notice that certain extended ascii characters that are supposed to work with your locale settings are not always displayed correctly.
VDF uses the Windows functions CharToOEM() and OEMToChar() to perform the translation, and these functions rely on the System Locale settings, rather than the Current User Locale as one might expect. So therefore the System Locale settings must be properly configured. Conflicting settings between Current User and System configurations are probably not intentional anyway, but it can easily happen by accident since these configurations in Windows are quite confusing. It's also a very common mistake when installing a US/English version of Windows, and then changing the Current User locale but forgetting to change the System locale, ending up with different and conflcting Ansi/OEM code pages.
Detecting Conflicting Code Page Settings
To simplify testing and verification of the configuration so that VDF operates correctly, I've put together a simple little test utility that detects conflicting code page configurations. The full source code is available as an attachment to this article. I've also provided a VDF 15.1 compiled executable version for convenience. The program requires no workspace and can be compiled and run anywhere.
This utility displays some information about the Current User and System locale configurations. With a standard US English configuration it looks like this:
Note that the sections for Current User and System info are identical. The Ansi and OEM code pages in the two respective sections are what's most important in regards to VDF OEM/Ansi translations.
If we change this to Swedish without properly changing the System locale, it might look like this:
Note that the OEM code page in the two sections are not the same. In fact, you'll notice that while the Current User section is configured for Swedish, the System section is still entirely configured for US English. It's very easy to miss this configuration since the date format and everything appears in Swedish. The only practical difference to most VDF programs is that it's still using the wrong Ansi and/or OEM code pages, which can produce the wrong result.
If conflicting settings are detected as in above, you'll be presented with this message:
Resolving the Problem and Getting Back on the Same Code Page
To resolve the conflict you have to change the System Locale so it matches your Current User locale. In Windows XP the System Locale setting can be found in Regional and Language Options, in the Advanced tab, select your specific language for non-unicode programs. And then also check the box in Default user account settings. Checking this box is what applies your changes to the System account also.
In Vista it's similar but with different names and actually better descriptions, on the Administrative tab, click on Change system locale to pick the same language, and also click Copy to reserved accounts.
In Windows versions prior to Vista and XP these settings may sometimes be called something else, like system default settings or something. It's all the same idea, just different names and descriptions for the same underlying settings. When the changes have been made, you usually have to reboot to apply them. You may even have to reboot between the different changes depending on the Windows version.
Once you've corrected the conflicting settings you can re-run the utility to verify that it's working correctly.