In the Windows 10 April 2018 Update (Version 1803), Microsoft introduced a change to the operating system that can affect the basic functionality and/or data in applications and systems across the IT landscape, including Microsoft’s own Visual Studio and SQL Server, DataFlex and more. The change is Microsoft’s unannounced introduction of a “Beta” setting found in the "Region" -> "Change system locale…" -> "Region Settings" which, in some cases may be enabled (checked) after an update (we are still trying to determine the conditions under which it may become set without being deliberately set by the user). This setting is present in all subsequent Windows 10 updates, but is still listed as “Beta”.

Click image for larger version. 

Name:	Region Settings Windows 10.jpg 
Views:	158 
Size:	50.6 KB 
ID:	13034

The problem caused by Microsoft’s new Beta setting is that Windows 10 systems are traditionally set to use single-byte OEM and ANSI codepages. If the above box is checked by default or by the user, the codepage for the system is set to 65001, the multi-byte UTF-8 codepage. This codepage change will impact all user accounts, drivers and applications, including DataFlex, that were previously operating on OEM and ANSI codepages and unpredictable results are likely to occur.

There is much more to know about this change and its impacts. At this time, we can state:

  • When the "Beta: Use Unicode UTF-8 for worldwide language support" setting is disabled (unchecked), DataFlex is supported by Data Access Worldwide on Windows 10 and will behave the same as with prior supported revisions of Windows.
  • When the "Beta: Use Unicode UTF-8 for worldwide language support" setting is enabled (checked), the change of the default codepages to 65001 (UTF-8) impacts fundamental system behaviors and DataFlex will not function properly. Windows 10 with the Beta option enabled (checked) is not supported by Data Access Worldwide.

Since the installation of some update (or other software) may, without warning, enable the UTF-8 option, we strongly advise that you verify that the option is turned off on any system running Windows 10 April 2018 Update (Version 1803) or higher.

Why Microsoft’s codepage change can affect DataFlex

DataFlex expects strings to be encoded internally as OEM and, when communicating with the operating system and other components, it performs appropriate conversions between OEM, ANSI and UTF-16. The Beta checkbox replaces both the traditional OEM and ANSI codepages with a UTF-8 codepage, allowing some data to come into the system as multi-byte.

In OEM and ANSI, only 255 characters are supported and are always represented by a single byte, while UTF-8 supports over a million characters and any character outside the basic ASCII set is represented as multiple bytes (up to 4). The current DataFlex runtime is not designed for UTF-8 and when it encounters data outside of the basic ASCII set (the first 128 characters in the codepage) the presence of incompatible multiple-byte characters will cause errors in many parts of the system, including collating sequences, string functions and comparisons that can produce undefined behaviors, including invalid data results, buffer overflows and crashes.

Windows 10 Builds Impacted Under this Notice

Version Servicing Option Availability Date OS Build
1903 Semi-Annual Channel 2019-05-21 18362.295
1809 Semi-Annual Channel 2019-03-28 17763.720
1809 Semi-Annual Channel (Targeted) 2018-11-13 17763.720
1803 Semi-Annual Channel 2018-07-10 17134.950
1803 Semi-Annual Channel (Targeted) 2018-04-30 17134.950