Development Team Blog

There are many aspects to the discussions about how the new font system in Visual DataFlex works when compared to the old. This is actually a complex topic with a lot of different perspectives – so this blog might be on the long side. I promise to do my best not to ramble (too much).

I’ll start by stating that the main goal in wirting this is to get everyone to a common understanding of the way fonts work: how the old system functioned, what we changed in moving to the new system and why those changes produce the current behaviors. We can then see what the options are for the product and developers; now and in the future.

Fonts are the driving factor in how things should appear (and scale) on Windows. Well, at least they are supposed to be. Rather than go through all the details on fonts in this document, it would help if you read the new section in the 17.0 help under “Windows Application Development | Using Fonts”. Go ahead; I’ll wait until you’re done…

Back already? You did read it, didn’t you? OK, onward. Let’s remember how the old font system works. All that good stuff you just read about everything adjusting and scaling based on font changes was not really functioning. Visual DataFlex was essentially a single font (MS San Serif) single size (8) system for many years (at least as far back as Visual DataFlex 7 and probably earlier). The default sizes for everything and the way we all (or at least most of us) designed our applications were almost solely based on the singular way our fonts worked best. Even for those of you that adopted other methods of dealing with fonts the underlying screen designs and default sizes play a large role in your applications today. For instance, have any of you adopted a form height ofother than 13 (our default since DataFlex for Windows)?

The existing font system had behaviors that have always been there, but the change to a significantly different (and larger) system font in Windows Vista was when those behaviors started to get more noticeable. We actually looked into changing the font system during the 12.x and 14.x projects (more on that later). The emergence of high-DPI displays on laptop computers was the real tipping point – we felt something had to be done to enable developers to create DPI-aware applications. The questions was; what and how?

The existing Visual DataFlex font system did not use the Windows system font. It had its own behaviors that weren’t based on Windows standards, was hard to manipulate and was not DPI-aware (so it would not scale). To complicate matters, COM controls (including, but not limited to, the CodeJock controls we use for menus and grids) do use the Windows system font and have behaviors that are based on Windows standards and are (or at least can be) DPI aware (and thus,scale). In addition to COM controls, if your application also uses Windows controls at a more basic level (popping up Windows Explorer dialogs, message boxes, etc.) all those will have their font and sizing based upon the host system font as well.

Adopting the Windows standards (using the system font and adopting DPI awareness) was the only logical choice.

The way the font system is supposed to work in Windows is that you specify a typeface (eg. Segoe UI) and a font point height (eg. 9) and, essentially, Windows does the rest based on dialog units. Windows then becomes the ultimate judge on how everything sizes. There were really two components to the change we needed to make; Visual DataFlex must use a font correctly (meaning that everything would scale properly off the combination of typeface and font point height) and Visual DataFlex must use the system font of the host system. This would mean that Visual DataFlex applications that used the new system would change from applications designed adn run mainly around MS Sans Serif 8 to applications that would run on Tahoma 8 or Segoe UI 9 (the two predominant system fonts). But it’s important to understand that using the new system would allow the application to adapt to any typeface and point height used by the system automatically.

I’ll use the venerable Order Entry main screen to illustrate the journey. We’ll start with XP (at both 100% and 125%) and then look at Vista / Windows 7 later.

The Old Font System on Windows XP

The image above shows that Visual DataFlex 16.1 on XP at 100% uses a combination of MS Sans Serif (VDF) & Tahoma (CJ) fonts because the Codejock grid is an example of a COM control that is (properly) using the system font. This illustrates one underlying issue; that the native VDF controls were using a different font fromCOM controls. Many developers lived with that difference because the two fonts were similar and the same size.

When we run the same application with Visual DataFlex 16.1 on XP at 125% (above) we see that the MS Sans Serif and Tahoma fonts are still used, but only the Codejock grid (Tahoma) is DPI-aware and scaled properly. Before high-DPI displays became commonplace, developers would just try to make sure their customers didn’t run with anything other than standard sizes. Those days are over…

The Start of the Change

We then made changes necessary to start using the Windows system font in all Visual DataFlex controls so that they would exhibit the same behaviors as the CJ COM controls. This is where we brought in the new Font Point Height support (necessary to make it all work) and designed the new system in such a way that the old font system behaviors could be preserved for compatibility.

After the changes were made (sounds like waving a magic wand, doesn’t it?) the first thing we verified was that we got the same results between 16.1 and 17.0 when the new font system was turned off. We did, so I’m not going reproduce any screen shots with the 17.0 / old font system combination.

Trust me

The New Font System (Take 1) on Windows XP

OK, so we turned on the new system (and really appreciated the design of setting a single property in the application object) and continued…

The image above shows Visual DataFlex 17.0 on XP at 100% using the new font system. The fonts are now all correctly using Tahoma 8, but you can see that there are issues with the horizontal sizing and spacing. All you have to do is look at how much the overall horizontal size of the view was reduced (while the vertical remained the same) to know that something has changed significantly.

This kind of sizing problem is the similar to issues we had in the past when we tried to change the font system. While even a view as simple as Order shows the horizontal sizing problem, the impact is much more profound as the layout is more complex and tightly-packed. Putting the horizontal sizing issue aside for a moment, we wanted to make sure that the changes made not only allowed the application to use the system font consistently but then would also scale correctly on high-DPI settings.

The image above shows Visual DataFlex 17.0 on XP at 125% using the new font system. Again we see that we are picking up the system font (Tahoma) everywhere and it is indeed scaling properly to 125%, but there is still an issue with the horizontal sizing and spacing.

Rather than show you how these behaviors manifested on Vista / Windows 7, I’ll stick with XP for now and explain what we did and what we found. We’ll then look at Windows 7 from top to bottom.

As mentioned earlier, the combination of typeface (now Tahoma) and font point height is, when working correctly, supposed to control everything at this point. Using the new system, based on Font Point Height, that was now true vertically, but not horizontally – so where was the problem? To understand what we found, we have to dive one layer deeper into how fonts really work.

While it’s true that everything is positioned using those “universal” dialog units, you eventually need to translate those dialog units to actual pixels to paint the control on the screen. This is actually the point at which the “magic” that makes things look good with different fonts and scaling properly happens – you do calculations on how the dialog units (the universal measurement) map to actual pixels (based on the specific settings of the system you are running on) and off you go.

So, we looked into how that calculation was being done. That code hadn’t changed from the way it worked in the old font system and had been in place for many, many years. In fact, it still worked great when the font was “our” system font (MS Sans Serif) and we couldn’t see anything obviously wrong with it. So, we went looking for how Microsoft does the conversion from dialog units to pixels and the formula was different from the one we use. Rather than try to adjust our formula we just decided to use the Microsoft calculation! So, we put the Microsoft recommended calculation in place (it sounds so easy when you say it that way) and ran the tests again…

The New Font System (Take 2) on Windows XP

Now, the image above shows Visual DataFlex 17.0 on XP at 100% using the new font system AND the Microsoft approved calculation for converting dialog units to pixels. We can see that the system font is properly used throughout and that both the vertical and horizontal scaling and positioning are “correct”. I quote the correct reference because even though the layout is properly resizing in both the vertical and horizontal plains we need to understand that the calculations that anyone (ourselves or Microsoft) uses are based on choices. What is the tallest character in the font? Should the horizontal plain use an “average” character width and how should that average be calculated? Should we “leave a little extra room” and, if so, how much? There is no such thing as a layout that will perfectly scale to all possible fonts and DPI settings – you design your application layout for the best look most of the time.

So, how will the Microsoft calculation hold up at 125%?

Much better! The image above shows Visual DataFlex 17.0 on XP at 125% using the new font system and the Microsoft approved calculation for converting dialog units to pixels. Even at 125% most everything is scaled properly and positioned well. It isn’t perfect, but what is?

So, it looks like we found that the calculation we had been using for many years didn’t do well when the fonts were changed (even “just” from MS Sans Serif 8 to Tahoma 8) and so the horizontal didn’t scale well at all. We dug deeper into the two calculations and determined that the difference was due to truncation (our old calculation) vs. rounding (the Microsoft calculation). It just so happened that for MS Sans Serif 8, there was no real difference between the calculations, regardless of which method they use. It actually depends on all the aspects of the font to determine if the difference between a truncate or round will tend to make things bigger or smaller than the original layout.

Now, how does all this work in Windows 7, where the system font is even less like our old standard font? Remember, instead of going from MS Sans Serif 8 to Tahoma 8 we’re going to Segoe UI 9. Let’s start with how Visual DataFlex 16.1 looked…

The Old Font System on Windows 7

The image above shows Visual DataFlex 16.1 on Windows 7 at 100% and uses a combination of MS Sans Serif 8 (VDF) and Segoe UI 9 (CJ). As noted earlier, because Vista and Windows 7 not only changed the typeface, but moved to a larger font point height, the differences between the VDF controls and COM controls are more noticeable. The font difference is even more profound at higher DPI settings…

The image above shows that Visual DataFlex 16.1 on Windows 7 at 125% uses the same combination of MS Sans Serif and Segoe UI, but just as with XP, the high-DPI scaling is only happening correctly with the COM controls. In addition, since the system font was already larger in the grid, the difference when it scales up to 125% (and the VDF controls don’t) is stunning (but not in a good way).

The New Font System on Windows 7

Using the same 17.0 change (before updating to the Microsoft conversion calculation) we saw the following behaviors on Windows 7:

Visual DataFlex 17.0 on Windows 7 at 100% using the new font system (with our legacy calculation) looks fairly reasonable, but you can still notice that the dialog didn’t properly scale horizontally. The design of this particular view is actually reasonably benign so its impact is misleading. This does illustrate one of the complicating factors of fonts and scaling: What looks acceptable in one instance can be just horrifying in others.

You might look at this Order view and say “Hey, with Segoe UI 9 the old calculation is fine (perhaps even better) than the Microsoft calculation, so why didn’t you stick with it?”

In response I give you Exhibit A…

The image above is running on Visual DataFlex 16.1 on Vista at 100%. This is a pretty simple test view that uses what most would consider pretty basic control positioning found in a more complex “main” screen. Available real estate for the layout is tight, but works. Here is the same image running with the new font system but our legacy calculation…

The image above is running on Visual DataFlex 17.0 on Windows 7 at 100% with our legacy calculation from dialog units to pixels. It’s pretty amazing how quickly something that is less than optimal (too much additional white space) can turn into something that is completely unacceptable. I’ve lost the overall aspect ratio of my meticulously designed view, Customer Number is completely cut off, I now have to scroll my tab pages where they used to fit with plenty of room and the labels for my important data are at best cramped and at worst cut off.

Imagine the volume and tone of the discussion we'd be having if this (or worse) happened to your applications?

So, how does this view look with the change to the Microsoft calculation?

The image above shows the test view running on Visual DataFlex 17.0 on Windows 7 at 100% with the Microsoft calculation.

Everything is as it should be – no clipping of labels, no scrolling of tab pages and my overall aspect ratio is essentially as I designed it.

I could go on and on with all kinds of layouts shown in all kinds of host environments but I think we’ve provided enough information and illustrations so that we’ve accomplished what we set out to: that common understanding of what was, what changed (and why) and that the new system, while certainly not perfect, is just as certainly not “broken” (just different). It is doing exactly what is it designed to do and doing so according to Microsoft’s own recommendations.

Additional Observations and Clarifications

Some of you noticed that the "extra" width is most prominent in date and numeric forms, especially those with prompt buttons. All the controls are actually "scaling up" based on the font is use the same way - there is nothing different done in the size calculations (50 dialog units wide is just that). So why is the effect more prominent with certain types of forms? First, the date is more narrow when compared to the average character width. Remember that the horizontal sizing of the controls are based on the dialog units times this "average" width. Where the data displayed is above the average (like a string of Ws) it will have to scroll, where it is below (like a string of 1s) it will not take up the full amount. Prompt buttons add to this in that they used to be just a fixed size (and fairly square) and now they are having their size properly calculated according to the same formula as everything else (which actually makes them a bit taller than wider).

Some of you have noticed that the new font system “works” on Windows Server 2008. This is because the Windows system font on that platform is 8 point, not 9 (it is also Tacoma and not Segoe UI - but the more important factor is actually the font point height). That common size between the design environment and the new environment (even though the typeface is changing) is what makes the result more pleasing. Any platform that has a system font of 8 point (Windows Server, XP, anything running Windows Classic as its theme, etc.) has a good chance of giving you a pleasing result, even though it is going through the same conversions and calculations; everything is indexed off the font point height. If you switch to any font that is the same basic size as your design environment the impact of control resizing is reduced. Of course, it is possible that a font could be so much wider than MS Sans Serif that even at 8 point it would have the same kind of sizing impact as Segoe UI 9 does.

All kinds of numbers are being thrown around about the percent difference - ranging from as small as 12% to as high as 20%. I'm certain we can all find examples to prove any particular point we wish (43.62 percent of all statistics are manufactured, you know) - but if we go back to my test view and use the overall size of the view (remember that everything scales up or down based on font size) as a reasonable example we can see that using the new formula we gained 14% vertically and 17% horizontally. In a perfect world we would have gained 14% in both, but the change in formula accounted for that extra gain in width.

Please keep in mind that my goal of trying to reach a common understanding isn’t meant to diminish any developer’s position or concern for the look and feel their application; it just gives us a better platform from which to have a discussion based on all the information.

But Why Didn't You... ?

Of course, understanding the behvaiors of both the old and new calculations leads us to the inevitable “Ok, so on platform A do this (instead of doing that) and then tweak it this way (but only when this happens), etc.” and on and on.

We’re certainly willing to entertain suggestions and continue to refine the new font system into the future (just like we do with all refinements) – but there is no silver bullet, no “one size fits all” or “correct” fix.

Take the two calculations we have at hand (our legacy calculation and the Microsoft recommendation) as an example. Our old calculation in the best of circumstances looks cramped but more often than not is simply unusable. Microsoft’s calculation is only sometimes a bit cramped (but never actually crosses into text clipping) and is often a bit too generous with the horizontal white space.

If you must pick between "more often than not simply unusable" and "often a bit too generous with horizontal white space", which would you choose?

The Microsoft calculation also is what Microsoft uses at the operating system level (but not always or consistently at the application level) so as Microsoft changes in the future using their calculations should put us (and you) in the best position to adapt.

What About Text Positioning?

If we put the horizontal scaling aside for a moment, there is another behavior that developers have commented on: how the text is positioned inside forms. As I mentioned back near the start of this discussion; we adopted a default height for the various forms of 13 dialog units. The decision was made so long ago it’s not clear if this was the Microsoft recommendation at the time (or even still is) but we have all (or most of us) designed our applications around that particular metric for as long as any of us remember. Developers have commented that, when using the new font system, they don’t like the increase in white space between the bottom of the characters and the bottom of the form. This behavior (a bit more space below than above) has always been there – it’s just that the new font size of 9 makes it more pronounced then is it at 8. This isn't even a byproduct of the change in calculation - even with our old formula the vertical white space is just as imbalanced.

There are really only two possible ways to address this concern; reduce the vertical size of the forms (probably from 13 to 12) or to try to center the text vertically in the control (so that the white space is balanced). We’ll look into both of these options for the future and seek your feedback on them.

Are You REALLY Sure About All This? Microsoft Applications Look Fine!

Lastly, some of you have commented that Microsoft applications don’t exhibit the same overall size and spacing behaviors and that those differences would indicate that “the problems” lay with Visual DataFlex. Again, this isn’t as cut and dried as it may seem. First, many (most) Microsoft applications don’t actually use the Windows system font. Even those that do often manipulate the size back down to 8 on Vista and Windows 7.

Why does Microsoft make these choices and implement some (many, most...) of their applications differently? We could speculate about that all day long but the reality is I simply can’t tell you what their motivations are.

What I can tell you is that if you want to see if the overall sizing is really based on the Windows system font it can be done by examing the size of text and buttons, etc. When we fisrt saw the sizes of the controls on Windows 7 (because of that font point height difference) we wanted to make sure Visual DataFlex was really “correct” based on the host system. We compared them to things like Windows message boxes. If you do so, you’ll see that we are indeed using the correct metrics.

What Now?

Some may think that using the sizing and spacing determined by the Windows system font is not always desirable but we believe it is the best option available. But developers do have choices:

• Continue to use the legacy font system where it provides you and your customers with the best path (for as long as that remains true).
• Adopt the new font system without going through a lot of redesign of your layout – this will work for some of you more often than others. Yes, the horizontal sizing may be different than it was when you designed it, but remember what it is doing (automatically) and why.
• Adopt the new font system and make adjustments to the layout of parts of your application. This will be a bigger job for some thant others and you may want to limit this investment by only making it to the main application screens (seeing "the customary" sizing layout in less-used parts of the application is not as pressing a need). Keep in mind that when designing applications to be DPI-aware there is no perfect layout, only one that you determine better presents your application in those environments that you think are important.
• Take more control over the use of fonts to determine your own standard (ala some Microsoft applications). You know that the new font system will actually allow you to do that morea easily and consistenly than the old. But you will have the ultimate responsibility to control not only the Visual DataFlex controls, but any COM controls you use (regardless of who provided them).

Thanks for taking the time to read through all this (there are so many elements at play) – I hope I didn’t meander too much…