PDA

View Full Version : SizeOfWString



wila
15-Nov-2019, 05:19 AM
Hi,

Reading the doc, so figured to install as I was curious about something in there.
Some interesting design decisions that have been made. Certainly within the spirit of the DataFlex language.

It mentions SizeOfWString

So I'm thinking .. hey earlier it was mentioned that length would work with all our strings.. so what is the difference?

Here's my first program in DF20



Use Windows.pkg

WString wsTest

Move (Repeat(Character(0),540)) to wsTest

Showln (SizeOfWString(wsTest))

Showln (Length(wsTest))

winput "test" windowindex


That didn't compile because


----Compiling testWString.src----
- Compiling Program: C:\DataFlex Projects\testing20\AppSrc\testWString.src
- Memory Available: 1195442176
- Using pre-compiled package WINDOWS.PKG
- Including file: windows.x64.pkd (C:\Program Files\DataFlex 20.0\Pkg\windows.x64.pkd)
- Warning 4533: C:\DataFlex Projects\testing20\AppSrc\testWString.src (ln 11) Obsolete command: Winput. Use Message_Box
-- Warning 4533: Obsolete command: GetAddress. Use AddressOf Function
-- Error 4539: Conversion from 64-bit pointer to integer is not allowed, use Longptr instead of Integer.
-- Error 4539: Illegal datatype conversion (would lead to a runtime error if not corrected)
- Summary
- Memory Available: 1192370176
- Total Warnings : 2
- Total Errors : 2
- Total Symbols : 9444
- Total Resources: 0
- Total Commands : 7973
- Total Windows : 0
- Total Pages : 0
- Static Data : 93735
- Message area : 44399
- Total Blocks : 5219
- 2 ERRORS HAVE BEEN FOUND.
----Compile Finished (with Errors and Warnings) ----
----Compiler Warning Summary----
- Warning 4533: C:\DataFlex Projects\testing20\AppSrc\testWString.src (ln 11) Obsolete command: Winput. Use Message_Box
-- Warning 4533: Obsolete command: GetAddress. Use AddressOf Function
----Compiler Error Summary----
- Error 4539: Conversion from 64-bit pointer to integer is not allowed, use Longptr instead of Integer.
-- Error 4539: Illegal datatype conversion (would lead to a runtime error if not corrected)


Hmm.. this works:



Use Windows.pkg

WString wsTest

Move (Repeat(Character(0),540)) to wsTest

Showln (SizeOfWString(wsTest))

Showln (Length(wsTest))

winput windowindex


The output is.....


80
80


Uhhh...

Moving all that global code over to a method gives:


Use Windows.pkg


Procedure test
WString wsTest

Move (Repeat(Character(0),540)) to wsTest

Showln (SizeOfWString(wsTest))

Showln (Length(wsTest))
End_Procedure

Send Test
winput windowindex



output is:


540
540


That's all very interesting...
1. globals are still causing trouble
2. Winput doesn't like the string prompts
3. Still not sure what the difference is between SizeOfWString and length except that the former is more explicit and will not trigger any implicit conversions to UTF-8

Now I guess that 3. is what answers my question after thinking about it, but it was an interesting test with surprising results, so figured to post it here.
--
Wil

starzen
15-Nov-2019, 05:25 AM
length is number of characters. size is number of bytes

Focus
15-Nov-2019, 05:25 AM
One counts characters and the other counts bytes which with unicode are no longer the same thing once you get past the first ASCII 127 characters

starzen
15-Nov-2019, 05:28 AM
in case of sizeofwstring count of wchars

wila
15-Nov-2019, 05:49 AM
in case of sizeofwstring count of wchars

Yeah.. not bytes and that's why the output on both is the same.
It's not strange to expect an output in bytes on a "SizeOf..." function though.
But length does an implicit conversion to UTF-8 to get to that same result.

--
Wil

wila
15-Nov-2019, 05:56 AM
The doc says about SizeOfWString:



This returns the number of WChars (codeunits / double-bytes) of a WString.



Yet WChar is not a supported type.
So I guess that there's also not a WChar array we can make unless we do something like a UChar[][] ?
--
Wil

starzen
15-Nov-2019, 05:57 AM
seems kind of odd. would think it would be better to have the ability to get the number of bytes but of course you can easily calculate that

wila
15-Nov-2019, 06:00 AM
seems kind of odd. would think it would be better to have the ability to get the number of bytes but of course you can easily calculate that Sure.

Now where is my like button?
Ah :cool:

--
Wil

Harm Wibier
15-Nov-2019, 06:00 AM
There is no WChar type and I would not recommend making it a two dimensional UChar array. I'd recommend using a UChar array and doubling the result of SizeOfWString. Alternatively you could use a UShort array.

The reason why SizeOfWString exists and returns the number of WChar entities is because that is what you need in a most cases when calling windows API's.

wila
15-Nov-2019, 06:08 AM
There is no WChar type and I would not recommend making it a two dimensional UChar array. I'd recommend using a UChar array and doubling the result of SizeOfWString. Alternatively you could use a UShort array.

The reason why SizeOfWString exists and returns the number of WChar entities is because that is what you need in a most cases when calling windows API's.

Thanks Harm,

Yeah I was shuddering when writing UChar[][] and already thought "not going to use that, but a UChar[] array double the size".

I think a WChar type would make sense though.
If only to be consistent and make code easier to read and see the output in the debugger.
A "UChar array double the size" is kind of hackish.

Then again, perhaps DataFlex is fine without.
--
Wil

Mike Peat
15-Nov-2019, 07:44 AM
Just having fun:


Use Windows.pkg
Use cApplication.pkg

Object oApplication is a cApplication
End_Object

Procedure Test
String sTest sPart
UChar[] ucaTest

Move "你好,世界" to sTest
Move (StringToUCharArray(sTest)) to ucaTest

Showln "== 'Hello World' in traditional Chinese =="
Showln sTest
Showln ("Size:" * String(SizeOfString(sTest)))
Showln ("Length:" * String(Length(sTest)))
Showln ("Bytes:" * String(SizeOfArray(ucaTest)))

Move (Mid(sTest, 3, 2)) to sPart

Showln
Showln "== Mid 3,2 =="
Showln sPart
Showln ("Size:" * String(SizeOfString(sPart)))
Showln ("Length:" * String(Length(sPart)))
Inkey Windowindex
End_Procedure

Send Test


Results in:


== 'Hello World' in traditional Chinese ==
你好,世界
Size: 15
Length: 5
Bytes: 15

== Mid 3,2 ==
好,世
Size: 9
Length: 3


Mike

wila
15-Nov-2019, 08:45 AM
Thanks Mike, so to continue the fun (and to welcome all that lovely unicode confusion) I changed it all back to WString...



Use Windows.pkg


Procedure Test
WString sTest sPart
UChar[] ucaTest

Move "你好,世界" to sTest
Move (StringToUCharArray(sTest)) to ucaTest

Showln "== 'Hello World' in traditional Chinese =="
Showln sTest
Showln ("Size:" * String(SizeOfWString(sTest)))
Showln ("Length:" * String(Length(sTest)))
Showln ("Bytes:" * String(SizeOfArray(ucaTest)))

Move (Mid(sTest, 3, 2)) to sPart

Showln
Showln "== Mid 3,2 =="
Showln sPart
Showln ("Size:" * String(SizeOfWString(sPart)))
Showln ("Length:" * String(Length(sPart)))
Inkey Windowindex
End_Procedure

Send test


output:


== 'Hello World' in traditional Chinese ==
你好,世界
Size: 5
Length: 5
Bytes: 15

== Mid 3,2 ==
好,世
Size: 3
Length: 3


;)

--
Wil

Mike Peat
15-Nov-2019, 08:58 AM
So SizeOfWString is pretty redundant (except in terms of consistency) since Length always returns the same value...

Or am I missing something?

It makes sense for String, since Length returns the number of characters, while SizeOfString returns the number of bytes those characters occupy.

Mike

Harm Wibier
15-Nov-2019, 09:11 AM
Well... most characters (including Chinese) are a single code unit in UTF-16, but non-BMP (https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane)character s and composite pairs are not. Try:


Move "����������" to sTest

Harm Wibier
15-Nov-2019, 09:12 AM
Whoops, it turns out our forum do not support non-bmp characters as well. Try some of the smilies from this page: https://en.wikipedia.org/wiki/Emoticons_(Unicode_block)

Mike Peat
15-Nov-2019, 09:36 AM
Ah ha: crying face:


Size: 4
WSize: 2
Length: 1
Bytes: 4


OK, fair enough. :)

Mike

wila
15-Nov-2019, 09:40 AM
Whoops, it turns out our forum do not support non-bmp characters as well. Try some of the smilies from this page: https://en.wikipedia.org/wiki/Emoticons_(Unicode_block)

Nice (ha DataFlex can do smilies, now THAT is worth a new release :p )

--
Wil

Marco
15-Nov-2019, 06:08 PM
Nice (ha DataFlex can do smilies, now THAT is worth a new release :p )

--
Wil

Yeah, but not in object names yet.
That would have really made it interesting [emoji6]