Unicode String Strategy

[Gary Beene]

I'm pondering the question of whether it's best to declare all strings as WString in original PBWin10 code, and also whether it's best to modify legacy code (when using the code in PBWin10) so that all strings are declared as WString?

Or is it just fine to have some strings declared as String and others as WString - according to your own personal taste?

The compiler will let you do both in an app, and PowerBASIC will do conversions for you in many cases, but are there considerations which make it advantageous to stick with all WString declarations?

One reason to support mixed declarations is to minimize the effort to convert legacy code to PBWin10. But I'd be willing to go to more conversion effort if I saw that there were some logical reasons to stick with a single type of declaration.

A downside to using WString declarations across the board is that legacy folks who have not updated to PBWin10 would have to make more changes in order to use posted PBWin10 code (only matters for code where PBWin9 supports the features used in that code).

In the PBWin10 samples, mixed declarations (String and WString) are found in most of the samples. So PowerBASIC Inc. doesn't appear to be promoting a single declaration strategy.

[Stuart McLachlan]

I think I'll follow PB's advice in Help - Characters, Strings, and Unicode

At this time, and for the foreseeable future, UTF-16 is the character set of choice for all of your applications. It is the best way to store all of your data to keep it secure and understandable.

[José Roca]

I use unicode strings in new code unless there is a good reason to use ansi.

[Fred Harris]

I have gone through the process of converting some of my older code to wide string, and what I have found for myself, is that it was easiest to leave code using text files alone, i.e., leave those variables as strictly ansi. In terms of general program logic code, wide string is just as easy as ansi, so going forward, wide string is more up to date. Likewise with Api code. I believe the general concensus is that all the ...A Api functions are just wrappers that convert ansi strings to wide, then call the ...W string versions.

Shortly after PBWin10 and CC6 came out I spent some time with this topic, and posted some code ideas here ...

http://www.powerbasic.com/support/pb...hlight=Unicode

[Gary Beene]

Hi Stuart,
Thanks for pointing out the Help comment. But this part of the quote didn't satisfy my intellectual itch:

It is the best way to store all of your data to keep it secure and understandable.

Surely that doesn't mean that all of the ANSI data we've been working with is non-secure and not understandable? I'd guess there's a more technical argument to made for adopting an all W String strategy?

And Fred,
Thanks for your thread. I scanned it briefly and it looks very useful. I'll go read it more carefully.

And Jose,
That was a tease!

... unless there is a good reason to use ansi.

What might be some of those reasons? With PowerBASIC's built in conversions, does what you say ever happen?

The heart of this question leads to perhaps a "best practice" issue. When I/we post code for folks here on the forum, we'll want to provide code that gives the best guidance possible. Right now, I'm mostly happy to use mix and match ANSI/Unicode variable declarations, mostly because it's slightly easier given all of the legacy code I work with. But if I understood that there were clear pitfalls in the approach, I'd be more motivated to make the switch over to an all-W coding approach not only for myself, but for the folks who use our posted code as well.

Hence, the digging nature of my questions here. Please take no offense anyone if I try to peel away at any answers given.

[Michael Mattias]

I believe the general concensus is that all the [WinAPI] ...A Api functions are just wrappers that convert ansi strings to wide, then call the ...W string versions.

Consensus? I surely hope not because that is incorrect.

These API functions are provided by Microsoft, and use different entry points within the named libraries. (Load up USER32.DLL or KERNEL32.DLL using Show exports and imports for PB/Win 6x, 7x (original by Torsten Reinow) and you will see what I mean).

"How" Microsoft implements these functions - by converting between Wide and ANSI (eg as a 'wrapper'), or by working on the "raw" data 'as is' - is proprietary. In any event, when you call any WinAPI function, you will pass or receive string data in the format documented by the publisher.

MCM

[Gary Beene]

Hey Stuart,
One point I was going to make earlier was this line of code from Help, which is an easy to find example where PowerBASIC continues to use ANSI Strings:

CONTROL ADD BUTTON, hDlg,  id&, txt$, x, y, xx,  yy

So when you posted the Help comment that Unicode was the way to go, I thought of this example where Help shows that even PowerBASIC has yet to fully adopt the all-W strategy (I do assume they are converting internally).

It may be that PowerBASIC is headed towards converting all of it's statements to use WString arguments. But for now, when Help is full of ANSI arguments, it's hard to put too much weight into the Help quote from your earlier post.

If ANSI was good enough to leave in many of the PowerBASIC statements, how can I argue to newbies that it's a non-preferred practice?

The work-in-progress approach is certainly one argument. But I'd like to have something stronger - such as demonstrable, negative side effects of staying ANSI or of using a mixed strategy.

[Michael Mattias]

But I'd like to have something stronger - such as demonstrable, negative side effects of staying ANSI or of using a mixed strategy.

In the "pure SDK" environment, other than 'maintainability' I can't think of any reason to go "all W", "all "A" or "six-five and pick 'em'"

(See Post #6 this thread)

The publisher of the compiler will have to speak for himself re "mixed datatype string operands in assignments and expressions."

I tried a little "mix and match" last week just because I could and had no problems. However, since I mostly get to write stuff "new" and I want to get to "all Wide Char" I do everything with WSTRING and WSTRINGZ types where I formerly would have used STRING and ASCIIZ types. YMMV.

MCM

[Fred Harris]

Because the actual implementations of the ...W verses ...A functions are proprietary, we can't really know what they do, naturally. However, I have seen Microsoft code demonstrating the technique of converting ansi strings to wide character strings, then calling W functions - surely not a difficult concept to comprehend. Because of this, it makes intuitive sense to just use wide character strings which don't require any further manipulations/conversions before being fed into some Api function. That's the only point I was trying to make Michael.

[José Roca]

What might be some of those reasons? With PowerBASIC's built in conversions, does what you say ever happen?

Indeed. To avoid conversions if I don't need unicode, to use a string as a byte buffer, to use third party libraries that don't support unicode, to save .bas files to be compiled with PB..

[Michael Mattias]

However, I have seen Microsoft code demonstrating the technique of converting ansi strings to wide character strings, then calling W functions - surely not a difficult concept to comprehend.

I think those examples may have been provided to help programmers use functions for which ONLY a "wide-char" interface is supported... e.g. I think some of the "Shxxxx" functions are like that. ( I know I've run into this).

MCM

[Jim Dunn]

Originally Posted by Gary Beene

... One reason to support mixed declarations is to minimize the effort to convert legacy code to PBWin10 ...

You could use all WSTRING in your examples, or a mix, and include a line at the top of your code:

'#define WSTRING STRING ' uncomment if not using PBWin10 or PBCC6

or

'MACRO WSTRING = STRING ' uncomment if not using PBWin10 or PBCC6

[José Roca]

Originally Posted by Fred Harris

Because the actual implementations of the ...W verses ...A functions are proprietary, we can't really know what they do, naturally.

We can. We don't have the source code, but we can know what it does.

Originally Posted by Fred Harris

However, I have seen Microsoft code demonstrating the technique of converting ansi strings to wide character strings, then calling W functions - surely not a difficult concept to comprehend. Because of this, it makes intuitive sense to just use wide character strings which don't require any further manipulations/conversions before being fed into some Api function. That's the only point I was trying to make Michael.

A typical example, the implementation of the API function CharLowerBuffA:

DWORD
WINAPI
CharLowerBuffA(LPSTR str, DWORD len)
{
    DWORD lenW;
    WCHAR *strW;
    if (!str) return 0;

    lenW = MultiByteToWideChar(CP_ACP, 0, str, len, NULL, 0);
    strW = HeapAlloc(GetProcessHeap(), 0, lenW * sizeof(WCHAR));
    if (strW) {
        MultiByteToWideChar(CP_ACP, 0, str, len, strW, lenW);
        CharLowerBuffW(strW, lenW);
        len = WideCharToMultiByte(CP_ACP, 0, strW, lenW, str, len, NULL, NULL);
        HeapFree(GetProcessHeap(), 0, strW);
        return len;
    }
    return 0;
}

It creates an unicode copy of the string with MultiByteToWideChar, calls CharLowerBuffW to do the conversion to lower case and then calls WideCharToMultiByte to convert it back to ansi.

[José Roca]

The point is that if someone is calling the API ansi functions thinking they should be faster and use less memory, he is wrong. They would be faster and use less memory if fully implemented in ansi, instead of being mere wrappers, but it is not the case.

[Francisco Castanedo]

I think all depends on the code of the program you're writting.

For example: I have a commercial program that uses ANSI only. It is for spanish speaking people only (it uses the same ASCII table as US English) so I haven't had the need to use UNICODE in it. I was when I implemented a security feature that uses a MySQL database that resides in my website's server when the need to use UNICODE to communicate with it arose. But only the portion of the program dealing with MySQL needed UNICODE and I just convert the WSTRINGs into STRINGs as needed.

On the other hand, I'm about to begin a multilingual project that will use web-based SQL databases. The only way I can accomplish this is by going UNICODE all the way. If I were to mix UNICODE and ANSI on this project it could lead to very annoying problems when showing text to the user and when communicating with the SQL database.

For in-house apps (utility programs) I use mostly ANSI and convert back-and-forth from/to UNICODE as needed.

Hope my point of view helps.