[hdf-forum] Unicode filenames on Windows?

[Andrew Collette]

Hi Quincey,

>        Hmm, I don't think we do anything special to the strings we pass to
> the file system.  Is there some particular problem you are seeing?

I can't figure out how to take an arbitrary sequence of Unicode code
points and create an HDF5 file with that name on Windows.

I have limited experience with Windows Unicode support, but I know
that the way Microsoft implements unicode is through a series of
wide-character (2-byte "UCS-2") APIs.  Unlike most UNIX platforms,
where you simply pass in a UTF-8 (or whatever) string through a char*,
I think you actually have to call a separate function (e.g. fopen vs.
_wfopen) to be able to handle generic Unicode filenames on Windows.
Otherwise Windows treats a simple char* string as extended-ASCII,
according to the current locale settings.  So if I'm on a French
computer, I can get HDF5 to generate an e-with-an-accent, but not (for
example) a name with Cyrillic letters.

Currently, as far as I've found out in my investigations, there's no
way to encode a generic Unicode string to char* on windows and have it
work with the filesystem; you have to use the UCS-2 functions.  I've
peeked at H5FDwindows.c and it looks like you're using the traditional
char* API.

I realize it's probably not a priority for HDF5 development, but it
would be nice if HDF5 could handle the full extent of names allowed by
the filesystem.  It seems like the correct place for that is the
Windows file driver.  One way would be to have two modes, perhaps set
by the file access property list; in the first, it passes the raw
bytes through to the filesystem (as is done now), and in the other, it
performs two-way translation between UTF-8 strings (HDF5 user side)
and the UCS-2/wchar API (Windows platform side).  This would have the
additional benefit of maintaining HDF5's internal standardization on
UTF-8.

Andrew

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.