Unicode support in RCSDK

The amount of effort required to support Unicode properly in a vast code project which has never been Unicode-aware can get really huge. So much, that if I had to explain what I’ve been through during the past 10 days, I wouldn’t even know where to begin. Of course our case may be a little special because at RandomControl every single line of code we use is proprietary, but yet, supporting Unicode may force you to carry out a massive amount of changes in your codebase. Actually, my (obvious) recommendation to anybody who’s working on a project that is intended to go international and last forever would be to support Unicode from the start.

I remember when I turned RCSDK and fryrender from 32-bit to 64-bit a few years ago. Everything went through clean and easy (ok… let’s ignore the fact that supporting x86 and x64 dooms you to manage twice as many project configurations and see your build times become twice as long…). But anyway, carefully crafted code (as I like to believe is ours) where you make no hard-coded assumptions on the size or memory alignment of data may require basically no changes at all to support 32/64-bit architectures. There are a couple of implications, such as serializing data in non-architecture-dependent ways, and if you’re ambitious, conditioning yourself to write code with data alignment in mind for better efficiency. But those implications are rather small. They are nothing compared to modifying your entire codebase so each and every single operation where you access the file system, or handle a text string is aware of Unicode and uses it properly.

If you are using Unicode-ready third-party libraries, or if you have always coded with Unicode support in mind, then you’re lucky. If not, tonight we dine in hell.

Unicode logo

If you look at them individually, none of the upgrades in RCSDK I’ve had to do lately is extraordinarily complicated. But the amount of them is beyond ridiculous. Just to mention a few of the things that I’ve had to work on during the past days:

– Multi-platform ASCII/Unicode conversion routines for strings and paths.
– UTF-8 transport in our XML I/O and generic marshalling modules.
– Unicode support for the command-line.
– Unicode glyph rasterization in our┬ámulti-platform UI manager.
– Unicode-aware code branches in our multi-platform file-system module.
– Find elegant ways to not need to compile RCSDK twice (with and without Unicode support), so a single build links with Unicode LIVE plugins (such as XSI and MAX 2013) and non-Unicode LIVE plugins (such as MAX 2010..12).
– …