Windows would be so much better if it were actually UTF-16. It's worse than that: it's from a world where Unicode thought "16-bits ought to be enough for anybody" and Windows NT baked that assumption deep into the system; it wasn't until 1996 that Unicode had to course-correct, and UTF-16 was carved out to be mostly compatible with the older standard (now known as UCS-2). For as long as you don't use the surrogate sequences in strings, you happen to be UTF-16 compatible; if you use the sequences appropriately, you happen to be UTF-16 compatible; if you use them in invalid ways to UTF-16, now you've got a mess that's a valid name on the operating system.

I can't really blame NT for this, it's unfortunate timing and it remains for backwards compatibility purposes. Java and JavaScript suffer similar issues for similar reasons.

I'll throw this out there too: UTF-8 isn't necessarily better than UTF-16; they both support the entirety of the Unicode character space.

UTF-8 is convenient on Unix systems since it fits into 8-bit character slots that were already in place; file systems have traditionally only forbidden the NULL byte and forward-slash, and all other characters are valid. From this fact, you can use UTF-8 in file names with ease on legacy systems, you don't need any operating system support for it.

UTF-8 is "space optimized" for ASCII text, while most extra-ASCII Latin, Cyrillic, Greek, Arabic characters need two bytes each (same as UTF-16); most of Chinese/Japanese/Korean script in the BMP requires three bytes in UTF-8, whereas you still only need two bytes in UTF-16. To go further beyond, all SMP characters (eg, most emoji) require four bytes each in both systems.

Essentially, UTF-8 is good space-wise for mostly-ASCII text. It remains on-par with UTF-16 for most western languages, and only becomes more inefficient than UTF-16 for east-Asian languages (in such regions, UTF-16 is already dominant).

Space savings are irrelavent. Text is small, and after gzip its going to be the same anyways.

Seriously, when was the last time where you both cared about saving a single kb, but compression was not an option. I'm pretty sure the answer is never. It was never in the 90s, it is extra never now that we have hard drives hundreds of gb big.

UTF-8 is better because bytes are a natural unit. You dont have to worry about surrogates. You dont have to worry about de-synchronization issues. You dont have to worry about byte order.

Backwards compatibility with ascii and basically every interface ever, also helps. (Well not 7bit smtp..). The fact is, ascii is everywhere. Being able to treat it as just a subset of utf-8 makes a lot of things easier.

> (eg, most emoji) require four bytes each in both systems.

This is misleading because most emoiji are not just a single astral character but a combination of many.

Saving disk space and synchronization is only important for network transmission. At the local level, you will need to convert to something where you can get/know positioning, utf8 does not allow for this given its variability, this means a lot of operations are more expensive and you will have to convert to utf16 anyway.

Interestingly in a Javascript or similar runtime most of text that hits the caches where the size actually matters is still ASCII even in far east because of identifiers. Utf8 for the win!