Why your string length is wrong: code units, code points and graphemes

The same emoji counts as 1, 2, 7 or 11 depending on what is counting — and each answer is right for something. Which to use when, and the characters that cause trouble.

Three different questions, three different answers

You can watch the disagreement inside this toolbox: the Word & Char Count tool reports UTF-16 code units, because that is what a text field and most length validators actually enforce, while the Unicode Inspector splits grapheme-aware and reports code points. Paste an emoji into both and the numbers differ — neither is broken.

The example that makes it concrete

Take the family emoji 👨‍👩‍👧‍👦. It is four person emoji joined by three zero-width joiners: 7 code points, 11 UTF-16 code units, 25 UTF-8 bytes — and 1 grapheme. A flag emoji is two regional-indicator letters. A skin-toned thumbs-up is a base emoji plus a modifier.

Every one of those numbers is the right answer to some question. The bug is using the wrong one: truncating at 20 code units can slice a surrogate pair in half and leave a replacement character at the end of the string.

The same letter, two spellings

é can be one code point (U+00E9) or two (a plain e followed by U+0301, a combining acute accent). They render identically and are not equal under a byte or code-point comparison. This is why user input that looks correct fails a lookup, and why macOS filenames historically compared unequal to the same name typed on Linux.

The fix is normalisation — String.prototype.normalize("NFC") — applied once at the boundary, before storing or comparing. Run both spellings through the Unicode Inspector and the difference is immediately visible in the code point list.

Characters that cause real trouble

When a string is behaving impossibly, inspect it rather than staring at it: the Unicode Inspector lists every code point with its UTF-8 bytes, and the Hex ↔ Text tool shows the raw bytes when you suspect an encoding problem rather than a character problem.

Bytes are a fourth count, and often the binding one

UTF-8 uses one byte for ASCII, two for most Latin accents and Greek, three for CJK and most symbols, four for emoji and rarer scripts. So a field limited to 255 bytes holds 255 English characters or about 63 emoji.

That distinction is where database limits bite. Postgres varchar(n) counts characters, MySQL's utf8mb4 counts characters but indexes have byte limits, and plenty of APIs document a character limit while enforcing a byte one. If a value fails validation only for non-English users, this is nearly always why.

What to do about it

Tools used in this guide