The answer first: byte counts come from the encoding, not the character
"A Chinese character takes two bytes" is only roughly true under legacy encodings such as GBK. The same '中' takes 2 bytes in GBK, 3 in UTF-8 and 4 in UTF-32. A character has no inherent byte size; its encoding does.
ASCII: 7 bits and 128 characters
ASCII defined 128 characters with 7 bits: uppercase and lowercase letters, digits, punctuation and control codes. It served English and could not hold other scripts — every compatibility problem that follows starts here.
'A' = 65 = 0x41 = 1000001 (1 byte, 7 significant bits)
Extended encodings: the breeding ground for mojibake
Each region defined what bytes above 0x80 meant: Western Europe chose ISO-8859-1, Chinese GBK/GB2312, Japanese Shift-JIS. These schemes are mutually incompatible: the byte run 0xE4 B8 AD (UTF-8 for '中') reads as a different character under GBK. Mojibake means the writing encoding and the reading encoding disagree; the character itself was never "broken".
Diagnose in this order:
- Get the raw bytes (
xxd,hexdump, or Python's.encode()) and establish which sequence they actually are; - Decode explicitly with the matching encoding instead of letting an editor or browser guess;
- Record the writer's encoding in the interface documentation so the next person does not have to guess.
Unicode and UTF-8: a character set versus an encoding
Unicode assigns every character a globally unique code point — '中' is U+4E2D — while UTF-8 specifies how those code points become bytes. The two are a specification and an implementation, not synonyms.
UTF-8 variable-length rules
| First byte | Bytes | Significant bits | Coverage |
|---|---|---|---|
0xxxxxxx |
1 | 7 | U+0000–007F (ASCII) |
110xxxxx |
2 | 11 | U+0080–07FF |
1110xxxx |
3 | 16 | U+0800–FFFF (incl. CJK) |
11110xxx |
4 | 21 | U+10000+ (emoji etc.) |
Continuation bytes are always 10xxxxxx. The first byte alone tells a decoder how many bytes follow, and pure ASCII is valid UTF-8 by construction.
Hand-computing '中': U+4E2D → E4 B8 AD
code point U+4E2D = 0x4E2D
binary 0100 1110 0010 1101 (16 bits → needs 3 bytes)
expand with 1110xxxx 10xxxxxx 10xxxxxx:
1110 0100 10 111000 10 101101
───────── ──────── ────────
E4 B8 AD
→ UTF-8 bytes: 0xE4 0xB8 0xAD
Sixteen bits fit exactly into the 16 significant bits of a three-byte sequence, and filling the template yields E4 B8 AD. That is the origin of "one Chinese character is 3 bytes in UTF-8".
UTF-16 and HTML entities: two other routes
- UTF-16: two bytes for Basic Multilingual Plane characters, four via surrogate pairs for supplementary planes. Windows and many systems use it internally; UTF-8 dominates on the network.
- HTML entity
中: writes the hex code point directly, so it displays regardless of file encoding; useful in constrained environments, at the cost of readability.
A checklist against mojibake
- [ ] All text files UTF-8, without a BOM;
- [ ] HTTP responses carry
Content-Type: text/html; charset=utf-8; - [ ] Tables, connection strings and collations all use
utf8mb4; - [ ] Both ends agree on UTF-8 for transport, which JSON already defaults to;
- [ ] On mojibake, first establish who wrote these bytes and who is reading them, then convert — never guess.
30-second verification
"你".encode("utf-8") # → b'\xe4\xbd\xa0'
"你".encode("utf-8").hex() # → 'e4bda0'
len("你") # → 1 (characters)
len("你".encode()) # → 3 (bytes)
你 is U+4F60, inside U+0800–U+FFFF, so UTF-8 spends 3 bytes on it (E4 BD A0). This is also the root of the "varchar by characters or bytes" argument: ten such characters cost 10 by count and 30 as UTF-8 bytes. Enter 你 in the text to Unicode tool to see the same code point escaped.