The answer first: the rule depends on which segment the character sits in
URL encoding is not "replace every special character with %xx". The same character has different legality in the path, the query and the fragment, so whether it is encoded — and into what — depends on position. One principle covers it: a reserved character stays literal where it is structural and must be encoded where it is data.
https://api.example.com:443/v1/search?q=caf%C3%A9&page=2#results
└─┬──┘ └──────┬──────┘└┬┘└────┬────┘ └────┬────┘ └───┬───┘
scheme host port path query fragment
Character classes and per-segment rules
A URL allows only a subset of ASCII. Unreserved characters A-Z a-z 0-9 - _ . ~ may appear literally anywhere; reserved characters : / ? # [ ] @ ! $ & ' ( ) * + , ; = appear literally only when they carry structure and must be encoded as data. Everything else — Chinese, spaces, emoji — is always encoded.
A per-segment reference table
| Character | Path | Query | Fragment | Notes |
|---|---|---|---|---|
Alphanumeric - _ . ~ |
✅ literal | ✅ literal | ✅ literal | Unreserved |
/ |
⚠️ delimiter | ✅ allowed | ✅ allowed | %2F when data |
? |
⚠️ starts query | ✅ allowed | ✅ allowed | %3F when data |
# |
❌ starts fragment | ❌ starts fragment | ❌ starts fragment | Always %23 as data |
& = |
✅ allowed | ⚠️ delimiter | ✅ allowed | %26 %3D in values |
+ |
⚠️ literal plus | ⚠️ space in forms | ✅ literal | See below |
% |
❌ escape prefix | ❌ escape prefix | ❌ escape prefix | %25 when literal |
| Space | %20 |
%20 or + |
%20 |
Paths require %20 |
| Non-ASCII | UTF-8 bytes, each encoded | same | same | See internationalisation |
# is special because its first occurrence takes everything after it: one unencoded # inside a query value turns the remaining parameters into a fragment the server never receives.
Base64URL is not URL encoding
URL encoding rewrites characters as % plus hex; Base64 turns binary into A-Za-z0-9+/=; Base64URL then swaps +// for -/_ and drops padding so the result can sit directly in a URL. JWT segments use Base64URL, which is why no extra percent-encoding layer is needed; decode by hand with the Base64 converter.
Building query strings
The query is the part after ?, made of &-separated pairs. There is no single standard here, only two conventions living side by side.
Spaces: where %20 ends and + begins
| Context | Space as | Origin |
|---|---|---|
| Percent-encoding | %20 |
RFC 3986 |
application/x-www-form-urlencoded |
+ |
HTML form submission |
URLSearchParams, Python's urlencode and Go's url.Values.Encode() follow the second convention, turning spaces into +; hand-built paths must use %20. Path and query may each use their own convention, but never mix them inside one segment.
Arrays and nesting: convention only
| Style | Example | Commonly supported by |
|---|---|---|
| Repeated keys | tag=a&tag=b |
Nearly everything (PHP needs tag[]) |
| Brackets | tag[]=a&tag[]=b |
PHP, Rails, Express (qs) |
| Dotted path | user.name=Tom |
Java Spring, some Go frameworks |
Cross-language integrations must agree explicitly, or the backend receives a string where it expected an array.
Use the standard library
JavaScript:
const params = new URLSearchParams({ name: 'John & Jane' });
const url = `/api?${params.toString()}`;
// /api?name=John+%26+Jane
Python:
from urllib.parse import urlencode, quote
query = urlencode({'name': 'John & Jane'}) # query: space → +
path = quote('John & Jane', safe='') # path segment: space → %20
Go:
import "net/url"
v := url.Values{}
v.Set("name", "John & Jane")
query := v.Encode() // query string
seg := url.PathEscape("John & Jane") // path segment
International URLs
Non-ASCII: UTF-8 first, then byte by byte
Two steps: convert the character to its UTF-8 byte sequence, then percent-encode each byte.
| Character | UTF-8 bytes | Encoded |
|---|---|---|
中 |
E4 B8 AD |
%E4%B8%AD |
é |
C3 A9 |
%C3%A9 |
€ |
E2 82 AC |
%E2%82%AC |
| 😀 | F0 9F 98 80 |
%F0%9F%98%80 |
An emoji occupies four bytes, so one emoji becomes twelve characters; budget URL length in bytes, not characters.
IDN and homographs
Domains use Punycode (RFC 3492): 中文.com becomes xn--fiq228c.com. Browsers display the original characters while requests use the Punycode form.
⚠️ Homograph attacks: Cyrillic а is visually identical to Latin a, so аpple.com can impersonate apple.com. Defend by displaying Punycode for user-supplied domains or validating against a character allow-list. Break a real link into its parts with the URL parser.
Encoding bugs, ordered by frequency and impact
1. Double encoding
Symptom: %25 appears — the % itself got encoded again.
Cause: the framework already encoded the value and the code called encodeURIComponent on top.
Check: decode twice and see whether it round-trips, or simply search for %25.
Fix: separate the layers — the data layer keeps raw values, the construction layer encodes once, the transport layer touches nothing.
Original: café
First pass: caf%C3%A9
Second pass: caf%25C3%25A9 ❌
2. Join first, encode second
Symptom: ?a=1&b=2 reaches the server as one parameter, or the value contains a%3D1%26b%3D2.
Cause: the whole pair string was encoded at once, so & and = stopped delimiting.
Check: print the final URL and count the & separators; the correct count is one, the broken count is zero.
Fix: encode each value separately, then join with the delimiters.
// ❌ Join first, then encode: the delimiters get encoded too, so the server sees one param
encodeURIComponent('a=1&b=2') // a%3D1%26b%3D2
// ✅ Encode each value, then join with delimiters
`?a=${encodeURIComponent('1')}&b=${encodeURIComponent('2')}`
3. Unencoded reserved characters rewrite the structure
Symptom: the path is truncated, parameters vanish, or the user lands on another site.
Cause: raw user input is concatenated into a URL, so ?, # or / changes the structure; splicing it into a next= parameter creates an open redirect.
Check: feed input containing ?, # and / and observe the path segment count and parameter count the server receives.
Fix: encode every value, and restrict redirect targets to allow-listed relative paths.
4. %2F turns routes into 404s
Symptom: an encoded slash returns 404 or matches the wrong route.
Cause: most servers and gateways decode the path before route matching, so %2F becomes / and the segment count changes.
Check: compare the decoded path in the access log against the raw path sent by curl --path-as-is.
Fix: move such values into the query string, switch to Base64URL, or stop the server decoding — Apache's AllowEncodedSlashes NoDecode, or Nginx keeping $request_uri.
5. Charset mismatch produces mojibake
Symptom: Chinese turns into replacement characters while ASCII stays intact.
Cause: one side writes UTF-8 and the other decodes as GBK, or the reverse.
Check: compare the original bytes with the decoded bytes, and inspect the response charset.
Fix: UTF-8 end to end with explicit declarations; specify the charset when decoding instead of relying on detection.
6. Missing normalisation breaks caching and signatures
Symptom: one resource yields several cache keys, or signature verification passes intermittently. Cause: hex case, default ports, parameter order and empty parameters differ across implementations. Check: normalise two visually identical URLs and compare them. Fix: decode once, lowercase scheme and host, sort query parameters alphabetically, drop default ports — then sign or build the cache key.
encodeURI vs encodeURIComponent
| Function | Purpose | Characters left alone |
|---|---|---|
encodeURI |
Encode a whole URL | A-Za-z0-9 ; , / ? : @ & = + $ - _ . ! ~ * ' ( ) # |
encodeURIComponent |
Encode a parameter value | A-Za-z0-9 - _ . ! ~ * ' ( ) |
new URL() |
Parse and normalise | Handles each segment automatically |
// Encode a whole URL (fixes illegal characters, preserves structure)
encodeURI('https://example.com/path?name=café')
// https://example.com/path?name=caf%C3%A9
// Encode a value (delimiters included)
encodeURIComponent('name=café&p=1')
// name%3Dcaf%C3%A9%26p%3D1
⚠️ encodeURIComponent lets ! ~ * ' ( ) through — RFC 3986 sub-delimiters. It rarely matters, but strict signing schemes (AWS SigV4, OAuth 1.0) need one extra pass:
const rfc3986 = (s) => encodeURIComponent(s)
.replace(/[!'()*]/g, (c) => '%' + c.charCodeAt(0).toString(16).toUpperCase());
When the server decodes
| Stage | Decodes? | Notes |
|---|---|---|
| Nginx / Apache route matching | ⚠️ Usually first | %2F becomes /, changing the segment count → 404 |
| Framework routing | ✅ Path segments | %2F is a common exception |
| Query parsing | ✅ Automatic | Both + and %20 become spaces |
| Request body | Depends on Content-Type | Forms decode; JSON does not |
Another boundary: nothing after # enters the request. Putting a token in the fragment only keeps it out of server logs; frontend JavaScript reads it freely and any XSS can steal it, so it is not a security measure.
When a route 404s but the URL looks right, check %2F, %5C (backslash) and %2E%2E (..) first — they are the encodings used for path traversal, and many WAFs block them outright. See the HTTP status code reference for related codes.
Length limits
No specification sets a maximum URL length, but reality does:
| Component | Limit |
|---|---|
| IE (historical) | 2,083 characters |
| Most CDNs / gateways | 8,192 characters |
Nginx default large_client_header_buffers |
8 KB (beyond it returns 414) |
| Practical guidance | Keep query strings under 2,000 characters |
GET requests with many filters hit this fastest; move to POST with a body.
Checklist
- [ ]
encodeURIComponentfor values,encodeURIonly for a whole URL - [ ]
%20for spaces in paths; let the standard library handle queries - [ ] Never mix
+and%20inside one segment - [ ] Encode user input before it enters a path, and reject
..and%2F - [ ] Normalise URLs before signing or caching
- [ ] UTF-8 end to end, with no double encoding (search for
%25) - [ ] Switch to POST once a query passes 2,000 characters
- [ ] Keep sensitive parameters out of the fragment
30-second verification
from urllib.parse import quote, quote_plus
s = '你好 world'
print(quote(s)) # → %E4%BD%A0%E5%A5%BD%20world (space kept as %20)
print(quote_plus(s)) # → %E4%BD%A0%E5%A5%BD+world (space becomes +)
The same input yields two valid but non-equivalent results: quote for path segments, quote_plus for query strings, with unquote / unquote_plus on the way back. The Chinese text becomes six percent-encoded bytes (two characters × three UTF-8 bytes each), visible right here. Encode online with the URL encoder and build URL-safe identifiers with the slug generator.