ASCII Collapse Vulnerability

You are a security engineer reviewing a web application. You have put in place a strong Content Security Policy. You sanitize all user input. You escape special characters. You feel confident.

Then an attacker sends a payload containing ＜script＞. Your filter looks for <script> and finds nothing. The browser reads ＜script＞ and interprets it as <script>. Your XSS protection has just been bypassed.

This is the reality of Unicode normalization vulnerabilities. When applications reduce non-ASCII characters to their ASCII equivalents, they can inadvertently enable XSS attacks that were previously blocked.

Important Disclaimer

This article is intended for educational and defensive purposes only. The techniques described here are shared to help security professionals understand emerging threats so they can better protect their systems.

Do not use these techniques against systems you do not own or do not have explicit written permission to test. Unauthorized testing is illegal in most jurisdictions and violates computer fraud and abuse laws.

The author assumes no liability for any damages, legal consequences, or other outcomes resulting from the use or misuse of this information. Always obtain proper authorization before conducting any security testing. Stay legal. Stay ethical. Stay responsible.

What Is the ASCII Collapse Vulnerability?

The ASCII Collapse Vulnerability happens when a system converts non-ASCII Unicode characters to their ASCII equivalents without thinking about the security implications of that transformation.

Unicode has many characters that look exactly like ASCII characters but have different code points. The fullwidth Latin characters in the U+FF00 to U+FF5E range have a direct 1:1 mapping to ASCII characters. The character ａ looks identical to a but is a completely different code point.

When an application normalizes Unicode input using NFKC normalization or similar transformations, it converts these visually identical characters to their ASCII equivalents. This is what researchers call "collapsing" the characters.

The problem shows up when the application applies security checks before normalization. A filter might block <script> but allow ＜script＞. The browser, however, performs its own normalization and interprets the non-ASCII characters as their ASCII equivalents. As soon as the tag <script> is executed, the XSS attack succeeds.

What makes this vulnerability particularly dangerous?

It Bypasses Input Filters

Typically, XSS filters search for <script>, onerror=, and javascript:. They do not look for fullwidth or other Unicode variants. Attackers can get around these filters by substituting Unicode characters that normalize to ASCII.

Normalization Is Inconsistent

Different parts of a system may normalize text differently. The application might not normalize at all. The browser might normalize during URL processing. The WAF could very well have its own normalization rules. This creates blind spots that attackers can exploit.

It Is Difficult to Spot

Normalized characters look exactly like their ASCII counterparts. A security reviewer looking at code might not notice that a character is non-ASCII. Automated scanners that rely on pattern matching may miss the attack entirely.

How the Attack Works

The Fullwidth Bypass

Latin fullwidth characters from the U+FF00 to U+FF5E code range are especially useful when conducting XSS attacks, as they can be mapped 1:1  to ASCII characters.

Payload Example:

https://example.com/?q=＜script＞alert(1)＜/script＞

What Happens:

• The application gets the fullwidth characters payload
• The XSS Filter detects <script> that one that doesn’t match its pattern
• The application allows the payload to go through
• The browser normalizes  <script> to <script>
• The script executes

The IDNA Bypass

Chromium-based browsers normalize certain Unicode characters during URL processing through IDNA/punycode normalization. This can bypass length restrictions or character filters.

Example:

// Application enforces a 6-character maximum for domain names
// The attacker uses 6 fullwidth characters that normalize to
 8 ASCII characters
unicode_domain = "\uff41\uff42\uff43\uff44\uff45\uff46"  // "abcdef"
// Chrome normalizes to "abcdef" (6 characters)
// But the application sees 6 Unicode code points

This technique can bypass domain length restrictions and character filters on domain names.

Bypassing the Character Filter

If the application blocks some characters from the domain name, the attacker may make use of the full-width equivalents of the blocked characters.

Example:

// Application blocks ‘x’ in domain names
// Attacker uses full width 'ｘ' (U+FF58)
url = "http://e\uff58ample.com/payload"
// Chrome normalizes to http://example.com/payload

Scenario 1: Comment System

The Setup

A web site has implemented a feature where users can write comments. It has a regex based XSS filter which scans for <script> tags. The filter also escapes special characters.

The Attack

The attacker leaves a comment that contains ＜script＞alert(1)＜/script＞ The filter detects full-width characters and rejects it.The comment is stored in the database.

The Result

If any other user views the comment, then the browser converts fullwidth characters into standard ones. The script executes. The attacker has achieved XSS.

The Aftermath

The attacker steals session cookies from victims' browsers, redirects them to malicious websites or defaces the web site. The reputation of the company is damaged.

Scenario 2: The URL Shortener

The Setup

A URL shortening service allows users to enter long URLs. It validates the domain name using the regular expression, which restricts some specific characters from the domain name.

The Attack

The attacker uses the presence of fullwidth characters to bypass the domain filter by inserting the URL of a malicious domain in fullwidth characters.

The Result

The service accepts the URL and generates a short link. Whenever the user clicks on the link, they are automatically redirected to the malicious website.

The Aftermath

The user is subjected to phishing and the company becomes responsible for the attack.

Scenario 3: The Login Page

The Setup

The login page is using Content Security Policy and only allowing scripts from certain domains. The CSP blocklist has blocked evil.com.

The Attack

The exploit makes use of full-width Unicode characters in the domain name ｅｖｉｌ.com.

The Result

The CSP detects evil.com and blocks it. Evil.com is not blocked since CSP normalizes the domain differently than the browser does.

The Aftermath

The attacker implements CSP Bypass to execute malicious scripts.

Testing for ASCII Collapse Vulnerabilities

Using Fuzzing on Normalization Mappings

It is possible to fuzz Unicode characters to find out those that will map to particular ASCII characters.

import unicodedata

target_char = 'a'
results = []
for cp in range(0x100, 0xffff):
    c = chr(cp)
    normalized = unicodedata.normalize('NFKC', c)
    if target_char in normalized:
        results.append(f"U+{cp:04X} ({c}) -> {target_char}")

Known Useful Mappings

Payloads for Testing

Your web application will require the following payloads to test:

＜script＞alert(1)＜/script＞
%uff1cscript%uff1ealert(1)%uff1c/script%uff1e
\uFF1Cscript\uFF1Ealert(1)\uFF1C/script\uFF1E

Countermeasures

1. Normalization First

Normalize the input data before executing any form of security filtering. Use either of the normalization types such as NFKC or NFC.

import unicodedata

def sanitize_input(user_input):
    normalized = unicodedata.normalize('NFKC', user_input)
    # Apply XSS filter to normalized string
    return xss_filter(normalized)

2. Use Normalization Consistently

All parts of the application must follow the same normalization technique. Any inconsistency will make your application vulnerable.

3. Restrict Valid Unicode Characters

Make sure the Unicode characters used are safe for use. Disallow fullwidth Latin letters, math symbols, , and other character ranges that can be abused.

4. Whitelist Characters and Patterns

Since whitelisting allows only the good characters and patterns to pass through, it is better than blacklisting.

5. Content Security Policy and Nonces

Nonces in CSP will stop XSS attacks. Even if the attacker tries to inject a script, he won’t be able to determine the nonce.

Conclusion

The ASCII Collapse vulnerability is a dangerous flaw that enables attackers to circumvent conventional defenses against XSS attacks. This is done through the use of Unicode characters whose normalization results in ASCII characters.

The reason for this is that while applications perform filtering on raw input data, the browser normalizes this data and executes it.

It is important to filter and validate data only after normalization. Validate Unicode code points. Use a whitelist approach.

FAQ Section

What is the ASCII Collapse Vulnerability?

The ASCII Collapse Vulnerability occurs when a system converts non-ASCII Unicode characters to their ASCII equivalents without considering the security implications. Attackers can use fullwidth Latin characters to bypass XSS filters.

How does the attack work?

Attackers send payloads containing Unicode characters that normalize to ASCII equivalents, such as ＜script＞. The filter sees the fullwidth characters and does not match them. The browser normalizes them to <script> and executes the script.

What characters are most commonly used?

Fullwidth Latin characters in the U+FF00 to U+FF5E range have a 1:1 mapping to ASCII characters. These are the most commonly used in attacks.

How can I check the presence of this vulnerability?

Fuzzing can be done for Unicode that gets collapsed into ASCII characters. The payloads to use include ＜script＞alert(1)＜/script＞ within your application.

How can I protect against the ASCII Collapse attack?

Unicode code point normalization should always precede any security checks. Consistent normalization should be done throughout all components. Validate all Unicode code points. Whitelist only.

Is this vulnerability common?

Yes. Most applications don’t perform input normalization before filtration. This vulnerability has been exploited in CTFs and even some real applications.