Unicode Normalization in Flask

Unicode normalization attacks in Flask applications often exploit the framework's handling of HTTP request parameters and route matching. Flask uses Python's built-in string handling, which can be vulnerable to homograph attacks when Unicode normalization isn't properly applied. A classic example is the use of visually identical characters from different Unicode blocks to bypass authentication or authorization checks.

@app.route('/account/<user_id>')
def get_account(user_id):
    # Vulnerable: no normalization, accepts visually identical IDs
    if user_id == session['current_user_id']:
        return fetch_account_data(user_id)
    return 'Unauthorized', 403

In this Flask route, an attacker could use a Unicode character that looks identical to the legitimate user ID but has a different code point. For instance, the Latin small letter 'a' (U+0061) versus the Cyrillic small letter 'a' (U+0430) appear identical but are different characters. Without normalization, Flask's string comparison would treat them as distinct, potentially allowing account enumeration or unauthorized access.

Another Flask-specific manifestation occurs in URL parameter handling. Flask's routing system doesn't automatically normalize Unicode characters in route parameters:

@app.route('/search')
def search():
    query = request.args.get('q', '')
    # Vulnerable: special Unicode characters in search queries
    results = search_database(query)
    return jsonify(results)

An attacker could craft search queries using Unicode characters that, when normalized differently, bypass search filters or access restricted content. This is particularly problematic in Flask applications that use search functionality for access control or data filtering.

Database queries in Flask applications also face normalization issues. When using ORMs like SQLAlchemy with Flask, Unicode characters in query parameters might not be normalized before database operations:

@app.route('/users/<username>')
def user_profile(username):
    user = User.query.filter_by(username=username).first()
    # Vulnerable: username comparison without normalization
    if user and user.id == session['user_id']:
        return render_template('profile.html', user=user)
    return 'Not found', 404

Here, an attacker could use Unicode variations of usernames to access other users' profiles if the database stores usernames without consistent normalization.

Detecting Unicode normalization vulnerabilities in Flask requires both static analysis and runtime testing. For static analysis, examine your Flask routes and request handling code for string comparisons and database queries that don't normalize input. Look for patterns where user input is directly compared to stored values without normalization.

middleBrick's API security scanner can detect these vulnerabilities by testing your Flask endpoints with Unicode variations. The scanner sends requests with homograph characters and checks for inconsistent behavior. For example, it might test if '/account/a' and '/account/а' (Cyrillic 'a') return different results, indicating a normalization vulnerability.

# Scan your Flask API with middleBrick
middlebrick scan https://yourapp.com/api

The scanner tests 12 security categories, including authentication bypasses that could reveal Unicode normalization issues. It specifically looks for cases where different Unicode representations of the same logical character produce different application behavior.

For manual testing, use Python's unicodedata module to generate test cases:

import unicodedata
import requests

def test_unicode_variants(base_url, endpoint, param):
    # Generate visually similar characters
    variants = [
        'a',  # Latin a
        'а',  # Cyrillic a
        'ɑ',  # Latin small letter alpha
    ]
    
    for variant in variants:
        url = f"{base_url}/{endpoint}/{variant}"
        response = requests.get(url)
        print(f"{variant}: {response.status_code}")

Run this against your Flask endpoints to identify inconsistent responses. Pay special attention to authentication endpoints, profile pages, and any resource access controlled by string identifiers.

middleBrick's OpenAPI analysis also helps detect normalization issues by examining your API specification. If your OpenAPI spec defines parameters without proper validation patterns, middleBrick flags these as potential normalization vulnerabilities.

Remediating Unicode normalization in Flask applications requires consistent normalization across all input handling points. The most effective approach is to normalize all incoming request data to a standard form before processing. Flask provides several ways to implement this.

For route parameters and query strings, create a normalization middleware or decorator:

import unicodedata
from functools import wraps

def normalize_unicode(f):
    @wraps(f)
    def decorated_function(*args, **kwargs):
        # Normalize all string parameters
        normalized_kwargs = {
            k: unicodedata.normalize('NFC', v) 
            if isinstance(v, str) else v 
            for k, v in kwargs.items()
        }
        return f(*args, **normalized_kwargs)
    return decorated_function

@app.route('/account/<user_id>')
@normalize_unicode
def get_account(user_id):
    # Now user_id is always in NFC form
    if user_id == session['current_user_id']:
        return fetch_account_data(user_id)
    return 'Unauthorized', 403

This decorator ensures all route parameters are normalized to NFC (Canonical Decomposition, followed by Canonical Composition) form before reaching your view function. Choose NFC for most cases, as it's the most compatible form.

For request data like JSON bodies and form data, normalize in a before_request handler:

@app.before_request
def normalize_request_data():
    if request.is_json:
        request._normalized_json = {
            k: unicodedata.normalize('NFC', v) 
            if isinstance(v, str) else v 
            for k, v in request.get_json().items()
        }
    elif request.form:
        request._normalized_form = {
            k: unicodedata.normalize('NFC', v) 
            if isinstance(v, str) else v 
            for k, v in request.form.items()
        }

Access normalized data through request._normalized_json or request._normalized_form in your view functions.

For database queries, ensure consistent normalization at both application and database levels. If using SQLAlchemy with Flask:

from sqlalchemy import func

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True, nullable=False)
    
    @classmethod
    def find_by_username(cls, username):
        # Normalize before querying
        normalized = unicodedata.normalize('NFC', username)
        return cls.query.filter(
            func.normalize('NFC', cls.username) == normalized
        ).first()

This approach normalizes both the input and the stored username in the database query, ensuring consistent matching regardless of how the username was originally stored.

For comprehensive protection, combine these approaches with middleBrick's continuous monitoring. The Pro plan can scan your Flask API on a schedule, alerting you if new normalization vulnerabilities are introduced during development.