Unicode Normalization in Django with Bearer Tokens

Unicode normalization inconsistencies can affect API authentication when bearer tokens are handled as Unicode strings and compared without normalization. In Django, HTTP Authorization headers are parsed as strings that may contain characters represented in multiple Unicode forms (e.g., composed vs. decomposed). If your token validation logic compares raw header values directly, an attacker could supply a visually identical token that differs in code point composition, potentially bypassing checks or causing inconsistent behavior.

Consider a Django view that extracts a bearer token from the Authorization header:

import re

def get_token_from_header(auth_header):
    # Naive extraction without normalization
    match = re.match(r'Bearer\s+(.+)', auth_header)
    return match.group(1) if match else None

If the token contains Unicode characters (rare but possible in opaque token schemes or custom schemes), the same logical token could be represented in multiple ways. Without normalization, an attacker might leverage homoglyphs or different combining sequences to evade logging, monitoring, or secondary validation steps that compare token strings without canonicalizing them.

Django's built-in token handling in packages like django-rest-framework typically treats bearer tokens as opaque byte sequences, but if you implement custom decoding or comparison, you must normalize. For example, before comparison or storage, apply Unicode normalization:

import unicodedata

def normalize_token(token: str) -> str:
    return unicodedata.normalize('NFC', token)

Applying NFC (or a consistent form such as NFKC depending on your threat model) ensures that equivalent tokens resolve to the same binary representation. This is especially important when tokens are logged, cached, or used in dictionary lookups. The risk is not that Django misparses standard JWTs (which are ASCII-safe), but that custom token schemes or edge cases in middleware could introduce subtle inconsistencies in authorization decisions.

Additionally, normalization should be applied early in request processing, ideally in middleware, to ensure consistent handling across all views and security checks. Combine this with constant-time comparison to avoid timing leaks, and ensure that any token introspection or validation logic operates on the normalized form.

To securely handle bearer tokens in Django, normalize the token string as soon as it is extracted from the Authorization header, and use constant-time comparison for any sensitive checks. Below is a robust middleware snippet that normalizes and stores the token for downstream use:

import re
import unicodedata
from django.utils.deprecation import MiddlewareMixin

class BearerTokenNormalizationMiddleware(MiddlewareMixin):
    def process_request(self, request):
        auth = request.META.get('HTTP_AUTHORIZATION', '')
        match = re.match(r'Bearer\s+(.+)', auth)
        if match:
            raw_token = match.group(1)
            request.normalized_bearer_token = unicodedata.normalize('NFC', raw_token)
        else:
            request.normalized_bearer_token = None

Use the normalized token in views or token validation utilities:

from django.conf import settings
import hmac
import unicodedata
import re

def verify_bearer_token(request, expected_token):
    raw = request.META.get('HTTP_AUTHORIZATION', '')
    match = re.match(r'Bearer\s+(.+)', raw)
    if not match:
        return False
    token = unicodedata.normalize('NFC', match.group(1))
    expected = unicodedata.normalize('NFC', expected_token)
    return hmac.compare_digest(token, expected)

If you use token libraries or JWT packages, ensure that any intermediate string handling also normalizes. For example, when passing a token to a third-party introspection endpoint, normalize before logging or caching:

import unicodedata

def log_token(token: str) -> None:
    safe_token = unicodedata.normalize('NFC', token)
    # Use safe_token for logging to ensure consistent identifiers

For projects using the middleBrick CLI to scan your Django API endpoints, you can integrate scans into your workflow with: middlebrick scan <url>. This helps detect whether your exposed endpoints rely on unvalidated or unnormalized token handling. Teams using the Pro plan benefit from continuous monitoring and can configure the GitHub Action to fail builds if new issues are introduced, while the Dashboard provides historical tracking of security scores.