Skip to content

Fix quadratic-time backtracking in hash header regex - #1672

Merged
waylan merged 1 commit into
Python-Markdown:masterfrom
naaz234:hash-header-trailing-hashes
Oct 23, 2026
Merged

waylan merged 1 commit into
Python-Markdown:masterfrom
naaz234:hash-header-trailing-hashes

Conversation

@naaz234

@naaz234 naaz234 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Description

HashHeaderProcessor.RE grows the header text one character at a time and after each step tries #*(?:\n|$) to see if only closing hashes are left. When that step lands inside a run of # that is followed by more text, the check scans to the end of the run and fails, so a run of n hashes costs O(n^2). With no extensions loaded, '# ' + '#' * 8000 + 'x' takes about 13 seconds in markdown.markdown() on CPython 3.11, and profiling puts all of it in re.search for this pattern. '#' * n + '\\' goes through the same path (over 9 s per search at n=8000).

This is not the same as #1545. There the time went to inline escape handling, while here it is spent inside the block regex.

The header now takes a whole run of hashes in one step when something other than a newline or the end of the block follows it (#+(?=[^#\n])), and # is removed from the single-character branch, so the lazy match can only stop at the start of a hash run. I compared the old and new patterns on every string of up to 8 characters over #, \, a and \n, plus about 470k random longer strings, and the span and groups matched in every case. The 8000-hash input now converts in about 1 ms and 100k hashes in about 0.3 s. The new test takes about 70 s against the old pattern and is instant with the fix. The full test suite and flake8 pass.

AI Assistance Disclosure

  • No AI tools were used in preparing this PR.

  • AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

    AI tools were use to assist with the following:

    • Code
    • Code Comments
    • Documentation
    • Changelog Entry
    • Commit Message
    • The Description Above
    • Other: finding the issue, the benchmarks and the equivalence check between the old and new regex

Checklist

`HashHeaderProcessor.RE` matches the header text lazily one character at
a time and, after each step, tries `#*(?:\n|$)` to see whether only
closing hashes remain. When the lazy match is inside a run of `#` that
is followed by more text, that check scans to the end of the run and
then fails, so a run of n hashes costs O(n^2). A line such as `# `
followed by 8000 `#` and an `x` takes about 13 seconds to convert. A
run of hashes ending in a backslash (`'#' * n + '\\'`) hits the same path.

Header text now consumes a run of hashes in one step when the run is
followed by something other than a newline or the end of the block, and
a lone `#` is no longer matched by the single-character branch. The
lazy match therefore only stops at the start of a hash run, never
inside one, and every match and capture group is unchanged. This was
checked against the old pattern on every string of up to 8 characters
over `#`, `\`, `a` and newline, plus random longer strings.

This differs from Python-Markdown#1545, where the time was spent in inline escape
handling; here the time is spent inside the block regex itself.
@waylan
waylan merged commit 0f0a860 into Python-Markdown:master Oct 23, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants