Repository navigation
Fix quadratic-time backtracking in hash header regex - #1672
Merged
waylan merged 1 commit intoOct 23, 2026
Merged
Conversation
`HashHeaderProcessor.RE` matches the header text lazily one character at a time and, after each step, tries `#*(?:\n|$)` to see whether only closing hashes remain. When the lazy match is inside a run of `#` that is followed by more text, that check scans to the end of the run and then fails, so a run of n hashes costs O(n^2). A line such as `# ` followed by 8000 `#` and an `x` takes about 13 seconds to convert. A run of hashes ending in a backslash (`'#' * n + '\\'`) hits the same path. Header text now consumes a run of hashes in one step when the run is followed by something other than a newline or the end of the block, and a lone `#` is no longer matched by the single-character branch. The lazy match therefore only stops at the start of a hash run, never inside one, and every match and capture group is unchanged. This was checked against the old pattern on every string of up to 8 characters over `#`, `\`, `a` and newline, plus random longer strings. This differs from Python-Markdown#1545, where the time was spent in inline escape handling; here the time is spent inside the block regex itself.
waylan
approved these changes
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
HashHeaderProcessor.REgrows the header text one character at a time and after each step tries#*(?:\n|$)to see if only closing hashes are left. When that step lands inside a run of#that is followed by more text, the check scans to the end of the run and fails, so a run of n hashes costs O(n^2). With no extensions loaded,'# ' + '#' * 8000 + 'x'takes about 13 seconds inmarkdown.markdown()on CPython 3.11, and profiling puts all of it inre.searchfor this pattern.'#' * n + '\\'goes through the same path (over 9 s per search at n=8000).This is not the same as #1545. There the time went to inline escape handling, while here it is spent inside the block regex.
The header now takes a whole run of hashes in one step when something other than a newline or the end of the block follows it (
#+(?=[^#\n])), and#is removed from the single-character branch, so the lazy match can only stop at the start of a hash run. I compared the old and new patterns on every string of up to 8 characters over#,\,aand\n, plus about 470k random longer strings, and the span and groups matched in every case. The 8000-hash input now converts in about 1 ms and 100k hashes in about 0.3 s. The new test takes about 70 s against the old pattern and is instant with the fix. The full test suite and flake8 pass.AI Assistance Disclosure
No AI tools were used in preparing this PR.
AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.
AI tools were use to assist with the following:
Checklist