Repository navigation
Conversation
The first scanner pass matched every byte of a string as a separate re2c token, and json_encode() copied the pending run each time it met a byte that is not plain ASCII, then decoded the character with php_next_utf8_char() and copied it on its own. - Scanner: before (re)entering the first string pass, skip the run of printable ASCII bytes other than quote and backslash, 16 bytes at a time with SSE2 and bytewise otherwise. - Encoder: skip runs of bytes that are always copied verbatim the same way, and with JSON_UNESCAPED_UNICODE keep well-formed UTF-8 sequences in the pending run. Anything else still takes the existing code path, so the output is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Profiling
json_validate()on string-heavy input showed ~63% of the time inphp_json_scan(): the first string pass (STR_P1) matches every byte as its own re2c token.json_validate()taking almost as long asjson_decode()confirms the scan, not value construction, dominates.On the encoding side,
json_encode($v, JSON_UNESCAPED_UNICODE)of non-ASCII text was ~3x slower than of ASCII text: for every non-ASCII characterphp_json_escape_string()flushes the pending run, decodes the character withphp_next_utf8_char()and appends it on its own.Changes:
STR_P1, skip the run of printable ASCII bytes other than"and\, 16 bytes at a time with SSE2 (baseline on x86_64) and bytewise otherwise. The re2c rules are unchanged; they are just entered less often.charmaplets through), and withJSON_UNESCAPED_UNICODEkeep well-formed UTF-8 sequences in the pending run instead of copying them one by one. Validation matchesphp_next_utf8_char()(no overlongs, surrogates or code points above U+10FFFF), and U+2028/U+2029 are only kept whenJSON_UNESCAPED_LINE_TERMINATORSis set. Everything else falls through to the existing code, so output and errors are unchanged.Results
Release build (
-O2), x86_64, median of 5 interleaved runs, 20 000-element payloads:json_decodejson_validatejson_encodejson_decodejson_encodejson_encode+JSON_UNESCAPED_UNICODEjson_decodejson_encodejson_decodejson_decodeWithout SSE2 (scalar fallback only, e.g. aarch64) the gains are smaller: ASCII decode 1.78x, UTF-8 encode with
JSON_UNESCAPED_UNICODE1.71x, but decoding escape-heavy strings is ~5% slower. A NEON variant of the skip loop could follow if wanted.Testing
ext/json/tests/string_scan_boundaries.phptputs each special byte (control characters, invalid UTF-8, quotes, escapes, U+2028, ...) at every position of strings of length 0–48, across 16-byte chunk boundaries, for decode, validate and encode with several flag sets. It passes before and after this change, and caught an off-by-one in an earlier version of the SIMD mask.json_decode()calls (4 flag sets) plusjson_validate()on random inputs, and 1 440 000json_encode()calls (12 flag sets), all identical including errors. Also with the SIMD paths disabled.ext/jsontests pass on release and debug builds, with and without the SIMD paths; no valgrind errors.