Repository navigation
Make check_stack_depth() effective for embedders; guard recursive output paths - #10
pyramation wants to merge 2 commits into
Conversation
…ive output paths libpg_query never calls set_stack_base(), so stack_base_ptr stays NULL and check_stack_depth() is a no-op: deep recursion in the JSON/protobuf output functions, the fingerprinter, the deparser and the PL/pgSQL JSON dump runs until the C stack overflows (a trap in WebAssembly builds). Add pg_query_set_stack_limit(bytes) so an embedder can set the reference point and limit at each entry, call check_stack_depth() at the top of _outNode, _fingerprintNode, dump_stmt/dump_expr and deparseExpr, and run pg_query_nodes_to_json in pg_query_parse_opts under PG_TRY so the 'stack depth limit exceeded' error is returned instead of being fatal.
|
Review complete. 🟠 2 high · 🟡 3 medium 💬 Inline comments (2)
📍 Findings outside the diff (3) — 🟠 2 high · 🟡 1 medium — defects on lines GitHub can't attach comments to🟠 High — Fingerprint entry points cannot catch new stack-depth error · pg_query_fingerprint.c:343–350 · // src/pg_query_fingerprint.c
343 if (parsetree_and_error.tree != NULL || result.error == NULL) {
344 FingerprintContext ctx;
345 XXH64_canonical_t chash;
346
347 _fingerprintInitContext(&ctx, NULL, printTokens);
348
349 if (parsetree_and_error.tree != NULL) {
350 _fingerprintNode(&ctx, parsetree_and_error.tree, NULL, NULL, 0);
🟠 High — Protobuf parse path lacks handler for stack-depth throw · pg_query_parse.c:195 · // src/pg_query_parse.c
195 result.parse_tree = pg_query_nodes_to_protobuf(parsetree_and_error.tree);The PR adds 🟡 Medium — Guard statement-level deparse recursion, not just // src/postgres_deparse.c
11841 static void deparseStmt(DeparseState *state, Node *node)
11842 {
11843 DeparseStateNestingLevel *parent_level = NULL;
11844
11845 // For statements that can be nested, push/pop is handled directly in the
11846 // respective deparse...Stmt methods
11847 bool skip_push_pop = IsA(node, SelectStmt) ||
11848 IsA(node, InsertStmt) ||
11849 IsA(node, UpdateStmt) ||
11850 IsA(node, DeleteStmt) ||
11851 IsA(node, MergeStmt);
11852 The PR adds The change introduces a public API to cap recursion depth and wires PostgreSQL's
Reviewed commit: 64e7eda |
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
There was a problem hiding this comment.
This PR adds configurable stack-depth guards (pg_query_set_stack_limit) and check_stack_depth() calls across libpg_query's parse, fingerprint, deparse, and serialization entry points to harden against stack exhaustion on deeply nested queries.
Key findings
- 🟠 Fingerprint entry points cannot catch new stack-depth error — pg_query_fingerprint.c:343
- 🟠 Protobuf parse path lacks handler for stack-depth throw — pg_query_parse.c:195
- 🟡 Guard statement-level deparse recursion, not just
deparseExpr— postgres_deparse.c:11841 - 🟡 New
check_stack_depthguards are inert; setter undeclared — src_backend_utils_misc_stack_depth.c:151 - 🟡 Serialize-on-parse-error path left outside
PG_TRY— pg_query_parse.c:165
| void | ||
| pg_query_set_stack_limit(size_t bytes) | ||
| { | ||
| char here; | ||
|
|
||
| stack_base_ptr = &here; | ||
| max_stack_depth_bytes = (ssize_t) bytes; | ||
| max_stack_depth = (int) (bytes / 1024); | ||
| } |
There was a problem hiding this comment.
🟡 bug · medium
New check_stack_depth guards are inert; setter undeclared
The PR adds check_stack_depth() to five recursive walkers (_fingerprintNode, _outNode in JSON and protobuf, dump_stmt/dump_expr in plpgsql JSON, deparseExpr), but stack_is_too_deep() returns false whenever stack_base_ptr == NULL (src/postgres/src_backend_utils_misc_stack_depth.c:123-127) and nothing in the library ever sets it. The only setter, pg_query_set_stack_limit (src/postgres/src_backend_utils_misc_stack_depth.c:151-159), is declared in no header and called nowhere, so embedders cannot invoke it portably; and because it writes __thread state, a limit configured on one thread leaves every other worker thread unprotected. The advertised stack-overflow protection is dead code until an undeclared, undocumented, per-thread function is called.
📋 Prompt for AI Agents
In the libpg_query repo: (1) add extern void pg_query_set_stack_limit(size_t bytes); to the public header pg_query.h alongside the other public API declarations; (2) make the stack-depth check active by default — e.g. in pg_query_enter_memory_context (src/pg_query.c), set stack_base_ptr via __builtin_frame_address(0) when it is still NULL so stack_is_too_deep() measures depth on every thread that enters a pg_query entry point, keeping pg_query_set_stack_limit as an override. Without this, the check_stack_depth() calls added in src/pg_query_fingerprint.c, src/pg_query_json_plpgsql.c, src/pg_query_outfuncs_json.c, src/pg_query_outfuncs_protobuf.c, and src/postgres_deparse.c never fire, and the new setter is unusable without a hand-written prototype.
| else | ||
| { | ||
| tree_json = pg_query_nodes_to_json(parsetree_and_error.tree); | ||
| result.parse_tree = strdup(tree_json); | ||
| pfree(tree_json); | ||
| } |
There was a problem hiding this comment.
🟡 bug · medium
Serialize-on-parse-error path left outside PG_TRY
In pg_query_parse_opts the new PG_TRY only covers the result.error == NULL branch; the else branch still calls pg_query_nodes_to_json on the partial tree with no handler (src/pg_query_parse.c:167-169). check_stack_depth() in _outNode (src/pg_query_outfuncs_json.c:293) can throw for a deeply nested partial statement on exactly this error path, and the uncaught throw skips pg_query_exit_memory_context and terminates the process. Restructuring the code so the error branch runs outside the try block creates the gap; the throw can be caught as easily as the success branch.
📋 Prompt for AI Agents
In src/pg_query_parse.c, pg_query_parse_opts lines 138-170: remove the if (result.error == NULL) ... else ... split and run the pg_query_nodes_to_json/strdup/pfree sequence unconditionally inside the existing PG_TRY block, keeping the PG_CATCH that builds a PgQueryError. When a parse error already exists, the partial tree is still serialized, but a check_stack_depth()/ereport throw during serialization is caught and reported instead of longjmping out of the library unguarded.
pg_query_plpgsql_reset_aliases() freed the names but kept the array, and nothing called it after the last compile, so valgrind reported a still-reachable block from pg_query_plpgsql_record_alias at exit. Free the array in reset and reset at the end of pg_query_parse_plpgsql().
|
Folded into #13 |
Summary
check_stack_depth()is compiled into libpg_query but is dead code:stack_base_ptris only set byset_stack_base(), which PostgreSQL calls frommain()and the library never calls. So deep recursion in the tree walkers runs until the C stack overflows — in the WebAssembly build that is aRuntimeError: memory access out of bounds(the Aug-2025 crash in libpg-query-node), and the instance is left corrupt.This PR makes the guard usable and turns those traps into PostgreSQL's normal
stack depth limit exceedederror:pg_query_set_stack_limit(size_t bytes)(insrc_backend_utils_misc_stack_depth.c): setsstack_base_ptrto the caller's frame andmax_stack_depth_bytes/max_stack_depthtobytes. An embedder calls it at every entry with however much stack it actually has left (libpg-query-node:emscripten_stack_get_free() - 512 KiB).check_stack_depth()at the top of the recursive entry points that had none:_outNode(JSON and protobuf),_fingerprintNode,dump_stmt/dump_expr(PL/pgSQL JSON),deparseExpr. Withstack_base_ptr == NULL(embedders that never call the setter) it stays a no-op, so nothing changes for them.pg_query_parse_opts:pg_query_nodes_to_jsonran outside thePG_TRYinpg_query_raw_parse, so anERRORraised while emitting JSON had no handler. It now runs under its ownPG_TRY/PG_CATCHand is returned asresult.error, like a parse error.Verified in libpg-query-node: with a Node stack large enough to exhaust the 32 MiB WASM stack (
ulimit -s unlimited; node --stack-size=100000),SELECT 1+1+…with 250k terms traps on the current build and returnsstack depth limit exceededwith this patch; the next call works.make build,make testpass.Also: free the PL/pgSQL alias list (pre-existing valgrind failure)
The
valgrindCI jobs were already red on unpatched18-constructive(verified on a clean worktree:test/parse_plpgsqlexits 1 with192 bytes … still reachablefrompg_query_plpgsql_record_alias):pg_query_plpgsql_reset_aliases()freed the names but never the array, and nothing called it after the last compile. Earlier fork PRs never ran this workflow, so it went unnoticed. Second commit:resetfrees the array too andpg_query_parse_plpgsql()resets at the end, so nothing outlives the call.make test VALGRIND=1is now clean (in use at exit: 0 bytes).Companion: libpg-query-node PR pganalyze#180 (wrapper calls the setter per export, bumps the pin). Same patch for
17-constructivein #11.Link to Devin session: https://app.devin.ai/sessions/2a9e06519f424ebab14772c74c05dd83
Open in Devin Desktop: https://app.devin.ai/desktop/session/2a9e06519f424ebab14772c74c05dd83?variant=devin
Requested by: @pyramation