Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions .changeset/eql-3-1-0-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,16 @@
'@cipherstash/stack-prisma': minor
---

Move the bundled EQL v3 migrations to **eql-3.1.0**. EQL 3.1.0 is a minor
release for the Rust `eql-bindings` crate. Its install SQL changes only the
version stamp: `eql_v3.version()` returns `'3.1.0'` and the `eql_v3` schema
comment reads `'3.1.0'`. No function, operator, domain or index behaviour
changes.
Move the bundled EQL v3 migrations to **eql-3.1.0**. Its install SQL changes
the version stamp (`eql_v3.version()` returns `'3.1.0'` and the `eql_v3`
schema comment reads `'3.1.0'`) and the ORE block comparator behind every
ordered ORE operator and the ORE operator class. The comparator now runs a
constant number of operations whatever two values share, closing a timing
leak. Ordering results are unchanged for well-formed values, but a block-ORE
text value with a malformed or mismatched-width term now raises on every
comparison against it. Check your `_ore` text columns before migrating; see
[U-001](../../../../packages/eql/docs/upgrading/v3.1.md#u-001-block-ore-text-comparisons-check-every-term).
No domain, wire format or index term changes.

Two artefacts carry the new bundle:

Expand Down Expand Up @@ -52,4 +57,6 @@ baseline. The team chose it on these numbers: `@cipherstash/stack-prisma` had
481 npm downloads in the 30 days to 3 October 2026, and 384 in the last 7 of
those days, 305 of them on the 1.2.0 release day. The append-only alternative,
a second `from: null` genesis edge, would add another permanent ~2.6 MB copy of
the bundle for a release that changes no SQL behaviour.
the bundle. That choice was made when 3.1.0 changed only the version stamp; the
comparator change came after it and does not alter the trade, since every
database reaches the same bundle either way.
21 changes: 21 additions & 0 deletions .changeset/eql-ore-block-256-constant-op-count.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
'@cipherstash/eql': minor
---

Security: the ORE block comparator (`eql_v3_internal.compare_ore_block_256_term`
and `compare_ore_block_256_terms`, behind every ordered ORE operator and the ORE
operator class) now runs a constant number of operations for operands of the
same shape (term count, term width, NULL positions), whether and wherever they
differ. Previously its run time depended on where two values first differ, so a
party able to time queries could learn how long a prefix their query value
shares with stored values. For text the effect was large: a comparison took
about 11 µs when the first term differs and 47 µs when only the sixth does.
Results are unchanged. Single-term comparisons cost about 6% more when the
terms differ and about 36% more (roughly 1 µs) when they are equal, since equal
terms now run the hash too; text comparisons now take about 18 µs whatever the
operands. Multi-term (text) values now check every paired term, so a
malformed or mismatched-width term after the first difference, or one paired
with a NULL element, raises instead of being skipped; see
[U-001](../../docs/upgrading/v3.1.md#u-001-block-ore-text-comparisons-check-every-term)
for how to find such rows before upgrading. A timing harness is in
`packages/eql/tests/timing/ore_block_256/`.
8 changes: 5 additions & 3 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -248,9 +248,10 @@ updates:

# ── Cargo (the root workspace — the stack-* crates) ─────────────
# The root workspace (the six stack-* crates and the three node binding
# crates) and the five detached workspaces beside it: the three cargo-fuzz
# crates and the two Go WASI guests. Each has its own Cargo.lock, and each
# is its own workspace root, so each is listed.
# crates) and the six detached workspaces beside it: the three cargo-fuzz
# crates, the two Go WASI guests and the EQL ORE timing harness's ciphertext
# generator. Each has its own Cargo.lock, and each is its own workspace
# root, so each is listed.
- package-ecosystem: cargo
directories:
- /
Expand All @@ -259,6 +260,7 @@ updates:
- /packages/stack-encrypt/fuzz
- /languages/golang/encrypt/guest
- /languages/golang/auth/guest
- /packages/eql/tests/timing/ore_block_256/ctgen
# Lockfile only, like the other two cargo entries. Every one of these
# workspaces shares crates by path with another: the fuzz crates and both
# guests depend on root-workspace crates, the encrypt guest also on
Expand Down
2 changes: 2 additions & 0 deletions docs/plans/2026-08-13-eql-monorepo-absorption.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,8 @@ That re-emit is the part worth recording, because the append-only rule says not

**Re-argued at 3.1.0, and re-emitted again — this time over a published baseline.** EQL 3.1.0 is a minor release for the Rust `eql-bindings` crate; its SQL changes only the version stamp, but the digest moves (`9b6dab78…` → `af316179…`), and the Version Packages PR (`cipherstash/stack#1054`) failed the lockstep test. Unlike 3.0.6, the baseline this rewrites is published: `@cipherstash/stack-prisma` 1.2.0 and 1.2.1 (2 October 2026) ship `2fdc7caf…`. The numbers: 481 npm downloads in the 30 days to 3 October 2026, 305 of them on the 1.2.0 release day — above the ~253/month that the 3.0.5 re-emit used, though the release-day spike is likely mostly automated. A new `20261005T0000_upgrade_eql_v3_3_1_0` edge carries existing databases; a 1.2.x database walks only it, and a 1.0.0 or 1.1.x database walks the 3.0.5, 3.0.6 and 3.1.0 edges. The append-only rule, and the second genesis edge once adoption is real, still stand for the next bump.

**Then 3.1.0 grew a SQL change before it shipped.** The constant-operation ORE block comparator landed after the bake, so the 3.1.0 bundle moved again (`af316179…` → `c9404de3…`) while the Version Packages PR was still open. Both 3.1.0 directories were re-emitted from that bundle, keeping their `createdAt`, and the changeset now names the comparator change instead of claiming a stamp-only release. Nothing had published, so this is a re-pin, not a third break. The lesson: a bake made ahead of a Version Packages PR assumes nothing else changes the bundle before it merges, and nothing checks that assumption except the lockstep test on the release PR itself.

The narrower lesson for the absorption: **absorbing a repo swaps "last published release" for "current HEAD" across every consumer**, and nothing in the dependency graph announces that. Only a digest pin caught it here. Any other package this repo absorbs by workspace link deserves the same check — diff the built artefact against the published tarball — before the link is treated as neutral.

### Three imported release workflows violate the no-caching rule — by omission, not by setting
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@
"cipherstash:upgrade-eql-v3-bundle-3.1.0-v1"
],
"createdAt": "2026-07-14T20:10:24.325Z",
"migrationHash": "f92f5be5239f3388239dfacb006420452003928ce9dc589a5a934486ada7ba22"
"migrationHash": "cd203837545c166b28762675b69591310ef35c4ad3783660b1bd7277e40d075c"
}

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
"cipherstash:upgrade-eql-v3-bundle-3.1.0-v1"
],
"createdAt": "2026-10-05T06:28:35.158Z",
"migrationHash": "73432f18671f840b97522ca8aafb3a3f021d91b38b1d05981a0efb71c13657a8"
"migrationHash": "dbf05d613aefeee865fe56b4eec40e9fc29a5261c9b435971d76fa39621fff13"
}

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@
"cipherstash:upgrade-eql-v3-bundle-3.1.0-v1"
],
"createdAt": "2026-07-14T20:10:24.325Z",
"migrationHash": "f92f5be5239f3388239dfacb006420452003928ce9dc589a5a934486ada7ba22"
"migrationHash": "cd203837545c166b28762675b69591310ef35c4ad3783660b1bd7277e40d075c"
}
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,14 @@
* stack-prisma had 481 npm downloads in the 30 days to 3 October 2026
* (384 in the last week, 305 on the 1.2.0 release day), against the
* ~253/month at which (2) was judged small and knowable. The team
* chose the re-emit on those numbers. 3.1.0 is a Rust-only minor
* release whose SQL changes only the version stamp, so a second
* genesis edge would again add a permanent ~2.6 MB bundle copy for no
* change in behaviour. A 1.2.x database walks the 3.1.0 self-edge
* only; a 1.0.0 / 1.1.x one walks 3.0.5, 3.0.6 and 3.1.0.
* chose the re-emit on those numbers, when 3.1.0's SQL changed only
* the version stamp, so a second genesis edge would again have added
* a permanent ~2.6 MB bundle copy for no change in behaviour. Before
* 3.1.0 shipped, its SQL also took the constant-operation ORE block
* comparator, and both 3.1.0 directories were re-emitted unpublished
* to bake it (`f92f5be5…` → the current hash). A 1.2.x database walks
* the 3.1.0 self-edge only; a 1.0.0 / 1.1.x one walks 3.0.5, 3.0.6
* and 3.1.0.
*
* Every re-emit changed the `migrationHash` and the baked
* `installSqlSha256`. Consumers must delete `migrations/cipherstash/` and
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
"cipherstash:upgrade-eql-v3-bundle-3.1.0-v1"
],
"createdAt": "2026-10-05T06:28:35.158Z",
"migrationHash": "73432f18671f840b97522ca8aafb3a3f021d91b38b1d05981a0efb71c13657a8"
"migrationHash": "dbf05d613aefeee865fe56b4eec40e9fc29a5261c9b435971d76fa39621fff13"
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,13 @@
/**
* Upgrade an existing EQL v3 installation to the pinned 3.1.0 bundle.
*
* 3.1.0 is a minor release for the Rust `eql-bindings` crate. Its install
* SQL changes only the version stamp (`eql_v3.version()` and the `eql_v3`
* schema comment), but the bundle digest still moves, so it is still a new
* bundle to install.
* 3.1.0's install SQL changes the version stamp (`eql_v3.version()` and
* the `eql_v3` schema comment) and the ORE block comparator, which now runs
* a constant number of operations whatever two values share. Ordering
* results are unchanged for well-formed values; a block-ORE text value with
* a malformed term now raises on every comparison (EQL's upgrade note
* U-001). This edge was first emitted when only the stamp changed, and
* re-emitted before publication to bake the comparator.
*
* A database that already recorded the earlier upgrade invariants must
* still traverse this edge, or it stays on an older EQL surface. Fresh
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -150,24 +150,28 @@ const PUBLISHED_MIGRATIONS = [
// baseline, `2fdc7caf…` → this hash. Taken in preference to a
// second `from: null` genesis edge duplicating the bundle SQL
// forever, while adoption is still small.
// Re-pinned once more before 3.1.0 published (`f92f5be5…` /
// `af316179…` → these), when 3.1.0's SQL took the
// constant-operation ORE block comparator; the 3.1.0 edge below
// moved with it (`73432f18…`).
// The baseline is the ONE migration whose baked digest tracks the
// pinned release rather than a historical one, which is why the
// lockstep test below has something to match.
createdAt: '2026-07-14T20:10:24.325Z',
migrationHash:
'f92f5be5239f3388239dfacb006420452003928ce9dc589a5a934486ada7ba22',
'cd203837545c166b28762675b69591310ef35c4ad3783660b1bd7277e40d075c',
installSqlSha256:
'af316179aba627347144f3d93e7e3410b46162e4af4332656178b6f9142f0745',
'c9404de3dccd6702c7addf9fa748402e7de2ac11fb2b0f6a751d7ec3813d870a',
},
{
dirName: CIPHERSTASH_V3_310_UPGRADE_MIGRATION_NAME,
metadata: v3Upgrade310Metadata,
ops: v3Upgrade310Ops,
createdAt: '2026-10-05T06:28:35.158Z',
migrationHash:
'73432f18671f840b97522ca8aafb3a3f021d91b38b1d05981a0efb71c13657a8',
'dbf05d613aefeee865fe56b4eec40e9fc29a5261c9b435971d76fa39621fff13',
installSqlSha256:
'af316179aba627347144f3d93e7e3410b46162e4af4332656178b6f9142f0745',
'c9404de3dccd6702c7addf9fa748402e7de2ac11fb2b0f6a751d7ec3813d870a',
},
{
dirName: CIPHERSTASH_V3_306_UPGRADE_MIGRATION_NAME,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ const STALE_1_2_BASELINE_HASH =
* only to show the fixtures are NOT the current artefact.
*/
const SHIPPED_BASELINE_HASH =
'f92f5be5239f3388239dfacb006420452003928ce9dc589a5a934486ada7ba22'
'cd203837545c166b28762675b69591310ef35c4ad3783660b1bd7277e40d075c'

/**
* What `@cipherstash/[email protected]` stored for the same directory, under
Expand Down
2 changes: 2 additions & 0 deletions packages/eql/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

91 changes: 91 additions & 0 deletions packages/eql/docs/upgrading/v3.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Upgrading to EQL 3.1

This guide covers behaviour in 3.1 that callers should know about. Nothing in
it requires you to change any SQL.

## TL;DR

1. Ordered comparisons of block-ORE text values now check every paired term,
so a malformed term anywhere in a stored value raises, where before it
raised only if the comparison reached it. Check your `_ore` text columns
before upgrading —
[U-001](#u-001-block-ore-text-comparisons-check-every-term).

## Compatibility

| Surface | Status |
| --- | --- |
| Ordering results of every block-ORE operator and the ORE operator class | **Unchanged** for well-formed values. |
| Block-ORE text values (`public.eql_v3_text_ord_ore`, `public.eql_v3_text_search_ore`) with a malformed or mismatched-width term | **Changed.** Every comparison against the value raises — see U-001. |
| Wire format, domains, index terms | **Unchanged.** No re-encryption, no data migration. |

## Upgrade notes

### U-001: Block-ORE text comparisons check every term

**What changed.** A block-ORE text value carries up to six ORE terms. The
comparator behind its ordered operators (`<`, `<=`, `>`, `>=`), `ORDER BY` and
the ORE operator class used to compare term by term and stop at the first
unequal pair. That made its run time reveal how long a prefix two values
share, so it now compares every paired term in a constant number of
operations. A consequence is that every paired term is now checked. A term
that is malformed (its length is not 49 × N + 16 bytes for some N ≥ 1) or
whose width differs from the term it is paired with raises:

```
ERROR: Malformed ORE term: 407 bytes
ERROR: Ciphertexts are different lengths
```

Before 3.1 the comparator raised only when it reached the bad term, which it
did not when an earlier term already settled the order, or when the bad term
was paired with a NULL element.

**Who is affected.** Only rows whose `ob` terms were not produced by a
CipherStash client: hand-edited payloads, truncated values, or payloads from
another encoder. Values encrypted by CipherStash clients are well formed, and
single-term domains (every block-ORE domain but text) are unaffected.

**Why it matters.** One bad row can now fail a whole query that orders or
range-scans the column, and `CREATE INDEX` or `REINDEX` on it, instead of
being skipped silently.

**What to do.** Before upgrading, list the term widths in each block-ORE text
column. Every term in a column should have the same width:

```sql
SELECT octet_length(term.bytes) AS bytes,
(octet_length(term.bytes) > 16
AND (octet_length(term.bytes) - 16) % 49 = 0) AS well_formed,
count(*) AS terms
FROM my_table t,
unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term
WHERE term.bytes IS NOT NULL
GROUP BY 1, 2
ORDER BY 3 DESC;
```

One row, with `well_formed` true, means the column is clean. For any other
width, find the rows that carry it and re-encrypt them:

```sql
SELECT t.ctid
FROM my_table t
WHERE EXISTS (
SELECT 1
FROM unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term
WHERE octet_length(term.bytes) <> 408 -- the width the first query reports
);
```

## Rollback

Installing the 3.0 bundle restores the term-by-term comparator. Applying any
EQL bundle drops and recreates its schemas, so re-apply grants and dependent
objects afterwards — see
[U-002 in the 3.0.5 guide](v3.0.5.md#u-002-grants-and-dependent-objects-do-not-survive-an-install).

## See also

- [`tests/timing/ore_block_256/`](../../tests/timing/ore_block_256/README.md):
the timing harness behind U-001, and its results.
Loading
Loading