Skip to content

Memoize repeated inference walks - #64880

Draft
Jake Bailey (jakebailey) wants to merge 1 commit into
microsoft:mainfrom
jakebailey:perf/inference-walk-memo
Draft

Memoize repeated inference walks#64880
Jake Bailey (jakebailey) wants to merge 1 commit into
microsoft:mainfrom
jakebailey:perf/inference-walk-memo

Conversation

@jakebailey

@jakebailey Jake Bailey (jakebailey) commented Oct 19, 2026

Copy link
Copy Markdown
Member

WIP

@typescript-automation typescript-automation Bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 19, 2026
@jakebailey

Jake Bailey (jakebailey) commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

typescript-automation Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 113,557k (± 0.06%) 113,543k (± 0.04%) ~ 113,454k 113,700k p=0.855 n=12
Memory Allocs 2,212,594 (± 0.01%) 2,216,866 (± 0.00%) +4,272 (+ 0.19%) 2,216,624 2,217,096 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.041s (± 5.67%) 0.039s (± 3.82%) ~ 0.036s 0.043s p=0.347 n=12
Bind Time 0.014s (±10.98%) 0.013s (± 9.68%) ~ 0.011s 0.017s p=0.392 n=12
Check Time 0.468s (± 0.79%) 0.463s (± 0.75%) -0.005s (- 1.09%) 0.456s 0.476s p=0.016 n=12
Emit Time 0.262s (± 1.78%) 0.271s (± 2.65%) +0.009s (+ 3.47%) 0.252s 0.291s p=0.037 n=12
Total Time 0.791s (± 0.95%) 0.793s (± 1.02%) ~ 0.773s 0.821s p=0.744 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 847,423 (± 0.05%) 846,519 (± 0.08%) -904 (- 0.11%) 844,779 847,818 p=0.039 n=12
Types 247,032 (± 0.00%) 247,033 (± 0.00%) ~ 247,030 247,042 p=0.283 n=12
Memory Used 596,390k (± 0.02%) 596,802k (± 0.03%) +412k (+ 0.07%) 596,336k 597,169k p=0.000 n=12
Memory Allocs 13,138,365 (± 0.02%) 13,196,866 (± 0.02%) +58,501 (+ 0.45%) 13,193,130 13,202,238 p=0.000 n=12
Config Time 0.017s (± 1.45%) 0.017s (± 1.99%) ~ 0.016s 0.017s p=0.193 n=12
Parse Time 0.220s (± 2.30%) 0.220s (± 1.91%) ~ 0.212s 0.236s p=0.898 n=12
Bind Time 0.085s (±11.29%) 0.079s (± 9.75%) ~ 0.059s 0.096s p=0.311 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.539s (± 1.75%) 1.531s (± 1.55%) ~ 1.474s 1.611s p=0.504 n=12
Total Time 1.880s (± 1.74%) 1.868s (± 1.61%) ~ 1.800s 1.950s p=0.541 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,757,463 1,757,463 ~ ~ ~ p=1.000 n=12
Types 943,966 943,966 ~ ~ ~ p=1.000 n=12
Memory Used 703,722k (± 0.01%) 703,878k (± 0.01%) +156k (+ 0.02%) 703,775k 704,052k p=0.000 n=12
Memory Allocs 13,291,034 (± 0.01%) 13,515,196 (± 0.02%) +224,162 (+ 1.69%) 13,512,263 13,525,061 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.069s (± 2.98%) 0.071s (± 3.09%) ~ 0.067s 0.077s p=0.557 n=12
Bind Time 0.020s (± 5.29%) 0.020s (± 5.64%) ~ 0.018s 0.023s p=0.906 n=12
Check Time 2.068s (± 0.56%) 2.075s (± 0.43%) ~ 2.054s 2.104s p=0.235 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.201s (± 0.53%) 2.212s (± 0.39%) ~ 2.194s 2.237s p=0.124 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,331,983 ~ ~ ~ p=1.000 n=12
Types 1,370,520 1,370,520 ~ ~ ~ p=1.000 n=12
Memory Used 2,624,615k (± 0.05%) 2,625,673k (± 0.04%) ~ 2,622,931k 2,627,637k p=0.242 n=12
Memory Allocs 27,706,524 (± 0.05%) 27,755,652 (± 0.07%) +49,128 (+ 0.18%) 27,717,138 27,805,863 p=0.000 n=12
Config Time 0.017s (± 1.07%) 0.017s (± 1.70%) ~ 0.016s 0.017s p=0.590 n=12
Parse Time 0.468s (± 2.35%) 0.471s (± 2.04%) ~ 0.439s 0.494s p=0.832 n=12
Bind Time 0.002s 0.002s (±24.21%) ~ 0.002s 0.005s p=1.000 n=12
Check Time 7.248s (± 0.40%) 7.190s (± 0.62%) -0.058s (- 0.80%) 7.138s 7.371s p=0.003 n=12
Emit Time 0.563s (± 9.91%) 0.556s (±12.47%) ~ 0.414s 0.662s p=0.855 n=12
Total Time 9.058s (± 0.32%) 8.986s (± 0.58%) -0.072s (- 0.79%) 8.794s 9.075s p=0.012 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 7,459,653 7,459,653 ~ ~ ~ p=1.000 n=12
Types 2,647,172 2,647,172 ~ ~ ~ p=1.000 n=12
Memory Used 3,803,318k (± 0.03%) 3,805,152k (± 0.02%) +1,833k (+ 0.05%) 3,803,702k 3,806,861k p=0.009 n=12
Memory Allocs 77,159,402 (± 0.09%) 77,407,594 (± 0.08%) +248,192 (+ 0.32%) 77,159,567 77,581,993 p=0.000 n=12
Config Time 0.060s (± 0.41%) 0.060s (± 0.47%) ~ 0.060s 0.061s p=1.000 n=12
Parse Time 1.021s (± 1.79%) 1.001s (± 2.80%) ~ 0.952s 1.104s p=0.062 n=12
Bind Time 0.324s (±13.05%) 0.362s (±10.79%) ~ 0.267s 0.451s p=0.139 n=12
Check Time 14.958s (± 0.78%) 14.787s (± 0.47%) -0.171s (- 1.14%) 14.657s 15.023s p=0.012 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 16.443s (± 0.65%) 16.293s (± 0.48%) -0.150s (- 0.91%) 16.128s 16.594s p=0.017 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 2,568,629 2,568,629 ~ ~ ~ p=1.000 n=12
Types 1,163,035 1,163,035 ~ ~ ~ p=1.000 n=12
Memory Used 1,213,017k (± 0.01%) 1,213,026k (± 0.01%) ~ 1,212,644k 1,213,526k p=0.977 n=12
Memory Allocs 13,980,169 (± 0.13%) 14,004,998 (± 0.13%) ~ 13,955,487 14,076,952 p=0.060 n=12
Config Time 0.017s (± 1.65%) 0.017s ~ ~ ~ p=0.217 n=12
Parse Time 0.349s (± 1.96%) 0.349s (± 2.13%) ~ 0.325s 0.374s p=0.966 n=12
Bind Time 0.082s (±13.78%) 0.081s (±14.36%) ~ 0.070s 0.139s p=0.898 n=12
Check Time 3.626s (± 0.46%) 3.557s (± 0.36%) -0.069s (- 1.90%) 3.520s 3.586s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 4.098s (± 0.59%) 4.026s (± 0.42%) -0.072s (- 1.76%) 3.976s 4.070s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,388,680 ~ ~ ~ p=1.000 n=12
Types 441,862 441,862 ~ ~ ~ p=1.000 n=12
Memory Used 1,312,271k (± 0.64%) 1,310,666k (± 0.54%) ~ 1,291,153k 1,330,310k p=0.977 n=12
Memory Allocs 78,193,067 (± 0.09%) 78,240,399 (± 0.13%) ~ 77,902,734 78,423,805 p=0.198 n=12
Config Time 0.004s (±11.30%) 0.005s (±18.62%) ~ 0.004s 0.009s p=0.169 n=12
Parse Time 0.306s (± 3.17%) 0.311s (± 2.07%) ~ 0.295s 0.329s p=0.541 n=12
Bind Time 0.000s (±167.50%) 0.000s (±217.90%) ~ 0.000s 0.001s p=0.739 n=12
Check Time 2.014s (± 0.65%) 1.996s (± 0.68%) -0.018s (- 0.89%) 1.963s 2.031s p=0.034 n=12
Emit Time 0.354s (± 2.89%) 0.358s (± 3.46%) ~ 0.330s 0.401s p=0.369 n=12
Total Time 23.856s (± 0.54%) 23.801s (± 0.63%) ~ 23.457s 24.115s p=0.590 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 334,953 ~ ~ ~ p=1.000 n=12
Types 197,616 197,616 ~ ~ ~ p=1.000 n=12
Memory Used 247,523k (± 0.03%) 247,642k (± 0.03%) +119k (+ 0.05%) 247,432k 247,795k p=0.028 n=12
Memory Allocs 4,329,877 (± 0.01%) 4,334,872 (± 0.02%) +4,994 (+ 0.12%) 4,333,250 4,336,444 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.115s (± 5.15%) 0.111s (± 4.13%) ~ 0.100s 0.122s p=0.202 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.966s (± 0.74%) 0.956s (± 0.61%) ~ 0.938s 0.967s p=0.066 n=12
Emit Time 0.149s (± 5.90%) 0.144s (± 8.93%) ~ 0.101s 0.164s p=0.680 n=12
Total Time 1.287s (± 1.05%) 1.268s (± 1.11%) -0.019s (- 1.50%) 1.222s 1.307s p=0.034 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 11,588,087 11,588,087 ~ ~ ~ p=1.000 n=12
Types 3,062,556 3,062,556 ~ ~ ~ p=1.000 n=12
Memory Used 3,946,155k (± 0.02%) 3,947,673k (± 0.02%) +1,518k (+ 0.04%) 3,945,875k 3,949,092k p=0.010 n=12
Memory Allocs 41,560,483 (± 0.05%) 42,056,076 (± 0.03%) +495,593 (+ 1.19%) 42,018,726 42,082,165 p=0.000 n=12
Config Time 0.006s 0.006s ~ ~ ~ p=1.000 n=12
Parse Time 0.571s (± 2.71%) 0.552s (± 2.33%) ~ 0.522s 0.582s p=0.066 n=12
Bind Time 0.177s (±13.21%) 0.196s (±15.50%) ~ 0.150s 0.277s p=0.522 n=12
Check Time 18.606s (± 0.19%) 17.640s (± 0.24%) 🟩-0.966s (- 5.19%) 17.499s 17.745s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 19.377s (± 0.14%) 18.413s (± 0.23%) 🟩-0.963s (- 4.97%) 18.238s 18.494s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 303,834 ~ ~ ~ p=1.000 n=12
Types 181,663 181,663 ~ ~ ~ p=1.000 n=12
Memory Used 215,259k (± 0.02%) 215,285k (± 0.03%) ~ 215,135k 215,417k p=0.410 n=12
Memory Allocs 1,401,544 (± 0.01%) 1,407,609 (± 0.01%) +6,065 (+ 0.43%) 1,407,209 1,408,179 p=0.000 n=12
Config Time 0.000s (±217.90%) 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.095s (± 4.23%) 0.095s (± 4.70%) ~ 0.086s 0.108s p=0.831 n=12
Bind Time 0.042s (± 7.76%) 0.043s (± 6.50%) ~ 0.038s 0.052s p=0.721 n=12
Check Time 0.744s (± 0.71%) 0.737s (± 0.79%) ~ 0.724s 0.754s p=0.085 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.896s (± 0.40%) 0.890s (± 0.62%) ~ 0.878s 0.906s p=0.116 n=12
vscode - native
Errors 398 398 ~ ~ ~ p=1.000 n=12
Symbols 12,486,816 12,486,816 ~ ~ ~ p=1.000 n=12
Types 4,082,751 4,082,751 ~ ~ ~ p=1.000 n=12
Memory Used 5,850,758k (± 0.02%) 5,851,575k (± 0.02%) ~ 5,848,899k 5,854,596k p=0.266 n=12
Memory Allocs 59,542,300 (± 0.01%) 59,940,991 (± 0.01%) +398,690 (+ 0.67%) 59,933,544 59,953,057 p=0.000 n=12
Config Time 0.076s (± 0.50%) 0.076s (± 0.41%) ~ 0.075s 0.076s p=0.278 n=12
Parse Time 1.859s (± 2.95%) 1.867s (± 2.48%) ~ 1.750s 1.970s p=0.920 n=12
Bind Time 0.652s (±17.20%) 0.624s (±16.55%) ~ 0.512s 0.912s p=0.899 n=12
Check Time 16.646s (± 1.20%) 14.716s (± 0.84%) 🟩-1.931s (-11.60%) 14.383s 14.892s p=0.000 n=12
Emit Time 5.575s (± 3.90%) 5.601s (± 4.77%) ~ 4.831s 6.520s p=1.000 n=12
Total Time 24.893s (± 0.61%) 22.967s (± 0.97%) 🟩-1.926s (- 7.74%) 22.321s 23.762s p=0.000 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,343,495 1,343,495 ~ ~ ~ p=1.000 n=12
Types 651,491 651,491 ~ ~ ~ p=1.000 n=12
Memory Used 855,319k (± 0.02%) 855,411k (± 0.02%) ~ 854,863k 855,814k p=0.514 n=12
Memory Allocs 6,769,540 (± 0.02%) 6,785,782 (± 0.03%) +16,241 (+ 0.24%) 6,782,520 6,792,218 p=0.000 n=12
Config Time 0.009s 0.009s ~ ~ ~ p=1.000 n=12
Parse Time 0.307s (± 3.92%) 0.310s (± 3.69%) ~ 0.280s 0.332s p=0.640 n=12
Bind Time 0.073s (± 4.05%) 0.075s (± 4.37%) ~ 0.068s 0.088s p=0.190 n=12
Check Time 2.783s (± 0.30%) 2.704s (± 0.33%) -0.080s (- 2.86%) 2.687s 2.742s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.209s (± 0.47%) 3.134s (± 0.36%) -0.075s (- 2.34%) 3.099s 3.153s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,060,810 ~ ~ ~ p=1.000 n=12
Types 389,671 389,671 ~ ~ ~ p=1.000 n=12
Memory Used 471,794k (± 0.02%) 472,713k (± 0.03%) +919k (+ 0.19%) 472,223k 473,173k p=0.000 n=12
Memory Allocs 4,217,940 (± 0.11%) 4,245,999 (± 0.08%) +28,059 (+ 0.67%) 4,235,542 4,257,008 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.106s (± 3.84%) 0.105s (± 2.62%) ~ 0.098s 0.111s p=0.966 n=12
Bind Time 0.035s (±14.42%) 0.035s (±13.13%) ~ 0.026s 0.050s p=0.767 n=12
Check Time 0.911s (± 0.91%) 0.911s (± 0.78%) ~ 0.900s 0.937s p=0.943 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.062s (± 0.94%) 1.062s (± 0.61%) ~ 1.043s 1.081s p=0.701 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

@frivanco97-cmd

frivanco97-cmd commented Oct 20, 2026

Copy link
Copy Markdown

Repositorios..

@jakebailey
Memoize repeated inference walks
ae113ed 
Some inference walks are long and are made again and again with the
same inputs. With Effect Schema, every call such as Schema.optional(x)
inside a Struct({ ... }) infers from the contextual Constraint type to
the combinator's return type, optional<S>, whose members span the large
schema interfaces. Each call repeats that walk from fresh inference
infos and always ends in the same candidates. Effect-heavy code like
t3code spends a large share of its check time this way.

When every inference info is still in its initial state, the outcome of
a walk is now remembered under its source, target, priority,
contravariance and type parameters, as long as the walk computed nothing
new and observed no transient state, so walking it again would take the
same path. Transient state is any read of an in-progress resolution,
signature resolution, variance computation, flow loop or flow type
cache, which are now counted.

Measured with --checkers 4 (n=8):

                      │ before.txt │            after.txt             │
                      │   sec/op   │   sec/op    vs base              │
    Tsc/bluesky         3.069 ± 4%   3.024 ± 1%  -1.47% (p=0.038 n=8)
    Tsc/sentry          12.74 ± 2%   12.62 ± 1%       ~ (p=0.382 n=8)
    Tsc/t3code-server   16.81 ± 3%   16.12 ± 4%  -4.12% (p=0.028 n=8)
    geomean             8.694        8.505       -2.17%

                      │  before.txt  │             after.txt              │
                      │    rss-B     │    rss-B      vs base              │
    Tsc/bluesky         1.422Gi ± 2%   1.413Gi ± 1%       ~ (p=0.959 n=8)
    Tsc/sentry          4.571Gi ± 8%   4.617Gi ± 4%       ~ (p=0.721 n=8)
    Tsc/t3code-server   4.378Gi ± 2%   4.650Gi ± 6%  +6.21% (p=0.038 n=8)
    geomean             3.053Gi        3.119Gi       +2.16%

                      │  before.txt  │             after.txt              │
                      │    heap-B    │    heap-B     vs base              │
    Tsc/bluesky         1.152Gi ± 0%   1.153Gi ± 0%       ~ (p=0.195 n=8)
    Tsc/sentry          3.620Gi ± 0%   3.622Gi ± 0%       ~ (p=0.195 n=8)
    Tsc/t3code-server   3.761Gi ± 0%   3.763Gi ± 0%  +0.04% (p=0.000 n=8)
    geomean             2.504Gi        2.504Gi       +0.03%
@jakebailey

Jake Bailey (jakebailey) commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test

@typescript-automation

typescript-automation Bot commented Oct 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test ✅ Started 👀 Results

@typescript-automation

typescript-automation Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 88,466 88,466 ~ ~ ~ p=1.000 n=12
Types 79,607 79,607 ~ ~ ~ p=1.000 n=12
Memory Used 109,198k (± 0.05%) 109,265k (± 0.07%) ~ 109,105k 109,456k p=0.266 n=12
Memory Allocs 2,196,706 (± 0.01%) 2,201,062 (± 0.01%) +4,357 (+ 0.20%) 2,200,886 2,201,452 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.040s (± 4.74%) 0.039s (± 5.21%) ~ 0.034s 0.046s p=0.364 n=12
Bind Time 0.014s (±11.01%) 0.013s (± 9.54%) ~ 0.011s 0.016s p=0.249 n=12
Check Time 0.477s (± 0.78%) 0.481s (± 0.76%) ~ 0.474s 0.492s p=0.061 n=12
Emit Time 0.270s (± 2.60%) 0.268s (± 2.71%) ~ 0.253s 0.291s p=0.788 n=12
Total Time 0.808s (± 0.88%) 0.808s (± 0.72%) ~ 0.797s 0.827s p=0.744 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 586,565 (± 0.01%) 586,546 (± 0.01%) ~ 586,216 586,727 p=1.000 n=12
Types 241,691 (± 0.00%) 241,692 (± 0.00%) ~ 241,690 241,694 p=0.315 n=12
Memory Used 551,158k (± 0.02%) 551,726k (± 0.02%) +567k (+ 0.10%) 551,449k 552,029k p=0.000 n=12
Memory Allocs 13,072,065 (± 0.03%) 13,129,893 (± 0.03%) +57,828 (+ 0.44%) 13,125,727 13,148,815 p=0.000 n=12
Config Time 0.020s 0.020s (± 1.21%) ~ 0.020s 0.021s p=0.478 n=12
Parse Time 0.280s (± 2.23%) 0.276s (± 3.01%) ~ 0.266s 0.312s p=0.192 n=12
Bind Time 0.110s (±16.54%) 0.117s (±10.19%) ~ 0.077s 0.142s p=0.965 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.815s (± 1.24%) 1.787s (± 1.05%) ~ 1.720s 1.815s p=0.213 n=12
Total Time 2.243s (± 1.14%) 2.220s (± 1.28%) ~ 2.125s 2.267s p=0.173 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 685,943 685,943 ~ ~ ~ p=1.000 n=12
Types 939,490 939,490 ~ ~ ~ p=1.000 n=12
Memory Used 533,204k (± 0.01%) 533,425k (± 0.01%) +221k (+ 0.04%) 533,151k 533,575k p=0.000 n=12
Memory Allocs 12,770,668 (± 0.01%) 12,994,577 (± 0.02%) +223,909 (+ 1.75%) 12,990,975 13,008,940 p=0.000 n=12
Config Time 0.003s 0.003s ~ ~ ~ p=1.000 n=12
Parse Time 0.088s (± 3.15%) 0.087s (± 3.84%) ~ 0.081s 0.098s p=0.503 n=12
Bind Time 0.029s (±13.09%) 0.031s (±15.11%) ~ 0.023s 0.042s p=0.920 n=12
Check Time 2.159s (± 0.68%) 2.164s (± 0.61%) ~ 2.131s 2.211s p=0.702 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.337s (± 0.48%) 2.348s (± 0.42%) ~ 2.325s 2.371s p=0.124 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 3,784,027 3,784,027 ~ ~ ~ p=1.000 n=12
Types 1,355,982 1,355,982 ~ ~ ~ p=1.000 n=12
Memory Used 2,200,760k (± 0.06%) 2,199,958k (± 0.04%) ~ 2,198,332k 2,202,644k p=0.242 n=12
Memory Allocs 26,481,738 (± 0.09%) 26,508,302 (± 0.11%) ~ 26,442,248 26,633,037 p=0.198 n=12
Config Time 0.017s (± 1.95%) 0.017s (± 1.45%) ~ 0.016s 0.017s p=0.371 n=12
Parse Time 0.468s (± 2.05%) 0.483s (± 1.73%) +0.015s (+ 3.24%) 0.456s 0.501s p=0.025 n=12
Bind Time 0.002s (±24.21%) 0.002s (±24.21%) ~ 0.002s 0.005s p=1.000 n=12
Check Time 5.943s (± 0.29%) 5.996s (± 0.25%) +0.053s (+ 0.89%) 5.963s 6.042s p=0.000 n=12
Emit Time 0.475s (± 4.21%) 0.494s (± 8.66%) ~ 0.416s 0.631s p=0.262 n=12
Total Time 7.674s (± 0.66%) 7.738s (± 0.55%) +0.065s (+ 0.84%) 7.658s 7.893s p=0.002 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 4,793,249 4,793,249 ~ ~ ~ p=1.000 n=12
Types 2,607,051 2,607,051 ~ ~ ~ p=1.000 n=12
Memory Used 3,207,841k (± 0.06%) 3,209,825k (± 0.05%) +1,985k (+ 0.06%) 3,207,049k 3,214,760k p=0.039 n=12
Memory Allocs 75,245,186 (± 0.05%) 75,525,499 (± 0.08%) +280,312 (+ 0.37%) 75,405,171 75,690,423 p=0.000 n=12
Config Time 0.061s (± 0.53%) 0.060s (± 0.54%) ~ 0.060s 0.061s p=1.000 n=12
Parse Time 1.009s (± 1.97%) 1.007s (± 1.77%) ~ 0.968s 1.069s p=0.944 n=12
Bind Time 0.333s (±13.97%) 0.321s (±10.47%) ~ 0.262s 0.415s p=0.452 n=12
Check Time 13.706s (± 0.52%) 13.752s (± 0.48%) ~ 13.622s 13.909s p=0.183 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 15.192s (± 0.50%) 15.222s (± 0.37%) ~ 15.071s 15.343s p=0.378 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,883,075 1,883,075 ~ ~ ~ p=1.000 n=12
Types 1,158,171 1,158,171 ~ ~ ~ p=1.000 n=12
Memory Used 1,057,456k (± 0.01%) 1,057,440k (± 0.02%) ~ 1,056,421k 1,057,813k p=0.551 n=12
Memory Allocs 13,608,281 (± 0.48%) 13,580,634 (± 0.13%) ~ 13,544,735 13,629,181 p=0.977 n=12
Config Time 0.014s 0.014s ~ ~ ~ p=1.000 n=12
Parse Time 0.286s (± 2.33%) 0.284s (± 2.40%) ~ 0.261s 0.298s p=0.921 n=12
Bind Time 0.067s (±12.40%) 0.068s (±17.43%) ~ 0.059s 0.114s p=0.287 n=12
Check Time 2.770s (± 0.78%) 2.733s (± 0.81%) -0.037s (- 1.33%) 2.670s 2.780s p=0.034 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.156s (± 0.58%) 3.118s (± 0.53%) -0.038s (- 1.19%) 3.080s 3.163s p=0.012 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,184,976 1,184,976 ~ ~ ~ p=1.000 n=12
Types 438,967 438,967 ~ ~ ~ p=1.000 n=12
Memory Used 1,275,208k (± 0.48%) 1,279,390k (± 0.58%) ~ 1,267,234k 1,302,163k p=0.410 n=12
Memory Allocs 77,388,286 (± 0.06%) 77,537,354 (± 0.11%) +149,068 (+ 0.19%) 77,301,634 77,735,491 p=0.002 n=12
Config Time 0.004s (±11.15%) 0.004s (± 7.33%) ~ 0.004s 0.005s p=0.574 n=12
Parse Time 0.308s (± 1.44%) 0.304s (± 2.68%) ~ 0.278s 0.321s p=0.599 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.954s (± 0.48%) 1.950s (± 0.58%) ~ 1.927s 1.981s p=0.514 n=12
Emit Time 0.372s (± 4.01%) 0.370s (± 4.50%) ~ 0.330s 0.425s p=0.943 n=12
Total Time 23.584s (± 0.45%) 23.531s (± 0.63%) ~ 23.172s 23.849s p=0.630 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 263,513 263,513 ~ ~ ~ p=1.000 n=12
Types 196,118 196,118 ~ ~ ~ p=1.000 n=12
Memory Used 236,080k (± 0.02%) 236,198k (± 0.03%) +118k (+ 0.05%) 235,970k 236,437k p=0.005 n=12
Memory Allocs 4,287,073 (± 0.01%) 4,292,922 (± 0.01%) +5,849 (+ 0.14%) 4,291,446 4,294,082 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.112s (± 4.82%) 0.110s (± 4.71%) ~ 0.099s 0.122s p=0.541 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.943s (± 0.53%) 0.942s (± 0.55%) ~ 0.928s 0.954s p=0.876 n=12
Emit Time 0.143s (± 6.02%) 0.144s (± 5.78%) ~ 0.121s 0.167s p=0.989 n=12
Total Time 1.254s (± 0.60%) 1.248s (± 1.01%) ~ 1.215s 1.289s p=0.284 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 4,350,705 4,350,705 ~ ~ ~ p=1.000 n=12
Types 3,040,691 3,040,691 ~ ~ ~ p=1.000 n=12
Memory Used 2,610,598k (± 0.01%) 2,611,468k (± 0.03%) ~ 2,610,080k 2,614,267k p=0.078 n=12
Memory Allocs 37,328,981 (± 0.04%) 37,779,821 (± 0.06%) +450,840 (+ 1.21%) 37,727,665 37,842,839 p=0.000 n=12
Config Time 0.006s 0.006s ~ ~ ~ p=1.000 n=12
Parse Time 0.568s (± 2.25%) 0.566s (± 3.51%) ~ 0.509s 0.622s p=0.876 n=12
Bind Time 0.175s (±11.28%) 0.187s (±16.04%) ~ 0.148s 0.278s p=0.921 n=12
Check Time 16.465s (± 0.27%) 15.454s (± 0.33%) 🟩-1.010s (- 6.14%) 15.310s 15.579s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 17.232s (± 0.24%) 16.230s (± 0.33%) 🟩-1.002s (- 5.81%) 16.101s 16.337s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 231,617 231,617 ~ ~ ~ p=1.000 n=12
Types 181,196 181,196 ~ ~ ~ p=1.000 n=12
Memory Used 203,541k (± 0.01%) 203,546k (± 0.02%) ~ 203,446k 203,665k p=0.854 n=12
Memory Allocs 1,358,518 (± 0.02%) 1,364,362 (± 0.02%) +5,844 (+ 0.43%) 1,363,869 1,365,085 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.097s (± 4.55%) 0.097s (± 4.30%) ~ 0.088s 0.109s p=0.966 n=12
Bind Time 0.041s (±11.95%) 0.043s (±10.24%) ~ 0.031s 0.054s p=0.417 n=12
Check Time 0.723s (± 0.98%) 0.723s (± 0.87%) ~ 0.709s 0.743s p=0.639 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.875s (± 0.77%) 0.877s (± 0.65%) ~ 0.865s 0.892s p=0.560 n=12
vscode - native
Errors 10 10 ~ ~ ~ p=1.000 n=12
Symbols 7,175,632 7,175,632 ~ ~ ~ p=1.000 n=12
Types 4,223,910 4,223,910 ~ ~ ~ p=1.000 n=12
Memory Used 5,355,247k (± 0.02%) 5,356,128k (± 0.02%) ~ 5,353,801k 5,359,375k p=0.291 n=12
Memory Allocs 61,912,166 (± 0.01%) 62,348,332 (± 0.02%) +436,166 (+ 0.70%) 62,330,506 62,387,984 p=0.000 n=12
Config Time 0.079s (± 0.39%) 0.078s (± 0.31%) -0.001s (- 0.64%) 0.078s 0.079s p=0.036 n=12
Parse Time 1.966s (± 2.68%) 1.949s (± 1.71%) ~ 1.867s 2.030s p=0.347 n=12
Bind Time 0.670s (±15.83%) 0.685s (±12.41%) ~ 0.557s 0.908s p=0.640 n=12
Check Time 17.002s (± 0.70%) 14.482s (± 0.75%) 🟩-2.520s (-14.82%) 14.238s 14.655s p=0.000 n=12
Emit Time 5.712s (± 7.73%) 5.660s (± 9.13%) ~ 4.900s 6.803s p=0.630 n=12
Total Time 25.511s (± 1.63%) 22.944s (± 2.03%) 🟩-2.567s (-10.06%) 22.251s 24.019s p=0.000 n=12
webpack - native
Errors 602 602 ~ ~ ~ p=1.000 n=12
Symbols 910,191 910,191 ~ ~ ~ p=1.000 n=12
Types 629,514 629,514 ~ ~ ~ p=1.000 n=12
Memory Used 745,147k (± 0.02%) 745,120k (± 0.02%) ~ 744,778k 745,588k p=0.799 n=12
Memory Allocs 6,441,827 (± 0.02%) 6,458,705 (± 0.02%) +16,878 (+ 0.26%) 6,456,283 6,462,561 p=0.000 n=12
Config Time 0.008s (± 2.29%) 0.008s (± 2.29%) ~ 0.007s 0.008s p=1.000 n=12
Parse Time 0.259s (± 2.28%) 0.248s (± 3.52%) ~ 0.230s 0.273s p=0.066 n=12
Bind Time 0.062s (± 8.19%) 0.061s (± 8.02%) ~ 0.053s 0.080s p=0.557 n=12
Check Time 2.235s (± 0.35%) 2.203s (± 0.38%) -0.031s (- 1.39%) 2.171s 2.222s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.595s (± 0.35%) 2.551s (± 0.45%) -0.044s (- 1.70%) 2.519s 2.576s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 640,494 640,494 ~ ~ ~ p=1.000 n=12
Types 387,992 387,992 ~ ~ ~ p=1.000 n=12
Memory Used 398,453k (± 0.02%) 399,328k (± 0.02%) +875k (+ 0.22%) 399,131k 399,499k p=0.000 n=12
Memory Allocs 3,990,902 (± 0.08%) 4,018,956 (± 0.08%) +28,054 (+ 0.70%) 4,009,485 4,024,548 p=0.000 n=12
Config Time 0.003s 0.003s ~ ~ ~ p=1.000 n=12
Parse Time 0.134s (± 1.66%) 0.137s (± 3.09%) ~ 0.126s 0.145s p=0.202 n=12
Bind Time 0.046s (±19.52%) 0.038s (±10.41%) ~ 0.033s 0.051s p=0.242 n=12
Check Time 0.949s (± 0.57%) 0.946s (± 0.49%) ~ 0.937s 0.958s p=0.417 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.138s (± 0.91%) 1.131s (± 0.58%) ~ 1.115s 1.151s p=0.224 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

lsp

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
CompilerLSP - native
Req 1 - updateOpen 14ms (±10.14%) 14ms (±13.76%) ~ 12ms 20ms p=0.956 n=12
Req 2 - geterr 343ms (± 0.92%) 345ms (± 0.81%) ~ 340ms 352ms p=0.297 n=12
Req 3 - references 59ms (± 3.35%) 60ms (± 2.42%) ~ 55ms 64ms p=0.086 n=12
Req 4 - navto 21ms (± 3.37%) 21ms (± 4.36%) ~ 20ms 25ms p=0.633 n=12
Req 5 - completionInfo count 1,525 1,525 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 23ms (±10.23%) 23ms (±11.56%) ~ 18ms 28ms p=0.850 n=12
Compiler-UnionsLSP - native
Req 1 - updateOpen 14ms (±11.92%) 13ms (±12.50%) ~ 11ms 17ms p=0.848 n=12
Req 2 - geterr 669ms (± 0.41%) 673ms (± 0.30%) +5ms (+ 0.70%) 669ms 679ms p=0.004 n=12
Req 3 - references 52ms (± 5.46%) 53ms (± 4.03%) ~ 50ms 61ms p=0.555 n=12
Req 4 - navto 19ms (± 1.28%) 19ms (± 2.53%) ~ 18ms 21ms p=0.568 n=12
Req 5 - completionInfo count 1,363 1,363 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 19ms (± 4.62%) 20ms (± 6.59%) ~ 17ms 25ms p=0.433 n=12
xstate-main-1-LSP - native
⚠️ Failed failed failed
System info unknown
Hosts
  • native
Scenarios
  • CompilerLSP - native
  • Compiler-UnionsLSP - native
  • xstate-main-1-LSP - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
lsp-startup - native
⚠️ Failed failed failed
tsgo-startup - native
⚠️ Failed failed failed
System info unknown
Hosts
  • native
Scenarios
  • lsp-startup - native
  • tsgo-startup - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

@jakebailey Jake Bailey (jakebailey)

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jakebailey @frivanco97-cmd