Skip to content

Experiment short-circuiting of union distribution #50640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

Zzzen
Copy link
Contributor

@Zzzen Zzzen commented Sep 5, 2022

Fixes #47511

Tried the following code on my machine, the check time is reduced from 24s to 6s. I'd like to see how it works for RWC.

type HugeUnion1 = "a" | "q" | ... [10,000 more];
type HugeUnion2 = "a" | ... [10,000 more which do not include 'q'];

type Square<T, U> = T extends U ? true : false;

type M = Square<HugeUnion1, HugeUnion2>;

@typescript-bot typescript-bot added the For Backlog Bug PRs that fix a backlog bug label Sep 5, 2022
@jakebailey
Copy link
Member

@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot perf test this faster
@typescript-bot test top100

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 9, 2022

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at cd31843. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 9, 2022

Heya @jakebailey, I've started to run the extended test suite on this PR at cd31843. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 9, 2022

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at cd31843. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 9, 2022

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at cd31843. You can monitor the build here.

Update: The results are in!

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 9, 2022

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at cd31843. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..50640

Metric main 50640 Delta Best Worst
Angular - node (v14.15.1, x64)
Memory used 338,844k (± 0.00%) 338,845k (± 0.01%) +1k (+ 0.00%) 338,744k 338,888k
Parse Time 2.07s (± 0.89%) 2.06s (± 0.24%) -0.00s (- 0.19%) 2.05s 2.07s
Bind Time 0.80s (± 0.46%) 0.79s (± 0.66%) -0.01s (- 0.63%) 0.78s 0.80s
Check Time 5.82s (± 0.42%) 5.81s (± 0.32%) -0.01s (- 0.24%) 5.78s 5.88s
Emit Time 6.13s (± 0.74%) 6.14s (± 0.49%) +0.00s (+ 0.05%) 6.08s 6.22s
Total Time 14.82s (± 0.41%) 14.80s (± 0.32%) -0.02s (- 0.15%) 14.73s 14.97s
Compiler-Unions - node (v14.15.1, x64)
Memory used 192,644k (± 0.02%) 192,842k (± 0.25%) +198k (+ 0.10%) 192,578k 194,784k
Parse Time 0.85s (± 0.72%) 0.85s (± 0.76%) +0.00s (+ 0.12%) 0.84s 0.87s
Bind Time 0.48s (± 0.70%) 0.48s (± 0.00%) -0.00s (- 0.62%) 0.48s 0.48s
Check Time 6.71s (± 0.50%) 6.71s (± 0.66%) +0.00s (+ 0.03%) 6.64s 6.83s
Emit Time 2.41s (± 0.71%) 2.43s (± 1.32%) +0.02s (+ 0.66%) 2.38s 2.54s
Total Time 10.46s (± 0.28%) 10.47s (± 0.59%) +0.02s (+ 0.15%) 10.38s 10.64s
Monaco - node (v14.15.1, x64)
Memory used 326,531k (± 0.01%) 326,522k (± 0.00%) -9k (- 0.00%) 326,499k 326,535k
Parse Time 1.57s (± 0.79%) 1.56s (± 0.31%) -0.01s (- 0.38%) 1.55s 1.57s
Bind Time 0.73s (± 0.65%) 0.72s (± 1.05%) -0.00s (- 0.28%) 0.71s 0.75s
Check Time 5.71s (± 0.54%) 5.71s (± 0.53%) +0.00s (+ 0.07%) 5.64s 5.77s
Emit Time 3.33s (± 0.63%) 3.31s (± 0.60%) -0.01s (- 0.39%) 3.26s 3.35s
Total Time 11.33s (± 0.33%) 11.31s (± 0.34%) -0.02s (- 0.16%) 11.22s 11.38s
TFS - node (v14.15.1, x64)
Memory used 289,654k (± 0.01%) 289,648k (± 0.01%) -7k (- 0.00%) 289,596k 289,713k
Parse Time 1.30s (± 0.64%) 1.30s (± 1.42%) +0.01s (+ 0.62%) 1.28s 1.37s
Bind Time 0.79s (± 1.01%) 0.79s (± 1.65%) -0.01s (- 0.76%) 0.74s 0.81s
Check Time 5.35s (± 0.23%) 5.35s (± 0.41%) -0.00s (- 0.02%) 5.30s 5.40s
Emit Time 3.57s (± 0.73%) 3.56s (± 0.79%) -0.01s (- 0.28%) 3.49s 3.62s
Total Time 11.00s (± 0.30%) 11.00s (± 0.24%) -0.01s (- 0.05%) 10.94s 11.04s
material-ui - node (v14.15.1, x64)
Memory used 436,462k (± 0.01%) 436,556k (± 0.01%) +94k (+ 0.02%) 436,494k 436,604k
Parse Time 1.85s (± 0.37%) 1.86s (± 0.56%) +0.01s (+ 0.43%) 1.83s 1.88s
Bind Time 0.58s (± 0.57%) 0.58s (± 0.00%) 0.00s ( 0.00%) 0.58s 0.58s
Check Time 12.85s (± 0.49%) 12.86s (± 0.56%) +0.01s (+ 0.08%) 12.74s 13.03s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.29s (± 0.40%) 15.31s (± 0.47%) +0.02s (+ 0.14%) 15.20s 15.47s
xstate - node (v14.15.1, x64)
Memory used 546,728k (± 0.00%) 546,768k (± 0.00%) +41k (+ 0.01%) 546,746k 546,791k
Parse Time 2.59s (± 0.45%) 2.59s (± 0.36%) +0.00s (+ 0.15%) 2.57s 2.61s
Bind Time 0.96s (± 0.38%) 0.97s (± 1.71%) +0.01s (+ 1.04%) 0.96s 1.03s
Check Time 1.51s (± 0.40%) 1.52s (± 0.43%) +0.01s (+ 0.46%) 1.51s 1.54s
Emit Time 0.07s (± 4.13%) 0.07s (± 3.14%) -0.00s (- 1.39%) 0.07s 0.08s
Total Time 5.14s (± 0.22%) 5.16s (± 0.42%) +0.02s (+ 0.33%) 5.12s 5.23s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50640 10
Baseline main 10

Developer Information:

Download Benchmark

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/50640/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/50640/merge:

Everything looks good!

@jakebailey
Copy link
Member

I feel like something's wrong with RWC; if doesn't reply?

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..50640

Metric main 50640 Delta Best Worst
Angular - node (v14.15.1, x64)
Memory used 338,937k (± 0.01%) 338,940k (± 0.01%) +2k (+ 0.00%) 338,885k 339,018k
Parse Time 2.06s (± 0.61%) 2.05s (± 0.33%) -0.00s (- 0.24%) 2.04s 2.07s
Bind Time 0.80s (± 1.03%) 0.80s (± 0.47%) -0.00s (- 0.25%) 0.79s 0.80s
Check Time 5.83s (± 0.42%) 5.84s (± 0.31%) +0.01s (+ 0.14%) 5.80s 5.88s
Emit Time 6.19s (± 0.43%) 6.17s (± 0.75%) -0.02s (- 0.26%) 6.04s 6.26s
Total Time 14.88s (± 0.27%) 14.86s (± 0.39%) -0.01s (- 0.08%) 14.70s 14.96s
Compiler-Unions - node (v14.15.1, x64)
Memory used 190,140k (± 0.01%) 192,402k (± 1.12%) +2,262k (+ 1.19%) 189,377k 195,989k
Parse Time 0.85s (± 0.40%) 0.90s (± 6.92%) +0.05s (+ 5.86%) 0.84s 1.13s
Bind Time 0.48s (± 0.77%) 0.53s (±10.02%) +0.04s (+ 8.87%) 0.48s 0.73s
Check Time 6.76s (± 0.70%) 7.19s (± 6.00%) +0.43s (+ 6.38%) 6.64s 8.28s
Emit Time 2.38s (± 0.92%) 2.52s (± 4.92%) +0.14s (+ 6.01%) 2.37s 2.94s
Total Time 10.48s (± 0.59%) 11.14s (± 5.72%) +0.66s (+ 6.33%) 10.37s 12.82s
Monaco - node (v14.15.1, x64)
Memory used 326,575k (± 0.01%) 326,548k (± 0.01%) -27k (- 0.01%) 326,446k 326,612k
Parse Time 1.59s (± 0.79%) 1.61s (± 3.97%) +0.03s (+ 1.57%) 1.57s 1.87s
Bind Time 0.73s (± 0.47%) 0.73s (± 1.21%) +0.01s (+ 0.96%) 0.72s 0.76s
Check Time 5.74s (± 0.53%) 5.75s (± 1.25%) +0.01s (+ 0.19%) 5.64s 5.99s
Emit Time 3.35s (± 0.65%) 3.33s (± 1.11%) -0.02s (- 0.54%) 3.26s 3.42s
Total Time 11.40s (± 0.41%) 11.42s (± 1.35%) +0.02s (+ 0.22%) 11.24s 11.99s
TFS - node (v14.15.1, x64)
Memory used 289,703k (± 0.01%) 289,694k (± 0.01%) -9k (- 0.00%) 289,640k 289,724k
Parse Time 1.31s (± 0.59%) 1.30s (± 0.46%) -0.01s (- 1.07%) 1.28s 1.31s
Bind Time 0.79s (± 2.25%) 0.80s (± 0.70%) +0.01s (+ 1.14%) 0.79s 0.81s
Check Time 5.35s (± 0.39%) 5.34s (± 0.34%) -0.01s (- 0.15%) 5.30s 5.38s
Emit Time 3.54s (± 0.89%) 3.54s (± 0.51%) +0.01s (+ 0.28%) 3.51s 3.58s
Total Time 10.98s (± 0.39%) 10.98s (± 0.25%) -0.00s (- 0.02%) 10.94s 11.07s
material-ui - node (v14.15.1, x64)
Memory used 436,655k (± 0.00%) 436,737k (± 0.00%) +83k (+ 0.02%) 436,690k 436,794k
Parse Time 1.88s (± 0.64%) 1.87s (± 0.76%) -0.00s (- 0.16%) 1.86s 1.92s
Bind Time 0.59s (± 1.16%) 0.59s (± 0.76%) +0.00s (+ 0.34%) 0.58s 0.60s
Check Time 12.91s (± 0.77%) 12.95s (± 1.02%) +0.05s (+ 0.35%) 12.78s 13.44s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.37s (± 0.71%) 15.41s (± 0.89%) +0.04s (+ 0.29%) 15.22s 15.89s
xstate - node (v14.15.1, x64)
Memory used 546,986k (± 0.00%) 547,013k (± 0.00%) +27k (+ 0.00%) 546,982k 547,055k
Parse Time 2.60s (± 0.43%) 2.61s (± 0.52%) +0.01s (+ 0.19%) 2.59s 2.64s
Bind Time 0.99s (± 1.23%) 0.98s (± 0.59%) -0.01s (- 1.21%) 0.97s 0.99s
Check Time 1.52s (± 0.49%) 1.53s (± 0.58%) +0.01s (+ 0.46%) 1.51s 1.55s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.19s (± 0.29%) 5.19s (± 0.29%) -0.00s (- 0.08%) 5.16s 5.23s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50640 10
Baseline main 10

Developer Information:

Download Benchmark

@sandersn sandersn added the Experiment A fork with an experimental idea which might not make it into master label Sep 21, 2022
@sandersn
Copy link
Member

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 21, 2022

Heya @sandersn, I've started to run the abridged perf test suite on this PR at cd31843. You can monitor the build here.

Update: The results are in!

@sandersn sandersn removed the Experiment A fork with an experimental idea which might not make it into master label Sep 21, 2022
@typescript-bot
Copy link
Collaborator

@sandersn
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..50640

Metric main 50640 Delta Best Worst
Angular - node (v14.15.1, x64)
Memory used 339,148k (± 0.01%) 339,161k (± 0.00%) +13k (+ 0.00%) 339,131k 339,180k
Parse Time 2.06s (± 0.44%) 2.06s (± 0.80%) +0.00s (+ 0.19%) 2.03s 2.11s
Bind Time 0.80s (± 0.75%) 0.79s (± 0.75%) -0.00s (- 0.13%) 0.78s 0.81s
Check Time 5.87s (± 0.54%) 5.86s (± 0.31%) -0.01s (- 0.10%) 5.82s 5.90s
Emit Time 6.23s (± 0.46%) 6.24s (± 0.45%) +0.01s (+ 0.11%) 6.18s 6.30s
Total Time 14.95s (± 0.29%) 14.96s (± 0.29%) +0.01s (+ 0.05%) 14.85s 15.07s
Compiler-Unions - node (v14.15.1, x64)
Memory used 190,791k (± 0.68%) 190,124k (± 0.13%) -667k (- 0.35%) 189,123k 190,350k
Parse Time 0.86s (± 0.43%) 0.85s (± 0.40%) -0.00s (- 0.35%) 0.85s 0.86s
Bind Time 0.48s (± 0.70%) 0.48s (± 0.70%) 0.00s ( 0.00%) 0.48s 0.49s
Check Time 6.77s (± 0.45%) 6.72s (± 0.30%) -0.04s (- 0.62%) 6.69s 6.77s
Emit Time 2.40s (± 0.92%) 2.43s (± 1.75%) +0.02s (+ 1.00%) 2.37s 2.58s
Total Time 10.51s (± 0.39%) 10.49s (± 0.51%) -0.02s (- 0.17%) 10.42s 10.68s
Monaco - node (v14.15.1, x64)
Memory used 326,591k (± 0.01%) 326,562k (± 0.01%) -29k (- 0.01%) 326,497k 326,602k
Parse Time 1.59s (± 0.83%) 1.58s (± 0.37%) -0.01s (- 0.94%) 1.57s 1.59s
Bind Time 0.73s (± 0.47%) 0.72s (± 0.41%) -0.01s (- 0.69%) 0.72s 0.73s
Check Time 5.73s (± 0.46%) 5.69s (± 0.48%) -0.04s (- 0.70%) 5.65s 5.74s
Emit Time 3.38s (± 0.76%) 3.38s (± 0.49%) -0.00s (- 0.12%) 3.35s 3.42s
Total Time 11.44s (± 0.38%) 11.37s (± 0.34%) -0.06s (- 0.54%) 11.31s 11.45s
TFS - node (v14.15.1, x64)
Memory used 289,679k (± 0.01%) 289,686k (± 0.00%) +7k (+ 0.00%) 289,663k 289,711k
Parse Time 1.30s (± 0.78%) 1.29s (± 0.46%) -0.01s (- 0.46%) 1.28s 1.31s
Bind Time 0.79s (± 0.56%) 0.79s (± 2.00%) -0.01s (- 0.88%) 0.74s 0.80s
Check Time 5.38s (± 0.52%) 5.39s (± 0.34%) +0.00s (+ 0.04%) 5.35s 5.44s
Emit Time 3.60s (± 0.80%) 3.59s (± 0.82%) -0.00s (- 0.06%) 3.55s 3.69s
Total Time 11.07s (± 0.37%) 11.06s (± 0.34%) -0.01s (- 0.13%) 10.96s 11.14s
material-ui - node (v14.15.1, x64)
Memory used 436,854k (± 0.00%) 436,912k (± 0.00%) +59k (+ 0.01%) 436,872k 436,958k
Parse Time 1.87s (± 0.70%) 1.86s (± 0.45%) -0.01s (- 0.54%) 1.85s 1.88s
Bind Time 0.58s (± 0.85%) 0.58s (± 0.81%) -0.01s (- 0.85%) 0.57s 0.59s
Check Time 12.85s (± 0.28%) 12.81s (± 0.39%) -0.03s (- 0.26%) 12.69s 12.93s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.30s (± 0.27%) 15.25s (± 0.30%) -0.05s (- 0.30%) 15.15s 15.36s
xstate - node (v14.15.1, x64)
Memory used 547,657k (± 0.00%) 547,693k (± 0.00%) +36k (+ 0.01%) 547,662k 547,725k
Parse Time 2.62s (± 0.58%) 2.61s (± 0.49%) -0.01s (- 0.34%) 2.59s 2.65s
Bind Time 0.98s (± 1.38%) 0.98s (± 1.23%) -0.00s (- 0.20%) 0.95s 1.01s
Check Time 1.51s (± 0.61%) 1.51s (± 0.45%) +0.00s (+ 0.13%) 1.50s 1.53s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.19s (± 0.40%) 5.18s (± 0.29%) -0.01s (- 0.19%) 5.16s 5.22s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50640 10
Baseline main 10

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member

Yeah, when I ran it, it didn't seem like we saw a boost, but we probably don't have something so bad in our test suite.

@jakebailey
Copy link
Member

jakebailey commented Jan 27, 2023

@Zzzen I took the liberty of merging main into this to retest it; hope you don't mind.

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the perf test suite on this PR at 8f978fd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..50640
Metric main 50640 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 359,834k (± 0.03%) 359,817k (± 0.01%) -16k (- 0.00%) 359,768k 359,897k
Parse Time 3.96s (± 0.87%) 3.97s (± 0.34%) +0.01s (+ 0.25%) 3.95s 3.99s
Bind Time 1.18s (± 0.69%) 1.17s (± 0.64%) -0.00s (- 0.42%) 1.16s 1.18s
Check Time 8.66s (± 0.31%) 8.67s (± 0.63%) +0.01s (+ 0.10%) 8.57s 8.72s
Emit Time 7.65s (± 0.93%) 7.61s (± 0.76%) -0.04s (- 0.52%) 7.51s 7.68s
Total Time 21.44s (± 0.27%) 21.41s (± 0.42%) -0.03s (- 0.12%) 21.30s 21.50s
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,493k (± 1.19%) 192,531k (± 0.06%) -963k (- 0.50%) 192,307k 192,609k
Parse Time 1.64s (± 1.37%) 1.65s (± 3.12%) +0.01s (+ 0.51%) 1.61s 1.72s
Bind Time 0.79s (± 0.69%) 0.79s (± 0.69%) 0.00s ( 0.00%) 0.79s 0.80s
Check Time 9.62s (± 0.60%) 9.66s (± 0.49%) +0.05s (+ 0.47%) 9.61s 9.73s
Emit Time 2.80s (± 0.43%) 2.87s (± 4.95%) +0.06s (+ 2.20%) 2.79s 3.15s
Total Time 14.86s (± 0.58%) 14.97s (± 0.62%) +0.11s (+ 0.76%) 14.89s 15.15s
Monaco - node (v18.10.0, x64)
Memory used 344,010k (± 0.01%) 343,990k (± 0.01%) -20k (- 0.01%) 343,922k 344,030k
Parse Time 2.94s (± 0.61%) 2.97s (± 0.84%) +0.03s (+ 0.96%) 2.94s 3.00s
Bind Time 1.03s (± 0.79%) 1.04s (± 1.59%) +0.00s (+ 0.16%) 1.02s 1.05s
Check Time 7.06s (± 0.65%) 7.07s (± 0.67%) +0.01s (+ 0.19%) 7.00s 7.13s
Emit Time 4.32s (± 0.74%) 4.35s (± 0.44%) +0.03s (+ 0.66%) 4.33s 4.37s
Total Time 15.35s (± 0.57%) 15.42s (± 0.35%) +0.07s (+ 0.46%) 15.33s 15.47s
TFS - node (v18.10.0, x64)
Memory used 300,383k (± 0.00%) 300,372k (± 0.00%) -11k (- 0.00%) 300,360k 300,383k
Parse Time 2.24s (± 1.53%) 2.25s (± 1.18%) +0.01s (+ 0.45%) 2.21s 2.29s
Bind Time 1.17s (± 0.70%) 1.17s (± 0.71%) +0.00s (+ 0.14%) 1.17s 1.19s
Check Time 6.52s (± 0.57%) 6.53s (± 0.42%) +0.00s (+ 0.08%) 6.48s 6.56s
Emit Time 3.91s (± 0.53%) 3.93s (± 1.27%) +0.02s (+ 0.55%) 3.88s 4.00s
Total Time 13.85s (± 0.58%) 13.89s (± 0.48%) +0.04s (+ 0.26%) 13.79s 13.96s
material-ui - node (v18.10.0, x64)
Memory used 476,755k (± 0.01%) 476,825k (± 0.02%) +70k (+ 0.01%) 476,715k 476,972k
Parse Time 3.55s (± 0.92%) 3.57s (± 1.04%) +0.03s (+ 0.75%) 3.51s 3.61s
Bind Time 0.96s (± 1.09%) 0.97s (± 0.84%) +0.01s (+ 0.86%) 0.96s 0.98s
Check Time 17.09s (± 0.49%) 17.25s (± 0.70%) +0.16s (+ 0.92%) 17.08s 17.39s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.60s (± 0.48%) 21.80s (± 0.51%) +0.19s (+ 0.90%) 21.66s 21.97s
xstate - node (v18.10.0, x64)
Memory used 548,660k (± 0.03%) 548,631k (± 0.02%) -29k (- 0.01%) 548,500k 548,869k
Parse Time 4.57s (± 0.52%) 4.60s (± 0.54%) +0.03s (+ 0.73%) 4.58s 4.65s
Bind Time 1.72s (± 0.48%) 1.72s (± 0.61%) -0.01s (- 0.58%) 1.70s 1.73s
Check Time 2.73s (± 0.80%) 2.77s (± 0.55%) +0.03s (+ 1.28%) 2.74s 2.78s
Emit Time 0.08s (± 0.00%) 0.08s (± 4.99%) +0.00s (+ 2.08%) 0.08s 0.09s
Total Time 9.12s (± 0.44%) 9.17s (± 0.41%) +0.06s (+ 0.66%) 9.14s 9.24s
Angular - node (v16.17.1, x64)
Memory used 359,218k (± 0.00%) 359,202k (± 0.01%) -15k (- 0.00%) 359,183k 359,238k
Parse Time 4.17s (± 0.20%) 4.20s (± 0.66%) +0.03s (+ 0.68%) 4.16s 4.23s
Bind Time 1.21s (± 0.43%) 1.22s (± 0.52%) +0.01s (+ 0.55%) 1.21s 1.23s
Check Time 9.41s (± 0.30%) 9.43s (± 0.52%) +0.02s (+ 0.16%) 9.35s 9.48s
Emit Time 8.06s (± 0.64%) 8.09s (± 1.07%) +0.03s (+ 0.31%) 8.01s 8.25s
Total Time 22.86s (± 0.35%) 22.93s (± 0.41%) +0.07s (+ 0.30%) 22.78s 23.03s
Compiler-Unions - node (v16.17.1, x64)
Memory used 194,269k (± 0.05%) 194,227k (± 0.03%) -43k (- 0.02%) 194,155k 194,340k
Parse Time 1.80s (± 0.23%) 1.80s (± 0.58%) +0.01s (+ 0.37%) 1.79s 1.82s
Bind Time 0.84s (± 0.97%) 0.85s (± 0.89%) +0.01s (+ 0.59%) 0.84s 0.86s
Check Time 10.37s (± 0.31%) 10.40s (± 0.77%) +0.03s (+ 0.29%) 10.27s 10.47s
Emit Time 3.06s (± 0.60%) 3.04s (± 0.90%) -0.01s (- 0.44%) 2.99s 3.06s
Total Time 16.07s (± 0.24%) 16.10s (± 0.69%) +0.04s (+ 0.22%) 15.90s 16.19s
Monaco - node (v16.17.1, x64)
Memory used 343,274k (± 0.02%) 343,264k (± 0.01%) -10k (- 0.00%) 343,218k 343,316k
Parse Time 3.17s (± 1.03%) 3.17s (± 1.34%) +0.00s (+ 0.05%) 3.12s 3.23s
Bind Time 1.11s (± 0.89%) 1.12s (± 0.67%) +0.01s (+ 0.90%) 1.11s 1.13s
Check Time 7.73s (± 0.25%) 7.73s (± 0.29%) +0.01s (+ 0.09%) 7.71s 7.77s
Emit Time 4.53s (± 0.95%) 4.55s (± 0.82%) +0.02s (+ 0.52%) 4.52s 4.62s
Total Time 16.53s (± 0.40%) 16.57s (± 0.39%) +0.04s (+ 0.26%) 16.47s 16.64s
TFS - node (v16.17.1, x64)
Memory used 299,756k (± 0.00%) 299,763k (± 0.00%) +8k (+ 0.00%) 299,750k 299,779k
Parse Time 2.48s (± 1.50%) 2.48s (± 1.38%) +0.01s (+ 0.27%) 2.43s 2.52s
Bind Time 1.25s (± 0.33%) 1.26s (± 0.32%) +0.01s (+ 0.53%) 1.25s 1.26s
Check Time 7.20s (± 0.38%) 7.17s (± 0.18%) -0.02s (- 0.35%) 7.15s 7.18s
Emit Time 4.22s (± 0.49%) 4.24s (± 0.62%) +0.02s (+ 0.55%) 4.20s 4.27s
Total Time 15.14s (± 0.37%) 15.15s (± 0.34%) +0.01s (+ 0.06%) 15.08s 15.21s
material-ui - node (v16.17.1, x64)
Memory used 476,178k (± 0.02%) 476,168k (± 0.01%) -11k (- 0.00%) 476,096k 476,248k
Parse Time 3.71s (± 0.36%) 3.72s (± 0.29%) +0.01s (+ 0.22%) 3.71s 3.74s
Bind Time 1.02s (± 0.00%) 1.02s (± 0.40%) +0.00s (+ 0.16%) 1.02s 1.03s
Check Time 18.11s (± 0.30%) 18.15s (± 0.26%) +0.03s (+ 0.19%) 18.10s 18.23s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.84s (± 0.27%) 22.89s (± 0.22%) +0.04s (+ 0.18%) 22.83s 22.98s
xstate - node (v16.17.1, x64)
Memory used 546,232k (± 0.01%) 546,320k (± 0.02%) +88k (+ 0.02%) 546,226k 546,436k
Parse Time 4.76s (± 0.29%) 4.76s (± 0.17%) 0.00s ( 0.00%) 4.75s 4.77s
Bind Time 1.80s (± 3.60%) 1.75s (± 5.62%) -0.05s (- 2.68%) 1.66s 1.85s
Check Time 2.98s (± 1.15%) 3.08s (± 3.16%) +0.10s (+ 3.47%) 2.98s 3.18s
Emit Time 0.09s (± 5.76%) 0.10s (± 5.76%) +0.00s (+ 0.00%) 0.09s 0.10s
Total Time 9.64s (± 0.42%) 9.70s (± 0.13%) +0.05s (+ 0.57%) 9.67s 9.70s
Angular - node (v14.15.1, x64)
Memory used 352,832k (± 0.01%) 352,848k (± 0.01%) +16k (+ 0.00%) 352,814k 352,913k
Parse Time 4.22s (± 0.79%) 4.24s (± 0.64%) +0.02s (+ 0.43%) 4.21s 4.29s
Bind Time 1.28s (± 0.49%) 1.28s (± 0.32%) +0.00s (+ 0.13%) 1.28s 1.29s
Check Time 9.70s (± 0.23%) 9.70s (± 0.52%) +0.00s (+ 0.03%) 9.63s 9.77s
Emit Time 8.40s (± 0.73%) 8.42s (± 0.51%) +0.02s (+ 0.20%) 8.35s 8.46s
Total Time 23.61s (± 0.17%) 23.64s (± 0.26%) +0.03s (+ 0.14%) 23.58s 23.72s
Compiler-Unions - node (v14.15.1, x64)
Memory used 189,203k (± 0.08%) 189,714k (± 0.66%) +511k (+ 0.27%) 189,031k 192,273k
Parse Time 1.85s (± 0.76%) 1.84s (± 0.72%) -0.01s (- 0.45%) 1.83s 1.86s
Bind Time 0.86s (± 1.20%) 0.87s (± 1.21%) +0.00s (+ 0.19%) 0.85s 0.88s
Check Time 10.46s (± 0.42%) 10.44s (± 0.36%) -0.03s (- 0.25%) 10.38s 10.48s
Emit Time 3.40s (± 4.99%) 3.36s (± 4.41%) -0.04s (- 1.13%) 3.17s 3.48s
Total Time 16.58s (± 1.00%) 16.50s (± 0.93%) -0.07s (- 0.45%) 16.27s 16.63s
Monaco - node (v14.15.1, x64)
Memory used 338,025k (± 0.00%) 338,032k (± 0.00%) +8k (+ 0.00%) 338,017k 338,046k
Parse Time 3.24s (± 0.94%) 3.23s (± 0.84%) -0.01s (- 0.26%) 3.19s 3.26s
Bind Time 1.11s (± 0.47%) 1.11s (± 0.68%) +0.00s (+ 0.45%) 1.10s 1.12s
Check Time 8.04s (± 0.29%) 8.08s (± 0.30%) +0.04s (+ 0.54%) 8.05s 8.11s
Emit Time 4.76s (± 0.59%) 4.77s (± 0.50%) +0.01s (+ 0.25%) 4.73s 4.80s
Total Time 17.15s (± 0.36%) 17.20s (± 0.23%) +0.05s (+ 0.29%) 17.14s 17.26s
TFS - node (v14.15.1, x64)
Memory used 294,583k (± 0.01%) 294,593k (± 0.01%) +10k (+ 0.00%) 294,574k 294,614k
Parse Time 2.74s (± 0.27%) 2.76s (± 0.37%) +0.01s (+ 0.55%) 2.74s 2.77s
Bind Time 1.10s (± 1.21%) 1.11s (± 1.05%) +0.01s (+ 0.91%) 1.09s 1.12s
Check Time 7.52s (± 0.58%) 7.53s (± 0.59%) +0.02s (+ 0.24%) 7.48s 7.60s
Emit Time 4.62s (± 0.71%) 4.63s (± 0.83%) +0.01s (+ 0.22%) 4.59s 4.69s
Total Time 15.98s (± 0.51%) 16.03s (± 0.35%) +0.05s (+ 0.33%) 15.94s 16.10s
material-ui - node (v14.15.1, x64)
Memory used 471,559k (± 0.01%) 471,609k (± 0.00%) +50k (+ 0.01%) 471,592k 471,624k
Parse Time 3.93s (± 0.67%) 3.96s (± 0.77%) +0.03s (+ 0.76%) 3.93s 4.01s
Bind Time 1.04s (± 0.94%) 1.05s (± 0.78%) +0.00s (+ 0.48%) 1.04s 1.06s
Check Time 19.00s (± 0.33%) 19.25s (± 0.72%) +0.25s (+ 1.32%) 19.06s 19.41s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 23.97s (± 0.34%) 24.26s (± 0.57%) +0.29s (+ 1.20%) 24.09s 24.44s
xstate - node (v14.15.1, x64)
Memory used 534,470k (± 0.01%) 534,475k (± 0.01%) +5k (+ 0.00%) 534,428k 534,549k
Parse Time 5.15s (± 1.24%) 5.19s (± 0.79%) +0.04s (+ 0.81%) 5.14s 5.23s
Bind Time 1.72s (± 0.49%) 1.73s (± 0.48%) +0.01s (+ 0.58%) 1.72s 1.74s
Check Time 3.09s (± 0.88%) 3.10s (± 0.67%) +0.00s (+ 0.05%) 3.07s 3.12s
Emit Time 0.10s (± 0.00%) 0.10s (± 0.00%) 0.00s ( 0.00%) 0.10s 0.10s
Total Time 10.06s (± 0.80%) 10.11s (± 0.59%) +0.05s (+ 0.55%) 10.01s 10.17s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50640 6
Baseline main 6

TSServer

Comparison Report - main..50640
Metric main 50640 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,531ms (± 1.10%) 2,505ms (± 1.77%) -27ms (- 1.05%) 2,465ms 2,579ms
Req 2 - geterr 5,517ms (± 0.63%) 5,512ms (± 0.45%) -5ms (- 0.09%) 5,488ms 5,550ms
Req 3 - references 368ms (± 1.18%) 368ms (± 1.21%) -1ms (- 0.23%) 364ms 375ms
Req 4 - navto 285ms (± 0.26%) 285ms (± 1.36%) -0ms (- 0.12%) 277ms 288ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 66ms (± 3.11%) 65ms (± 1.79%) -1ms (- 1.76%) 63ms 66ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,707ms (± 1.14%) 2,745ms (± 1.26%) +37ms (+ 1.38%) 2,699ms 2,786ms
Req 2 - geterr 4,057ms (± 0.65%) 4,052ms (± 0.70%) -5ms (- 0.12%) 4,023ms 4,095ms
Req 3 - references 370ms (± 0.65%) 368ms (± 0.64%) -2ms (- 0.45%) 365ms 371ms
Req 4 - navto 299ms (± 1.49%) 299ms (± 0.33%) -0ms (- 0.06%) 297ms 300ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 79ms (± 8.12%) 83ms (± 1.46%) +4ms (+ 4.42%) 81ms 84ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,507ms (± 0.52%) 3,531ms (± 0.50%) +24ms (+ 0.68%) 3,498ms 3,549ms
Req 2 - geterr 1,426ms (± 0.89%) 1,421ms (± 0.89%) -5ms (- 0.34%) 1,409ms 1,443ms
Req 3 - references 105ms (± 1.90%) 104ms (± 1.66%) -1ms (- 1.11%) 101ms 106ms
Req 4 - navto 358ms (± 0.63%) 359ms (± 0.82%) +1ms (+ 0.28%) 356ms 364ms
Req 5 - completionInfo count 3,171 (± 0.00%) 3,171 (± 0.00%) 0 ( 0.00%) 3,171 3,171
Req 5 - completionInfo 470ms (± 0.67%) 469ms (± 1.60%) -1ms (- 0.11%) 459ms 480ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,770ms (± 1.04%) 2,778ms (± 0.47%) +9ms (+ 0.31%) 2,765ms 2,799ms
Req 2 - geterr 5,852ms (± 0.45%) 5,899ms (± 0.28%) +47ms (+ 0.79%) 5,870ms 5,918ms
Req 3 - references 374ms (± 1.02%) 379ms (± 1.37%) +5ms (+ 1.25%) 373ms 386ms
Req 4 - navto 278ms (± 0.71%) 279ms (± 0.42%) +2ms (+ 0.60%) 278ms 281ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 77ms (± 8.12%) 73ms (± 4.62%) 🟩-4ms (- 4.79%) 69ms 77ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,934ms (± 0.52%) 2,962ms (± 0.92%) +28ms (+ 0.95%) 2,932ms 3,004ms
Req 2 - geterr 4,403ms (± 0.38%) 4,394ms (± 0.56%) -9ms (- 0.21%) 4,365ms 4,423ms
Req 3 - references 380ms (± 1.02%) 378ms (± 0.41%) -2ms (- 0.57%) 376ms 380ms
Req 4 - navto 289ms (± 0.72%) 288ms (± 0.84%) -1ms (- 0.29%) 285ms 291ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 71ms (± 0.72%) 72ms (± 0.00%) +1ms (+ 0.93%) 72ms 72ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,631ms (± 0.41%) 3,628ms (± 0.25%) -3ms (- 0.07%) 3,617ms 3,641ms
Req 2 - geterr 1,571ms (± 1.09%) 1,566ms (± 1.52%) -4ms (- 0.28%) 1,525ms 1,586ms
Req 3 - references 110ms (± 2.15%) 112ms (± 1.19%) +2ms (+ 1.67%) 110ms 113ms
Req 4 - navto 342ms (± 1.04%) 342ms (± 1.09%) 0ms ( 0.00%) 337ms 346ms
Req 5 - completionInfo count 3,171 (± 0.00%) 3,171 (± 0.00%) 0 ( 0.00%) 3,171 3,171
Req 5 - completionInfo 478ms (± 0.79%) 477ms (± 0.71%) -0ms (- 0.07%) 474ms 482ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,906ms (± 0.35%) 2,917ms (± 0.23%) +11ms (+ 0.38%) 2,907ms 2,926ms
Req 2 - geterr 6,264ms (± 0.51%) 6,308ms (± 1.16%) +43ms (+ 0.69%) 6,218ms 6,433ms
Req 3 - references 388ms (± 0.70%) 393ms (± 1.02%) +5ms (+ 1.29%) 389ms 401ms
Req 4 - navto 282ms (± 1.73%) 283ms (± 1.32%) +1ms (+ 0.30%) 278ms 289ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 85ms (± 6.38%) 88ms (± 8.76%) +4ms (+ 4.34%) 73ms 93ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,074ms (± 0.40%) 3,084ms (± 0.42%) +10ms (+ 0.31%) 3,065ms 3,097ms
Req 2 - geterr 4,694ms (± 0.37%) 4,721ms (± 0.26%) +27ms (+ 0.56%) 4,712ms 4,744ms
Req 3 - references 403ms (± 0.65%) 405ms (± 1.07%) +2ms (+ 0.41%) 398ms 410ms
Req 4 - navto 293ms (± 0.91%) 293ms (± 0.78%) +0ms (+ 0.11%) 289ms 296ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 81ms (± 1.01%) 81ms (± 0.64%) 0ms ( 0.00%) 80ms 81ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,962ms (± 0.47%) 4,000ms (± 0.43%) +37ms (+ 0.94%) 3,973ms 4,023ms
Req 2 - geterr 1,524ms (± 2.03%) 1,513ms (± 0.54%) -11ms (- 0.70%) 1,500ms 1,522ms
Req 3 - references 132ms (± 2.25%) 132ms (± 2.85%) -0ms (- 0.13%) 127ms 137ms
Req 4 - navto 374ms (± 0.57%) 375ms (± 0.29%) +1ms (+ 0.22%) 374ms 377ms
Req 5 - completionInfo count 3,171 (± 0.00%) 3,171 (± 0.00%) 0 ( 0.00%) 3,171 3,171
Req 5 - completionInfo 505ms (± 2.35%) 509ms (± 2.14%) +4ms (+ 0.83%) 492ms 525ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50640 6
Baseline main 6

Startup

Comparison Report - main..50640
Metric main 50640 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 142.62ms (± 0.19%) 142.52ms (± 0.18%) -0.10ms (- 0.07%) 141.55ms 146.56ms
tsserver-startup - node (v16.17.1, x64)
Execution time 226.34ms (± 0.18%) 229.15ms (± 0.50%) +2.81ms (+ 1.24%) 225.18ms 234.99ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 229.01ms (± 0.23%) 231.56ms (± 0.40%) +2.54ms (+ 1.11%) 227.83ms 242.62ms
typescript-startup - node (v16.17.1, x64)
Execution time 209.44ms (± 0.17%) 210.16ms (± 0.26%) +0.72ms (+ 0.34%) 208.62ms 216.53ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 50640 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member

@typescript-bot test this
@typescript-bot test top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot test tsserver top100
@typescript-bot run dt

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at 8f978fd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the extended test suite on this PR at 8f978fd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at 8f978fd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the diff-based user code test suite (tsserver) on this PR at 8f978fd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the diff-based top-repos suite (tsserver) on this PR at 8f978fd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 27, 2023

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at 8f978fd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/50640/merge:

Everything looks good!

1 similar comment
@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/50640/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Heya @jakebailey, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/50640/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/50640/merge:

Something interesting changed - please have a look.

Details

palantir/blueprint

⚠️ Note that built also had errors ⚠️
Req #19430 - completionEntryDetails
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77733:33)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77779:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at getEffectsSignature (/typescript-main/built/local/tsserver.js:68256:22)
    at getTypeAtFlowCall (/typescript-main/built/local/tsserver.js:68595:25)
    at getTypeAtFlowNode (/typescript-main/built/local/tsserver.js:68481:18)
    at getFlowTypeOfReference (/typescript-main/built/local/tsserver.js:68438:45)
    at checkIdentifier (/typescript-main/built/local/tsserver.js:69687:22)
    at checkExpressionWorker (/typescript-main/built/local/tsserver.js:77745:16)
    at checkExpression (/typescript-main/built/local/tsserver.js:77699:32)
    at checkNonNullExpression (/typescript-main/built/local/tsserver.js:72180:29)
    at checkPropertyAccessExpression (/typescript-main/built/local/tsserver.js:72259:162)
Req #19430 - completionEntryDetails
    at getContextualTypeForArgument (/typescript-50640/built/local/tsserver.js:70378:40)
    at getContextualType2 (/typescript-50640/built/local/tsserver.js:70822:16)
    at getApparentTypeOfContextualType (/typescript-50640/built/local/tsserver.js:70758:120)
    at getContextualType2 (/typescript-50640/built/local/tsserver.js:70837:22)
    at getApparentTypeOfContextualType (/typescript-50640/built/local/tsserver.js:70758:120)
    at getContextualSignature (/typescript-50640/built/local/tsserver.js:71125:18)
    at getNarrowedTypeOfSymbol (/typescript-50640/built/local/tsserver.js:69574:39)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69666:16)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)
    at getTypeAtFlowCall (/typescript-50640/built/local/tsserver.js:68616:25)
    at getTypeAtFlowNode (/typescript-50640/built/local/tsserver.js:68502:18)
    at getFlowTypeOfReference (/typescript-50640/built/local/tsserver.js:68459:45)
    at checkIdentifier (/typescript-50640/built/local/tsserver.js:69708:22)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77766:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at checkPropertyAccessExpression (/typescript-50640/built/local/tsserver.js:72280:162)
    at checkExpressionWorker (/typescript-50640/built/local/tsserver.js:77800:16)
    at checkExpression (/typescript-50640/built/local/tsserver.js:77720:32)
    at checkNonNullExpression (/typescript-50640/built/local/tsserver.js:72201:29)
    at getEffectsSignature (/typescript-50640/built/local/tsserver.js:68277:22)

That is a filtered view of the text. To see the raw error text, go to RepoResults4/palantir.blueprint.rawError.txt in the artifact folder

Last few requests

{"seq":19427,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/site/docs/versions/2/docs-app.js","line":1,"offset":278158,"includeExternalModuleExports":false,"includeInsertTextCompletions":true,"triggerKind":1}}
{"seq":19428,"type":"request","command":"completionEntryDetails","arguments":{"file":"@PROJECT_ROOT@/site/docs/versions/2/docs-app.js","line":1,"offset":278158,"entryNames":["_"]}}
{"seq":19429,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/site/docs/versions/2/docs-app.js","line":1,"offset":280850,"includeExternalModuleExports":false,"includeInsertTextCompletions":true,"triggerKind":1}}
{"seq":19430,"type":"request","command":"completionEntryDetails","arguments":{"file":"@PROJECT_ROOT@/site/docs/versions/2/docs-app.js","line":1,"offset":280850,"entryNames":["_"]}}

Repro Steps

  1. git clone https://github.com/palantir/blueprint --recurse-submodules
  2. In dir blueprint, run git reset --hard f3be31b619cde60772150ce5e08972f9e6de5138
  3. In dir blueprint, run yarn install --ignore-engines --ignore-scripts --silent
  4. Back in the initial folder, download RepoResults4/palantir.blueprint.replay.txt from the artifact folder
  5. npm install --no-save @typescript/server-replay
  6. npx tsreplay ./blueprint ./palantir.blueprint.replay.txt path/to/tsserver.js
  7. npx tsreplay --help to learn about helpful switches for debugging, logging, etc

Copy link
Member

@ahejlsberg ahejlsberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is superceded by the more general #53192.

@ahejlsberg ahejlsberg removed their assignment Mar 14, 2023
@ahejlsberg ahejlsberg added Declined The issue was declined as something which matches the TypeScript vision and removed For Backlog Bug PRs that fix a backlog bug labels Mar 14, 2023
@typescript-bot typescript-bot added the For Backlog Bug PRs that fix a backlog bug label Mar 14, 2023
@sandersn sandersn closed this Mar 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Declined The issue was declined as something which matches the TypeScript vision For Backlog Bug PRs that fix a backlog bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Experiment short-circuiting of union distribution
5 participants