🌍 #349 Gen performance enhancements#351
Merged
Merged
Conversation
nth-commit
commented
Aug 7, 2022
nth-commit
commented
Owner
- 🌍 Generator performance #349 Prefer Lazy > WeakMap, use GenProvider everywhere to get laziness for free
- 🌍 Generator performance #349 Use a list to internally build the list gen, ensure immutability with a readonly interface
- 🌍 Generator performance #349 Make list shrinking lazier, computation deferred for current level
- 🌍 Generator performance #349 Simplify example space merging by lifting the merge values function out
- 🌍 Generator performance #349 Improve merge performance by deferring more shrink computations
- 🌍 Generator performance #349 Simplify ListGen by handling empty/non-empty cases distinctly
…ty with a readonly interface Was using an ImmutableList but it's pretty slow for the use-case of iteratively adding to it. ImmutableArray has similar performance to List, but stutters when generating generating lists with a high count. When we simply use a List, we can allocate the capacity upfront. Previous (ImmutableList): ``` | Method | size | Mean | Error | StdDev | Median | |-------- |----- |-----------:|----------:|----------:|-----------:| | ListGen | 10 | 141.0 us | 12.04 us | 35.32 us | 138.7 us | | ListGen | 50 | 547.0 us | 15.31 us | 44.16 us | 542.7 us | | ListGen | 100 | 1,170.3 us | 50.84 us | 147.50 us | 1,129.4 us | | ListGen | 500 | 6,209.3 us | 119.20 us | 117.07 us | 6,193.8 us | ``` Now (List): ``` | Method | size | Mean | Error | StdDev | Median | |-------- |----- |-----------:|---------:|---------:|------------:| | ListGen | 10 | 112.0 us | 8.93 us | 26.34 us | 98.88 us | | ListGen | 50 | 400.4 us | 7.96 us | 14.55 us | 400.79 us | | ListGen | 100 | 775.7 us | 14.95 us | 16.00 us | 775.65 us | | ListGen | 500 | 5,837.2 us | 73.30 us | 64.98 us | 5,823.66 us | ``` And ImmutableArray, for interest: ``` | Method | size | Mean | Error | StdDev | Median | |-------- |----- |-----------:|----------:|----------:|-----------:| | ListGen | 10 | 114.9 us | 8.20 us | 24.18 us | 105.7 us | | ListGen | 50 | 403.2 us | 8.06 us | 15.72 us | 404.6 us | | ListGen | 100 | 790.7 us | 14.71 us | 13.76 us | 791.4 us | | ListGen | 500 | 6,485.2 us | 128.36 us | 120.07 us | 6,467.7 us | ```
…evel Results in about a 10% increase in performance for lists of < 100. Also made the list gen benchmark exercise shrinks more by not declaring a constant (non-shrinking) count. Before: ``` | Method | size | Mean | Error | StdDev | |-------- |----- |------------:|----------:|----------:| | ListGen | 10 | 95.34 us | 7.731 us | 22.80 us | | ListGen | 50 | 317.55 us | 10.500 us | 30.96 us | | ListGen | 100 | 577.27 us | 11.460 us | 30.59 us | | ListGen | 500 | 3,969.47 us | 78.922 us | 223.89 us | ``` After: ``` | Method | size | Mean | Error | StdDev | |-------- |----- |------------:|----------:|----------:| | ListGen | 10 | 86.97 us | 6.249 us | 18.43 us | | ListGen | 50 | 282.29 us | 5.469 us | 13.00 us | | ListGen | 100 | 548.28 us | 10.949 us | 28.84 us | | ListGen | 500 | 3,806.14 us | 75.494 us | 212.93 us | ```
About 20-25% performance increase. Affects ListGen, SetGen, InfiniteGen, and all the other gens that depend on those (i.e. StringGen, EnumGen). Before: ``` | Method | size | Mean | Error | StdDev | |-------- |----- |------------:|----------:|----------:| | ListGen | 10 | 94.92 us | 7.037 us | 20.53 us | | ListGen | 50 | 294.40 us | 5.845 us | 15.19 us | | ListGen | 100 | 579.21 us | 11.517 us | 26.69 us | | ListGen | 500 | 3,969.76 us | 83.782 us | 232.16 us | ``` After: ``` | Method | size | Mean | Error | StdDev | |-------- |----- |------------:|----------:|----------:| | ListGen | 10 | 79.05 us | 4.358 us | 12.85 us | | ListGen | 50 | 244.06 us | 6.232 us | 17.68 us | | ListGen | 100 | 465.13 us | 9.205 us | 24.57 us | | ListGen | 500 | 2,656.37 us | 53.010 us | 150.38 us | ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.