perf(coderd/util/slice): refactor unique method for large lists #8925
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces an improved version of the
Unique
method, which aims to significantly enhance the performance of processing large lists. The original implementation relied on a genericContains
function, resulting in suboptimal performance for large datasets. The proposed modification leverages a more efficient approach using a map-based algorithm, delivering notable speed gains.Benchmark
The benchmark was conducted under controlled conditions with a list size of one million elements (1,000,000). Two different implementations of the
Unique
method were tested:BenchmarkUniqueImproved
andBenchmarkUnique
. The updated version,BenchmarkUniqueImproved
, utilizes a more efficient map-based approach for duplicate detection, whileBenchmarkUnique
represents the original implementation with a genericContains
function.The results showed that the
BenchmarkUniqueImproved
version outperformed theBenchmarkUnique
implementation significantly. The average elapsed time per operation forBenchmarkUniqueImproved
was approximately 6.12 milliseconds (6,120,000 nanoseconds), whileBenchmarkUnique
took significantly longer with an average elapsed time of approximately 1,493.25 milliseconds (1,493,250,000 nanoseconds).By adopting the optimized map-based algorithm, the updated
Unique
method demonstrated notable performance improvements, especially for large lists.