8000 Implementation - palmr by Palmr · Pull Request #19 · gunnarmorling/1brc · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Implementation - palmr #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 3, 2024
Merged

Implementation - palmr #19

merged 6 commits into from
Jan 3, 2024

Conversation

Palmr
Copy link
Contributor
@Palmr Palmr commented Jan 2, 2024

My implementation: CalculateAverage_palmr
Was using Zulu21.30+15-CA (build 21.0.1+12-LTS) for JDK but nothing specific to Zulu.

Memory mapped file, single-pass parsing, custom hash map, fixed size thread pool
The threading was a hasty addition and needs work. On a single thread we got it down to ~40s locally.

An assumption has been made that measurement values are all 1 decimal place. Hoping that's fair 🤞

Locally seeing on Linux 6.1.8 (Fedora 36), AMD Ryzen 9 5950X, 128GB RAM, NVMe disk (AKA not representative of the SUT at all, but hey):

#  ./calculate_average.sh
real    2m3.411s
user    2m1.455s
sys     0m2.821s

# ./calculate_average_bjhara.sh
real    0m10.318s
user    4m22.220s
sys     0m10.747s

#  ./calculate_average_palmr.sh
real    0m5.787s
user    0m42.884s
sys     0m0.796s

Palmr added 6 commits January 2, 2024 20:48
…d pool

The threading was a hasty addition and needs work
We only need it sorted for output, so only construct a treemap for output
Benchmark                                                           Mode  Cnt    Score    Error   Units
DoubleParsingBenchmark.ourToDouble                                 thrpt   10  569.771 ±  7.065  ops/us
DoubleParsingBenchmark.ourToDoubleAccumulateInToDouble             thrpt   10  648.026 ±  7.741  ops/us
DoubleParsingBenchmark.ourToDoubleDivideInsteadOfMultiply          thrpt   10  570.412 ±  9.329  ops/us
DoubleParsingBenchmark.ourToDoubleNegative                         thrpt   10  512.618 ±  8.580  ops/us
DoubleParsingBenchmark.ourToDoubleNegativeAccumulateInToDouble     thrpt   10  565.043 ± 18.137  ops/us
DoubleParsingBenchmark.ourToDoubleNegativeDivideInsteadOfMultiply  thrpt   10  511.228 ± 13.967  ops/us
DoubleParsingBenchmark.stringToDouble                              thrpt   10   52.310 ±  1.351  ops/us
DoubleParsingBenchmark.stringToDoubleNegative                      thrpt   10   50.785 ±  1.252  ops/us
@lobaorn
Copy link
lobaorn commented Jan 3, 2024

Shamelessly sharing this idea for JVM/GC tuning in another PR/discussion? #15 (comment)

@Palmr
Copy link
Contributor Author
Palmr commented Jan 3, 2024

Shamelessly sharing this idea for JVM/GC tuning in another PR/discussion? #15 (comment)

I think the ZGC parts are moot for my solution, since it doesn't GC. I should probably use Epsilon GC but when trying locally it didn't impact the runtime.

I've not seen anything about Project Leyden before, so something to play around with later.
It looks similar to Azul ReadyNow? I did consider trying but avoided since it's paid only.

@lobaorn
Copy link
lobaorn commented Jan 3, 2024

You can check the JEP here: https://openjdk.org/projects/leyden/

Yeah, it would have some intersections of functionality, but in a more general direction. There are two talks from the end of last year that are really great about the subject:

https://www.youtube.com/watch?v=lnth19Kf-x0

https://www.youtube.com/watch?v=O1Oz2-AXKKM

P.S.: I know that for many implementations ZGC is not the case, however it was just to illustrate the importance (many times) of proper JVM/GC tuning, there are so many that once the implementation reachs a threshold, then it would be tuning of the platform/OS itself.

@gunnarmorling
Copy link
Owner

Nice one, clocking in at 00:18.865. Squashing and merging. Exploring Leyden would be great, it was one of the thing I was hoping someone would do!

@gunnarmorling gunnarmorling merged commit 8e6298c into gunnarmorling:main Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0