8000 Stream par optimizations a3 by eyalfa · Pull Request #9919 · zio/zio · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Stream par optimizations a3 #9919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: series/2.x
Choose a base branch
from
Open

Stream par optimizations a3 #9919

wants to merge 6 commits into from

Conversation

eyalfa
Copy link
Contributor
@eyalfa eyalfa commented May 31, 2025 •
edited
Loading

This work introduces couple of optimizations to ZChannel.mapOutZIOPar that sums up to roughly an X3.5 benchmark improvement.

  1. fibers management, instead of using forkScoped, fibers are forked as daemons and stored in a local array using a cyclic-buffer pattern. This is possible since the number of fibers at any given time is bound by bufferSize + 2 where the extra 2 are the fiber about to be offered to the queue and the last fiber removed from the queue.
    a. furthermore, fibers are responsible for removing themselves from the array once completed.
    b. no race over array 'adds', potential race on read (due to fibers nulling their slot in the array) is not an issue since the reader can identify fibers which are already terminated.
    c. this array is used to interrupt pending fibers at end of execution, it's also used to await pending fiber in the 'happy case' completion.
    d. fibers are stored in the array before actually started, this guarantees no fibers will be missed once the array is examined (either for interrupting or awaiting all fibers).
  2. simplified semaphore implementation optimized for the single consumer case.
    a. consumer starts with all available permits, once exhausted it'd acquire all currently available permits from the
    semaphore (potentially blocking).
    b. acquired permits are transfered to the forked fibers which release them back to the semaphore upon completion, potentially unblocking the forker.
  3. remove the latches, these served two purposes: prevent the forker from exceeding the number of permits while ensuring ordering (prevent a later fiber from acquiring the last permit before a preceding fiber, enforced by a test case for n=1) and guarantee all started fibers have actually started running which was important for the way the previous implementation coordinated successful completion.
    a. in the new implementation the forker acquires a permit from the semaphore and transfers it to the forked fiber, this effectively enforces the ordering between fibers (later tasks can't 'still' a permit from earlier tasks).
    b. successful completion is now coordinated by leveraging the array tracking inflight fibers. once upstream completed there will be no more forked fibers and no more insertions into this array so it can be used to obtain a snapshot of still running fibers which are then awaited (notice the snapshot is not accurate and may include already completed fibers, but awaiting these is a no-op).

so numbers!

baseline:

Benchmark                              (chunkCount)  (chunkSize)  (parChunkSize)   Mode  Cnt  Score   Error  Units
StreamParBenchmark.zioMapPar                  10000         5000              50  thrpt   20  0.638 ± 0.102  ops/s
StreamParBenchmark.zioMapParUnordered         10000         5000              50  thrpt   20  0.726 ± 0.028  ops/s
Total time: 158 s (0:02:38.0), completed 5 Jun 2025, 11:06:06

this branch:
Benchmark                              (chunkCount)  (chunkSize)  (parChunkSize)   Mode  Cnt  Score   Error  Units
StreamParBenchmark.zioMapPar                  10000         5000              50  thrpt   20  2.388 ± 0.090  ops/s
StreamParBenchmark.zioMapParUnordered         10000         5000              50  thrpt   20  0.735 ± 0.030  ops/s
  • zioMapParUnordered is next on my list 😎

@CLAassistant
Copy link
CLAassistant commented May 31, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

eyalfa added 2 commits May 31, 2025 16:03
@eyalfa
Squashed commit of the following:
fa1ab62 
commit 5d4d9fb
Merge: dd9a4d4 d2bc5fd
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Sat May 31 10:50:23 2025 +0300

    Merge branch 'series/2.x' into stream_par_optimizations_a3

commit dd9a4d4
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Sat May 31 10:48:54 2025 +0300

    stream_par_optimizations_a3: fmt

commit 1776963
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Fri May 30 19:41:59 2025 +0300

    stream_par_optimizations_a3: fix an elusive race condition, fibers may be interrupted BEFORE starting, and actually never started, resulting with a hung fiber

commit 01eca4e
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Fri May 30 17:34:28 2025 +0300

    stream_par_optimizations_a3: fix one more issue... (99 remaining..)

commit c455134
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Fri May 30 15:34:35 2025 +0300

    stream_par_optimizations_a3: use the light semaphore for the ordered version as well

commit 56f57fe
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Fri May 30 11:44:28 2025 +0300

    stream_par_optimizations_a2: remove View import which is broken in scala 2.12, optimize imports, fmt

commit 3b9f217
Merge: e57471d 29b3055
Author: Eyal Farago-Hagag <eyal.farago@gmail.com>
Date:   Fri May 30 11:24:39 2025 +0300

    Merge branch 'series/2.x' into stream_par_optimizations_a2

commit e57471d
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Thu May 29 15:48:24 2025 +0300

    stream_par_optimizations_a2: fmt

commit c9bcc82
Author: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>
Date:   Thu May 29 01:17:27 2025 +0000

    Update zio-http to 3.3.2 (zio#9905)

    Co-authored-by: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>

commit 91fb5c5
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Wed May 28 13:40:31 2025 -0700

    fix(deps): update tailwindcss monorepo to v4.1.8 (zio#9904)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 207a2d0
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Thu May 29 00:11:32 2025 +1000

    fix(deps): update dependency @zio.dev/zio-http to v3.3.2 (zio#9903)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit bb2de19
Author: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>
Date:   Wed May 28 10:48:50 2025 +1000

    Update zio-http to 3.3.1 (zio#9897)

    Co-authored-by: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>

commit db4df92
Author: kyri-petrou <67301607+kyri-petrou@users.noreply.github.com>
Date:   Tue May 27 20:58:54 2025 +0300

    Rollback update to `sbt-ci-release` (zio#9894)

commit a9ea879
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Tue May 27 09:54:26 2025 -0700

    fix(deps): update dependency @zio.dev/zio-http to v3.3.1 (zio#9892)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 29e7cfd
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Tue May 27 09:31:18 2025 -0700

    chore(deps): update dependency @types/react to v19.1.6 (zio#9891)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit b099630
Author: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>
Date:   Mon May 26 19:10:18 2025 +0300

    Update sbt-ci-release to 1.11.0 (zio#9885)

    Co-authored-by: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>

commit 7a4704a
Author: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>
Date:   Fri May 23 18:02:25 2025 -0700

    Update zio-http to 3.3.0 (zio#9884)

    Co-authored-by: zio-scala-steward[bot] <145262613+zio-scala-steward[bot]@users.noreply.github.com>

commit be2cf76
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Wed May 21 11:03:43 2025 -0700

    fix(deps): update dependency @zio.dev/zio-http to v3.3.0 (zio#9879)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 40c16ac
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Wed May 21 10:17:40 2025 -0700

    chore(deps): update dependency @types/react to v19.1.5 (zio#9876)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit f7b2a19
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Mon May 19 10:16:48 2025 -0700

    fix(deps): update dependency @zio.dev/zio-schema to v1.7.2 (zio#9873)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit b2bbd09
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri May 16 08:06:45 2025 -0600

    fix(deps): update dependency @zio.dev/zio-schema to v1.7.1 (zio#9865)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 1362c98
Author: kyri-petrou <67301607+kyri-petrou@users.noreply.github.com>
Date:   Thu May 15 20:59:36 2025 -0600

    Ignore failing Cause test (zio#9863)

commit 2f8bc64
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Thu May 15 16:41:45 2025 +0000

    fix(deps): update tailwindcss monorepo to v4.1.7 (zio#9861)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit f544529
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Tue May 13 14:52:11 2025 +1000

    chore(deps): update dependency @types/react to v19.1.4 (zio#9856)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 19f6dd4
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat May 10 14:49:56 2025 -0600

    fix(deps): update dependency @zio.dev/zio-json to v0.7.43 (zio#9852)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit bbc9846
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri May 9 14:44:51 2025 -0700

    fix(deps): update tailwindcss monorepo to v4.1.6 (zio#9849)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 68ff3b5
Author: ​Andrzej Ressel <github@andrzejressel.pl>
Date:   Tue May 6 16:59:36 2025 +0200

    Replace <:<[R0 & out, R] with proper proof (zio#9840)

    * Reapply "Replace <:<[R0 & Any, R] with proper proof"

    This reverts commit 52b5db8.

    * Fix order of operations

    * Replace throw with errorAndAbort

    * Inline given

commit f41354e
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Tue May 6 06:47:06 2025 -0700

    chore(deps): update dependency @types/react to v19.1.3 (zio#9841)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 291c39e
Author: Georgi Krastev <joro.kr.21@gmail.com>
Date:   Tue May 6 11:43:28 2025 +0300

    Extend causes in the Exit.getOrThrow* methods with stack traces (zio#9386)

    * Extend causes in the Exit.getOrThrow* methods with stack traces

    The idea is that most users call those methods in interop code
    and since they are leaving the ZIO runtime, they need a stack trace.

    Also override `printStackTrace` in `FiberFailure` by delegating
    to a new `Cause.prettyPrintWith` method which allows for a custom
    `println` function. This is necessary because the original method
    uses the private `stackTrace` field, not `getStackTrace`.

    Updated `Cause` to remove the "Exception in thread" part as this
    shouldn't be part of the exception `toString`, it's added later and
    was causing it to be printed twice.

    Last thing to node is that I didn't change `FiberFailure.toString`
    for caution of breaking user code but it's not in line with Java
    exceptions, which don't include the stack trace in their `toString`.

    * Extract unfoldPull

    * Fix broken tests

    * Add tests for getOrThrow stack traces

    * Test entire traces in StackTracesSpec

    * Move unfoldPull to steam package object

    Rename `prettyPrintWith` argument, make it `private[zio]`

commit 141211d
Author: ​Andrzej Ressel <github@andrzejressel.pl>
Date:   Mon May 5 08:39:39 2025 +0200

    Replace <:<[R0 & Any, R] with <:<[R0 & out, R] (zio#9838)

    * Replace <:<[R0 & Any, R] with <:<[R0 & out, R]

    * Replace <:<[R0 & Any, R] with proper proof

    * Revert "Replace <:<[R0 & Any, R] with proper proof"

    This reverts commit 4dbeab8.

commit 5932fbc
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Thu May 29 09:48:48 2025 +0300

    stream_par_optimizations_a2_mapZIOParUnordered: document mapOutZIOPar

commit a1261e6
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Mon May 26 08:19:15 2025 +0300

    stream_par_optimizations_a2: eliminate latches in mapZIOPar

commit 48d1872
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Mon May 26 07:44:48 2025 +0300

    stream_par_optimizations_a2_mapZIOParUnordered: fix mapOutZIOPar

commit 9d1ce86
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Sun May 25 09:24:20 2025 +0300

    stream_par_optimizations_a2: eliminate childScope (par ordered)

commit f09e3fb
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Fri May 2 18:12:44 2025 +0300

    stream_par_optimizations_a2: correct version

commit 66a09e5
Author: Eyal Farago-Hagag <efaragohagag@zscaler.com>
Date:   Fri May 2 16:55:50 2025 +0300

    stream_par_optimizations_a2: strm.mapZIOPar: safely use forkDaemon
@eyalfa
stream_par_optimizations_a3: CI failures fixes
72383ef
@eyalfa eyalfa force-pushed the stream_par_optimizations_a3 branch from 5d4d9fb to 72383ef Compare May 31, 2025 13:13
@eyalfa eyalfa marked this pull request as ready for review June 5, 2025 08:44
@eyalfa
Copy link
Contributor Author
eyalfa commented Jun 5, 2025

CC @guizmaii , @regiskuckaertz , @kyri-petrou
you seem to be the last ones to touch this method, so I'd appreciate your reviews.

@eyalfa
Copy link
Contributor Author
eyalfa commented Jun 30, 2025

@kyri-petrou , any chance to get this reviewed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@eyalfa @CLAassistant
0