feat: parallel rails execution #1234
Open
+578
−40
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
commit 6ab101ce510c52a89f613e183b8ac485f2d98e08
Author: Christian Schüller <cschueller@nvidia.com>
Date: Thu Jun 19 11:17:10 2025 +0200
Revert abc bot changes
commit 6c8ff5788bd9de9375a4dcc165a633b540fbb3b1
Author: Christian Schüller <cschueller@nvidia.com>
Date: Thu Jun 19 11:07:05 2025 +0200
Add unit tests
commit 43e83fe1df21fa335d3c0afdf7bc90da854594bc
Author: Christian Schüller <cschueller@nvidia.com>
Date: Thu Jun 19 10:53:56 2025 +0200
Fix event order in processing log
commit b2ac8c2d4ba0be4dbeb50b076ef0018e1fe1bc53
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 15:44:03 2025 +0200
Fix bug in task cancellation
commit aa53bd4ce857d2dd399a76ef1077dd6e5bfc0418
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 15:30:54 2025 +0200
Handle CancelledError
commit a37c890a40bd13f1263a6d61ef296b5d7ae63e6a
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 11:26:54 2025 +0200
Revert bot config changes
commit 2ebeee89d409bf2d844656176337a887fbf68024
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 11:00:02 2025 +0200
Fix types for Python 3.9
commit 410ee0e3a5eaa3c24749fd291a6d8da4cc3c06d0
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 10:17:59 2025 +0200
Add support for parallel output rails
commit 5760f9a29cb723f8a9049aef6b8efe45c98e0c23
Author: Christian Schüller <cschueller@nvidia.com>
Date: Wed Jun 18 10:13:00 2025 +0200
Fix event history creation
commit 3b9193a391f19e3e22ebe56c4dca23fe416fcbae
Author: Christian Schüller <cschueller@nvidia.com>
Date: Tue Jun 17 16:51:03 2025 +0200
Add support for tracing events
commit ba20b5b39f55a366919a8e5bb038ad3dc350b162
Author: Christian Schüller <cschueller@nvidia.com>
Date: Tue Jun 17 15:39:09 2025 +0200
Fix tracing
commit b807f28b67c8347593f2b925733ec80a81f67449
Author: Christian Schüller <cschueller@nvidia.com>
Date: Mon Jun 16 13:43:50 2025 +0200
Improve parallel task handling
commit 2516d4f030ff6dcfd64d39708eb2bb36aa36bbc0
Author: Christian Schüller <cschueller@nvidia.com>
Date: Fri Jun 13 11:54:36 2025 +0200
Add support for rail parameters
commit b70942d
Author: Razvan Dinu <rdinu@nvidia.com>
Date: Wed Jun 4 15:11:52 2025 +0300
feat: poc for parallel rails
The approach is to add a custom action called `run_flows_in_parallel` which can be used for both input and output rails.
Small tweak needed for the logic for starting new flows.
Only basic manual testing done to check that two self-check input flows run in paralle.
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for parallel execution of input and output rails via a new run_flows_in_parallel action.
- Introduces a
parallelflag in the rails config to toggle between sequential and parallel processing. - Implements
_run_flows_in_parallel,_run_input_rails_in_parallel, and_run_output_rails_in_parallelin the runtime, and registers them as actions. - Adds end-to-end tests and test configs that verify success and failure behaviors under parallel rail execution.
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_parallel_rails.py | New async tests covering parallel-rails success and failure paths |
| tests/test_configs/parallel_rails/config.yml | Bot config with parallel: True and lists of rails to run |
| tests/test_configs/parallel_rails/blocked_terms.co | Definitions for blocked-term subflows |
| tests/test_configs/parallel_rails/prompts.yml | Prompt templates for self-check input/output |
| tests/test_configs/parallel_rails/actions.py | Simulated actions with delays to test concurrency |
| nemoguardrails/rails/llm/llm_flows.co | Branches on parallel flag to call new parallel-run actions |
| nemoguardrails/rails/llm/config.py | Adds parallel field to InputRails model |
| nemoguardrails/logging/processing_log.py | Extends ignored actions to hide parallel-run helpers |
| nemoguardrails/colang/v1_0/runtime/runtime.py | Registers the new parallel-run actions and unpacks history events |
| nemoguardrails/colang/v1_0/runtime/flows.py | Updates flow-start logic to respect explicit start_flow events |
Comments suppressed due to low confidence (1)
tests/test_configs/parallel_rails/config.yml:30
- The third input rail is duplicated here instead of the intended 'generate user intent' subflow, which will cause the test to fail at index 2. Please replace this entry with the correct flow name.
- check blocked input terms
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Christian Schüller <160150754+schuellc-nvidia@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Christian Schüller <160150754+schuellc-nvidia@users.noreply.github.com>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1234 +/- ##
===========================================
+ Coverage 68.65% 68.92% +0.27%
===========================================
Files 161 161
Lines 15978 16065 +87
===========================================
+ Hits 10969 11073 +104
+ Misses 5009 4992 -17
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
Thank you @schuellc-nvidia, I'm going to review it gradually 👍🏻 Have you considered adding tests that compares sequential and parallel execution, I see that following test passes but would be great to address the differences in their behavior @pytest.mark.asyncio
async def test_sequential_rails_execution():
# Test sequential execution - rails should run one after another and take longer
config = RailsConfig.from_path(os.path.join(CONFIGS_FOLDER, "parallel_rails"))
config.rails.input.parallel = False
config.rails.output.parallel = False
chat = TestChat(
config,
llm_completions=[
"No",
"Hi there! How can I assist you with questions about the ABC Company today?",
"No",
],
)
chat >> "hi"
result = await chat.app.generate_async(messages=chat.history, options=OPTIONS)
# Assert the response is correct
assert (
result
and result.response[0]["content"]
== "Hi there! How can I assist you with questions about the ABC Company today?"
)
# Check that all rails were executed sequentially
assert result.log.activated_rails[0].name == "self check input"
assert (
result.log.activated_rails[1].name == "check blocked input terms $duration=1.0"
)
assert len(result.log.activated_rails[1].executed_actions) == 1
assert (
result.log.activated_rails[2].name == "check blocked input terms $duration=1.0"
)
assert result.log.activated_rails[3].name == "generate user intent"
assert result.log.activated_rails[4].name == "self check output"
assert (
result.log.activated_rails[5].name == "check blocked output terms $duration=1.0"
)
assert (
result.log.activated_rails[6].name == "check blocked output terms $duration=1.0"
)
assert len(result.log.activated_rails[5].executed_actions) == 1
# Sequential execution should take approximately 2 seconds for each set of rails:
# Input rails: 1s + 1s = 2s (sequential)
# Output rails: 1s + 1s = 2s (sequential)
assert (
result.log.stats.input_rails_duration >= 2
and result.log.stats.output_rails_duration >= 2
), "Sequential rails processing should take longer than parallel processing." |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This MR enables parallel input/output rail execution (in Colang 1) through a new action called 'run_flows_in_parallel'.
run_input_rails_in_parallel,run_output_rails_in_parallel) to simplify the parallel rails calling and to create the necessary tracing events:StartInputRail,InputRailFinished,StartOutputRail,OutputRailFinished.Related Issue(s)
Checklist