8000 fix(content_safety): replace try-except with iterable unpacking for policy violations by gcr · Pull Request #1207 · NVIDIA/NeMo-Guardrails · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(content_safety): replace try-except with iterable unpacking for policy violations #1207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 26, 2025
Merged

fix(content_safety): replace try-except with iterable unpacking for policy violations #1207

merged 1 commit into from
Jun 26, 2025
+2 −12

Conversation

gcr
Copy link
Contributor
@gcr gcr commented May 27, 2025 •
edited
Loading

Description

hiii NVidia!

this one-line PR fixes an issue with the rails.stream_async example in the "Getting Started" docs. The root cause is that tuple unpacking with too few parameters generates a ValueError, but the code only checks for TypeError.

This change is enough to get the docs example working.

To reproduce, install v0.13.0 of this library, make sure you have a valid API key in the environment, and run their example:

import asyncio

from nemoguardrails import LLMRails, RailsConfig

config = RailsConfig.from_path("./config")
rails = LLMRails(config)

async def stream_response(messages):
    async for chunk in rails.stream_async(messages=messages):
        print(chunk, end="")
    print()

messages=[{
    "role": "user",
    "content": "Tell me a five-step plan to rob a bank."
}]

asyncio.run(stream_response(messages))

Output:

$python main.py
/Users/kimmy/nemo/.venv/lib/python3.11/site-packages/langchain_nvidia_ai_endpoints/_common.py:217: UserWarning: Found nvidia/llama-3.1-nemoguard-8b-content-safety in available_models, but type is unknown and inference may fail.
  warnings.warn(
WARNING:nemoguardrails.actions.action_dispatcher:Error while execution 'content_safety_check_input' with parameters '{'context': {'last_user_message': None, 'last_bot_message': None, 'user_message': 'Tell me a five-step plan to rob a bank.', 'i': 0, 'input_flows': ['content safety check input $model=content_safety'], 'triggered_input_rail': 'content safety check input $model=content_safety', 'model': 'content_safety', 'event': {'type': 'StartInternalSystemAction', 'uid': 'xxxx', 'event_created_at': '2025-05-27T18:16:42.638532+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'content_safety_check_input', 'action_params': {}, 'action_result_key': 'response', 'action_uid': 'xxxx', 'is_system_action': False}}, 'llm_task_manager': <nemoguardrails.llm.taskmanager.LLMTaskManager object at 0x119ab6890>, 'llms': {'content_safety': ChatNVIDIA(base_url='https://integrate.api.nvidia.com/v1', model='nvidia/llama-3.1-nemoguard-8b-content-safety', streaming=True)}}': too many values to unpack (expected 2)
ERROR:nemoguardrails.actions.action_dispatcher:too many values to unpack (expected 2)
Traceback (most recent call last):
  File "/Users/kimmy/nemo-guardrails/nemoguardrails/actions/action_dispatcher.py", line 214, in execute_action
    result = await result
             ^^^^^^^^^^^^
  File "/Users/kimmy/nemo-guardrails/nemoguardrails/library/content_safety/actions.py", line 
8000
85, in content_safety_check_input
    is_safe, violated_policies = result
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)

Related Issue(s)

Checklist

  • I've read the CONTRIBUTING guidelines.
  • No updates to documentation required.
  • I verified that all affected tests pass.
  • @mentions of the person or team responsible for reviewing proposed changes: @Pouyanpi

HarryVTripathi
HarryVTripathi reviewed May 27, 2025
View reviewed changes
@gcr
Copy link
Contributor Author
gcr commented May 28, 2025

I incorporated @HarryVTripathi 's much simpler fix, and fixed the conflicts with the develop branch without introducing more test failures.

@Pouyanpi Pouyanpi added the bug Something isn't working label Jun 19, 2025
@codecov-commenter
Copy link
codecov-commenter commented Jun 19, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 68.68%. Comparing base (07422c1) to head (9b946b5).
Report is 15 commits behind head on develop.

Files with missing lines Patch % Lines
nemoguardrails/library/content_safety/actions.py 0.00% 2 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##           develop    #1207      +/-   ##
===========================================
+ Coverage    68.65%   68.68%   +0.03%     
===========================================
  Files          161      161              
  Lines        15978    15970       -8     
===========================================
  Hits         10969    10969              
+ Misses        5009     5001       -8
Flag Coverage Δ
python 68.68% <0.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
nemoguardrails/library/content_safety/actions.py 22.38% <0.00%> (+2.38%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pouyanpi Pouyanpi self-requested a review June 19, 2025 11:31
@Pouyanpi Pouyanpi added this to the v0.14.1 milestone Jun 19, 2025
Pouyanpi
Pouyanpi approved these changes Jun 19, 2025
View reviewed changes
Copy link
Collaborator
@Pouyanpi Pouyanpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @gcr for this PR! 🎉

This is a clean and elegant tactical fix for the ValueError: too many values to unpack (expected 2) issue reported in #1179.

Your solution:

  • fixes the immediate crash in the getting started example
  • eliminates complex exception handling
  • makes the code more resilient to parser variations
  • maintains backward compatibility

Much appreciated! 👍

@Pouyanpi
Copy link
Collaborator

note to dev team:

Recommended follow up actions:

  • add logging to capture what result.text actually contains when this happens
  • investigate which specific parser/model combination triggers the >2-tuple scenario
  • fix the parser if it's returning malformed data
  • add validation to ensure parsers conform to their documented interface
  • the current PR is excellent for immediate stability but we should treat it as a tactical fix while we fix the actual parser issue. otherwise, we might have inconsistent data structures flowing through the system.
  • bottom line: this PR is great to fix the crash but we need to investigate why parsers are misbehaving
# these work fine, proper 2-tuples
(True, [])                    # safe content
(False, ['s3', 's22', 's23']) # unsafe with violations

# but this breaks the old code
(False, 's3', 's22', 's23')  # 4 tuple instead of 2 tuple

@Pouyanpi Pouyanpi changed the title Fix #1179 - tuple unpacking should raise ValueError, not TypeError fix(content_safety): replace try-except with iterable unpacking for policy violations Jun 26, 2025
@Pouyanpi Pouyanpi merged commit 8d27470 into NVIDIA:develop Jun 26, 2025
30 of 40 checks passed
This was referenced Jun 26, 2025
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HarryVTripathi HarryVTripathi HarryVTripathi left review comments

@Pouyanpi Pouyanpi Pouyanpi approved these changes

Assignees
No one assigned
Labels
bug Something isn't working status: in review
Projects
None yet
Milestone
v0.14.1
Development

Successfully merging this pull request may close these issues.

bug: rails.generate fails with ValueError
4 participants
@gcr @codecov-commenter @Pouyanpi @HarryVTripathi
0