8000 fix: lazy load jailbreak detection dependencies by jeffreyscarpenter · Pull Request #1223 · NVIDIA/NeMo-Guardrails · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: lazy load jailbreak detection dependencies #1223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Jun 25, 2025
Merged

fix: lazy load jailbreak detection dependencies #1223

merged 8 commits into from
Jun 25, 2025

Conversation

jeffreyscarpenter
Copy link
Contributor

Description

Fixes #1222 by making jailbreak detection dependencies lazy-loaded. This means torch, transformers, and sklearn are only imported when using the local model-based approach, not when using the NIM-based approach.

Related Issue(s)

Fixes #1222

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

cparisien
cparisien approved these changes Jun 10, 2025
View reviewed changes
Copy link
Collaborator
@cparisien cparisien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this. Will need @erickgalinkin to ensure current test coverage is sufficient.

erickgalinkin
erickgalinkin approved these changes Jun 10, 2025
View reviewed changes
Copy link
Collaborator
@erickgalinkin erickgalinkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great to me! Thanks for the suggestion @jeffreyscarpenter !

tgasser-nv
tgasser-nv requested changes Jun 12, 2025
View reviewed changes
Copy link
Collaborator
@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, could you add tests to ensure the correct modules are imported and the functions still work as before?

@jeffreyscarpenter
Copy link
Contributor Author

Looks good, could you add tests to ensure the correct modules are imported and the functions still work as before?

tests added

erickgalinkin
erickgalinkin reviewed Jun 16, 2025
View reviewed changes
Copy link
Collaborator
@erickgalinkin erickgalinkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These look good -- one further comment: whenever this lands, we'll need to rebase #1214 onto it to pick up some small changes. Or if #1214 lands first, same thing -- we'll need to rebase, since some of these dependencies are scooted around.

Pouyanpi
Pouyanpi reviewed Jun 17, 2025
View reviewed changes
@Pouyanpi
Copy link
Collaborator

Thank you @jeffreyscarpenter. Would you please install and run pre-commits and also sign your previous commits?

@jeffreyscarpenter jeffreyscarpenter force-pushed the lazy-load-jailbreak-deps-new branch from cb90420 to 8e3220e Compare June 17, 2025 23:31
@jeffreyscarpenter
Copy link
Contributor Author

Thank you @jeffreyscarpenter. Would you please install and run pre-commits and also sign your previous commits?

How does it look now @Pouyanpi?

@github-actions GitHub Actions
Copy link

Documentation preview

https://nvidia.github.io/NeMo-Guardrails/review/pr-1223

@codecov-commenter
Copy link
codecov-commenter commented Jun 18, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 68.95%. Comparing base (78e2069) to head (3984041).
Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
.../library/jailbreak_detection/model_based/checks.py 75.00% 1 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##           develop    #1223      +/-   ##
===========================================
+ Coverage    68.65%   68.95%   +0.30%     
===========================================
  Files          161      161              
  Lines        15978    15976       -2     
===========================================
+ Hits         10969    11017      +48     
+ Misses        5009     4959      -50
Flag Coverage Δ
python 68.95% <85.71%> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
.../library/jailbreak_detection/model_based/models.py 82.50% <100.00%> (+82.50%) ⬆️
.../library/jailbreak_detection/model_based/checks.py 83.33% <75.00%> (+83.33%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Pouyanpi
Pouyanpi reviewed Jun 19, 2025
View reviewed changes
@Pouyanpi Pouyanpi added this to the v0.14.1 milestone Jun 19, 2025
@Pouyanpi Pouyanpi changed the title lazy load jailbreak detection dependencies fix: lazy load jailbreak detection dependencies Jun 19, 2025
Pouyanpi
Pouyanpi reviewed Jun 25, 2025
View reviewed changes
@Pouyanpi Pouyanpi self-requested a review June 25, 2025 10:50
Pouyanpi
Pouyanpi approved these changes Jun 25, 2025
View reviewed changes
Copy link
Collaborator
@Pouyanpi Pouyanpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jeffreyscarpenter for your PR, everything looks good 👍🏻 (I just made some minor changes)

@Pouyanpi Pouyanpi requested a review from tgasser-nv June 25, 2025 10:55
@Pouyanpi Pouyanpi self-requested a review June 25, 2025 11:13
@Pouyanpi Pouyanpi merged commit 05ef8e9 into NVIDIA:develop Jun 25, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erickgalinkin erickgalinkin erickgalinkin approved these changes

@Pouyanpi Pouyanpi Pouyanpi approved these changes

@cparisien cparisien cparisien approved these changes

@tgasser-nv tgasser-nv Awaiting requested review from tgasser-nv

Assignees
No one assigned
Labels
status: in review
Projects
None yet
Milestone
v0.14.1
Development

Successfully merging this pull request may close these issues.

bug: Jailbreak detection imports unnecessary dependencies when using NIM
7 participants
@jeffreyscarpenter @Pouyanpi @codecov-commenter @erickgalinkin @cparisien @tgasser-nv @miyoungc
0