fix: prod optimizations (GCS ValueError, dynamic profile for fuzzers with big wordlists) #707
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThis update modifies the Cloud Build configuration to use different substitution values, tightens MongoDB-related logic in Celery tasks to execute only within worker processes, changes the GCS upload method to use file objects, adds a Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant HttpFuzzer
User->>HttpFuzzer: Call profile(opts)
HttpFuzzer->>HttpFuzzer: dynamic_profile(opts)
HttpFuzzer->>HttpFuzzer: Determine wordlist size
alt size > 5 MB
HttpFuzzer-->>User: Return "cpu"
else size <= 5 MB
HttpFuzzer-->>User: Return "io"
end
Possibly related PRs
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
secator/tasks/_categories.py (1)
99-99: Consider using a static method call instead of lambda.While the lambda function works, a direct static method call would be more readable and avoid the lambda anti-pattern.
- profile = lambda opts: HttpFuzzer.dynamic_profile(opts) # noqa: E731 + profile = staticmethod(lambda opts: HttpFuzzer.dynamic_profile(opts))Or better yet, use a property-like approach if the framework supports it.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
cloudbuild.yaml(1 hunks)secator/celery.py(3 hunks)secator/hooks/gcs.py(1 hunks)secator/hooks/mongodb.py(1 hunks)secator/tasks/_categories.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: integration (3.11, ubuntu-latest)
🔇 Additional comments (6)
cloudbuild.yaml (1)
11-12: Verify the intended deployment target.The substitution values have been changed from production-ready (
latest/lite) to development-oriented (dev/full). Ensure this aligns with the intended deployment target and that production builds continue to use appropriate values.secator/hooks/gcs.py (1)
45-50: LGTM! Good fix for potential GCS upload issues.The changes from
upload_from_filename()toupload_from_file()with explicit file handling address potential ValueError issues with GCS uploads. The explicitseek(0)ensures the file pointer is at the beginning, and using binary mode ('rb') is appropriate for file uploads. The updated log message is also more accurate.secator/hooks/mongodb.py (1)
161-177: LGTM! Good enhancement to duplicate tagging flexibility.The addition of the
full_scanparameter provides useful flexibility for duplicate tagging operations. The implementation correctly applies the_taggedfilter only whenfull_scanisFalse, allowing for both incremental (default) and comprehensive duplicate checking. The parameter documentation is clear and the default value is appropriate.secator/celery.py (3)
243-243: LGTM! Proper gating of MongoDB operations to worker processes.The addition of
IN_CELERY_WORKER_PROCESSchecks ensures MongoDB operations only execute within appropriate Celery worker contexts. This prevents potential issues with MongoDB hooks running in non-worker environments and maintains consistency with the distributed processing model.
275-275: LGTM! Consistent worker process gating.The MongoDB result retrieval and UUID extraction logic is correctly gated to only run within Celery worker processes, maintaining consistency with the other MongoDB operations in this file.
Also applies to: 281-281
303-303: LGTM! Consistent MongoDB operation gating.The MongoDB operations in the completion handler are properly gated to worker processes, completing the consistent pattern established throughout the file.
Also applies to: 309-309
🤖 I have created a release *beep* *boop* --- ## [0.16.5](v0.16.4...v0.16.5) (2025-06-25) ### Bug Fixes * **celery:** pass mongodb uuids when enabled ([#701](#701)) ([64b43e8](64b43e8)) * **ci:** add apt update in ci ([261d1e8](261d1e8)) * prod optimizations (GCS ValueError, dynamic profile for fuzzers with big wordlists) ([#707](#707)) ([bcd6024](bcd6024)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Summary by CodeRabbit