feat: cli multipart uploads #33546
feat: cli multipart uploads #33546
Conversation
There was a problem hiding this comment.
PR Summary
Added multipart upload support to the PostHog CLI to resolve source map upload failures due to 21MB file size limits (HTTP 413 errors).
- Modified
cli/Cargo.tomlto enable reqwest's multipart feature for chunked file uploads - Updated
cli/src/commands/sourcemap/upload.rsto implement multipart form functionality with amultipart=trueparameter - Added multipart handling in
posthog/api/error_tracking.pywhile maintaining backwards compatibility - Added TODOs around release object handling that require discussion
- Implementation needs additional error handling around multipart responses
3 files reviewed, 1 comment
Edit PR Review Bot Settings | Greptile
| let mut params: Vec<(&'static str, &str)> = | ||
| vec![("chunk_id", &upload.chunk_id), ("multipart", "true")]; |
There was a problem hiding this comment.
style: Vec type could be simplified to just Vec<(&str, &str)> since 'static isn't needed here - the strings don't need static lifetime
| let mut params: Vec<(&'static str, &str)> = | |
| vec![("chunk_id", &upload.chunk_id), ("multipart", "true")]; | |
| let mut params: Vec<(&str, &str)> = | |
| vec![("chunk_id", &upload.chunk_id), ("multipart", "true")]; |
cli/Cargo.toml
Outdated
There was a problem hiding this comment.
Wasn't really sure based on the CONTRIBUTING guide if I was supposed to bump the version in this PR or create a new one after this merges
There was a problem hiding this comment.
Either works - the release flow is actually triggered by the tag push, not the version bump, so you can do whichever you like
Problem
Users are complaining that their source map uploads are hitting 413 limits. We have a file limit of 21MB when not using multipart file uploads
Changes
Changes
Possibly an easier way to do this but wanted to be certain that the same content was being uploaded, so...
Picked a random symbol set from S3 which had already been injected with a chunk id (given that is randomly generated)
Downloaded that file and got the contents of the source & map
Created the two files in a
./chunksdirectory and ranposthog-cli sourcemap upload --directory ./chunksVerified that the
content_hashof the created symbol set record on my local machine matched that of the remote one e.g. multipart upload worked!