8000 Reduce part size limit, add troubleshooting info for bad gateway error by muhsinking · Pull Request #299 · runpod/docs · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Reduce part size limit, add troubleshooting info for bad gateway error #299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 23, 2025
Merged

Reduce part size limit, add troubleshooting info for bad gateway error #299

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c4c485c
Add troubleshooting section
muhsinking Jun 20, 2025
95a75cf
Remove troubleshooting section
muhsinking Jun 20, 2025
e74128d
Move tip
muhsinking Jun 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions serverless/storage/s3-api.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,17 @@ aws s3 rm --region [DATACENTER] \
s3://[NETWORK_VOLUME_ID]/remote-file.txt
```

<Tip>

If you encounter a 502 "bad gateway" error during file transfer, try increasing `AWS_MAX_ATTEMPTS` to 10 or more:

```bash
export AWS_RETRY_MODE=standard
export AWS_MAX_ATTEMPTS=10
```

</Tip>

### Sync directories

This command syncs a local directory (source) to a network volume directory (destination):
Expand Down Expand Up @@ -314,9 +325,9 @@ Large file handling is supported through multipart uploads, allowing you to tran
- **Storage capacity**: Network volumes have a fixed storage capacity, unlike the virtually unlimited storage of standard S3 buckets. The `CopyObject` and `UploadPart` actions do not check for available free space beforehand and may fail if the volume runs out of space. This behavior is similar to applying a size quota in S3.
- **Maximum file size:** 4TB (the maximum size of a network volume).
- **Multipart uploads**:
- Parts from multipart uploads are stored on disk until either `CompleteMultipartUpload` or `AbortMultipartUpload` is called.
- The S3-compatible API enforces the 5GB maximum single file part upload size, but not the 5TB maximum file size.
- The S3-compatible API enforces a 500MB maximum on upload part size.
- The 5MB minimum part size for multipart uploads is not enforced.
- Parts from multipart uploads are stored on disk until either `CompleteMultipartUpload` or `AbortMultipartUpload` is called.
- **Object names**: Unlike traditional S3 key-value stores, object names in the Runpod S3-compatible API correspond to actual file paths on your network volume. Object names containing special characters (e.g., `#`) may need to be URL encoded to ensure proper processing.
- **Time synchronization**: Requests that are out of time sync by 1 hour will be rejected. This is more lenient than the 15-minute window specified by the AWS SigV4 authentication specification.
- **Unsupported S3 features:**
Expand Down
0