[async] Include prediction id upload request #1788
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
technillogue
reviewed
Jul 8, 2024
2da3de0 to
c337191
Compare
Something off with the go deps, I'll rebase and try again. |
|
@technillogue fix for CI is here #1797 |
c337191 to
62851fe
Compare
7d5160e to
5665632
Compare
5665632 to
c7fce45
Compare
|
@technillogue would you mind reviewing this when you have a moment. I'd like to republish real-esrgan with a cut of a new alpha release that supports the new uploader tomorrow if possible. |
aron
commented
Jul 16, 2024
| @@ -62,7 +72,7 @@ def webhook_headers() -> "dict[str, str]": | |||
|
|
|||
| async def on_request_trace_context_hook(request: httpx.Request) -> None: | |||
| ctx = current_trace_context() or {} | |||
| request.headers.update(ctx) | |||
| request.headers.update(cast(Mapping[str, str], ctx)) | |||
There was a problem hiding this comment.
Fixes the linting issue on async.
aron
commented
Jul 16, 2024
aron
commented
Jul 16, 2024
Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service.
c7fce45 to
97ce06c
Compare
aron
commented
Jul 16, 2024
python/cog/server/clients.py
Outdated
Comment on lines
202
to
212
| class ChunkFileReader: | ||
| async def __aiter__(self) -> AsyncIterator[bytes]: | ||
| fh.seek(0) | ||
| while 1: | ||
| chunk = fh.read(1024 * 1024) | ||
| if isinstance(chunk, str): | ||
| chunk = chunk.encode("utf-8") | ||
| if not chunk: | ||
| log.info("finished reading file") | ||
| break | ||
| yield chunk |
There was a problem hiding this comment.
The previous implementation didn't reset the file handle on subsequent calls which resulted in an httpx.StreamConsumed exception. I added a test and implemented this fix which will ensure a new iterator is created for each request.
There was a problem hiding this comment.
FWIW I believe the function-based implementation would work with a fh.seek(0) at the top of the function, but this is great/cleaner
mattt
reviewed
Jul 17, 2024
|
@mattt thanks! Looks great! |
technillogue
pushed a commit
that referenced
this pull request
Jul 18, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
technillogue
pushed a commit
that referenced
this pull request
Jul 18, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
mattt
added a commit
that referenced
this pull request
Jul 18, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
mattt
added a commit
that referenced
this pull request
Jul 19, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
technillogue
pushed a commit
that referenced
this pull request
Jul 24, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
technillogue
pushed a commit
that referenced
this pull request
Jul 25, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
technillogue
pushed a commit
that referenced
this pull request
Jul 25, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the as 6D40 set in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
technillogue
pushed a commit
that referenced
this pull request
Jul 26, 2024
* Cast TraceContext into Mapping[str, str] to fix linter * Include prediction id upload request Based on #1667 This PR introduces two small changes to the file upload interface. 1. We now allow downstream services to include the destination of the asset in a `Location` header, rather than assuming that it's the same as the final upload url (either the one passed via `--upload-url` or the result of a 307 redirect response. 2. We now include the `X-Prediction-Id` header in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service. * Extract ChunkFileReader into top-level class --------- Co-authored-by: Mattt Zmuda <mattt@replicate.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on #1667
This PR introduces two small changes to the file upload interface.
We now allow downstream services to include the destination of the asset in a
Locationheader, rather than assuming that it's the same as the final upload url (either the one passed via--upload-urlor the result of a 307 redirect response.We now include the
X-Prediction-Idheader in upload request, this allows the downstream client to potentially do configuration/routing based on the prediction ID. This ID should be considered unsafe and needs to be validated by the downstream service.