Description
What happened + What you expected to happen
Our Request flow:
Client calls our ingress app (which is a ray serve app wrapped in a FastAPI ingress) which then calls another serve app using handle.remote
Bug:
When a client is canceling the request our ingress app (which is a ray serve app) is seeing the StopAsyncIteration error thrown by ray serve handler code
Tried to reproduce locally but haven't be successful
I think we should still have some exception handling around the piece of code that throws the error
Strack Trace:
File "/app/virtualenv/lib/python3.10/site-packages/ray/serve/handle.py", line 404, in await result = yield from replica_result.get_async().await() File "/app/virtualenv/lib/python3.10/site-packages/ray/serve/_private/replica_result.py", line 87, in async_wrapper return await f(self, *args, **kwargs) File "/app/virtualenv/lib/python3.10/site-packages/ray/serve/_private/replica_result.py", line 117, in get_async return await (await self.to_object_ref_async()) File "/app/virtualenv/lib/python3.10/site-packages/ray/serve/_private/replica_result.py", line 179, in to_object_ref_async self._obj_ref = await self._obj_ref_gen.anext() File "python/ray/_raylet.pyx", line 343, in anext File "python/ray/_raylet.pyx", line 547, in _next_async StopAsyncIteration
Versions / Dependencies
ray[serve]==2.42.1
python==3.10.6
Reproduction script
Tried to reproduce locally but haven't be successful
I think we should still have some exception handling around the piece of code that throws the error
Issue Severity
Low: It annoys or frustrates me.