Closed
Description
What happened + What you expected to happen
Getting the following AttributeError when attempting to use serve with a kuberay cluster and FastAPI per the docs:
(ServeController pid=3863) INFO 2025-06-23 10:29:57,794 controller 3863 -- Deploying new version of Deployment(name='MyFastAPIDeployment', app='default') (initial target replicas: 1).
(ProxyActor pid=3927) INFO 2025-06-23 10:29:57,798 proxy 10.42.0.44 -- Got updated endpoints: {Deployment(name='MyFastAPIDeployment', app='default'): EndpointInfo(route='/hello', app_is_cross_language=False)}.
(ProxyActor pid=3927) INFO 2025-06-23 10:29:57,811 proxy 10.42.0.44 -- Started <ray.serve._private.router.SharedRouterLongPollClient object at 0x7f62a0726ec0>.
(ServeController pid=3863) INFO 2025-06-23 10:29:57,899 controller 3863 -- Adding 1 replica to Deployment(name='MyFastAPIDeployment', app='default').
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::SERVE_REPLICA::default#MyFastAPIDeployment#b13y75ge:ServeReplica:default:MyFastAPIDeployment.__init__() (pid=1611, ip=10.42.0.41, actor_id=0a93a43e2b80703667bbb6ea01000000, repr=<ray.serve._private.replica.ServeReplica:default:MyFastAPIDeployment object at 0x7efb962599c0>)
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) return self.__get_result()
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) raise self._exception
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) File "/usr/local/lib/python3.10/dist-packages/ray/serve/_private/replica.py", line 924, in __init__
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) deployment_def = cloudpickle.loads(serialized_deployment_def)
(ServeReplica:default:MyFastAPIDeployment pid=1611, ip=10.42.0.41) AttributeError: Can't get attribute 'Router.app' on <module 'starlette.routing' from '/usr/local/lib/python3.10/dist-packages/starlette/routing.py'>
Current version of ray is 0.46.0 and FastAPI is 0.115.2.
We have an existing application working with FastAPI at 0.108.0 but need to upgrade FastAPI due to another required dependency. I've tried upgrading Ray from 0.44.1 to 0.46.0 but neither seem to work with a version of FastAPI past 0.108.0.
Versions / Dependencies
ray: 0.46.0
FastAPI: 0.115.2
pydantic: 2.9.2
python: 3.10.12
Reproduction script
from fastapi import FastAPI
import ray
from ray import serve
ray.init(address='ray://ray-head:10001', log_to_driver=True)
app = FastAPI()
@serve.deployment
@serve.ingress(app)
class MyFastAPIDeployment:
@app.get("/")
def root(self):
return "Hello, world!"
@app.post("/{subpath}")
def root(self, subpath: str):
return f"Hello from {subpath}!"
serve.run(MyFastAPIDeployment.bind(), route_prefix="/hello")
Issue Severity
None