Open
Description
What happened + What you expected to happen
Use the same code snippet in #24116 but ensure to use default implementation of from ray.serve.drivers import DAGDriver
Also, run the test with parameterize, NOTE: running the test with use_build only as True or False will pass, but only fails when we run both.
def maybe_build(node: ClassNode, use_build: bool) -> Union[Application, ClassNode]:
if use_build:
return Application.from_dict(build_app(node).to_dict())
else:
return node
@pytest.mark.parametrize("use_build", [False, True])
def test_two_dags_shared_instance(serve_instance, use_build):
Observed issue
On the second run, the body of DAGDriver deployment turned to have a path of types.DAGDriver that cannot be initialized.
Workaround
Bringing my own implementation without subclassing anything works however, I suspect the issue is related to subclass of our default DAGDriver.
@serve.deployment
class DAGDriver:
def __init__(self, dag_handle):
self.dag_handle = dag_handle
async def predict(self, inp):
"""Perform inference directly without HTTP."""
return await self.dag_handle.remote(inp)
async def __call__(self, request: starlette.requests.Request):
"""HTTP endpoint of the DAG."""
input_data = await request.json()
return await self.predict(input_data)
Versions / Dependencies
.
Reproduction script
.
Issue Severity
Medium: It is a significant difficulty but I can work around it.