8000 [ray|llm] ray lora DiskMultiplexConfig loss load from local path to disk_cache · Issue #53315 · ray-project/ray · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
[ray|llm] ray lora DiskMultiplexConfig loss load from local path to disk_cache #53315
New issue
New issue
Open
Open
[ray|llm] ray lora DiskMultiplexConfig loss load from local path to disk_cache#53315
Labels
enhancementRequest for new feature and/or capabilityfeature-requestllmserveRay Serve Related Issueusability
@ZHAOL0302

Description

@ZHAOL0302
ZHAOL0302
opened on May 26, 2025

Description

https://github.com/ray-project/ray/blob/master/python/ray/llm/_internal/serve/deployments/llm/multiplex/lora_model_loader.py#L63

DiskMultiplexConfig is retrieved from memory and will be downloaded remotely if there is no memory available. It does not support loading through local paths ？

Use case

ray==2.46.0
ray[llm]==2.46.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementRequest for new feature and/or capabilityfeature-requestllmserveRay Serve Related Issueusability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0