8000 [dashboard] Wonky GPU display · Issue #14664 · ray-project/ray · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
[dashboard] Wonky GPU display #14664
Open
@DmitriGekhtman

Description

@DmitriGekhtman

What is the problem?

Two bugs -

(1) The GPU field for a n-gpu node looks like this -- '[0]: N/A [1]: N/A [2]: N/A ... [n-1]: N/A'
which isn't too informative.
Hovering mouse over each index shows a tooltip with the type of the GPU.

(2) If you launch a multi-GPU head (e.g. g4dn.12xlarge) and a single-GPU worker (e.g. p2.xlarge),
the info rows for the head and worker may swap with each other every few seconds, which makes it hard to read the dashboard.
I saw this when launching on AWS and K8s a few hours ago.
The very last time I tried this a few minutes ago, this bug didn't appear.

Screen Shot 2021-03-13 at 5 17 46 AM

Ray version and other system information (Python version, TensorFlow version, OS):
cluster launcher 2.0.0dev, rayproject/ray:nightly-gpu docker image

Reproduction (REQUIRED)

Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments):

If the code snippet cannot be run by itself, the issue will be closed with "needs-repro-script".

  • I have verified my script runs in a clean environment and reproduces the issue.
  • I have verified the issue also occurs with the latest wheels.

Metadata

Metadata

Labels

P1Issue that should be fixed within a few weeksdashboardIssues specific to the Ray DashboardenhancementRequest for new feature and/or capabilityobservabilityIssues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    0