[Feature]: Plot training and validation losses overlaid on same chart automatically by writing code (not manually in web interface) #7887
Comments
|
Hello @aj96 , thank you for writing in and for the very detailed information. Could you please try the following if these will help your case: Workspaces API and more examples here: https://github.com/wandb/wandb-workspaces |
|
@JoanaMarieL I did Also, please correct me if I'm wrong, but the examples you provided only allow me to write via code how I want the sections and panels to be, but none of the examples show how to write code to plot multiple y values on the same panel, which is what is necessary for overlaying two plots on the same chart. The examples shown with the pictures show making a section with multiple panels, where each panel is its own separate x vs y plot, so even if I stopped getting errors, I don't think these examples will work. To be clear, I would like to plot two y's against the same x on the same chart/panel by writing code and I would like to update the log in real-time each time I log each step, just like in tensorboard. |
|
Hi @aj96 , "I did pip install wandb-workspaces and then tried making my own custom workspace with a custom section and a custom panel containing a LinePlot following the example in the readme here: https://github.com/wandb/wandb-workspaces And for plotting multiple line plot, you may refer to these:
|
|
Hi @aj96 , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know! |
Description
In Tensorboard, when you have separate tensorboard writers, one for train and another for validation, it automatically overlays scalars with the same name on the same chart (see picture attached). This is useful because it makes it easier to spot overfitting. I have tried looking up how to do this using wandb, but it seems you can only manually do this in the web interface. This is cumbersome because I do not want to have to manually click and type multiple times every time I run a new experiment to overlay my train and val losses, and most times there are multiple losses plotted as I plot each subloss that contributes to the overall summed total loss. So if I have 5 sublosses and a total loss, I would have to manually overlay the losses 6 times each time I run an experiment. This is too cumbersome, and this is core functionality that any ML engineer would want.
If there is already a solution to automatically overlay scalars on the same chart by configuring it to do so in the python code, please tell me. But I looked at this open issue that is a year and a half old which asked for this functionality (link: #4449), and the wandb reply was to provide a link to the documentation (https://docs.wandb.ai/guides/app/features/panels/line-plot#compare-two-metrics-on-one-chart) on how to manually use the web interface to overlay two metrics, which is not what was asked for.
Suggested Solution
Possible solutions would be to have the ability to define multiple writers like in tensorboard or to add a group argument to wandb.log() method so we can specify which scalars should be grouped onto the same chart.
Alternatives
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered: