Description
Very cool project! I have a bunch of questions that I figured I'll put into a single issue if that's okay:
-
Is this just a "demo" type project, similar to Teachable Machine, or is it something that Google is looking to expand into a "proper" platform - kind of like a visual Colab/JupyterLab, or something? (Hoping it's the latter!)
-
Can you add other popular model runners like ORT Web, or do you only intend to support Google formats?
-
Is it just for inference, or is training/fine-tuning something that might be on the horizon?
-
I wasn't able to find a generic "execute custom JS code" node? I found a "code editor" node, and a "shader processing" node that the custom code can feed into, but no custom JS code node that allows you to define some inputs, and some outputs, and some processing code. This code can of course be run inside a web worker (or sandboxed iframe) for sandboxing, so it's safe to allow adding custom code like this at "runtime" as opposed to requiring that all nodes are pre-defined.
-
Related to points 3 & 4, maybe a pipe dream, but it would be really cool if training/fine-tuning were supported, and the graph contained within JAX/TF/etc. models could be displayed and visually edited - I made a demo editor here using ONNXScript and Pyodide (browser-based Python runtime) which illustrates what I mean: https://github.com/josephrocca/onnxscript-editor But the editing would be two-directional. I.e. edit the JAX code of a visual node would propagate those changes back to the source code, and vice versa.
- (Edit) Related: I just noticed that MediaPipe now has a "Model Maker" tool that is focused on transfer learning.