Project Manager support for Workflows #4338
Comments
|
I'm not sure if a I also kind of don't think a form is the right UI - likely some sort of drag & drop VISIO type worfkflow builder is what I'd be looking for. |
|
Having a Drag and Drop kinda UI would have been awesome, but I think of another way of doing it which is a little static -- We can create 4 launcher dropdowns (static part):
Instead of having a Launch button below each script, like we have now; there will be a |
|
I don't think we should limit it into this specific 3 step workflow. If we support workflows, they should be generic so we can support any number of steps. We shouldn't pigeon hole ourselves into this specific worfkflow. |
|
I believe generic workflows will be too much. Instead, users can just run Pegasus or something similar on a node to achieve this. Earlier idea was "All to All" scenario, where all the main scripts depend on all the pre-processing scripts and so on. What could be done extra here is -- "Draw lines between Launchers in Pre-processing to Main Script / Main-Script to Post Processing Launchers", therefore we change it to "Any to Any" dependency. It could be achieved easily with HTML Canvas, and in the backend we can send the <Launcher_1, Launcher_2> edges to describe these dependencies. This could be our sweet spot instead of
8000
a full-fledged workflow with drag-and-drop. Also, implementation is easier using layering (submit all Preprocessing launchers -> then Main Script -> finally Post Processing), just need to attach the job-id (via |
I don't think we should box ourselves in to this specific workflow. What if they need 2 steps for pre-processing? What about 10? I myself maintain a chain that has about 10 steps. In that workflow, I'm not even sure what would be defined as 'pre' and 'post'. Which is to say - I've got something I need to do, but this is now forcing me to do it under a specific paradigm, not mine, not the one I've already got.
I think the UI is separate issue. I know the UI is going to be a big undertaking - but I just know users don't want to be forced into a specific workflow. I know for sure they will request for this to be expanded and I don't want to be in a spot where we either can't do it or it's very hard to refactor it all. |
|
This is the design documentation for the Workflow feature. Please post your review comments. |
This Epic is to track a new feature in PM that will allow users to specify Pre-processing, Post-processing scripts. Workflows will be added inside Launchers via
SmartAttributes.Add new
SmarAttributethat will be similar toauto_environment_variableto allow user to choosekey(ie, what type of script it is --after, afterok, afternook) and invalueuser has to select any script from the dropdown list.In
Launchers.rb:submit(), add the driver code to make it work. It will haveif-elsestatement to submit preprocessing scripts before then gathering the job-ids and appending those to the main script. Post-processing scripts will include the job-id of the main script.Unlike other existing SmartAttributes, this
SmartAttributecan be added more than once. Allowing it to be populated once for now and then in a separate PR will make it appendable multiple times.Other attributes, such as
cores / nodeswill all be part of the main script. Newly added scripts can either follow these or not (in that case, the user needs to specify the resources inside the script). Add a checkbox in theSmartAttributeto describe this functionality.Handle errors from all kinds of scripts that are being executed and populate them nicely on the UI.
OodCore::Job::Adaptors::submit()already has functionality to specify job-ids of the scripts that the current job is dependent upon, thus we don't have to make any changes there to fulfill this feature.The text was updated successfully, but these errors were encountered: