Description
I have encountered an issue where a file is pruned before all tasks that consume this file have been completed. This can be see in the diagram below. A Temp file, F1, will be create by task 1 on worker 1. This file is used as an input file for tasks 2 and 3 to create files and F2 and F3 respectively. Before task 3 completes, the manager directs all workers which have F1 to unlink the file. This causes task 3 to fail and subsequent recovery tasks to be scheduled. This only occurs while pruning is activated. The workflow completes as normal when the pruning is not activated.
Some hypotheses discussed:
I can confirm that the task is not a recovery task.
Need to confirm if somehow these temp files are marked uncacheable
Need to confirm if somehow pruning is happening too early