Open
Description
TrainableUtil.get_checkpoints_paths does not recognize the .tune_metadata files due to the fact that glob.glob does not match files starting with ., see the end of the glob documentation.
ray/python/ray/tune/utils/trainable.py
Lines 151 to 156 in e0573df
Possible solution: It seems that only .tune_metadata files are used throughout the library, so one could just omit glob.glob, e.g.
metadata_file = os.path.join(chkpt_dir, ".tune_metadata")
if not os.path.isfile(metadata_file):
raise ValueError(
"{} has no tune_metadata.".format(chkpt_dir))
Ray version: 1.0.1.post1
Python version: 3.7.3 (issue should be independent of the python version)