Releases: riga/law
Releases · riga/law
v0.1.20
Breaking changes
Features & improvements
- Accelerate local workflow completion checks. (0f619e5)
- Add optional_existing attribute to target collections. (25b489b, 3d38873)
- Improve local root lookup for mirrored targets. (6b1143c, 4048d2b)
- Add wrapper task output filtering. (1e8cf64)
- Move workflow output caching to base workflow. (920401f)
- Cache and bookkeep existing branches in remote workflow. (37592c9)
- Refactor target collection state checks. (bca2d55, 6f73b18)
- Speedup NestedSiblinFileCollection when possible (afef50f)
- Improve parameter description update on copy. (b5be40c)
- Add --clear-logs option to remote workflows. (540f48f)
- Cleanup log handling in remote workflows. (530e26a)
- Warn on bad rendering options in JobInputFile. (921efca)
- Allow skipping completion. (2fe969d)
- Fix child target creation in MirroredDirectoryTarget. (0aae7e6)
- Allow defining eager requirements for dynamic workflow conditions. (7cc152c)
Fixes
- Recompute workflow resources until branch map is cached. (e323f29)
- Fix basename comparison for sibling file collections. (d918c03)
- Consistent workflow attributes, avoid falling back to base remote workflow. (0156263)
- Fix open and localize of mirrored targets. (fb914c4, 0f5e25c)
contrib packages
- [slurm] Fix log output handling. (8aa3ffe)
- [htcondor] Properly use transfer_output_remap. (3a8a165)
- [htcondor, slurm, lsf] Separate log and output directories in htcondor, lsf and slurm workflows. (2c671e0)
- [htcondor] Fix submission exception handling, close #194. (9ac6dbc)
- [pyarrow, roow] Improve path handling in pyarrow and root contrib packages. (bf19a78)
v0.1.19
Breaking changes
- Rename abs_dirname -> absdirname in file targets. (5593f18)
Features & improvements
- Add 'law luigid' sub cli. (bd0cd88)
- Adapt live task resources in remote workflow. (88c4215)
- Unified permission support for target formatter dump. (976d53b)
- Automatic pattern detection in multi_match. (673c2ac)
- Add mirrored targets. (f4a0203)
- Human readable temporary job submission directories (4a05454, 93574bd)
- Improve node naming in ForestMerge. (7ff7513)
- Remove potentially expensive check in ForestMerge. (f295d05)
- Add keep_missing flag to mask_struct util. (e22d2bd)
- Additional early stopping in remote workflow. (245e0a7)
- Improve remote fs defaults. (893eb5f)
- Unify sandbox variable parsing, allow pre and post setup commands. (44b34c5)
- Improve workflow and sandbox proxy handling. (1ac4f1a)
- Add option to eval workflow condition as workflow only. (bb7b363)
Fixes
- Handle file collisions for grouped job submissions, fix #183. (d60ea71)
- Fix mirrored target checks in collections. (87b4017)
- Fix root path check in mirrored targets. (ff594e7)
- Fix TargetCollection directory check. (291723b)
- Fix empty workflows. (f57a677)
- Fix target makedirs. (2f4e664)
- Fix indentation in print-status output. (ea3585c)
- Fix workflow resources for branches. (a179eae)
- Fix workflow attribute fallbacks. (83d9023)
- Fix deferred config resolution. (4389a86)
- Fix TypeError causing function call (#175, 1848c57)
- Remove redundant global args from remote commands. (ab5bb48)
- Drop effective_workflow in remote command. (62d4576)
- Fix local target copy with non-root fs. (e02edde)
contrib packages
- [htcondor] Grouped submission using job clusters (682c983, 34ba133, c475ebb)
- [htcondor] Allow -spool jobs. (e9408bf)
- [htcondor] Fix htcondor log files. (08804d1)
- [pandas] Add pandas contrib with file formatter (8ac97e0, ca4e237, @Nollde)
- [singularity] Fix law sw forwarding into singularity sandboxes (#190, @EthanMarx, e97ab26)
- [cms] Fix crab wrapper file incrementation. (af59c5e)
- [cms] Add option to pass args to custom cmssw setup. (64dbe8d)
- [matplotlib] Set default file permissions in matplotlib formatter (caacf90)
- [slurm] Change tmp dir usage (#184, 6c714a8)
Contributors
Nollde and EthanMarx
Assets 2
v0.1.18
Breaking changes
None
Features & improvements
- Refactor dynamic workflow condition. (95bedec, 306046d)
- Place remote workflow run() into optional context. (2146c7a)
- Add local_workflow_pre_run hook to local workflow proxy. (bda49ee)
- Add mypy to future-proof images. (05292ba, fbaaa5b)
- Change docs requirements. (84c0a0c)
- Add range expansion to range parameters. (5d0f2ed)
Fixes
contrib packages
None
v0.1.17
v0.1.16
v0.1.15
Breaking changes
None
Features & improvements
- Split image workflows. (65e6b32)
- Streamline docker image builds, add alma9 images. (#169) (e4afe3a)
- Add custom parameter base class. (e16012b)
- Preserve job data of skipped jobs. (57b5328)
Fixes
- Fix parameter encoding for law_run(). (4a0372b)
- Fix localize decorator. (9c2df74)
- Update CmdlineParser patch. (6d39db0)
contrib packages
- [htcondor] Improve error extraction from query response. (4d843dd)
v0.1.14
Breaking changes
None
Features & improvements
- Add dynamic_workflow_condition. (5173f91)
- Handle potential pathlib.Path's throughout law, close #167. (dca607b)
- Add 'law quickstart' cli, close #164. (ce82c5f)
- Improve config handling in remote file interface. (6fb41c1)
- Improve config variable expansion in sandboxing. (e2fc8fc)
- Simplify log file handling and extra data for remote jobs. (e1ccb60)
- Improve sandbox key parsing. (e300819)
- Print contrib package location via 'law location'. (bae1576)
Fixes
- Explicit sandbox config section names, close #165. (52fe3f9)
- Fix sandbox stage-in vs. localization interplaye, fix #168. (d301b94)
contrib packages
v0.1.13
Breaking changes
- Combine {cached_,}workflow_property. (d4b12ff)
- Rename cfg.get_expanded_boolean -> cfg.get_expanded_bool. (4c3a503)
Features and improvements
Larger enhancements
- Crab job submission and workflows (c7f39b3) 🎉
- Workflow parameters (#159) (cec32d9)
- Pass through requested workflow (#155) (f54534a)
- Add support for group jobs. (9cd02d9, 8f2d086, f8ce46c) 🎉
- Deferred config values. (18e32ed)
Workflow and job handling
- Add mechanism to register contrib install scripts. (c718d67)
- Improve separating of workflow and branch attributes. (7d4f679)
- Improve parameter parsing. (d62f1f9)
- Add --assistant to global cli params. (3e5c233)
- Add options to read stdin in interruptable_popen. (8e281ff)
- Allow poll_callback return value to stop polling loop. (574ed58)
- Add additional hook for adding workflow control output postfixes. (aaaa6f6)
- Allow delay before job completeness checks in remote workflow. (b18d4ce)
- Add switch to check task completion after remote jobs finsihed. (e75728b)
General improvements
- Improve generic job script. (8bd879c)
- Mark job section in crab job wrapper. (a2d1d05)
- Add hook to setup job managers. (55515ba)
- Update job error summary grouping. (db1e74c)
- Improve error logs during job status checks in remote workflow. (4caf842)
- Make sandbox executable configurable. (a0c559c)
- Add make_set to utils. (c414b45)
- Add convert_types to util.mask_struct. (71f3796)
- Add classmethod checking to util. (e56e6cf)
- Support brace expansion in module list for 'law index'. (43fee29)
- Improve interactive task status formatting. (91de7b6)
- Better "optional" handling in collections (#149) (bdb908d, 752b1a5)
- Streamline sandbox env caching and activity status. (cca3e22)
- Improve control over repr parameters when empty. (991208f)
- Refactor sandbox target stage-in/-out. (a87377d)
- Allow single value parsing in CSVParameter and MultiCSVParameter. (cc2aa0a)
Improvements in contrib packages
- Consistent naming of utils in arc contrib. (d56380d)
- Refactor voms proxy renewal and myproxy delegation. (f9b1d7d)
- Add myproxy delegation tools to wlcg and cms contribs. (d696c93)
- Add rlocks to guard checksum creation in contrib bundle tasks. (04fedbf)
- Catch xrootd listdir error in gfal interface. (f823440)
Fixes
- Backwards compatible lookup of log file in job data. (d067e02)
- Fix username in myproxy_info util in wlcg contrib. (2ae185f)
- Fix is_classmethod util, close #162. (02a71d4)
- Fix disabling of instance caching for proxy tasks. (2751204)
- Fix gfal_transfer_overwrite (#160) (6199d42)
- Consistent usage of destination_info. (02d504c)
- Patch luigi run result in sandboxes. (6c02198)
- Fix inheritance issue in contrib ForestMerge task. (a892b9f)
- Fix entry point in example image, close #154. (e470fd3)
- Fix deps in docker and singularity sandboxes. (3b18aff)
- Fix readable_popen util. (e9100d4)
- Fix interactive messages. (cf1dfa3)
- Fix slurm job status queries. (22ced6c)
- Fix bookkeeping of finished jobs in remote workflow. (eee5d26, 6d88cf4)
- Fix job file merging setting in htcondor workflow, fix job id assignment. (fb58ec4)
- Fix cached workflow forwarding (#147) (2f99321, f334bd3)
- Fix duration parsing. (09caa5d)
- Fix branch selection workflows. (098ec08)
Misc
v0.1.12
New features
- Add load and dump support for directory targets. (fa18d32)
- Add localization support for directory targets. (2108295)
- Add TFSavedModelFormatter to tensorflow contrib. (78e5ef7)
- Add DaskAwkwardFormatter to awkward contrib. (a36219a)
- Add notebook example. (e92fb58)
Improvements
- Forward some requests from remote to local fs. (585e3de)
- Refactor parquet file merging in pyarrow contrib. (f13d6a5)
- Improve %ilaw ipython magic. (e4a213a)
- Parse left-over string parameters. (b1abe37)
Breaking changes
None.