Automatically clean up old ELT logs #2420
Comments
|
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the |
|
Still relevant |
|
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the |
|
Proposing here that this should be $ meltano logs clear |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2465
Originally created by @ivanovzlatan2 on 2020-12-01 17:52:27
Problem to solve
Meltano generates log files in the following directories:
.../.meltano/run/elt/...
.../.meltano/run/airflow/logs
The log files are taking up a lot of space, that's why it would be good to have a solution of regular cleanup of old log files.
Target audience
Target audience is everyone that is using Meltano scheduler and has limited disk space.
Proposal
Introducing a custom config for Meltano automatic log cleanup and retaining with options for how many days to retain the log files (7d/30d/60d).
Current workaround
One can set up own cron job to automatically delete files with an old creation/modification date
Links / references
The text was updated successfully, but these errors were encountered: