8000 [feature] affected or status (staged) files token · Issue #1865 · moonrepo/moon · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[feature] affected or status (staged) files token #1865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
muuvmuuv opened this issue Mar 11, 2025 · 13 comments
Open

[feature] affected or status (staged) files token #1865

muuvmuuv opened this issue Mar 11, 2025 · 13 comments
8000
Labels
enhancement New feature or request

Comments

@muuvmuuv
Copy link

Is your feature request related to a problem? Please describe.

I do know my dev-workflow is kind of wild here but maybe some can relate.

While I have set up hooks I noticed that affected and status=staged have no possibility to just lint these affected files. In other words I cannot develop 4 features simultaneously when some files create errors that I have not staged yet but came across and error that I was able to fix on-the-fly in the same application.

Describe the solution you'd like

A token or something like @affected with a fallback option like @affected(@in(0)). That will just append all affected files or fall back to what is described.

Describe alternatives you've considered

vcs:
  hooks:
    pre-commit:
      - 'moon run :lint --affected --status=staged'
      - 'gitleaks git --pre-commit --redact --staged --verbose'
# ....
  run-eslint:
    command:
      - 'eslint'
      - '--cache'
      - if:
          - '@status(staged)'
        else:
          - '.'
    inputs:
      - '@group(app)'
      - '@group(sources)'
      - '@group(tests)'
      - '**/eslint.config.mjs'

Additional context

@muuvmuuv muuvmuuv added the enhancement New feature or request label Mar 11, 2025
@milesj
Copy link
Collaborator
milesj commented Mar 11, 2025

Not sure if this will solve this problem exactly, but have you tried this task option? https://moonrepo.dev/docs/config/project#affectedfiles

@muuvmuuv
Copy link
Author

How should that actually work, when I enable this option it will also pass other files to e.g. ESLint which will prints a lot of warnings for JSON files, but these are relevant for the group "app" which I wanted to re-use for eslint. I might have to be more explicit with my groups then.

@muuvmuuv
Copy link
Author

Hm, no this is not what I was looking for, but maybe I just need some help settings this up.

So this is for ex 8000 ample for stylelint task:

  run-stylelint:
    command:
      - 'stylelint'
      - '--ignore-path'
      - '@in(0)'
      - '--ignore-path'
      - '@in(1)'
      - '--allow-empty-input'
      - '--cache'
    options:
      affectedFiles: true
    inputs:
      - '/.gitignore'
      - '/.stylelintignore'
      - '**/.stylelintrc.json'
      - '@group(styles)'

Stylelint will exit with 1 if it cannot parse a file. So running moon :run-stylelint will fail due to some affected file are HTML or JSON files.

I thought inputs would also "filter" affected files, or did I missunderstood that?

@muuvmuuv
Copy link
Author

Oh, ok, got it now I guess. Stylelint is special. I had to explicitly ignore these kind of files because implicit inputs were passed to it when using it from the cli not the hook. Now it works as far as I can see.

  run-eslint:
    command:
      - 'eslint'
      - '--cache'
    options:
      affectedFiles: true
    inputs:
      - '**/eslint.config.mjs'
      - '@group(sources)'


  run-stylelint:
    command:
      - 'stylelint'
      - '--cache'
      - '--ignore-path'
      - '@in(0)'
      - '--ignore-path'
      - '@in(1)'
      - '--allow-empty-input'
    options:
      affectedFiles: true
      affectedPassInputs: true
    inputs:
      - '/.gitignore'
      - '/.stylelintignore'
      - '**/.stylelintrc.json'
      - '@group(styles)'

*** Insert "What a Week, Huh?" for "Web programming" ***

@muuvmuuv
Copy link
Author

No, now CI fails, because Moon seems to also add deleted files to the list of affected files.

   xxx:run-prettier | [error] No files matching the pattern were found: "./src/app/components/planning-group/README.md".

I am also seeing it passing files not actually in "inputs" (see config above):

 app-i18n:run-eslint | /Users/marvin/Developer/xxx/libs/app-i18n/moon.yml
 app-i18n:run-eslint |   0:0  warning  File ignored because no matching configuration was supplied

@milesj
Copy link
Collaborator
milesj commented Mar 13, 2025

That's unfortunate. Let me think about a way to handle this.

@muuvmuuv
Copy link
Author

I think "inputs" are not the way to handle actual "inputs" for a program, these are 2 different things, one that says "this tool should run again" and "this tool has these inputs". I might be wrong, but from my persepctive this seems more correct.

So MAYBE it would be more appropriate to allow @group(sources) in command and then apply the affected files appon it other then using the input as the filter for affected.

@milesj
Copy link
Collaborator
milesj commented Mar 14, 2025

@muuvmuuv You can use @group(sources), just place it in args.

@muuvmuuv
Copy link
Author

Isnt that just a little bit of a workaround

@milesj
Copy link
Collaborator
milesj commented Mar 14, 2025

Just a temporary solution. You could create a file group that only has files that are valid for the specific command (eslint, etc)?

@muuvmuuv
Copy link
8000
Author

Yeah that was what I did before and then I tried "affected" so that it only lint affected files on git hooks but both together does not really work like I expected.

@muuvmuuv
Copy link
Author

It seems to somewhat work, all warnings about files not beeing processed by tool X can be ignored (could not try with args but guess this will solve that) but that it also passes deleted files makes it always fail in CI.

Truncated trace logs:
; MOON_LOG=trace moon ci app-storage:lint --cache=write --base 82b41789f2ee1869fd16ca6b33856757d660afe5

[DEBUG 17:30:58.527] moon_cache::state_engine  Creating states engine  states_dir="/Users/PATH/.moon/cache/states"
[TRACE 17:30:58.527] starbase_utils::fs_lock  Locking file  file="/Users/PATH/.moon/cache/locks/proto-install.lock"
[TRACE 17:30:58.527] starbase_utils::fs  Creating file without truncating  file="/Users/PATH/.moon/cache/locks/proto-install.lock"
[DEBUG 17:30:58.526] moon_process::signal::unix  Listening for SIGINT, SIGQUIT, and SIGTERM signals
[TRACE 17:30:58.527] starbase_utils::fs_lock  Waiting to acquire lock  lock="/Users/PATH/.moon/cache/locks/proto-install.lock"
[TRACE 17:30:58.527] starbase_utils::fs_lock  Acquired lock, writing PID  lock="/Users/PATH/.moon/cache/locks/proto-install.lock" pid=68145
[DEBUG 17:30:58.527] moon_app::systems::analyze  Checking if proto is installed  proto="/Users/marvin/.proto/tools/proto/0.47.7/proto"
[TRACE 17:30:58.527] starbase_utils::fs_lock  Unlocking path  path="/Users/PATH/.moon/cache/locks/proto-install.lock"
[TRACE 17:30:58.527] starbase_utils::fs  Removing file  file="/Users/PATH/.moon/cache/locks/proto-install.lock"
[DEBUG 17:30:58.527] moon_app::systems::analyze  Registering platforms based on toolchain configuration  platforms=[Node]
[DEBUG 17:30:58.527] moon_plugin::plugin_registry  Creating plugin registry  plugin="toolchain"
[TRACE 17:30:58.527] warpgate::loader  Creating plugin loader  cache_dir="/Users/marvin/.moon/plugins/toolchains"
[DEBUG 17:30:58.527] moon_toolchain_plugin::toolchain_registry  Loading all toolchain plugins
[DEBUG 17:30:58.528] moon_plugin::plugin_registry  Attempting to load and register plugin  plugin="toolchain" id="typescript"
[TRACE 17:30:58.528] warpgate::loader  Loading plugin typescript  id="typescript" locator="https://github.com/moonrepo/plugins/releases/download/typescript_toolchain-v0.1.3/typescript_toolchain.wasm"

[TRACE 17:30:59.170] moon_process::exec_command  Ran command in 46.792791ms  pid=68215
[DEBUG 17:30:59.171] moon_app::queries::touched_files  Filtering based on touched status all
[DEBUG 17:30:59.171] moon_app::queries::touched_files  Found touched files  files=[s", "libs/app-storage/src/storage.service.ts", "libs/ui-forms/src/inputs/text/text.component.ts", "libs/app-core/tsconfig.json", "apps/us
[DEBUG 17:30:59.171] moon_action_graph::action_graph_builder  Building action graph

[DEBUG 17:31:00.109] moon_task_runner::command_executor  Running task (attempt 1 of 1)  task_target="app-storage:run-stylelint" command="stylelint"
[DEBUG 17:31:00.115] moon_process::exec_command  Running command stylelint --cache --ignore-path ../../.gitignore --ignore-path ../../.stylelintignore --allow-empty-input ./src/**/*.{css,scss} ./package.json  pid=68258 env={"MOON_PROJECT_ROOT": "/Users/PATH/libs/app-storage", "MOON_PROJECT_SNAPSHOT": "/Users/PATH/.moon/cache/states/app-storage/snapshot.json", "MOON_PROJECT_ID": "app-storage", "MOON_WORKSPACE_ROOT": "/Users/PATH", "MOON_PROJECT_SOURCE": "libs/app-storage", "MOON_TARGET": "app-storage:run-stylelint", "MOON_WORKING_DIR": "/Users/PATH", "MOON_CACHE_DIR": "/Users/PATH/.moon/cache"} cwd="/Users/PATH/libs/app-storage"
[DEBUG 17:31:00.116] moon_task_runner::command_executor  Running task (attempt 1 of 1)  task_target="app-storage:run-prettier" command="prettier"
[TRACE 17:31:00.121] moon_process::exec_command  Ran command in 14.878458ms  pid=68257
[DEBUG 17:31:00.121] moon_process::exec_command  Running command prettier --cache --ignore-path ../../.gitignore --ignore-path ../../.prettierignore --check ./package.json ./src/index.ts ./src/provide-storage.ts ./src/storage.service.ts ./src/with-sync-storage.ts  pid=68259 env={"MOON_PROJECT_ROOT": "/Users/PATH/libs/app-storage", "MOON_PROJECT_SNAPSHOT": "/Users/PATH/.moon/cache/states/app-storage/snapshot.json", "MOON_PROJECT_ID": "app-storage", "MOON_WORKSPACE_ROOT": "/Users/PATH", "MOON_PROJECT_SOURCE": "libs/app-storage", "MOON_TARGET": "app-storage:run-prettier", "MOON_WORKING_DIR": "/Users/PATH", "MOON_CACHE_DIR": "/Users/PATH/.moon/cache", "MOON_AFFECTED_FILES": "package.json,src/index.ts,src/provide-storage.ts,src/storage.service.ts,src/with-sync-storage.ts"} cwd="/Users/PATH/libs/app-storage"
[TRACE 17:31:00.121] moon_hash::hasher  Adding content to hasher  label="RunTask(app-storage:run-eslint)"
[TRACE 17:31:00.121] starbase_utils::json  Formatting JSON
[TRACE 17:31:00.121] starbase_utils::fs  Traversing upwards to find a file/root  file="pnpm-lock.yaml" dir="/Users/PATH/libs/app-storage"
[TRACE 17:31:00.121] starbase_utils::fs  Traversing upwards to find a file/root  file="pnpm-lock.yaml" dir="/Users/PATH/libs"
[TRACE 17:31:00.121] starbase_utils::fs  Traversing upwards to find a file/root  file="pnpm-lock.yaml" dir="/Users/PATH"
[TRACE 17:31:00.122] moon_hash::hasher  Adding content to hasher  label="RunTask(app-storage:run-eslint)"
[TRACE 17:31:00.122] starbase_utils::json  Formatting JSON

[DEBUG 17:31:00.125] moon_task_runner::command_executor  Running task (attempt 1 of 1)  task_target="app-storage:run-eslint" command="eslint"
[DEBUG 17:31:00.130] moon_process::exec_command  Running command eslint --cache ./src/**/*.{js,ts,html} ./package.json ./src/index.ts ./src/provide-storage.ts ./src/storage.service.ts ./src/with-sync-storage.ts  pid=68264 env={"MOON_PROJECT_ROOT": "/Users/PATH/libs/app-storage", "MOON_PROJECT_SNAPSHOT": "/Users/PATH/.moon/cache/states/app-storage/snapshot.json", "MOON_PROJECT_ID": "app-storage", "MOON_WORKSPACE_ROOT": "/Users/PATH", "MOON_PROJECT_SOURCE": "libs/app-storage", "MOON_TARGET": "app-storage:run-eslint", "MOON_WORKING_DIR": "/Users/PATH", "MOON_CACHE_DIR": "/Users/PATH/.moon/cache"} cwd="/Users/PATH/libs/app-storage"
▪▪▪▪ app-storage:run-stylelint (26d4192c)
▪▪▪▪ app-storage:run-prettier (37ad4395)
▪▪▪▪ app-storage:run-eslint (4ebacb29)
[TRACE 17:31:00.627] moon_process::exec_command  Ran command in 511.071125ms  pid=68259
[DEBUG 17:31:00.627] moon_task_runner::command_executor  Ran task, checking conditions  task_target="app-storage:run-prettier" command="prettier" exit_code=2
[DEBUG 17:31:00.627] moon_task_runner::command_executor  Task was unsuccessful, failing as we hit our max attempts  task_target="app-storage:run-prettier"
[TRACE 17:31:00.628] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-prettier/stderr.log"
[TRACE 17:31:00.628] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-prettier/stdout.log"
[DEBUG 17:31:00.628] moon_cache_item::cache_item  Writing cache item  cache="/Users/PATH/.moon/cache/states/app-storage/run-prettier/lastRun.json"
[TRACE 17:31:00.628] starbase_utils::json  Writing JSON file  file="/Users/PATH/.moon/cache/states/app-storage/run-prettier/lastRun.json"
[TRACE 17:31:00.628] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-prettier/lastRun.json"
[TRACE 17:31:00.628] moon_action_pipeline::action_runner  Failed to run action RunTask(app-storage:run-prettier)  index=16 status=Failed
app-storage:run-prettier | Checking formatting...
▪▪▪▪ app-storage:run-prettier (511ms, 37ad4395)
app-storage:run-prettier | [warn] src/index.ts
app-storage:run-prettier | [error] No files matching the pattern were found: "./src/storage.service.ts".
app-storage:run-prettier | [warn] Code style issues found in the above file. Run Prettier with --write to fix.
[TRACE 17:31:00.812] moon_process::exec_command  Ran command in 686.820542ms  pid=68264
[DEBUG 17:31:00.812] moon_task_runner::command_executor  Ran task, checking conditions  task_target="app-storage:run-eslint" command="eslint" exit_code=2
[DEBUG 17:31:00.812] moon_task_runner::command_executor  Task was unsuccessful, failing as we hit our max attempts  task_target="app-storage:run-eslint"
[TRACE 17:31:00.812] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-eslint/stderr.log"
[TRACE 17:31:00.812] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-eslint/stdout.log"
[DEBUG 17:31:00.813] moon_cache_item::cache_item  Writing cache item  cache="/Users/PATH/.moon/cache/states/app-storage/run-eslint/lastRun.json"
[TRACE 17:31:00.813] starbase_utils::json  Writing JSON file  file="/Users/PATH/.moon/cache/states/app-storage/run-eslint/lastRun.json"
[TRACE 17:31:00.813] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/run-eslint/lastRun.json"
[TRACE 17:31:00.813] moon_action_pipeline::action_runner  Failed to run action RunTask(app-storage:run-eslint)  index=15 status=Failed
app-storage:run-eslint | Oops! Something went wrong! :(
app-storage:run-eslint |
app-storage:run-eslint | ESLint: 9.22.0
app-storage:run-eslint |
app-storage:run-eslint | No files matching the pattern "./src/storage.service.ts" were found.
app-storage:run-eslint | Please check for typing mistakes in the pattern.
▪▪▪▪ app-storage:run-eslint (686ms, 4ebacb29)
[TRACE 17:31:01.123] moon_process::exec_command  Ran command in 1.013319625s  pid=68258
[DEBUG 17:31:01.123] moon_task_runner::command_executor  Ran task, checking conditions  task_target="app-storage:run-stylelint" command="stylelint" 

[DEBUG 17:31:01.124] moon_task_runner::task_runner  Skipping task  task_target="app-storage:lint"
[DEBUG 17:31:01.124] moon_cache_item::cache_item  Writing cache item  cache="/Users/PATH/.moon/cache/states/app-storage/lint/l
96BB
astRun.json"
[TRACE 17:31:01.124] starbase_utils::json  Writing JSON file  file="/Users/PATH/.moon/cache/states/app-storage/lint/lastRun.json"
[TRACE 17:31:01.124] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/states/app-storage/lint/lastRun.json"
[TRACE 17:31:01.124] moon_action_pipeline::action_runner  Ran action RunTask(app-storage:lint) in 581.417µs  index=14 status=Skipped
[DEBUG 17:31:01.124] moon_action_pipeline::action_pipeline  Finished pipeline, received all results
▪▪▪▪ app-storage:run-stylelint (1s 13ms, 26d4192c)
▪▪▪▪ app-storage:lint (skipped)

 REVIEW

▪▪▪▪ app-storage:run-prettier
Checking formatting...
[warn] src/index.ts
[error] No files matching the pattern were found: "./src/storage.service.ts".
[warn] Code style issues found in the above file. Run Prettier with --write to fix.

▪▪▪▪ app-storage:run-eslint
Oops! Something went wrong! :(

ESLint: 9.22.0

No files matching the pattern "./src/storage.service.ts" were found.
Please check for typing mistakes in the pattern.


 SUMMARY

pass SyncWorkspace (27ms)
skip SetupToolchain(node:^22) (skipped, 416ms)
skip InstallWorkspaceDeps(node:^22) (skipped)
pass SyncProject(node, moon-project) (2ms)
pass SyncProject(node, app-common) (4ms)
pass SyncProject(node, app-http) (9ms)
pass SyncProject(node, app-theme) (10ms)
pass SyncProject(node, types) (10ms)
pass SyncProject(node, app-environment) (2ms)
pass SyncProject(node, ui-forms) (7ms)
pass SyncProject(node, ui-components) (8ms)
pass SyncProject(node, app-core) (6ms)
pass SyncProject(node, utils) (3ms)
pass SyncProject(node, app-storage) (4ms)
fail RunTask(app-storage:run-prettier) (648ms, 37ad4395)
[DEBUG 17:31:01.124] moon_action_pipeline::subscribers::reports_subscriber  Creating run report
[DEBUG 17:31:01.124] moon_action_pipeline::reports::estimate  Calculating a comparison estimate against other build systems
[DEBUG 17:31:01.124] moon_cache::cache_engine  Writing cache  cache="/Users/PATH/.moon/cache/ciReport.json"
[TRACE 17:31:01.124] starbase_utils::json  Writing JSON file  file="/Users/PATH/.moon/cache/ciReport.json"
[TRACE 17:31:01.125] starbase_utils::fs  Writing file  file="/Users/PATH/.moon/cache/ciReport.json"
[DEBUG 17:31:01.125] moon_action_pipeline::subscribers::cleanup_subscriber  Cleaning stale cache
[DEBUG 17:31:01.125] moon_cache::cache_engine  Cleaning up and deleting stale cached artifacts older than "7 days"
[TRACE 17:31:01.125] starbase_utils::fs  Removing stale contents from directory  dir="/Users/PATH/.moon/cache/hashes"
[TRACE 17:31:01.125] starbase_utils::fs  Reading directory  dir="/Users/PATH/.moon/cache/hashes"

[DEBUG 17:31:01.141] moon_cache::cache_engine  Deleted 0 artifacts and saved 0 bytes
[TRACE 17:31:01.141] starbase::app  Setting exit code  code=1
[TRACE 17:31:01.141] starbase::app  Running shutdown phase
[TRACE 17:31:01.141] moon_console::console  Closing console and flushing buffered output
[TRACE 17:31:01.141] moon_console::buffer  Closing stderr stream
[TRACE 17:31:01.141] moon_console::buffer  Closing stdout stream
fail RunTask(app-storage:run-eslint) (833ms, 4ebacb29)
pass RunTask(app-storage:run-stylelint) (1s 144ms, 26d4192c)
skip RunTask(app-storage:lint) (skipped)

 STATS

Actions: 13 completed, 2 failed, 3 skipped
   Time: 1s 631ms

@muuvmuuv
Copy link
Author
muuvmuuv commented Mar 25, 2025

I am big impressed by your rust programming btw. Just scrolled a bit through it, really like https://github.com/moonrepo/moon/blob/master/crates/app/src/queries/touched_files.rs#L148. I think its valid to pass deleted files as well for other tools, not linters, like for danger-js. So I guess besides affectedFiles it needs something like affectedFilesStatus as an array of possible values from the linked file. Something similar to moon run --status=modified but then it would also work for moon ci, isn't it? Only way I could think of because these task just cannot handle deleted files or process them in any way.
Example:

tasks:
  format:
    options:
      mergeArgs: prepend
      runDepsInParallel: false
    deps:
      - target: 'run-eslint'
        args: '--fix'

  run-eslint:
    command:
      - 'eslint'
      - '--cache'
    options:
      affectedFiles: 'args'
      affectedFilesStatus:
        - modified
        - added
    args:
      - '@group(app)'
      - '@group(sources)'
    inputs:
      - '**/eslint.config.mjs'
      - '/libs/plugin-eslint/**/*.js'
      - '@group(configs)'
      - '@group(app)'
      - '@group(sources)'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

2 participants
0