8000 Optimizing DevOps Pipeline Jobs · Issue #808 · gitgitgadget/gitgitgadget · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Optimizing DevOps Pipeline Jobs #808

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
softworkz opened this issue Dec 18, 2021 · 4 comments
Open

Optimizing DevOps Pipeline Jobs #808

softworkz opened this issue Dec 18, 2021 · 4 comments

Comments

@softworkz
Copy link
Contributor

Most pipeline definition include the steps:

  • Use Node 16.x
  • Pull GitGitGadget, npm install & build

This wastes execution time on every single pipeline run and it also complicates the job definitions unnecessarily.

My suggestion (and what I have done) is this:

  • Remove these steps from all jobs
  • Use a 3rd folder that is located like the others:
    ggg_dir => $(Agent.HomeDirectory)/../ggg
  • Install Node.js locally on the build agent
  • Add a new pipeline that runs on push to the gitgitgadget repo
    • It checks out the repo normally ("Get Sources")
    • Build with node
    • Copy result to $GGG_DIR
  • Change all scripts to execute like node $GGG_DIR/build/misc_helper.js ...
@softworkz
Copy link
Contributor Author
softworkz commented Dec 18, 2021 •
edited
Loading

Re-use non-expired app access tokens

The tasks Obtain GitHub App token are slow (I'm seeing like 5s for each run)

Those tokens are valid for 1h, so it's not needed to request a new token when the previous token is still valid.

The idea would be to save the creation time alongside the token after obtaining and check that time before querying a new one.
(haven't done that yet).

@dscho
Copy link
Member
dscho commented Dec 20, 2021

My suggestion (and what I have done)

The reason I did not do this is that I wanted to keep the Pipeline easy to run on any new VM (as long as it is registered as a build agent).

Those tokens are valid for 1h, so it's not needed to request a new token when the previous token is still valid.

The idea would be to save the creation time alongside the token after obtaining and check that time before querying a new one.

That's a good idea.

@softworkz
Copy link
Contributor Author
softworkz commented Dec 20, 2021 •
edited
Loading

Octokit can do that checking (renew or re-use) for user auth automatically, but I'm not sure whether it can to that for app installation auth.
The potential gain is between 2 and 6 seconds, and thart's a lot, now that I am below 20.

Unfortunately, the quick turnaround times are coming with a new problem: The display of the "Checks" progress is pretty much out of sync. It's a few seconds late in showing and the same in hiding.

I guess I'll better do this manually instead of having it done by DevOps..

7859

@softworkz
Copy link
Contributor Author

The reason I did not do this is that I wanted to keep the Pipeline easy to run on any new VM (as long as it is registered as a build agent).

I think you have crossed that border since quite a while already.. 😆
(but for good reasons)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dscho @softworkz
0