8000 [probes] Add BROWSER probe type by manugarg · Pull Request #942 · cloudprober/cloudprober · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[probes] Add BROWSER probe type #942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Dec 11, 2024
Merged

[probes] Add BROWSER probe type #942

merged 14 commits into from
Dec 11, 2024

Conversation

manugarg
Copy link
Contributor
@manugarg manugarg commented Dec 10, 2024 •
edited
Loading

Add BROWSER probe type to Cloudprober.

  • We use playwright to run browser probes. Users will provide a playwright test spec, cloudprober will run it, using either the playwright that comes with the cloudprober playwright image (-pw tags) or user provided playwright installation.
  • Browser probe runs usually generate artifacts like screenshots and test reports. Cloudprober supports handling those artifacts in the following manner:
    • You can store them locally and serve them using an built-in web server which can be enabled with a config flag.
    • You can store them in S3 or GCS (azure support coming soon).
    • Built-in cleanup functionality to clean-up local artifacts automatically.
    • See [probes.browser] Add support for storing artifacts in S3. #865 for how these artifacts look like.
  • Test results are exported as metrics. You get a dedicated metric for each individual test and optionally each test step. Metrics include success/failure counters as well as test duration.
  • Provide a pre-built Cloudprober docker image that comes with playwright pre-installed. This image will have -pw tag suffix.

Example config:

probe {
  type: BROWSER
  name: "browser"

  interval: "60s"
  timeout: "30s"
  
  browser_probe {
    # Look for example.spec.ts in the same directory as config file
    test_spec: "example.spec.ts"

    # By default screenshots are saved on failures only
    save_screenshots_for_success: true
   
    # Export metrics at per-step granularity
    test_metrics_options {
      enable_step_metrics: true
    }
  
    artifacts_options {
      serve_on_web: true
      storage {
        local_storage_dir: "{{configDir}}/reports"
      }
    }

    workdir_cleanup_options {
      max_age_sec: 120
    }
  }
}

Example test spec (example.spec.ts)

import { test, expect } from '@playwright/test';

test('has title', async ({ page }) => {
  await page.goto('https://cloudprober.org/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Cloudprober/);
});

@manugarg
[probes.browser] First working draft of the browser probe. (#846)
2085c4a 
- It uses playwright for running browser tests. It expects playwright to be already installed. We'll eventually provide a docker image with playwright installed.
- Probe config takes a test spec. Each test can contain multiple tests and steps. We export metrics for all tests defined in the test spec. We achieve this by defining a custom reporter.
- Config also takes the output directory for generated test assets like screenshots, and traces (need to be enabled through config). If output directory is not defined, a temporary directory will be created. However, we'll strongly encourage users to provide a persistent directory so that screenshots are preserved. More work here:
   - Serve screenshots through cloudprober web server, unless disabled.
   - Implement retention period for assets.

Warning: this is not ready for use right now, that's why integrating into a dev branch for now.
@manugarg
[probes] Use common command execution functionality. (#855)
64f74f5 
- Now that we have factored out command execution part (#854), use that.
@manugarg
[probes.browser] Add a web server to serve test artifacts. (#863)
5301961 
- Add a web server to serve test artifacts. Web server serves artifacts
  at /artifacts/<probename> path by default. You can change the path
  through config.
- Arrange artifacts by date.
- Add long term storage options, not implemented yet though.
@manugarg
[probes.browser] Add support for storing artifacts in S3. (#865)
8000 1f3d848 
- Create the abstration of artifacts handler that will be initialized at the time of
   probe initialization. Artifacts handler in turn will initialize various storage backends
   based on the configuration.

- Probe calls handle() on the artifacts handler in a goroutine after each probe run
  with the output directory path. Artifacts handler will then upload the artifacts to
  all the configured storage backends.

- Tested with a local minio setup and AWS S3 bucket.

- Other changes: enable HTML reporter and persist only reports as these reports
   provide a good overview and contain all the assets like screenshots, videos and
   traces.
@manugarg
[probes.browser] Initial implementation of workdir cleanup (#870)
0227912 
- To ensure that workdir doesn't grow indefinitely, you can configure cleanup handler that will periodically delete files older than a certain age, e.g. you can run cleanup handler every hour to delete files older than 1 day.
@manugarg
[probes.browser] Build docker image with playwright. (#878)
6b618f5
@manugarg
[playwright.docker] Install playwright at /playwright.
6c86a5c
@manugarg
[probes.browser] Implement local artifacts storage (#885)
0890fd0 
- Factor out common code for walking through artifacts directory and add tests for it.
- Add test for local store file saving.
@manugarg
[probes.browser] Provide a way to pass target info and add tests (#888)
2186309 
- This adds non-capitalized env variables for playwright: target_name, target_ip, target_port, target_label_<label>.
- Also add tests to verify that correct command line is being generated.
@manugarg
[probes.browser] Add support for uploading artifacts to GCS (#891)
7bb0652 
- Tested with a GCP account.
@manugarg
[probes.browser] Improve artifacts path (#895)
e9df3b0 
- Also add one more test.
@manugarg
[probes.browser] Improve playwright directory discovery. (#909)
44159ab 
- Instead of assuming default playwright installation path to be /playwright, use environment variable $PLAYWRIGHT_DIR and set this variable in our container image.
@manugarg
[probes.browser] Improve how we find test specs.
fb49e07 
- Instead of copying test specs to temporary work dir, use them at their
  original location itself. This will allow tests to refer to other
  files, etc.
- Assume test specs to be located at the same location as config file
  itself. Provide an option to specify the test directory. For example,
  one could say all tests are at /tests.
@manugarg
[probes.browser] Fix default test_dir and add tests. (#936)
73a1af1
@manugarg manugarg merged commit 4727269 into master Dec 11, 2024
12 of 13 checks passed
@manugarg manugarg deleted the browser_dev branch December 11, 2024 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
Implement browser probe
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@manugarg
0