This is an autonomous image scraper developed using TypeScript, Deno, and GitHub Actions. It was purpose-built to document the historic Formula 1 track construction in Las Vegas, Nevada, slated to host the inaugural Heineken Silver Grand Prix on November 18th. The images will be stitched together to form timelapse videos of the track's lifecycle.
| Estimated Top Speed | Circuit Length | Corners | Straights | DRS Zones |
|---|---|---|---|---|
| 212 mph • 342 km/h | 3.8 miles • 6.12 km | 17 | 3 | 2 |
The first scrape happened on June 3rd, 2023. As of October 1st, it has amassed over 16,000 photos from two build sites, equivalent to over 1.0GB of input material for the timelapse process.
The photos are stored in the public GitHub Repository, thanks to GitHub's wonderful free storage for open source projects. They're also persisted to a FoundationDB-backed Deno KV database.
Tools Used
-
Deno v1.37.1 - Rust-based JS runtime, sandboxed, with great TS/TSX support.
- Provides the tools for network and file system operations.
-
TypeScript 5.2.2 - Superset of JavaScript featuring advanced static typechecking.
- Better type safety means more readable and maintainable code.
-
GitHub Actions - Provides free macOS virtual machines powering the scraper.
- Responsible for scheduled execution of the scraper workflow
- Temporarily stores the image artifacts
-
Deno KV(currently in beta) - Provides us with global data persistence and caching
-
ffmpeg(timelapse feature coming soon) - Leveraged by GitHub Actions to compile timelapse videos
The data source on the scraped images is an updating live photo feed sourced from the official Formula 1 website. As long as it remains up and transmitting data, this project will continue to auto-update.
This project is maintained by Nicholas Berlette, developed as an open source project using TypeScript, Deno, Deno KV, and GitHub Actions.
⚠️ This non-commercial project is for educational purposes only.
| October 6th | October 5th | October 4th | October 3rd |
|---|---|---|---|
| October 2nd | September 28th | September 24th | September 20th |
| September 16th | September 12th | September 8th | September 4th |
The majority of the work happens in main.ts, despite it only being
3 lines of code. It is responsible for invoking the scraper located in
src/scrape.ts, and is ran every 10 minutes by a GitHub Action
defined by the workflow in main.yml.
📖 Click here for an in-depth explanation of the scrape process
Images are named after their capture time as a JPEG file in UTC. For
example, an image captured at 2023-07-09T04:28:57 would be saved as
./assets/2023-07-09/04_28_57.jpg.
The latest image is always saved as ./assets/latest.jpg for easy
access.
- GitHub Actions runs the scrape workflow every ~10 minutes, depending on traffic.
- The runner checks out the repository and installs Deno.
- The command
deno task scrapeis run, invoking themain.tsfile in the project root. main.tsimportsscrapeand double-checks that it is only being run as the main module. If so, it immediately callsscrape(), which contains two inner functions,readandwrite. The following steps are taken:read()is called withIMAGE_URLwrite()is called with theImageinstance returned from step 1 as its only argument.- The image is compared for equality against the latest image, using a
timing-safe equality check to avoid exposure to timing-based attacks.
- If they are equal, the image has not updated at the origin. The scrape will start over at step 4 and repeat until a new image is found.
- If the maximum number of
ATTEMPTSis reached and no new image was found, the job will terminate unsuccessfully.
- If we've made it this far, we have a fresh image and we need to store
it. It is written to
Deno KVviaImage.write(). - The image is written to a file by
Image.writeFile()with our naming conventions, and logged tostdout. - The image is written to
./assets/latest.jpg, and the size difference is logged tostdoutand GitHub Actions Outputs. - The
setOutputmethod is called with all of the metadata for the image, so the runner can process it further.
- The image is compared for equality against the latest image, using a
timing-safe equality check to avoid exposure to timing-based attacks.
- The scrape is now complete and the runner proceeds to the next step.
- The photo is stored as a GitHub Workflow artifact for 90 days.
- The changes are committed and pushed to the repository.
- The job completes and the runner is terminated successfully.
AI-generated F1 art created with SDXL 1.0 and the prompt
"Formula 1 cars on the Las Vegas Strip"
The camera is at the "Harmon Paddock" zone of the track, located just southeast of "The Strip" (Las Vegas Blvd) near Harmon Avenue and Koval Lane. The camera itself is mounted directly on the roof of the Paddock building, facing south-southeast. It gives a great view of the grandstands (bleachers) and what appears to be shaping up to be the start/finish line of the track.
Something which is completely out of my control is anything related to the actual camera itself. That being said, when the construction crew suddenly chose to relocate it on August 15th, it effectively split the timelapse part of this project into two parts.
The camera was facing north-northwest, giving a bird-eye view of the Paddock area and the track's first corner, with the iconic Las Vegas Strip in the background. As of August 15th, however, the camera is now facing south-southeast towards the Harry Reid International Airport, with the MGM Grand in the background to the right.
It's not all bad though - in my opinion, the new angle is far superior to the previous one, and it provides quite a bit more content for us since it's in an area with much more activity.
The first angle will be made into a separate timelapse video, from June 3rd, 2023 through August 15th, 2023. The second angle will be from August 15th, 2023 through the completion of the track.
A frontend display for viewing the images as a growing timelapse is under
development. This will also help create a timelapse video once the track build
is complete. Check n.berlette.com/f1 for updates.
MIT © Nicholas Berlette • Made with ♥️ in Las Vegas, NV
This project is not affiliated with Formula 1 and has no commercial interests.