About Testing: E2E #21
Replies: 3 comments 2 replies
-
|
Hi @davidescapolan01 ,
How should these tests be structured and written? How should we run the tests? |
Beta Was this translation helpful? Give feedback.
-
|
i'll explore around github hook when PR is raised which builds all the images and test making sure build is going through. later on will add unit tests also to the same actions |
Beta Was this translation helpful? Give feedback.
-
|
Hey @nishanb , I went through your suggestions, and I think there’s a bit of mixing between different layers of testing. Just to clarify and ensure we approach this in the best way: 1. Docker Compose Build & Container StartupThis should be handled via integration or smoke tests, not E2E. E2E tests shouldn't be responsible for verifying if the 2. Lab Startup (from the app)Here, E2E can simulate the user clicking “Start Lab” and then wait until the lab is ready (based on UI signals or API responses). However, E2E tests shouldn’t directly inspect Docker or container internals — they can only validate the outcome (e.g., lab status is "ready", UI elements are visible, etc.). 3. Solving QuestionsRight now, the correct answers seem to be stored in Markdown, which is hard to interpret programmatically. For automation, we should consider switching to (or supplementing with) a structured JSON format that defines the correct answers. This would make E2E automation feasible and robust. 4. Test StrategyA good approach could be:
5. Running the TestsInstead of running the tests manually on local, I’d suggest we set them up in a way that’s already compatible with the CI/CD environment we plan to use later (e.g., Travis, GitHub Actions, etc.). That way, we can ensure consistency from the beginning and avoid rework later. Even for local runs, we could containerize the tests or use the same setup scripts we’d use in CI. Let me know if you'd like help drafting the JSON format for lab answers or setting up the first E2E test based on this structure. |
Beta Was this translation helpful? Give feedback.
-
|
That should be the way forward. thanks for summarizing !! Also for 2 & 3 we would be needing a containers running with docker compose to perform verification. good that you're planning to segregate them !! |
Beta Was this translation helpful? Give feedback.
-
|
Alright, sounds good — we’ll proceed assuming that the docker-compose setup is already running and that all containers are healthy during test execution. The E2E tests will focus purely on simulating user interaction with the frontend and, at most, verifying API responses received by the UI. As for testing the actual Lab behavior (like container creation or cluster logic), I’m not entirely sure that falls under E2E testing. That might be better suited for an integration test module that runs closer to the virtualized mini-cluster itself. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone 👋
I've taken on the task of writing some end-to-end (E2E) tests using Playwright for this project, and I’d like to open up a discussion to align on a few key aspects before diving deeper.
Topics to discuss:
I'd like to identify the core flows and critical functionality worth testing.
Looking for conventions or best practices on organizing tests, naming, tools, etc.
Currently, there's no CI/CD setup, but GitHub Actions are planned for the future and will eventually run these tests automatically.
For now, the repo owner (@nishanb ) suggested that contributors should run the E2E tests locally before opening PRs. I think that’s a fair compromise at this stage, and I’d like to offer a few thoughts and suggestions on that:
Local Testing Infrastructure (e.g., using Gitea)
We could encourage contributors to clone the repo into their own self-hosted Gitea instance to run the tests. Here’s a quick breakdown of pros and cons:
Pros:
Cons:
Possible Solutions
CONTRIBUTING.md.That said, for more complex pipelines, Gitea’s limitations might be a hard stop — it’s something we might not be able to work around easily.
Alternative CI Options
While Gitea might work for now, we could explore other CI tools as well:
Let me know what you think, especially regarding:
Happy to collaborate on defining a sustainable approach 🚀
Beta Was this translation helpful? Give feedback.
All reactions