Description
My proposal is to split the entire set of mutators in a sub-project. Not only Infection\Mutator\ namespace, but the bulk of related machinery too (so tests can be run), while also opening up the API for use in other projects (by removing @private annotations first and foremost).
I had this idea for some time. For one because I wanted a tool which I could use in place of PHPUnit executable to do mutation testing against a single test suite e.g.
infection-tool tests/Example/SomethingTest.php
regardless of an effective coverage (just assuming that if a test says it @covers something, it does it for 100%). I don't think this feature is impossible, but this appears to take much more effort I can get by with they way I want it to be with the current state of affairs (Infection would need to have quite drastic changes).
For another, because Infection is very tightly integrated with PHPUnit's outputs, and not like this is bad, but this makes it very hard to integrate other testing frameworks which use different means for estimating coverage or even don't have the standard coverage logs at all. If mutators were a sub-project, then there could be a more tightly-integrated framework-specific tool for, say, Behat, that would function very different from Infection, but still use the same time-proven and thoroughly tested mutators.
Last straw for me was this video of a talk on mutation testing given at Badoo (is it like Tinder on steroids? I'm out of a dating scene for a long time, but it seems like they're kind of big) posted like yesterday, which mentions problems Badoo had with their codebase. This video is in Russian and quite long, so, saving you the trouble watching it, in the nutshell, they couldn't use Infection because collecting PHPUnit's standard coverage takes several hours for their codebase, and it takes around 30 Gb in disk space alone (slide link). They also had some other issues with the tooling they use. In the end they took mutators from Infection (because "they're very solid and concisely written", kudos to all of you!), making their own simplified tool (an obligatory screenshot) (another slide). They could also had used mutators from Infection more easily if they were a sub-project, and even contribute to it in code or else.
One of more or less annoying issues when developing Infection is very long test run times. If mutators were a sub-project, then adding a mutator would not require running the whole Infection's test suite with endless E2E tests in multitude of PHP versions, and such (they're for purpose, so no offence!). Only standard unit tests would have to be run.
In the end this change should make Infection an even more valuable project to the industry.
As this is a quite formidable undertaking, here I ask for your support and approval.
What do you all think about this idea?
Especially @infection/core I want your opinion. Should we do this?
If there's an agreement, here's what to be done:
- Make a dedicated repository, say
infection/mutators(needs admin privileges). - Clone the entire Infection into it (but not fork from GitHub UI!).
- Change project name in composer.json to match the repository name; git tags cleanup.
- Proceed with stripping everything not related to mutators or mutating files.
- PR to Infection with that new project as a dependency, with all mutators etc removed from Infection itself.