This software should be compatible with the Python version.
I wrote this one to get more insights on the original version, and trying to shed some lights on the internals. There are many assumptions on the inner workings that I needed to dig into the code to know how they worked.
My goals are:
- get a compatible
ftwversion, with no dependencies and easy to deploy - be CI/CD extremely friendly
- be fast (if possible)
- add features like:
- syntax checking on the test files
- use docker API to get logs (if possible), so there is no need to read files
- add different outputs for CI (junit xml?, github, gitlab, etc.)
Just get the binary file for your architecture, and run it!
To run tests you need:
- a WAF (doh!)
- a file where the waf stores the logs
- a config file, or environment variables, with the information to get the logs and how to parse them (I might embed this for the most commonly used, like Apache/Nginx)
By default, ftw would search for a file in $PWD with the name .ftw.yaml. An example configuration is:
---
logfile: '../coreruleset/tests/logs/modsec2-apache/apache2/error.log'
logtype:
name: 'apache'
timeregex: '\[([A-Z][a-z]{2} [A-z][a-z]{2} \d{1,2} \d{1,2}\:\d{1,2}\:\d{1,2}\.\d+? \d{4})\]'
timeformat: '%a %b %d %H:%M:%S.%f %Y'I normally perform my testing using the Core Rule Set.
You can start the containers from that repo using docker-compose:
git clone https://github.com/coreruleset/coreruleset.git
docker-compose -f tests/docker-compose.yml up -d modsec2-apacheThis is the help for the run command:
❯ ./ftw run -h
Run all tests below a certain subdirectory. The command will search all y[a]ml files recursively and pass it to the test engine.
Usage:
ftw run [flags]
Flags:
-d, --dir string recursively find yaml tests in this directory (default ".")
--exclude string exclude tests matching this Go regexp (e.g. to exclude all tests beginning with "91", use "91.*")
-h, --help help for run
--id string set test id to run
-q, --quiet do not show test by test, only results
-t, --time show time spent per test
Global Flags:
--cfg string override config file (default is $PWD/.ftw.yaml)
--debug debug output
Then run all tests (slightly modified, see here) in the ModSecurity Core Rule set using:
ftw run -d tests -t
And the result should be similar to:
❯ ./ftw run -d tests -t
🛠️ Starting tests!
🚀 Running!
👉 executing tests in file 911100.yaml
running 911100-1: ✔ passed 6.382692ms
running 911100-2: ✔ passed 4.590739ms
running 911100-3: ✔ passed 4.833236ms
running 911100-4: ✔ passed 4.675082ms
running 911100-5: ✔ passed 3.581742ms
running 911100-6: ✔ passed 6.426949ms
...
running 944300-322: ✔ passed 13.292549ms
running 944300-323: ✔ passed 8.960695ms
running 944300-324: ✔ passed 7.558008ms
running 944300-325: ✔ passed 5.977716ms
running 944300-326: ✔ passed 5.457394ms
running 944300-327: ✔ passed 5.896309ms
running 944300-328: ✔ passed 5.873305ms
running 944300-329: ✔ passed 5.828122ms
➕ run 2354 total tests in 18.923445528s
⏭ skept 7 tests
🎉 All tests successful!
Happy testing!
