8000 Go Test WAF bypasses · Issue #1991 · coreruleset/coreruleset · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
8000

Go Test WAF bypasses #1991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ghost opened this issue Jan 27, 2021 · 15 comments
Closed

Go Test WAF bypasses #1991

ghost opened this issue Jan 27, 2021 · 15 comments
Assignees
@fzipi
Labels
➖ False Negative - Evasion

Comments

@ghost
Copy link
ghost commented Jan 27, 2021

Description

https://github.com/wallarm/gotestwaf#testing-on-owasp-modsecurity-core-rule-set

Confirmation

[X] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.
@dune73 dune73 mentioned this issue Mar 15, 2021
Closed
@dune73 dune73 mentioned this issue Apr 16, 2021
Closed
@github-actions
Copy link
Contributor

This issue has been open 120 days with no activity. Remove the stale label or comment, or this will be closed in 14 days

@github-actions github-actions bot added the ⌛ Stale issue This issue has been open 120 days with no activity. label May 28, 2021
@fzipi fzipi self-assigned this May 29, 2021
@fzipi fzipi removed the ⌛ Stale issue This issue has been open 120 days with no activity. label Jun 4, 2021
@fzipi
Copy link
Member
fzipi commented Jun 4, 2021

I will take care of this one.

@dune73
Copy link
Member
dune73 commented Jun 21, 2021

Thank you

@fzipi
Copy link
Member
fzipi commented Jul 13, 2021

Ok, so I found some time to run some tests with gotestwaf.

Tests were made using the same docker-compose we use for testing using ftw/go-ftw, with just one modification:

❯ diff tests/gotestwaf.yml tests/docker-compose.yml
14c14
<       MODSEC_RULE_ENGINE: "On"
---
>       MODSEC_RULE_ENGINE: DetectionOnly

Here are the results (they differ depending on the url used):

  1. Test with CRS in paranoia level 4:
❯ docker run -v ${PWD}/reports:/go/src/gotestwaf/reports --network="host" gotestwaf --url=http://127.0.0.1:80 --verbose
GOTESTWAF : 2021/07/13 19:46:21.360383 main.go:62: Test cases loading started
GOTESTWAF : 2021/07/13 19:46:21.365254 main.go:69: Test cases loading finished
GOTESTWAF : 2021/07/13 19:46:21.365312 main.go:79: Scanned URL: http://127.0.0.1:80
GOTESTWAF : 2021/07/13 19:46:21.389021 main.go:102: WAF pre-check: OK. Blocking status code: 403
GOTESTWAF : 2021/07/13 19:46:21.389219 main.go:114: WebSocket pre-check. URL to check: ws://127.0.0.1:80
GOTESTWAF : 2021/07/13 19:46:21.400405 main.go:118: WebSocket pre-check: connection is not available, reason: websocket: bad handshake
GOTESTWAF : 2021/07/13 19:46:21.400871 main.go:146: Scanning http://127.0.0.1:80
GOTESTWAF : 2021/07/13 19:46:21.401003 scanner.go:138: Scanning started
GOTESTWAF : 2021/07/13 19:46:32.876719 scanner.go:143: Scanning Time:  11.4756551s
GOTESTWAF : 2021/07/13 19:46:32.876838 scanner.go:174: Scanning finished

Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-lfi         |                100.00 |                     6 |                     0 |                     0 |
| community             | community-rce         |                100.00 |                    42 |                     0 |                     0 |
| community             | community-sqli        |                100.00 |                    48 |                     0 |                     0 |
| community             | community-xss         |                100.00 |                   304 |                     0 |                     0 |
| community             | community-xxe         |                100.00 |                     4 |                     0 |                     0 |
| owasp                 | ldap-injection        |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | mail-injection        |                 66.67 |                     6 |                     3 |                     3 |
| owasp                 | nosql-injection       |                 66.67 |                     6 |                     3 |                     9 |
| owasp                 | path-traversal        |                 71.43 |                    10 |                     4 |                    10 |
| owasp                 | shell-injection       |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | sql-injection         |                 60.00 |                    12 |                     8 |                    12 |
| owasp                 | ss-include            |                 54.55 |                     6 |                     5 |                     9 |
| owasp                 | sst-injection         |                 54.55 |                     6 |                     5 |                     9 |
| owasp                 | xml-injection         |                100.00 |                    12 |                     0 |                     0 |
| owasp                 | xss-scripting         |                 62.50 |                    10 |                     6 |                    12 |
| owasp-api             | graphql               |                100.00 |                     1 |                     0 |                     0 |
| owasp-api             | rest                  |                100.00 |                     2 |                     0 |                     0 |
| owasp-api             | soap                  |                100.00 |                     2 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF AVERAGE SCORE:   |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |        81.65%         |   485/523 (92.73%)    |    38/523 (7.27%)     |    68/591 (11.51%)    |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                  0.00 |                     8 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF POSITIVE SCORE:  | FALSE POSITIVE (RES): | TRUE POSITIVE (RES):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |         0.00%         |     8/8 (100.00%)     |      0/8 (0.00%)      |      0/8 (0.00%)      |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

PDF report is ready: reports/waf-evaluation-report-generic-2021-July-13-19-46-32.pdf
  1. Test using /get (we are using httpbin, so "GET" methods would return 200)
❯ docker run -v ${PWD}/reports:/go/src/gotestwaf/reports --network="host" gotestwaf --url=http://127.0.0.1:80/get --verbose
GOTESTWAF : 2021/07/13 19:45:54.855380 main.go:62: Test cases loading started
GOTESTWAF : 2021/07/13 19:45:54.857172 main.go:69: Test cases loading finished
GOTESTWAF : 2021/07/13 19:45:54.857237 main.go:79: Scanned URL: http://127.0.0.1:80/get
GOTESTWAF : 2021/07/13 19:45:54.881865 main.go:102: WAF pre-check: OK. Blocking status code: 403
GOTESTWAF : 2021/07/13 19:45:54.881901 main.go:114: WebSocket pre-check. URL to check: ws://127.0.0.1:80/get
GOTESTWAF : 2021/07/13 19:45:54.888902 main.go:118: WebSocket pre-check: connection is not available, reason: websocket: bad handshake
GOTESTWAF : 2021/07/13 19:45:54.889415 main.go:146: Scanning http://127.0.0.1:80/get
GOTESTWAF : 2021/07/13 19:45:54.889526 scanner.go:138: Scanning started
GOTESTWAF : 2021/07/13 19:46:05.828922 scanner.go:143: Scanning Time:  10.9393588s
GOTESTWAF : 2021/07/13 19:46:05.829073 scanner.go:174: Scanning finished

Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-lfi         |                100.00 |                     6 |                     0 |                     0 |
| community             | community-rce         |                100.00 |                    42 |                     0 |                     0 |
| community             | community-sqli        |                100.00 |                    48 |                     0 |                     0 |
| community             | community-xss         |                100.00 |                   304 |                     0 |                     0 |
| community             | community-xxe         |                100.00 |                     4 |                     0 |                     0 |
| owasp                 | ldap-injection        |                100.00 |                     4 |                     0 |                     4 |
| owasp                 | mail-injection        |                100.00 |                     6 |                     0 |                     6 |
| owasp                 | nosql-injection       |                100.00 |                     6 |                     0 |                    12 |
| owasp                 | path-traversal        |                100.00 |                    10 |                     0 |                    14 |
| owasp                 | shell-injection       |                100.00 |                     4 |                     0 |                     4 |
| owasp                 | sql-injection         |                 75.00 |                    12 |                     4 |                    16 |
| owasp                 | ss-include            |                100.00 |                     6 |                     0 |                    14 |
| owasp                 | sst-injection         |                100.00 |                     6 |                     0 |                    14 |
| owasp                 | xml-injection         |                100.00 |                    12 |                     0 |                     0 |
| owasp                 | xss-scripting         |                100.00 |                    10 |                     0 |                    18 |
| owasp-api             | graphql               |                100.00 |                     1 |                     0 |                     0 |
| owasp-api             | rest                  |                100.00 |                     2 |                     0 |                     0 |
| owasp-api             | soap                  |                100.00 |                     2 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF AVERAGE SCORE:   |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |        98.61%         |   485/489 (99.18%)    |     4/489 (0.82%)     |   102/591 (17.26%)    |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                  0.00 |                     8 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF POSITIVE SCORE:  | FALSE POSITIVE (RES): | TRUE POSITIVE (RES):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |         0.00%         |     8/8 (100.00%)     |      0/8 (0.00%)      |      0/8 (0.00%)      |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

PDF report is ready: reports/waf-evaluation-report-generic-2021-July-13-19-46-05.pdf
  1. Test using /post (we are using httpbin, so "POST" methods would return 200)
❯ docker run -v ${PWD}/reports:/go/src/gotestwaf/reports --network="host" gotestwaf --url=http://127.0.0.1:80/post --verbose
GOTESTWAF : 2021/07/13 19:46:46.834235 main.go:62: Test cases loading started
GOTESTWAF : 2021/07/13 19:46:46.837989 main.go:69: Test cases loading finished
GOTESTWAF : 2021/07/13 19:46:46.838045 main.go:79: Scanned URL: http://127.0.0.1:80/post
GOTESTWAF : 2021/07/13 19:46:46.890657 main.go:102: WAF pre-check: OK. Blocking status code: 403
GOTESTWAF : 2021/07/13 19:46:46.890697 main.go:114: WebSocket pre-check. URL to check: ws://127.0.0.1:80/post
GOTESTWAF : 2021/07/13 19:46:46.900418 main.go:118: WebSocket pre-check: connection is not available, reason: websocket: bad handshake
GOTESTWAF : 2021/07/13 19:46:46.901251 main.go:146: Scanning http://127.0.0.1:80/post
GOTESTWAF : 2021/07/13 19:46:46.901354 scanner.go:138: Scanning started
GOTESTWAF : 2021/07/13 19:46:58.460956 scanner.go:143: Scanning Time:  11.559542s
GOTESTWAF : 2021/07/13 19:46:58.461000 scanner.go:174: Scanning finished

Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-lfi         |                100.00 |                     6 |                     0 |                     0 |
| community             | community-rce         |                100.00 |                    42 |                     0 |                     0 |
| community             | community-sqli        |                100.00 |                    48 |                     0 |                     0 |
| community             | community-xss         |                100.00 |                   304 |                     0 |                     0 |
| community             | community-xxe         |                100.00 |                     4 |                     0 |                     0 |
| owasp                 | ldap-injection        |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | mail-injection        |                100.00 |                     6 |                     0 |                     6 |
| owasp                 | nosql-injection       |                 66.67 |                     6 |                     3 |                     9 |
| owasp                 | path-traversal        |                 71.43 |                    10 |                     4 |                    10 |
| owasp                 | shell-injection       |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | sql-injection         |                 75.00 |                    12 |                     4 |                    16 |
| owasp                 | ss-include            |                100.00 |                     6 |                     0 |                    14 |
| owasp                 | sst-injection         |                100.00 |                     6 |                     0 |                    14 |
| owasp                 | xml-injection         |                100.00 |                    12 |                     0 |                     0 |
| owasp                 | xss-scripting         |                100.00 |                    10 |                     0 |                    18 |
| owasp-api             | graphql               |                100.00 |                     1 |                     0 |                     0 |
| owasp-api             | rest                  |                100.00 |                     2 |                     0 |                     0 |
| owasp-api             | soap                  |                100.00 |                     2 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF AVERAGE SCORE:   |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |        91.47%         |   485/500 (97.00%)    |    15/500 (3.00%)     |    91/591 (15.40%)    |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                  0.00 |                     8 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF POSITIVE SCORE:  | FALSE POSITIVE (RES): | TRUE POSITIVE (RES):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |         0.00%         |     8/8 (100.00%)     |      0/8 (0.00%)      |      0/8 (0.00%)      |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

PDF report is ready: reports/waf-evaluation-report-generic-2021-July-13-19-46-58.pdf

I'm attaching all PDFs is someone wants to review.

@fzipi
Copy link
Member
fzipi commented Jul 13, 2021

waf-evaluation-report-generic-2021-July-13-19-46-05.pdf
waf-evaluation-report-generic-2021-July-13-19-46-32.pdf
waf-evaluation-report-generic-2021-July-13-19-46-58.pdf

@fzipi
Copy link
Member
fzipi commented Jul 13, 2021

This is with PARANOIA_LEVEL=2:

❯ docker run -v ${PWD}/reports:/go/src/gotestwaf/reports --network="host" gotestwaf --url=http://127.0.0.1:80 --verbose
GOTESTWAF : 2021/07/13 20:01:44.828664 main.go:62: Test cases loading started
GOTESTWAF : 2021/07/13 20:01:44.830095 main.go:69: Test cases loading finished
GOTESTWAF : 2021/07/13 20:01:44.830160 main.go:79: Scanned URL: http://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:01:44.864749 main.go:102: WAF pre-check: OK. Blocking status code: 403
GOTESTWAF : 2021/07/13 20:01:44.864854 main.go:114: WebSocket pre-check. URL to check: ws://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:01:44.873899 main.go:118: WebSocket pre-check: connection is not available, reason: websocket: bad handshake
GOTESTWAF : 2021/07/13 20:01:44.874756 main.go:146: Scanning http://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:01:44.874823 scanner.go:138: Scanning started
GOTESTWAF : 2021/07/13 20:01:55.336768 scanner.go:143: Scanning Time:  10.4618945s
GOTESTWAF : 2021/07/13 20:01:55.336795 scanner.go:174: Scanning finished

Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-lfi         |                100.00 |                     6 |                     0 |                     0 |
| community             | community-rce         |                100.00 |                    42 |                     0 |                     0 |
| community             | community-sqli        |                100.00 |                    48 |                     0 |                     0 |
| community             | community-xss         |                100.00 |                   304 |                     0 |                     0 |
| community             | community-xxe         |                100.00 |                     4 |                     0 |                     0 |
| owasp                 | ldap-injection        |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | mail-injection        |                 66.67 |                     6 |                     3 |                     3 |
| owasp                 | nosql-injection       |                 66.67 |                     6 |                     3 |                     9 |
| owasp                 | path-traversal        |                 71.43 |                    10 |                     4 |                    10 |
| owasp                 | shell-injection       |                 66.67 |                     4 |                     2 |                     2 |
| owasp                 | sql-injection         |                 60.00 |                    12 |                     8 |                    12 |
| owasp                 | ss-include            |                 54.55 |                     6 |                     5 |                     9 |
| owasp                 | sst-injection         |                 54.55 |                     6 |                     5 |                     9 |
| owasp                 | xml-injection         |                100.00 |                    12 |                     0 |                     0 |
| owasp                 | xss-scripting         |                 62.50 |                    10 |                     6 |                    12 |
| owasp-api             | graphql               |                100.00 |                     1 |                     0 |                     0 |
| owasp-api             | rest                  |                100.00 |                     2 |                     0 |                     0 |
| owasp-api             | soap                  |                100.00 |                     2 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF AVERAGE SCORE:   |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |        81.65%         |   485/523 (92.73%)    |    38/523 (7.27%)     |    68/591 (11.51%)    |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                  0.00 |                     8 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF POSITIVE SCORE:  | FALSE POSITIVE (RES): | TRUE POSITIVE (RES):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |         0.00%         |     8/8 (100.00%)     |      0/8 (0.00%)      |      0/8 (0.00%)      |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

PDF report is ready: reports/waf-evaluation-report-generic-2021-July-13-20-01-55.pdf

@fzipi
Copy link
Member
fzipi commented Jul 13, 2021

And the one that I really like, is the one that comes from adding --nonBlockedAsPassed:

❯ docker run -v ${PWD}/reports:/go/src/gotestwaf/reports --network="host" gotestwaf --url=http://127.0.0.1:80 --verbose --nonBlockedAsPassed
GOTESTWAF : 2021/07/13 20:09:04.688795 main.go:62: Test cases loading started
GOTESTWAF : 2021/07/13 20:09:04.690489 main.go:69: Test cases loading finished
GOTESTWAF : 2021/07/13 20:09:04.690541 main.go:79: Scanned URL: http://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:09:04.712868 main.go:102: WAF pre-check: OK. Blocking status code: 403
GOTESTWAF : 2021/07/13 20:09:04.712913 main.go:114: WebSocket pre-check. URL to check: ws://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:09:04.720378 main.go:118: WebSocket pre-check: connection is not available, reason: websocket: bad handshake
GOTESTWAF : 2021/07/13 20:09:04.720943 main.go:146: Scanning http://127.0.0.1:80
GOTESTWAF : 2021/07/13 20:09:04.720990 scanner.go:138: Scanning started
GOTESTWAF : 2021/07/13 20:09:15.790390 scanner.go:143: Scanning Time:  11.1036227s
GOTESTWAF : 2021/07/13 20:09:15.790484 scanner.go:174: Scanning finished

Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-lfi         |                100.00 |                     6 |                     0 |                     0 |
| community             | community-rce         |                100.00 |                    42 |                     0 |                     0 |
| community             | community-sqli        |                100.00 |                    48 |                     0 |                     0 |
| community             | community-xss         |                100.00 |                   304 |                     0 |                     0 |
| community             | community-xxe         |                100.00 |                     4 |                     0 |                     0 |
| owasp                 | ldap-injection        |                100.00 |                     6 |                     0 |                     2 |
| owasp                 | mail-injection        |                100.00 |                     9 |                     0 |                     3 |
| owasp                 | nosql-injection       |                166.67 |                    15 |                    -6 |                     9 |
| owasp                 | path-traversal        |                142.86 |                    20 |                    -6 |                    10 |
| owasp                 | shell-injection       |                100.00 |                     6 |                     0 |                     2 |
| owasp                 | sql-injection         |                120.00 |                    24 |                    -4 |                    12 |
| owasp                 | ss-include            |                136.36 |                    15 |                    -4 |                     9 |
| owasp                 | sst-injection         |                136.36 |                    15 |                    -4 |                     9 |
| owasp                 | xml-injection         |                100.00 |                    12 |                     0 |                     0 |
| owasp                 | xss-scripting         |                137.50 |                    22 |                    -6 |                    12 |
| owasp-api             | graphql               |                100.00 |                     1 |                     0 |                     0 |
| owasp-api             | rest                  |                100.00 |                     2 |                     0 |                     0 |
| owasp-api             | soap                  |                100.00 |                     2 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF AVERAGE SCORE:   |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |        113.32%        |   553/523 (105.74%)   |   -30/523 (-5.74%)    |    68/591 (11.51%)    |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                  0.00 |                     8 |                     0 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |       WAF NAME:       |  WAF POSITIVE SCORE:  | FALSE POSITIVE (RES): | TRUE POSITIVE (RES):  |      UNRESOLVED:      |
|      2021-07-13       |        GENERIC        |         0.00%         |     8/8 (100.00%)     |      0/8 (0.00%)      |      0/8 (0.00%)      |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+

@fzipi
Copy link
Member
fzipi commented Jul 13, 2021

In summary: there might be some bypasses in PL4 that we might want to look, but probably I would say the bypass is wrong.

@dune73
Copy link
Member
dune73 commented Jul 16, 2021

Thank you very much @fzipi. I do not really get all of your statistical tables above, but I was able to extract useful information from the PDF reports and validated by hand. I looked at the false positives and the bypasses (but not the unresolved cases).

False Positives at PL1

932150
	curl localhost -d 'foo=time he came'

	[2021-07-16 08:18:09.376018] [-:error] 127.0.0.1:55142 YPEkoRy0s3c8Xzd5uRp5MQAAABU [client 127.0.0.1] ModSecurity: Warning. Pattern match "(?:^|=)\\\\s*(?:{|\\\\s*\\\\(\\\\s*|\\\\w+=(?:[^\\\\s]*|\\\\$.*|\\\\$.*|<.*|>.*|\\\\'.*\\\\'|\\".*\\")\\\\s+|!\\\\s*|\\\\$)*\\\\s*(?:'|\\")*(?:[\\\\?\\\\*\\\\[\\\\]\\\\(\\\\)\\\\-\\\\|+\\\\w'\\"\\\\./\\\\\\\\]+/)?[\\\\\\\\'\\"]*(?:l[\\\\\\\\'\\"]*(?:s(?:[\\\\\\\\'\\"]*(?:b[\\\\\\\\'\\"]*_[\\\\\\\\'\\"]*r[\\\\\\\\'\\"]*e[\\\\\\\\'\\"]*l[\\\\\\\\' ..." at ARGS:foo. [file "/home/dune73/data/git/crs-official/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "467"] [id "932150"] [msg "Remote Command Execution: Direct Unix Command Execution"] [data "Matched Data: time  found within ARGS:foo: time he came"] [severity "CRITICAL"] [ver "OWASP_CRS/3.4.0-dev"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/"] [unique_id "YPEkoRy0s3c8Xzd5uRp5MQAAABU"]

942190
	curl localhost -d 'foo=union was a great select'

	[2021-07-16 08:19:27.279243] [-:error] 127.0.0.1:55158 YPEk7xy0s3c8Xzd5uRp5MQAAABc [client 127.0.0.1] ModSecurity: Warning. Pattern match "(?i:(?:[\\"'`](?:;?\\\\s*?(?:union\\\\b\\\\s*?(?:(?:distin|sele)ct|all)|having|select)\\\\b\\\\s*?[^\\\\s]|\\\\s*?!\\\\s*?[\\"'`\\\\w])|(?:c(?:onnection_id|urrent_user)|database)\\\\s*?\\\\([^\\\\)]*?|u(?:nion(?:[\\\\w(\\\\s]*?select| select @)|ser\\\\s*?\\\\([^\\\\)]*?)|s(?:chema\\\\s*?\\\\ ..." at ARGS:foo. [file "/home/dune73/data/git/crs-official/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "201"] [id "942190"] [msg "Detects MSSQL code execution and information gathering attempts"] [data "Matched Data: union was a great select found within ARGS:foo: union was a great select"] [severity "CRITICAL"] [ver "OWASP_CRS/3.4.0-dev"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/66"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/"] [unique_id "YPEk7xy0s3c8Xzd5uRp5MQAAABc"]

932110
	curl localhost -d "foo=--I think it would; copy should"

	[2021-07-16 08:27:28.301498] [-:error] 127.0.0.1:55372 YPEm0By0s3c8Xzd5uRp5MQAAABY [client 127.0.0.1] ModSecurity: Warning. Pattern match "(?i)(?:;|\\\\{|\\\\||\\\\|\\\\||&|&&|\\\\n|\\\\r|`)\\\\s*[\\\\(,@\\\\'\\"\\\\s]*(?:[\\\\w'\\"\\\\./]+/|[\\\\\\\\'\\"\\\\^]*\\\\w[\\\\\\\\'\\"\\\\^]*:.*\\\\\\\\|[\\\\^\\\\.\\\\w '\\"/\\\\\\\\]*\\\\\\\\)?[\\"\\\\^]*(?:m[\\"\\\\^]*(?:y[\\"\\\\^]*s[\\"\\\\^]*q[\\"\\\\^]*l(?:[\\"\\\\^]*(?:d[\\"\\\\^]*u[\\"\\\\^]*m[\\"\\\\^]*p(?:[\\"\\\\^]*s[\\"\\\\^ ..." at ARGS:foo. [file "/home/dune73/data/git/crs-official/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "259"] [id "932110"] [msg "Remote Command Execution: Windows Command Injection"] [data "Matched Data: ; copy found within ARGS:foo: --I think it would; copy should"] [severity "CRITICAL"] [ver "OWASP_CRS/3.4.0-dev"] [tag "application-multi"] [tag "language-shell"] [tag "platform-windows"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/"] [unique_id "YPEm0By0s3c8Xzd5uRp5MQAAABY"]

	curl localhost -d "foo=a hero; more than"

	[2021-07-16 08:29:29.784646] [-:error] 127.0.0.1:55400 YPEnSRy0s3c8Xzd5uRp5MgAAABc [client 127.0.0.1] ModSecurity: Warning. Pattern match "(?i)(?:;|\\\\{|\\\\||\\\\|\\\\||&|&&|\\\\n|\\\\r|`)\\\\s*[\\\\(,@\\\\'\\"\\\\s]*(?:[\\\\w'\\"\\\\./]+/|[\\\\\\\\'\\"\\\\^]*\\\\w[\\\\\\\\'\\"\\\\^]*:.*\\\\\\\\|[\\\\^\\\\.\\\\w '\\"/\\\\\\\\]*\\\\\\\\)?[\\"\\\\^]*(?:m[\\"\\\\^]*(?:y[\\"\\\\^]*s[\\"\\\\^]*q[\\"\\\\^]*l(?:[\\"\\\\^]*(?:d[\\"\\\\^]*u[\\"\\\\^]*m[\\"\\\\^]*p(?:[\\"\\\\^]*s[\\"\\\\^ ..." at ARGS:foo. [file "/home/dune73/data/git/crs-official/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "259"] [id "932110"] [msg "Remote Command Execution: Windows Command Injection"] [data "Matched Data: ; more than found within ARGS:foo: a hero; more than"] [severity "CRITICAL"] [ver "OWASP_CRS/3.4.0-dev"] [tag "application-multi"] [tag "language-shell"] [tag "platform-windows"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/"] [unique_id "YPEnSRy0s3c8Xzd5uRp5MgAAABc"]

932100
	curl localhost -d "foo=a hero; more than"
		
	[2021-07-16 08:29:29.784520] [-:error] 127.0.0.1:55400 YPEnSRy0s3c8Xzd5uRp5MgAAABc [client 127.0.0.1] ModSecurity: Warning. Pattern match "(?:;|\\\\{|\\\\||\\\\|\\\\||&|&&|\\\\n|\\\\r|\\\\$\\\\(|\\\\$\\\\(\\\\(|`|\\\\${|<\\\\(|>\\\\(|\\\\(\\\\s*\\\\))\\\\s*(?:{|\\\\s*\\\\(\\\\s*|\\\\w+=(?:[^\\\\s]*|\\\\$.*|\\\\$.*|<.*|>.*|\\\\'.*\\\\'|\\".*\\")\\\\s+|!\\\\s*|\\\\$)*\\\\s*(?:'|\\")*(?:[\\\\?\\\\*\\\\[\\\\]\\\\(\\\\)\\\\-\\\\|+\\\\w'\\"\\\\./\\\\\\\\]+/)?[\\\\\\\\'\\"]*(?:l[\\\\\\\\'\\"]* ..." at ARGS:foo. [file "/home/dune73/data/git/crs-official/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "124"] [id "932100"] [msg "Remote Command Execution: Unix Command Injection"] [data "Matched Data: ; more than found within ARGS:foo: a hero; more than"] [severity "CRITICAL"] [ver "OWASP_CRS/3.4.0-dev"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/"] [unique_id "YPEnSRy0s3c8Xzd5uRp5MgAAABc"]

False Negatives at PL1

Caught at PL4

        curl localhost -d 'foo=\r\nV100 CAPABILITY\r\nV101 FETCH 4791'

        curl localhost -d 'foo=\r\nQUIT\r\n'

        curl localhost -d 'foo=\nRCPT TO: test@evil.com\n'

Caught at PL3

        curl localhost -d "foo=.../.../WINDOWS/win.ini"

Caught at PL2

        curl localhost -d "foo=true, $where: '99 == 88'"

        curl localhost -d "foo=db.injection.insert({success:1});"

        curl localhost -d "foo=', $or: [ {}, { 'order':'order"

        curl localhost -d "foo=', $or: [ {}, { 'order':'order"

        curl localhost -d 'foo={{_self.env.registerUndefinedFilterCallback("exec")}}{{_self.env.getFilter("id")}}'

        curl localhost -d 'foo={{+''.__class__.__mro__[2].__subclasses__()[40]('/test/aaaa').read()+}}'

@dune73
Copy link
Member
dune73 commented Jul 16, 2021

For the false negatives, I am unsure what to do. The mail injection stuff that we only catch at PL4 (and only by accident) is asking for a new rule I guess.

The Windows RCE that we only catch at PL3 is substantial. If this tripple dot work in Windows, we should catch it at PL1 I guess.

The stuff that we catch at PL2 is not overly noteworthy, I guess.

@fzipi
Copy link
Member
fzipi commented Jul 16, 2021

Excellent!

I've been playing with the idea of adding fuzzing to go-ftw. I guess from this one that it might make some sense after all...

@dune73
Copy link
Member
dune73 commented Jul 16, 2021

So you would take the payload in an existing test and start to fuzz it?

@fzipi
Copy link
Member
fzipi commented Jul 16, 2021

Yes, that's the idea. Provided that can be done.

@dune73 dune73 mentioned this issue Jul 19, 2021
Closed
@lifeforms lifeforms mentioned this issue Jul 19, 2021
Closed
3 tasks
@dune73 dune73 mentioned this issue Aug 16, 2021
Closed
@dune73
Copy link
Member
dune73 commented Sep 3, 2021

Any status update, @fzipi?

Also: I guess we need a volunteer for the

  • Add Windows triple dot to a PL1 rule
  • New PL2/3 rule that catches the mail injections quoted above

This was referenced Sep 19, 2021
Closed
Closed
Closed
@dune73
Copy link
Member
dune73 commented Sep 19, 2021

The fuzzing is a separate idea. But I have opened two new issues that replace this issues. I'm thus closing this.

@dune73 dune73 closed this as completed Sep 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fzipi fzipi

Labels
➖ False Negative - Evasion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dune73 @fzipi
0