8000 Lots of false positives with 932236 · Issue #3220 · coreruleset/coreruleset · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Lots of false positives with 932236 #3220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
EsadCetiner opened this issue May 22, 2023 · 10 comments · Fixed by #3274
Closed

Lots of false positives with 932236 #3220

EsadCetiner opened this issue May 22, 2023 · 10 comments · Fixed by #3274
Assignees
@franbuehler
Labels
➕ False Positive PR available this issue is referenced by an active pull request

Comments

@EsadCetiner
Copy link
Member

Description

Rule 932236 is triggered frequently when using words like mail or tasks mixed in with capital letters, spaces, or numbers. for example:
MailerUI
tasksListView

I've tuned away this false positive but this rule is too aggressive in my opinion, at least for PL-2.

How to reproduce the misbehavior (-> curl call)

curl https://example.com/?args=MailerUI
curl https://example.com/?args=tasksListView

Logs

---cu9fDCI1---A--
[22/May/2023:17:09:31 +1000] 168473937193.473129 0.0.0.0 27354 0.0.0.0 443
---cu9fDCI1---B--
POST /SOGo/so/postmaster@example.com/Calendar/saveSelectedList HTTP/2.0
sec-gpc: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
sec-fetch-site: same-origin
sec-ch-ua-mobile: ?0
origin: https://0.0.0.0
x-xsrf-token: 3c6a9920cb5dda8ba494e9ede93dfffed5339c2e
accept-language: en-US,en
content-type: application/json;charset=UTF-8
accept: application/json, text/plain, */*
sec-ch-ua: "Brave";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-platform: "Windows"
referer: https://0.0.0.0/SOGo/so/postmaster@example.com/Calendar/view
content-length: 24
host: 0.0.0.0
sec-fetch-mode: cors
sec-fetch-dest: empty
accept-encoding: gzip, deflate, br

---cu9fDCI1---D--

---cu9fDCI1---E--
<html>\x0d\x0a<head><title>403 Forbidden</title></head>\x0d\x0a<body>\x0d\x0a<center><h1>403 Forbidden</h1></center>\x0d\x0a<hr><center>nginx</center>\x0d\x0a</body>\x0d\x0a</html>\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a

---cu9fDCI1---F--
HTTP/2.0 403
Server: nginx
Date: Mon, 22 May 2023 07:09:31 GMT
Content-Length: 548
Content-Type: text/html
Connection: close

---cu9fDCI1---H--
ModSecurity: Warning. Matched "Operator `Rx' with parameter `(?i)(?:(?:^|=)[\s\v]*(?:t[\"'\)\[-\x5c]*(?:(?:(?:\|\||&&)[\s\v]*)?\$[!#\(\*\-0-9\?-@_a-\{]*)?\x5c?i[\"'\)\[-\x5c]*(?:(?:(?:\|\||&&)[\s\v]*)?\$[!#\(\*\-0-9\?-@_a-\{]*)?\x5c?m[\"'\)\[-\x5c]*(?:(?:(?:\|\ (5267 characters omitted)' against variable `ARGS:json.list' (Value: `tasksListView' ) [file "/etc/nginx/modsecurity/coreruleset/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "1216"] [id "932236"] [rev ""] [msg "Remote Command Execution: Unix Command Injection (command without evasion)"] [data "Matched Data: task found within ARGS:json.list: tasksListView"] [severity "2"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/2"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "0.0.0.0"] [uri "/SOGo/so/postmaster@example.com/Calendar/saveSelectedList"] [unique_id "168473937193.473129"] [ref "o0,4v10,13"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `5' ) [file "/etc/nginx/modsecurity/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "176"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "0"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [hostname "0.0.0.0"] [uri "/SOGo/so/postmaster@example.com/Calendar/saveSelectedList"] [unique_id "168473937193.473129"] [ref ""]

---P1Yt9ImY---A--
[22/May/2023:17:05:35 +1000] 168473913548.175735 0.0.0.0 27311 0.0.0.0 443
---P1Yt9ImY---B--
POST /SOGo/so/postmaster@example.com/labels HTTP/2.0
sec-gpc: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
sec-fetch-site: same-origin
sec-ch-ua-mobile: ?0
origin: https://0.0.0.0
x-xsrf-token: 3c6a9920cb5dda8ba494e9ede93dfffed5339c2e
accept-language: en-US,en
content-type: application/json;charset=UTF-8
accept: application/json, text/plain, */*
sec-ch-ua: "Brave";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-platform: "Windows"
referer: https://0.0.0.0/SOGo/so/postmaster@example.com/Preferences
content-length: 24
host: 0.0.0.0
sec-fetch-mode: cors
sec-fetch-dest: empty
accept-encoding: gzip, deflate, br

---P1Yt9ImY---D--

---P1Yt9ImY---E--
<html>\x0d\x0a<head><title>403 Forbidden</title></head>\x0d\x0a<body>\x0d\x0a<center><h1>403 Forbidden</h1></center>\x0d\x0a<hr><center>nginx</center>\x0d\x0a</body>\x0d\x0a</html>\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a<!-- a padding to disable MSIE and Chrome friendly error page -->\x0d\x0a

---P1Yt9ImY---F--
HTTP/2.0 403
Server: nginx
Date: Mon, 22 May 2023 07:05:35 GMT
Content-Length: 548
Content-Type: text/html
Connection: close

---P1Yt9ImY---H--
ModSecurity: Warning. Matched "Operator `Rx' with parameter `(?i)(?:(?:^|=)[\s\v]*(?:t[\"'\)\[-\x5c]*(?:(?:(?:\|\||&&)[\s\v]*)?\$[!#\(\*\-0-9\?-@_a-\{]*)?\x5c?i[\"'\)\[-\x5c]*(?:(?:(?:\|\||&&)[\s\v]*)?\$[!#\(\*\-0-9\?-@_a-\{]*)?\x5c?m[\"'\)\[-\x5c]*(?:(?:(?:\|\ (5267 characters omitted)' against variable `ARGS:json.framework' (Value: `MailerUI' ) [file "/etc/nginx/modsecurity/coreruleset/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "1216"] [id "932236"] [rev ""] [msg "Remote Command Execution: Unix Command Injection (command without evasion)"] [data "Matched Data: Mail found within ARGS:json.framework: MailerUI"] [severity "2"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/2"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "0.0.0.0"] [uri "/SOGo/so/postmaster@example.com/labels"] [unique_id "168473913548.175735"] [ref "o0,4v15,8"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `5' ) [file "/etc/nginx/modsecurity/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "176"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "0"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [hostname "0.0.0.0"] [uri "/SOGo/so/postmaster@example.com/labels"] [unique_id "168473913548.175735"] [ref ""]

Your Environment

  • CRS version: Latest CRS 4 Dev version as of the creation of this issue
  • Paranoia level setting: PL-2
  • ModSecurity version: 3.0.9
  • Web Server: Nginx 1.18.0
  • Operating System and version: Ubuntu 22.04

Confirmation

[X] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.
@franbuehler
Copy link
Contributor
franbuehler commented May 22, 2023 •
edited
Loading

Thank you for your report and sorry that you have issues with the rule.
I can confirm these two false positives. Even if they are at PL2 I would suggest eliminating them as this is easily solved.
They could be resolved by adding a word boundary after the regex, by adding a \b as a suffix comment in https://github.com/coreruleset/coreruleset/blob/v4.0/dev/regex-assembly/932236.ra:

##! Please refer to the documentation at
##! https://coreruleset.org/docs/development/regexp_assemble/.

##!+ i

##!> assemble
  ##!> assemble
    ##!> include unix-shell-evasion-prefix-start-of-string
  ##!<

  ##!> assemble
    ##!> include unix-shell-evasion-prefix
  ##!<
##!<
##!=>

##!> include unix-shell-upto3-with-params
##!> include unix-shell-4andup-with-params

##! Add suffix word boundary:
##!$ \b

If you all agree, I can open a PR for that.

@theseion
Copy link
Contributor

I'd be careful with the word boundary. It might be better to adjust the internal matching logic there.

@dune73
Copy link
Member
dune73 commented Jun 1, 2023

Not sure I follow you @theseion. What is the problem with the word boundary in this regex? Do we have shell command name substrings in the list?

@theseion
Copy link
Contributor
theseion commented Jun 3, 2023

Some of the words are already being matched with logic that requires some specific token to follow the word. Simply adding a word boundary match would

  1. change the semantics for all words without special matching
  2. match a word boundary after the the tokens that must follow some of the words

This is like using a single screw driver for two sizes of screws.

@dune73
Copy link
Member
dune73 commented Jun 5, 2023

Got you. Thanks. Do you have a proposal with adjusting the internal matching logic?

@theseion
Copy link
Contributor
theseion commented Jun 6, 2023

I suggest we run the list through the english word filter (spell.sh) and then add boundary modifiers to all of the matches (what that looks like depends on the files).

@franbuehler
Copy link
Contributor
franbuehler commented Jun 8, 2023 •
edited
Loading

Result of extended spell.sh:

$ bash util/fp-finder/spell.sh regex-assembly/include/unix-shell-upto3-with-params.ra 
   - found English word: 7z
   - found English word: apt
   - found English word: ash
   - found English word: dig
   - found English word: ls
   - found English word: rev
   - found English word: tee
   - found English word: top
   - found English word: w3m
   - found English word: who
   - found English word: yum
   - found English word: zip

$ bash util/fp-finder/spell.sh regex-assembly/include/unix-shell-4andup-with-params.ra 
   - found English word: apt-get
   - found English word: aria2c
   - found English word: bash
   - found English word: builtin
   - found English word: check_cups
   - found English word: check_log
   - found English word: check_memory
   - found English word: check_raid
   - found English word: composer
   - found English word: curl
   - found English word: easy_install
   - found English word: emacs
   - found English word: function
   - found English word: gawk
   - found English word: hostname
   - found English word: ispell
   - found English word: mail
   - found English word: nroff
   - found English word: perms
   - found English word: run-parts
   - found English word: sendmail
   - found English word: shell
   - found English word: start-stop-daemon
   - found English word: strings
   - found English word: task
   - found English word: telnet
   - found English word: troff
   - found English word: uncompress
   - found English word: unzip
   - found English word: update-alternatives
   - found English word: volatility
   - found English word: xterm
   - found English word: yarn

Ah, some of the commands in those 2 files are already extended with a {{space-or-redirect}} to prevent false positives, for example: ab{{space-or-redirect}}

This means, I have to take the original list (without the {{space-or-redirect}}) and add {{space-or-redirect}} to the commands that are listed by spell.sh.

Does this sound like a good plan?

@franbuehler franbuehler self-assigned this Jun 8, 2023
@theseion
Copy link
Contributor
theseion commented Jun 8, 2023

Yes. Why are there weird words in the output from spell.sh? 7z and w3m should not be matched as English words IMO.

Also, see the PR I just opened: #3238.

@dune73
Copy link
Member
dune73 commented Jun 12, 2023

New output when using PR in #3238:

$ ./spell.sh ../../regex-assembly/include/unix-shell-upto3-with-params.ra
-> checking unix-shell-upto3-with-params.ra
   `- found English word: apt
   `- found English word: ash
   `- found English word: dig
   `- found English word: ls
   `- found English word: rev
   `- found English word: tee
   `- found English word: top
   `- found English word: who
   `- found English word: yum
   `- found English word: zip
$ ./spell.sh ../../regex-assembly/include/unix-shell-4andup-with-params.ra 
-> checking unix-shell-4andup-with-params.ra
   `- found English word: bash
   `- found English word: builtin
   `- found English word: composer
   `- found English word: curl
   `- found English word: emacs
   `- found English word: function
   `- found English word: gawk
   `- found English word: hostname
   `- found English word: ispell
   `- found English word: mail
   `- found English word: nroff
   `- found English word: perms
   `- found English word: sendmail
   `- found English word: shell
   `- found English word: strings
   `- found English word: task
   `- found English word: telnet
   `- found English word: troff
   `- found English word: uncompress
   `- found English word: unzip
   `- found English word: volatility
   `- found English word: xterm
   `- found English word: yarn

So the situation improved when compared to above.

@fzipi fzipi mentioned this issue Jul 17, 2023
Closed
@franbuehler
Copy link
Contributor

I'll work on this after my vacation...

@franbuehler franbuehler mentioned this issue Jul 31, 2023
Merged
@franbuehler franbuehler added the PR available this issue is referenced by an active pull request label Jul 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@franbuehler franbuehler

Labels
➕ False Positive PR available this issue is referenced by an active pull request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(932236): add space to keywords mail and task franbuehler/coreruleset
4 participants
@theseion @dune73 @franbuehler @EsadCetiner
0