8000 Fake bot plugin · Issue #2228 · coreruleset/coreruleset · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fake bot plugin #2228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
azurit opened this issue Oct 12, 2021 · 6 comments
Closed

Fake bot plugin #2228

azurit opened this issue Oct 12, 2021 · 6 comments
Labels
👍 Feature Request

Comments

@azurit
Copy link
Member
azurit commented Oct 12, 2021 •
edited
Loading

Motivation

As we all know, many bots are fakeing User-Agent header to look like a, i.e., Googlebot. In lots of cases, users are trusting this header and, for example, are not applying so high restrictions for such requests. Anyway, behavior where anyone is fakeing anything to get something more is considered bad (or should be). This plugin is able to detect and block such requests.

Proposed solution

This is an alternative and more general version of Fake Googlebot plugin.

As @theMiddleBlue suggested, this version of plugin is using DNS PTR records for checking if a bot is real or fake. Plugin consists of these parts:

  • blocking rule
  • Lua script

Currently, plugin is able to detect these fake bots:

  • Googlebot
  • Bingbot
  • Facebookbot

Prototype of this plugin is ready.

Plugin registry PR: coreruleset/plugin-registry#4.
@azurit
Copy link
Member Author
azurit commented Oct 13, 2021 •
edited
Loading

As Luasocket library is using POSIX function getnameinfo for getting PTR records, it is not possible to set any timeout - there is a chance that this might be abused to do a DoS attack. I retested with few IP addresses which are really slow to resolve and, thanks to DNS cache, it should't be a big deal as only first resolve is slow, all other are read from cache.

IPs with slow reverse record resolving:

  • 192.165.48.59 (about 3 seconds)
  • 45.141.84.214 (about 8 seconds)

@theMiddleBlue
Copy link
Contributor

wow, that's true... I didn't think about this kind of problem!

➜  ~ time host -t PTR 192.165.48.59 1.1.1.1
Host 59.48.165.192.in-addr.arpa not found: 2(SERVFAIL)
0,00s user 0,01s system 0% cpu 3,941 total

➜  ~ time host -t PTR 45.141.84.214 1.1.1.1
Host 214.84.141.45.in-addr.arpa not found: 2(SERVFAIL)
0,00s user 0,00s system 0% cpu 8,516 total

weird... why this happens?

@azurit
Copy link
Member Author
azurit commented Oct 17, 2021 •
edited
Loading

Because of the slow / not working DNS servers which are set for resolving PTR records of these IPs. For example, PTR record for IP 45.141.84.214 is saved (or should be) in these DNS servers:
ns1.dns-sshvps.net
ns2.dns-sshvps.net

None of them is working, at least from my location.

$ dig -t ptr 214.84.141.45.in-addr.arpa @ns1.dns-sshvps.net

; <<>> DiG 9.16.1-Ubuntu <<>> -t ptr 214.84.141.45.in-addr.arpa @ns1.dns-sshvps.net
;; global options: +cmd
;; connection timed out; no servers could be reached


$ dig -t ptr 214.84.141.45.in-addr.arpa @ns2.dns-sshvps.net

; <<>> DiG 9.16.1-Ubuntu <<>> -t ptr 214.84.141.45.in-addr.arpa @ns2.dns-sshvps.net
;; global options: +cmd
;; connection timed out; no servers could be reached

@azurit azurit mentioned this issue Nov 15, 2021
Closed
@fzipi fzipi mentioned this issue Dec 5, 2021
Closed
@lifeforms lifeforms mentioned this issue Dec 20, 2021
Closed
@azurit azurit mentioned this issue Dec 21, 2021
Merged
@azurit
Copy link
Member Author
azurit commented Dec 21, 2021

Plugin registry PR: coreruleset/plugin-registry#4

@azurit
Copy link
Member Author
azurit commented Dec 21, 2021

@dune73 @lifeforms --^

@fzipi fzipi mentioned this issue Jan 2, 2022
Closed
@dune73
Copy link
Member
dune73 commented Jan 17, 2022

Repo for new plugin created https://github.com/coreruleset/fake-bot-plugin
@azurit has been made maintainer.

I have also added the plugin into the registry, which is formally too early, but that way @azurit can get on with the work.

@dune73 dune73 closed this as completed Jan 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
👍 Feature Request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dune73 @azurit @theMiddleBlue
0