Scouter is a simple containerized API that allows you to perform various troubleshooting utilities remotely.
It runs common Linux based troubleshooting util 8000 ities such as ping, traceroute and a few others that are documented further on. These utilities support multiple options, so creating specific types of tests is very easy.
It deploys easily to any container infrastructure.
The most powerful aspect of Scouter is its automation and monitoring potential.
These instructions cover building and using the docker container.
You'll need docker installed in order to run this container.
You'll also need to sign up for a GeoLite2 account and generate a license key. License keys can be generated free of charge and are used to authorize access to download MaxMind databases.
The GeoLite2-ASN MaxMind database is used to translate source IP addresses in both the traceroute and dns_traceroute utilities into their associated Autonomous System Numbers.
Run the following to build the Scouter container from source.
git clone https://github.com/stackpath/scouter.git
docker build -t scouter .If you'd like to skip building the container from source then pull the latest version of Scouter from Docker Hub:
Coming soon!SCOUTER_API_SECRET- Specify the shared API secret to be used to authenticate every API call.MMDB_LICENSE_KEY- Specify one of your valid GeoLite2 license keys to be used when pulling the latest GeoLite2-ASN.mmdb.
SCOUTER_MAX_TEST_COUNT- Specify the maximum number of individual tests in a single test payload. Defaults to 10.SCOUTER_MAX_PROCESS_COUNT- Specify the maximum number of parallel processes to be used in test execution. Defaults to 10.API_PORT- Specify the port that Nginx will be listening on. Defaults to 8000.UWSGI_WORKERS- Specify the number of Uwsgi worker processes to use. Defaults to 3.UWSGI_CACHE_ITEMS- Specify the maximum number of Uwsgi cache items. Defaults to 100.UWSGI_CACHE_BLOCKSIZE- Specify the maximum Uwsgi cache size. Defaults to 1000000.BUP_PROXY_TTL- Specify the maximum amount of time that a BUP proxy is allowed to live. Defaults to 300 seconds.BUP_PROXY_PORT_RANGE- Specify the range of ports reserved for BUP proxies. Also serves a pseudo rate limiter. Defaults to 9001-9005.
First, start the Scouter container:
$ docker run --rm --name "scouter" -e "SCOUTER_API_SECRET=secret" -p 8000:8000 scouterOnce started, ensure that everything is working correctly by checking the /api/v1.0/status endpoint:
$ curl -X GET -H "Authorization: secret" "http://localhost:8000/api/v1.0/status" | jqThis produces a response that contains internal information about the Scouter service.
{
"total_requests": 0,
"worker_status": [
{
"avg_rt": 0,
"delta_requests": 0,
"exceptions": 0,
"id": 1,
"last_spawn": 1569856797,
"pid": 41,
"requests": 0,
"respawn_count": 0,
"rss": 0,
"running_time": 0,
"signals": 0,
"status": "busy",
"tx": 0,
"vsz": 0
},
{
"avg_rt": 0,
"delta_requests": 0,
"exceptions": 0,
"id": 2,
"last_spawn": 1569856797,
"pid": 42,
"requests": 0,
"respawn_count": 0,
"rss": 0,
"running_time": 0,
"signals": 0,
"status": "idle",
"tx": 0,
"vsz": 0
},
{
"avg_rt": 0,
"delta_requests": 0,
"exceptions": 0,
"id": 3,
"last_spawn": 1569856797,
"pid": 43,
"requests": 0,
"respawn_count": 0,
"rss": 0,
"running_time": 0,
"signals": 0,
"status": "idle",
"tx": 0,
"vsz": 0
}
]
}Once you've confirmed the API is up and running you can execute tests.
Scouter supports the ability to send multiple tests in a single payload. These tests are
executed in parallel, the number of which is configured with the SCOUTER_MAX_THREAD_COUNT
environment variable.
Here's an example of how to create a new test to perform both an http_request and a dns_lookup:
First start by creating a test via POST:
$ curl -X POST \
-H "Authorization: secret" \
-H "Content-Type: application/json" \
-d '{"http_request": [{"url": "example.com"}], "dns_lookup": [{"qname": "example.com"}]}' \
"http://localhost:8000/api/v1.0/tests" | jqThis command returns a receipt that's used to retrieve test results:
{
"receipt": "78e473ed3397c8fb02b9c9c9b21a9ae1"
}Pass the test ID via the receipt parameter in a subsequent API call:
$ curl -X GET \
-H "Authorization: secret" \
"http://localhost:8000/api/v1.0/tests?receipt=78e473ed3397c8fb02b9c9c9b21a9ae1" | jqThe response contains the test result's details:
{
"is_running": false,
"receipt": "c37f83382242675804820562d2a44210",
"results": {
"dns_lookup": [
{
"failed": false,
"id": "38e618",
"message": null,
"result": {
"answer": [
{
"rclass": "IN",
"rdata": "93.184.216.34",
"rdlen": null,
"rrname": "example.com.",
"ttl": 86400,
"type": "A"
}
],
"elapsed_time": 0.06239771842956543,
"failed": false,
"ns": "169.254.169.254",
"question": {
"qclass": "IN",
"qname": "example.com.",
"qtype": "A"
},
"rcode": "ok",
"timeout_count": 0
}
}
],
"http_request": [
{
"failed": false,
"id": "4051b9",
"message": null,
"result": {
"comment": null,
"failed": false,
"headers": {
"accept-ranges": "bytes",
"age": "325974",
"cache-control": "max-age=604800",
"content-encoding": "gzip",
"content-length": "648",
"content-type": "text/html; charset=UTF-8",
"date": "Wed, 05 Feb 2020 19:57:20 GMT",
"etag": "\"3147526947\"",
"expires": "Wed, 12 Feb 2020 19:57:20 GMT",
"last-modified": "Thu, 17 Oct 2019 07:18:26 GMT",
"server": "ECS (oxr/830C)",
"x-cache": "HIT"
},
"method": "HEAD",
"reason": "OK",
"speed_download": 0,
"status": 200,
"time_appconnect": "0.000",
"time_connect": "0.018",
"time_namelookup": "0.016",
"time_starttransfer": "0.020",
"time_total": "0.023",
"url": "example.com",
"version": "1.1"
}
}
]
}
}There is a lot of data from these two tests. Use a tool like jq to manually parse the output.
Test results can also be deleted. Results automatically expire in 10 minutes if not deleted manually:
$ curl -X DELETE \
-H "Authorization: secret" \
"http://localhost:8000/api/v1.0/tests?receipt=78e473ed3397c8fb02b9c9c9b21a9ae1" | jqTest deletions should produce a success message:
{
"message": "Provided 'receipt' has been successfully deleted."
}| Description | HTTP method | Request path |
|---|---|---|
| Create a new test. | POST | /api/v1.0/tests |
| Retrieve test results. | GET | /api/v1.0/tests?receipt=receipt_id |
| Delete test results. | DELETE | /api/v1.0/tests?receipt=receipt_id |
| Retrieve API status. | GET | /api/v1.0/status |
| Test type | Required | Optional |
|---|---|---|
| browser_request | * url - The webpage URL to attempt to load via the emulated browser. |
* * * |
| http_request | * url - The URL to cURL. |
* * * * * * |
| dns_lookup | * qname - The Domain name that you would like perform a DNS lookup for. |
* * * |
| dns_traceroute | * qname - The domain name to use when crafting the DNS UDP packet. |
* * * |
| ping | * dst - The destination address to ping. Can be either a FQDN or an IP address. |
* * * |
| traceroute | * dst - The destination address to trace to. Can be either a FQDN or an IP address. |
* * * * * |
- @aaroncouch - Initial work
This project is written in Python Version 3.7.3 and uses the PEP 8 style guide. The use of a code linter such as Pylint is highly recommended to keep the code as consistent as possible. All docstrings and comments follow the Chromium Python Style Guidelines.