Cloud ready L4 TCP proxy with first-class support for dynamic listeners.
Exposing dynamic backends on the static frontend ports is the bread-and-butter of any modern proxy. Load balancing multiple dynamic backends from one ingress point using on-demand opened ports is something that, for a good reason as it might poise certain security concerns, is not that simple. But when you exactly know what you are doing you are pretty much on your own.
- dill
- Table of Contents
- Installation
- Getting Started
- Routing
- Configuration
- Values
- listeners.allowed
map - listeners.port_min
integer - listeners.port_max
integer - peek.listener
string - runtime.gomaxprocs
integer - routing.file.path
string - routing.file.watch
bool - routing.http.endpoint
string - routing.http.poll_interval
duration - routing.http.poll_timeout
duration - routing.consul.address
string
- listeners.allowed
- Formats
- Values
- Project status
- Alternatives
You can find pre-built binaries in Releases.
$ make build
Compiled binary will be available inside dist/ directory.
$ docker pull fwkz/dill
or build the image yourself from the sources
make image
First of all you have to define how you want to route the incoming traffic. dill provides multiple routing providers that allow you to apply live changes to the routing configuration. By far simplest approach is routing.file.
# /etc/dill/config.toml
[routing.file]
path = "/etc/dill/routing.toml"
watch = true # /etc/dill/routing.toml
[[services]]
name = "foobar"
listener = "any:1234"
backends = ["192.168.10.1:5050"]$ dill -config /etc/dill/config.tomlAnd that's it! dill will bind service running on 192.168.10.1:5050 to 0.0.0.0:1234 on the host running the dill. Make sure to read more about interface labels, e.g., any, local
dill offers multiple routing providers that allow you to apply live changes to the routing configuration.
It is the simplest way of defining routing. All routing logic is being kept in a separate config file. By setting routing.file.watch = true you can also subscribe to changes made to the routing configuration file which would give you the full power of dill's dynamic routing capabilities.
[routing.file]
path = "/etc/dill/routing.toml"
watch = true dill can poll the HTTP endpoint for its routing configuration with a predefined time interval. Fetched data should be compliant with routing configuration schema and it will be parsed based on the response Content-Type header.
[routing.http]
endpoint = "http://127.0.0.1:8000/config/routing.json"
poll_interval = "5s"
poll_timeout = "5s"dill can build its routing table based on services registered in Consul. All you need to do in order to expose Consul registered service in dill instace is to add appropriate tags.
dilltag registers service and its updates withdillinstance.dill.listenerbinds, based on predefined listeners declared bylisteners.allowed, service to specific address and port.
{
"service": {
"tags": [
"dill",
"dill.listener=local:5555",
],
}
}In order to pass traffic via proxy make sure to add dill.proxy tag:
{
"service": {
"tags": [
"dill",
"dill.listener=local:5555",
"dill.proxy=socks5://admin:password@192.168.10.11:1080"
],
}
}dill distributes load across the backends using round-robin strategy
The routing configuration should be compliant with following schema:
# /etc/dill/routing.toml
[[services]]
name = "foo"
listener = "local:1234"
backends = ["127.0.0.1:5050"]
proxy = "socks5://user:pass@127.0.0.1:1080" # optional
[[services]]
name = "bar"
listener = "any:4444"
backends = ["127.0.0.1:4000", "127.0.0.1:4001"]or equivalent in different format e.g. JSON, YAML:
{
"services": [
{
"name": "foo",
"listener": "local:1234",
"backends": [
"127.0.0.1:5050"
],
"proxy": "socks5://user:pass@127.0.0.1:1080"
},
{
"name": "bar",
"listener": "any:4444",
"backends": [
"127.0.0.1:4000",
"127.0.0.1:4001"
]
}
]
}services:
- name: foo
listener: local:1234
backends:
- 127.0.0.1:5050
proxy: socks5://user:pass@127.0.0.1:1080
- name: bar
listener: any:4444
backends:
- 127.0.0.1:4000
- 127.0.0.1:4001Name of the service.
Listener that binds, based on predefined list declared by listeners.allowed, backend to specific address and port.
List of backend services that will be load balanced.
Proxy address if you want to tunnel the traffic.
dill is capable of tunneling traffic to backend services using SOCKS proxy.
# /etc/dill/routing.toml
[[services]]
name = "foobar"
listener = "any:4444"
backends = ["192.168.10.11:4001"]
proxy = "socks5://user:password@192.168.10.10:1080" incoming ┌───────────┐ ┌─────────────┐ ┌───────────────┐
─────connection────►4444 dill ├────────►1080 SOCKS ├─────────►4001 Backend │
└───────────┘ └─────────────┘ └───────────────┘
192.168.10.10 192.168.10.11
dill already comes with sane defaults but you can adjust its behaviour providing configuration file
$ dill -config config.tomlor use environment variables
$ export DILL_CONSUL_ADDRESS="http://127.0.0.1:8500"
$ DILL_LISTENERS_PORT_MIN=1234 dillInterface addresses that are allowed to be bind to by upstream services. Address labels (keys in the map) are opaque for dill.
Imagine that a machine hosting dill has two interfaces, one is internal (192.168.10.10) and the other is external (12.42.22.65). You might want to use the following setup
[listeners.allowed]
internal = "192.168.10.10"
public = "12.42.22.65"with such configuration, upstream services that want to be accessible on 12.42.22.65:5555 can use the public listener in Consul tags dill.listener=public:5555.
default: {"local": "127.0.0.1", "any": "0.0.0.0"}
Minimal port value at which it will be allowed to expose upstream services. Backends requesting to be exposed on lower ports will be dropped from routing.
default: 1024
Maximum port value at which it will be allowed to expose upstream services. Backends requesting to be exposed on higher ports will be dropped from routing.
default: 49151
Address on which Peek will be exposed. Peek is a TCP debug server spawned alongside the dill. Connecting to it will return the current state of the routing table. By default Peek is turned off.
default: ""
$ nc 127.0.0.1 2323
0.0.0.0:4444
├ round_robin
├──➤ 192.168.10.17:1234
├──➤ 192.168.10.23:2042
0.0.0.0:8088
├ round_robin
├──➤ 192.168.10.11:5728
├──➤ 192.168.65.87:5942
Value of Go's runtime.GOMAXPROCS()
default: equals to runtime.NumCPU()
Location of routing configuration file.
Subscribe to changes made to the routing configuration file which would give you the full power of dill's dynamic routing capabilities.
default: true
Endpoint which http provider will poll for routing configuration
How often http provider will poll endpoint for routing configuration
default: 5s
Maximum time http provider will wait when fetching routing configuration
default: 5s
Consul address from which dill will fetch the updates and build the routing table.
Configuration is powered by Viper so it's possible to use format that suits you best.
reading from JSON, TOML, YAML, HCL, envfile and Java properties config files
dill uses the following precedence order:
- environment variable
- config file
- default value
[listeners]
port_min = 1024
port_max = 49151
[listeners.allowed]
local = "127.0.0.1"
any = "0.0.0.0"
[routing.consul]
address = "http://127.0.0.1:8500"
[peek]
listener = "127.0.0.1:4141"
[runtime]
gomaxprocs = 4listeners:
port_min: 1024
port_max: 49151
allowed:
local: "127.0.0.1"
any: "0.0.0.0"
routing:
http:
endpoint: "http://127.0.0.1:8000/config/routing.json"
poll_interval: "5s"
poll_timeout: "5s"
peek:
listener: "127.0.0.1:4141"
runtime:
gomaxprocs: 4{
"listeners": {
"port_min": 1024,
"port_max": 49151,
"allowed": {
"local": "127.0.0.1",
"any": "0.0.0.0"
}
},
"routing": {
"file": {
"path": "/Users/fwkz/Devel/dill/configs/routing.toml",
"watch": true
}
},
"peek": {
"listener": "127.0.0.1:4141"
},
"runtime": {
"gomaxprocs": 4
}
}Variables should be prefixed with DILL and delimited with underscore e.g. consul.address becomes DILL_CONSUL_ADDRESS.
$ export DILL_CONSUL_ADDRESS="http://127.0.0.1:8500"
$ DILL_LISTENERS_PORT_MIN=1234 dillConcept of dynamic listeners is experimental and should be used with responsibility. There might be some breaking changes in the future.