Fetches public address data for use by the grasshopper-loader.
- node
- GDAL 1.11.2
- Docker
- Optional, removes need for preinstalled node and GDAL
- Clone the repo
- Install GDAL 1.11.2
- Install node 0.12.x
npm install
docker build --rm --tag=<your-image:your-tag> .
docker pull hmda/retriever:0.3
The retriever is a command-line application that can be used in the following ways:
Run ./retriever.js with the following options:
- -f, --file Required The metadata file that contains data source informatation. See
data.jsonfor example schemas. - -b, --bucket An AWS S3 bucket where the data should be loaded after retrieval.
- -d, --directory The directory relative to the current directory or an S3 bucket's root, where data should be deposited. If neither a bucket not a directory is provided, the retriever will check for file changes but will not load them anywhere. This is useful for monitoring endpoints via a scheduler. If a bucket is provided with no directory, data will be loaded to the bucket's root.
- -p, --profile Default: default The aws credentials profile in ~/.aws/credentials to use. Environment variables will override this option.
- -m, --match A string or regex that
namesfrom the metafile must match in order to be retrieved. - -q, --quiet Default: false Suppresses logging. Useful for automated error testing without polluting logs.
- Run
./docker-run.sh <image-name:tag-name> <rest.. of.. args.. from.. above> - The shell script is used to automatically set up a volume containing your
~/.aws/credentialsfile
npm test
./docker-test.sh <image-name:tag-name>