mapper-fivetran is a Singer mapper for Fivetran.
Built with the Meltano Mapper SDK for Singer Mappers.
mapper-fivetran maps incoming data to a Fivetran-compatible format. This involves:
- Flattening top-level properties
- Converting property names to snake-case
- Adding properties pertaining to Fivetran system columns
_fivetran_id: MD5-hash of a record, added when nokey_propertiesare defined for the stream_fivetran_synced: ISO8601 timestamp of when the record was processed by the mapper
A full list of supported settings and capabilities for this mapper is available by running:
mapper-fivetran --aboutThis Singer mapper will automatically import any environment variables within the working directory's
.env if the --config=ENV is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the .env file.
You can easily run mapper-fivetran by itself or in a pipeline using Meltano.
mapper-fivetran --version
mapper-fivetran --helpFollow these instructions to contribute to this project.
Prerequisites:
- Python 3.9+
- uv
uv syncCreate tests within the tests subfolder and
then run:
uv run pytestYou can also test the mapper-fivetran CLI interface directly using uv run:
uv run mapper-fivetran --helpTesting with Meltano
Note: This mapper will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.
Next, install Meltano (if you haven't already) and any needed plugins:
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd mapper-fivetran
meltano installNow you can test and orchestrate using Meltano:
# Run a test `run` pipeline:
meltano run tap-smoke-test mapper-fivetran target-jsonlSee the dev guide for more instructions on how to use the SDK to develop your own taps, targets, and mappers.