Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Latest commit

 

History

History
100 lines (66 loc) · 2.59 KB

README.md

File metadata and controls

100 lines (66 loc) · 2.59 KB
Raw

tap-sparkthink

tap-sparkthink is a Singer tap for sparkthink.

Built with the Meltano SDK for Singer Taps.

Installation

  • Developer TODO: Update the below as needed to correctly describe the install procedure. For instance, if you do not have a PyPi repo, or if you want users to directly install from your git repo, you can modify this step as appropriate.
pipx install tap-sparkthink

Configuration

Accepted Config Options

  • Developer TODO: Provide a list of config options accepted by the tap.

A full list of supported settings and capabilities for this tap is available by running:

tap-sparkthink --about

Source Authentication and Authorization

  • Developer TODO: If your tap requires special access on the source system, or any special authentication requirements, provide those here.

Usage

You can easily run tap-sparkthink by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-sparkthink --version
tap-sparkthink --help
tap-sparkthink --config CONFIG --discover > ./catalog.json

Developer Resources

  • Developer TODO: As a first step, scan the entire project for the text "TODO:" and complete any recommended steps, deleting the "TODO" references once completed.

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the tap_sparkthink/tests subfolder and then run:

poetry run pytest

You can also test the tap-sparkthink CLI interface directly using poetry run:

poetry run tap-sparkthink --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Open the meltano.yml and follow any "TODO" items listed in the file.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-sparkthink
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-sparkthink --version
# OR run a test `elt` pipeline:
meltano elt tap-sparkthink target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.