Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Can't install meltano-map-transformer on Meltano v3 #8224

Open
peterspiro opened this issue Oct 11, 2023 · 3 comments
Open

bug: Can't install meltano-map-transformer on Meltano v3 #8224

peterspiro opened this issue Oct 11, 2023 · 3 comments
Labels

Comments

@peterspiro
Copy link

Meltano Version

3.1.0

Python Version

3.9

Bug scope

CLI (options, error messages, logging, etc.)

Operating System

OSX

Description

I'm getting the error:

Mapper 'meltano-map-transformer' is not known to Meltano. Try running `meltano lock --update --all` to ensure your plugins are up to date.

when I try to either install meltano-map-transformer or run a job containing a mapping. The error isn't resolved by running meltano lock --update --all.

The error doesn't occur with Meltano v2. It looks like it may have been introduced by #7992.

Code

mappers:
    - name: meltano-map-transformer
      pip_url: "git+https://github.com/MeltanoLabs/meltano-map-transform.git"
      executable: meltano-map-transform
      mappings:
        - name: rename_column
          config:
            stream_maps:
              demographics:
                foo_text: foo
                foo: null
@peterspiro peterspiro added kind/Bug Something isn't working valuestream/Meltano labels Oct 11, 2023
@edgarrmondragon edgarrmondragon changed the title bug: meltano-map-transformer doesn't work on Meltano v3 bug: Can't install meltano-map-transformer on Meltano v3 Oct 11, 2023
@edgarrmondragon
Copy link
Collaborator

Here's a potential workaround:

  1. Remove meltano-map-transformer from the project

  2. Use --from-ref to add the plugin

    meltano add mapper meltano-map-transformer --from-ref https://raw.githubusercontent.com/meltano/hub/main/_data/meltano/mappers/meltano-map-transformer/meltano.yml
  3. Add your mappings

@peterspiro
Copy link
Author

Thanks @edgarrmondragon, that worked. It seems that the namespace config element is now required.

@edgarrmondragon
Copy link
Collaborator

edgarrmondragon commented Oct 11, 2023

I'd rather keep this open since the workaround makes the mapper a custom plugin. We probably want the plugin to work when added with meltano add mapper meltano-map-transformer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants