Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] IndexError on OracleDB yaml data exporter when key column or only datetime as column type is assigned #4587

Open
cristopheridlc opened this issue Feb 15, 2024 · 3 comments · May be fixed by #4646
Assignees
Labels
bug Something isn't working

Comments

@cristopheridlc
Copy link

Mage version

v0.9.63

Describe the bug

When setting a key column and/or setting a column as datetime only for a pipeline block involving an OracleDB yaml data exporter, the block execution fails with an IndexError: list index out of range exception. This occurs despite successful block execution when no key column is assigned.

To reproduce

1. Configure a Python data integrations OracleDB exporter block with a key column in a pipeline.
How I created the block (there is no OracleDB exporter block template in Data explorer -> Databases):
image

Stream schema settings:
image

3. Execute the block.
4. Observe that the execution fails with an IndexError.
image

Expected behavior

The data exporter block should handle the key column assignment without causing an IndexError. The expected behavior is a successful execution of the block and proper export of the data to the OracleDB with the assigned key column.

Screenshots

No response

Operating system

  • OS: Red Hat Enterprise Linux release 8.9 (Ootpa)
  • podman version 4.6.1

Additional context

  1. The issue arises specifically either when assigning a key column, when assigning a column as datetime only, or both, using the OracleDB yaml data exporter.
  2. The error message indicates a list index out of range, suggesting that the code might be attempting to access a list element that does not exist.
  3. @johnson-mage suggested that the error might be due to the type not being set for certain columns in the catalog.json.
    image
  4. The temporary workaround suggested by @johnson-mage (adding a string type to datetime columns) has not led to errors, but resulted in the columns with both string and datetime types becoming CHAR(52) in the Oracle DB, in my specific case, which is not ideal.
    image
@cristopheridlc cristopheridlc added the bug Something isn't working label Feb 15, 2024
@johnson-mage
Copy link
Member

Tagging @matrixstone to help with this.

@johnson-mage
Copy link
Member

The IndexError isn't specific to OracleDB. It could occur with other SQL destinations like MySQL.

@matrixstone matrixstone linked a pull request Feb 25, 2024 that will close this issue
6 tasks
@MartinLoeper
Copy link

Can confirm: Ran into the same issue with postgres

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants