Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL DOUBLE accepts deprecated params that dont propagate from the base DOUBLE type, document this caveat #11132

Open
MacBath opened this issue Mar 10, 2024 · 3 comments
Labels
datatypes things to do with database types, like VARCHAR and others documentation mysql PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers
Milestone

Comments

@MacBath
Copy link

MacBath commented Mar 10, 2024

Describe the bug

Hi im trying to get startet with sql alchemy and alembic and just found a bug.

during the translation of a double precision=17, and decimal_return_scale=7 into mysql i always get an exception.
sqlalchemy.exc.ArgumentError: You must specify both precision and scale or omit both altogether.

When i check the kw at dialects/mysql/types.py i see that the parameter there is still the parameter decimal_return_scale=7, but named argument scale is None.

Optional link from https://docs.sqlalchemy.org which documents the behavior that is expected

No response

SQLAlchemy Version in Use

2.0.28

DBAPI (i.e. the database driver)

mysqlclient

Database Vendor and Major Version

MariaDB 10.*

Python Version

3.11

Operating system

Windows

To Reproduce

from sqlalchemy.orm import DeclarativeBase, MappedAsDataclass, Mapped, mapped_column
from sqlalchemy.types import DOUBLE
from datetime import datetime
from sqlalchemy import create_engine


class Base(MappedAsDataclass, DeclarativeBase):
    """Base Class for ORM Table Modell"""

    pass


metadata = Base.metadata


class mytable(Base):
    __tablename__ = "tablename"

    _unix_dt: Mapped[float] = mapped_column(
        type_=DOUBLE(precision=17, decimal_return_scale=7),
        primary_key=True,
    )
    _dt: Mapped[datetime] = mapped_column(nullable=True, server_default=None)


def main():
    engine = create_engine("mysql+mysqldb://root:secret@localhost/dummdb")
    Base.metadata.create_all(engine)


if __name__ == "__main__":
    main()

Error

raceback (most recent call last):
  File "HOMEDIR\AppData\Local\Programs\Python\Python311\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "HOMEDIR\AppData\Local\Programs\Python\Python311\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy\__main__.py", line 39, in <module>
    cli.main()
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 430, in main
    run()
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "Project.git\classes\dummy.py", line 32, in <module>
    main()
  File "Project.git\classes\dummy.py", line 28, in main
    Base.metadata.create_all(engine)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\schema.py", line 5825, in create_all
    bind._run_ddl_visitor(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 3254, in _run_ddl_visitor
    conn._run_ddl_visitor(visitorcallable, element, **kwargs)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 2460, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 918, in visit_metadata
    self.traverse_single(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 956, in visit_table
    )._invoke_with(self.connection)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 314, in _invoke_with
    return bind.execute(self)
           ^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 1421, in execute
    return meth(
           ^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 180, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 1529, in _execute_ddl
    compiled = ddl.compile(
               ^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\elements.py", line 307, in compile
    return self._compiler(dialect, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 69, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 865, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 910, in process
    return obj._compiler_dispatch(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 141, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 6601, in visit_create_table
    processed = self.process(
                ^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 910, in process
    return obj._compiler_dispatch(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 141, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 6632, in visit_create_column
    text = self.get_column_specification(column, first_pk=first_pk)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\base.py", line 1818, in get_column_specification        
    column.type._unwrapped_dialect_impl(self.dialect),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 882, in _unwrapped_dialect_impl
    return self.dialect_impl(dialect)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 868, in dialect_impl
    return self._dialect_info(dialect)["impl"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 977, in _dialect_info
    impl = self._gen_dialect_impl(dialect)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 992, in _gen_dialect_impl
    return dialect.type_descriptor(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\default.py", line 589, in type_descriptor
    return type_api.adapt_type(typeobj, self.colspecs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 2323, in adapt_type
    return typeobj.adapt(impltype)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 1032, in adapt
    return util.constructor_copy(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\util\langhelpers.py", line 1414, in constructor_copy
    return cls(*args, **kw)
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\types.py", line 185, in __init__
    super().__init__(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\types.py", line 42, in __init__
    raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: You must specify both precision and scale or omit both altogether.

Additional context

No response

@MacBath MacBath added the requires triage New issue that requires categorization label Mar 10, 2024
@CaselIT
Copy link
Member

CaselIT commented Mar 16, 2024

Hi,

Sorry I guess this issue was missed.
I think you will need to use the mysql double directly here.
Also note that the setting decimal_return_scale is ignored if you don't set asdecimal=True

@zzzeek Do you think that double should do a better job here while adapting to the driver native type?

@CaselIT CaselIT added question issue where a "fix" on the SQLAlchemy side is unlikely, hence more of a usage question mysql datatypes things to do with database types, like VARCHAR and others and removed requires triage New issue that requires categorization labels Mar 16, 2024
@zzzeek
Copy link
Member

zzzeek commented Mar 17, 2024

https://dev.mysql.com/doc/refman/8.3/en/floating-point-types.html

"FLOAT(M,D)and DOUBLE(M,D) are nonstandard MySQL extensions; and are deprecated. You should expect support for these variants to be removed in a future version of MySQL. "

Since these are deprecated, it's not critical that our own generic DOUBLE doesnt support these. I would say the answer is to not use these deprecated extensions. We can document this caveat

@zzzeek zzzeek added documentation and removed question issue where a "fix" on the SQLAlchemy side is unlikely, hence more of a usage question labels Mar 17, 2024
@zzzeek zzzeek changed the title MySQL Dialect Double Precision scale and decimal_return_scale missmatch MySQL DOUBLE accepts deprecated params that dont propagate from the base DOUBLE type, document this caveat Mar 17, 2024
@zzzeek
Copy link
Member

zzzeek commented Mar 17, 2024

we should likely alter the exception message in mysql.DOUBLE also to reflect this

@CaselIT CaselIT added the PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers label Apr 12, 2024
@CaselIT CaselIT added this to the 2.0.x milestone Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datatypes things to do with database types, like VARCHAR and others documentation mysql PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers
Projects
None yet
Development

No branches or pull requests

3 participants