Possible type mapping error in recent version

[connorwang]

Going from 3.17 to 3.20 gave us this error. I believe it's a mysql to postgres type mapping error - is there any soultion other than swithing to mysql?

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2023-04-29T22:45:50.102988894Z INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 27ed76f75106 -> 1b8aba201c94, add user group support
2023-04-29T22:45:50.170636020Z INFO  [alembic.runtime.migration] Running upgrade 1b8aba201c94 -> ec2f32c25f34, add more metadata support
INFO  [alembic.runtime.migration] Running upgrade ec2f32c25f34 -> 7f6cdb3621f7, add data element support
Traceback (most recent call last):
2023-04-29T22:45:50.183143817Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
    self.dialect.do_execute(
2023-04-29T22:45:50.183514717Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
2023-04-29T22:45:50.183806625Z     cursor.execute(statement, parameters)
2023-04-29T22:45:50.183813095Z psycopg2.errors.SyntaxError: type modifier is not allowed for type "text"
2023-04-29T22:45:50.183815845Z LINE 10:  description TEXT(16777215), 
2023-04-29T22:45:50.183818745Z                       ^
2023-04-29T22:45:50.183820935Z 

2023-04-29T22:45:50.183825305Z The above exception was the direct cause of the following exception:
2023-04-29T22:45:50.183827365Z 
Traceback (most recent call last):
2023-04-29T22:45:50.183832296Z   File "/usr/local/bin/alembic", line 8, in <module>
    sys.exit(main())
2023-04-29T22:45:50.183846516Z   File "/usr/local/lib/python3.9/site-packages/alembic/config.py", line 590, in main
2023-04-29T22:45:50.184246757Z     CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python3.9/site-packages/alembic/config.py", line 584, in main
2023-04-29T22:45:50.184339779Z     self.run_cmd(cfg, options)
2023-04-29T22:45:50.184347999Z   File "/usr/local/lib/python3.9/site-packages/alembic/config.py", line 561, in run_cmd
2023-04-29T22:45:50.184554975Z     fn(
  File "/usr/local/lib/python3.9/site-packages/alembic/command.py", line 322, in upgrade
    script.run_env()
2023-04-29T22:45:50.184727820Z   File "/usr/local/lib/python3.9/site-packages/alembic/script/base.py", line 569, in run_env
2023-04-29T22:45:50.184848543Z     util.load_python_file(self.dir, "env.py")
2023-04-29T22:45:50.184856483Z   File "/usr/local/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
2023-04-29T22:45:50.184939825Z     module = load_module_py(module_id, path)
2023-04-29T22:45:50.184958256Z   File "/usr/local/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
2023-04-29T22:45:50.185042088Z     spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
2023-04-29T22:45:50.185186562Z   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "migrations/env.py", line 76, in <module>
2023-04-29T22:45:50.185336086Z     run_migrations_online()
2023-04-29T22:45:50.185342366Z   File "migrations/env.py", line 70, in run_migrations_online
    context.run_migrations()
2023-04-29T22:45:50.185441029Z   File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.9/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
2023-04-29T22:45:50.185796469Z     self.get_context().run_migrations(**kw)
2023-04-29T22:45:50.185802449Z   File "/usr/local/lib/python3.9/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
2023-04-29T22:45:50.186010484Z     step.migration_fn(**kw)
2023-04-29T22:45:50.186016655Z   File "/opt/querybook/querybook/migrations/versions/7f6cdb3621f7_add_data_element_support.py", line 20, in upgrade
    op.create_table(
  File "<string>", line 8, in create_table
2023-04-29T22:45:50.186161368Z   File "<string>", line 3, in create_table
2023-04-29T22:45:50.186167439Z   File "/usr/local/lib/python3.9/site-packages/alembic/operations/ops.py", line 1254, in create_table
2023-04-29T22:45:50.186557469Z     return operations.invoke(op)
  File "/usr/local/lib/python3.9/site-packages/alembic/operations/base.py", line 399, in invoke
2023-04-29T22:45:50.186706733Z     return fn(self, operation)
2023-04-29T22:45:50.186712783Z   File "/usr/local/lib/python3.9/site-packages/alembic/operations/toimpl.py", line 114, in create_table
2023-04-29T22:45:50.186834507Z     operations.impl.create_table(table)
2023-04-29T22:45:50.186839967Z   File "/usr/local/lib/python3.9/site-packages/alembic/ddl/impl.py", line 354, in create_table
2023-04-29T22:45:50.186985971Z     self._exec(schema.CreateTable(table))
  File "/usr/local/lib/python3.9/site-packages/alembic/ddl/impl.py", line 195, in _exec
2023-04-29T22:45:50.187016831Z     return conn.execute(construct, multiparams)
2023-04-29T22:45:50.187020342Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1306, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
2023-04-29T22:45:50.187637198Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
2023-04-29T22:45:50.187773572Z     return connection._execute_ddl(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1398, in _execute_ddl
2023-04-29T22:45:50.188419659Z     ret = self._execute_context(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
    self._handle_dbapi_exception(
2023-04-29T22:45:50.189378445Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
2023-04-29T22:45:50.190416674Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
2023-04-29T22:45:50.191240136Z     self.dialect.do_execute(
2023-04-29T22:45:50.191255156Z   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
2023-04-29T22:45:50.191638147Z     cursor.execute(statement, parameters)
2023-04-29T22:45:50.191671888Z sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) type modifier is not allowed for type "text"
2023-04-29T22:45:50.191676028Z LINE 10:  description TEXT(16777215), 
2023-04-29T22:45:50.191678808Z                       ^
2023-04-29T22:45:50.191680958Z 
2023-04-29T22:45:50.191683318Z [SQL: 
2023-04-29T22:45:50.191685768Z CREATE TABLE data_element (
2023-04-29T22:45:50.191687998Z 	id SERIAL NOT NULL, 
2023-04-29T22:45:50.191690228Z 	created_at TIMESTAMP WITHOUT TIME ZONE, 
	updated_at TIMESTAMP WITHOUT TIME ZONE, 
2023-04-29T22:45:50.191702008Z 	created_by INTEGER, 
2023-04-29T22:45:50.191704558Z 	metastore_id INTEGER NOT NULL, 
	name VARCHAR(255) NOT NULL, 
2023-04-29T22:45:50.191709059Z 	type VARCHAR(4096) NOT NULL, 
2023-04-29T22:45:50.191711299Z 	description TEXT(16777215), 
	properties JSON, 
	PRIMARY KEY (id), 
	FOREIGN KEY(created_by) REFERENCES "user" (id) ON DELETE SET NULL, 
	FOREIGN KEY(metastore_id) REFERENCES query_metastore (id) ON DELETE CASCADE
)
2023-04-29T22:45:50.191724839Z 
2023-04-29T22:45:50.191727029Z ]
2023-04-29T22:45:50.191729789Z (Background on this error at: https://sqlalche.me/e/14/f405)
2023-04-29T22:45:52.507559149Z /opt/querybook
2023-04-29T22:45:52.510807637Z Running: python3 querybook/server/runweb.py --debug --webpack 10001
2023-04-29T22:45:55.810651964Z  * Restarting with stat
Traceback (most recent call last):
  File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links
2023-04-29T22:45:55.814841114Z   File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links
2023-04-29T22:45:55.814845694Z AssertionError: (None, <callback at 0x7fd45147ab80 args=([],)>)
2023-04-29T22:45:55.814946447Z 2023-04-29T22:45:55Z <callback at 0x7fd45147ab80 args=([],)> failed with AssertionError
2023-04-29T22:45:55.814954527Z 

[fp57918]

+1 here with postgres, same issue

INFO [alembic.runtime.migration] Running upgrade ec2f32c25f34 -> 7f6cdb3621f7, add data element support
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.SyntaxError: type modifier is not allowed for type "text"
LINE 10: description TEXT(16777215),

[little-eyes]

+1 here. same issue when using postgresql 15 as the backend db.

2023-11-21 20:56:17.765 | Traceback (most recent call last):
  | 2023-11-21 20:56:17.765 | File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
  | 2023-11-21 20:56:17.765 | self.dialect.do_execute(
  | 2023-11-21 20:56:17.765 | File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
  | 2023-11-21 20:56:17.766 | cursor.execute(statement, parameters)
  | 2023-11-21 20:56:17.766 | psycopg2.errors.SyntaxError: type modifier is not allowed for type "text"
  | 2023-11-21 20:56:17.766 | LINE 10:  description TEXT(16777215),

[VictorMikhaylov]

+1
same issue

master branch
postgres:16.1-alpine3.17

[shakeelansari63]

Same issue for us. We are only allowed to use Postgres and we cannot use MySql/MariaDb. This makes it impossible for us to use Querybook in any way.

Please fix the issue or provide some workaround..

[shakeelansari63]

Built a custom image from Querybook image to workaround this issue.
And used sed to remove the length from text fields.

FROM querybook/querybook:latest

RUN sed -Ei 's/sa.Text\(length=[0-9]+\)/sa.Text()/g' /opt/querybook/querybook/migrations/versions/*.py