Support SQLAlchemy 2.0.x #938

Ahmad-Wahid · 2023-12-19T15:12:51Z

Description

This PR updates SQLAlchemy dependency from version 1.4 to 2.0. The update underwent thorough testing to ensure compatibility, and all the tests passed successfully.

How to test

Run make install-for-dev to install the required packages. Do the testing using the UI and CLI.

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

…nto feature/update-sqlalchemy-2.0 # Conflicts: # requirements/3.10/app.txt

…hemy-2.0

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Signed-off-by: F.N. Claessen <felix@seita.nl>

…hemy-2.0

nhoening · 2024-02-06T21:07:31Z

We can run the script @Nischay-Pro wrote to update the dependencies for all supported versions.

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

flexmeasures/data/models/time_series.py

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

flexmeasures/cli/data_delete.py

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x

Please don't be scared by the high number of comments I made. Most of them are just duplicate suggestions to reduce the verbosity of the queries where possible (I was just keeping track of occurrences).

The code would probably also become cleaner if we refactor a bit, but probably that would be better as a follow-up PR.

I did find a couple of potentially serious issues with some of the delete statements (those that filter by name rather than by ID). Here we should take a critical look, but I believe the fixes should be easy.

All in all I think you did a great job on this PR. This looked like hard work. Some of these queries weren't trivial (e.g. in the migration files).

flexmeasures/ui/crud/users.py

flexmeasures/api/dev/sensors.py

flexmeasures/api/dev/tests/conftest.py

flexmeasures/api/v3_0/assets.py

flexmeasures/data/services/scheduling.py

flexmeasures/data/services/users.py

flexmeasures/data/tests/test_annotations.py

flexmeasures/data/tests/test_scheduling_repeated_jobs.py

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid · 2024-02-14T16:44:17Z

@Flix6x Some tests are failing, please fix them if you can. I have tried but couldn't understand them.

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

flexmeasures/data/services/users.py

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Signed-off-by: F.N. Claessen <felix@seita.nl>

…hemy-2.0

Signed-off-by: F.N. Claessen <felix@seita.nl>

…n IDs Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: fix typos Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: add readme comment on updating requirements Signed-off-by: F.N. Claessen <felix@seita.nl> * chore: run ci/update-packages.sh Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: F.N. Claessen <felix@seita.nl>

Ahmad-Wahid added 9 commits December 16, 2023 17:56

import declarative object from orm

8a90135

use db.session.get instead of query.get

2289efb

refactor users to use session.get()

68adac5

refactor assets and view_utils to use session.get and session.execute

8314f1c

refactor generic_assets to use session.get and session.execute

401cce9

refactor data-show to use session.get and session.execute

859092d

remove commented in code

da190ee

refactor account to use session.get()

c3423c2

adopt sqlalchemy 2 features in data_add

8da4eef

Ahmad-Wahid self-assigned this Dec 19, 2023

Ahmad-Wahid added the dependency-hygiene label Dec 19, 2023

Ahmad-Wahid added 4 commits December 19, 2023 16:41

refactore cli scripts to support sqlalchemy 2.0

c6b0ea5

update v3_0/assets

5a0483e

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

update api to use sqlalchemy 2.0

3a996d1

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

update models to support sqlalchemy 2.0

bac86d7

Ahmad-Wahid linked an issue Dec 19, 2023 that may be closed by this pull request

Support SQLAlchemy 2 #607

Closed

Ahmad-Wahid and others added 14 commits December 20, 2023 17:58

update migration scripts

568f1f5

update api

6905157

update ui to use sqlalchemy 2.0

f2366e0

update cli queries to sqlalchemy 2.0 version

20bb740

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

migrate queries to sqlalchemy 2.0

a3b98c7

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Merge branch 'main' into feature/update-sqlalchemy-2.0

b06b5de

update requirements

6083cbc

Merge remote-tracking branch 'origin/feature/update-sqlalchemy-2.0' i…

57afa7d

…nto feature/update-sqlalchemy-2.0 # Conflicts: # requirements/3.10/app.txt

use session.execute instead of engine.execute

83b450f

Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

501c56b

…hemy-2.0

first check for a table if exists then data

b1ec01d

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

add zone to session before returning it

0e97f52

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

update last value datatime query

07b866b

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

update data_gen.py queries

16e9774

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x added 3 commits February 6, 2024 21:08

chore: update Python 3.9 requirements

de7f685

Signed-off-by: F.N. Claessen <felix@seita.nl>

fix: more type annotations for Python 3.8

14be2ff

Signed-off-by: F.N. Claessen <felix@seita.nl>

Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

f8f1c8c

…hemy-2.0

update timerange query

ffb0fb5

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x mentioned this pull request Feb 6, 2024

Chore: update dependencies for #938 #973

Merged

Flix6x added this to the 0.19.0 milestone Feb 6, 2024

Flix6x reviewed Feb 7, 2024

View reviewed changes

flexmeasures/data/models/time_series.py Outdated Show resolved Hide resolved

fix: typo

50d0abf

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x reviewed Feb 8, 2024

View reviewed changes

flexmeasures/cli/data_delete.py Outdated Show resolved Hide resolved

revert nan and unchanged beleifs delete queries

a194d04

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid requested a review from Flix6x February 12, 2024 13:17

Flix6x requested changes Feb 13, 2024

View reviewed changes

refactor changes

bf81165

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

refactor epex_da sensor query

cf4b3da

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid requested a review from Flix6x February 14, 2024 18:13

Flix6x reviewed Feb 15, 2024

View reviewed changes

flexmeasures/data/services/users.py Outdated Show resolved Hide resolved

Ahmad-Wahid and others added 3 commits February 15, 2024 09:08

fix queries

1af5ff8

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

docs: changelog entry

ac04574

Signed-off-by: F.N. Claessen <felix@seita.nl>

Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

2ee2ac8

…hemy-2.0

Flix6x added Data and removed Still Needs Changelog Entry labels Feb 15, 2024

Flix6x and others added 4 commits February 15, 2024 11:16

fix: deletion of role account association for a given account

9664250

Signed-off-by: F.N. Claessen <felix@seita.nl>

style: black

465c8f7

Signed-off-by: F.N. Claessen <felix@seita.nl>

fix: test missed adding an object to session before flushing to assig…

c647c44

…n IDs Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x approved these changes Feb 15, 2024

View reviewed changes

Flix6x merged commit 071a4de into main Feb 15, 2024
9 checks passed

Flix6x deleted the feature/update-sqlalchemy-2.0 branch February 15, 2024 10:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support SQLAlchemy 2.0.x #938

Support SQLAlchemy 2.0.x #938

Ahmad-Wahid commented Dec 19, 2023 •

edited

nhoening commented Feb 6, 2024

Flix6x left a comment •

edited

Ahmad-Wahid commented Feb 14, 2024

Support SQLAlchemy 2.0.x #938

Support SQLAlchemy 2.0.x #938

Conversation

Ahmad-Wahid commented Dec 19, 2023 • edited

Description

How to test

nhoening commented Feb 6, 2024

Flix6x left a comment • edited

Choose a reason for hiding this comment

Ahmad-Wahid commented Feb 14, 2024

Ahmad-Wahid commented Dec 19, 2023 •

edited

Flix6x left a comment •

edited