Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UI for data upload #563

Draft
wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

UI for data upload #563

wants to merge 19 commits into from

Commits on Dec 26, 2022

  1. Tech spike: upload CSV files and parse as pandas DataFrames 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    75bdd98 View commit details
    Browse the repository at this point in the history

  2. Parse uploaded CSV files as BeliefsDataFrames 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    43b7993 View commit details
    Browse the repository at this point in the history

  3. Validate sensor id 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    a004e17 View commit details
    Browse the repository at this point in the history

  4. UI styling for upload widget: moved to new side panel 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    77ed733 View commit details
    Browse the repository at this point in the history

  5. Reload data after uploading 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    e485998 View commit details
    Browse the repository at this point in the history

  6. Actually save the data 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 26, 2022
    Configuration menu
    Copy the full SHA
    67eee73 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2022

  1. Upgrade timely-beliefs 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    e3f43cb View commit details
    Browse the repository at this point in the history

  2. Remove upload form from asset page (leftover from tech spike) 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    2f14c1f View commit details
    Browse the repository at this point in the history

  3. Deal with "empty" file list actually containing a FileStorage with an… 

    … empty filename

Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    8610373 View commit details
    Browse the repository at this point in the history

Commits on Dec 28, 2022

  1. Marshmallow file validation 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    98caefd View commit details
    Browse the repository at this point in the history

  2. Marshmallow file deserialization 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    1c98cb4 View commit details
    Browse the repository at this point in the history

  3. Move processing of deserialized data from validation to post_load 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    632fbff View commit details
    Browse the repository at this point in the history

  4. Use MultiDict, because files can have duplicate keys, and we don't wa… 

    …nt them to overwrite each other when calling .update() on the dictionary

Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    fa491ae View commit details
    Browse the repository at this point in the history

  5. Add docstrings 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    3abd802 View commit details
    Browse the repository at this point in the history

  6. Repurpose validator to focus on the uploaded_files field only 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    3e02afb View commit details
    Browse the repository at this point in the history

  7. Show errors per uploaded file 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    a37df6e View commit details
    Browse the repository at this point in the history

  8. Processing file contents can also lead to a ValidationError 

    Signed-off-by: F.N. Claessen <felix@seita.nl>Processing file contents can also lead to a ValidationError

Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 28, 2022
    Configuration menu
    Copy the full SHA
    2260299 View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2022

  1. Use f-string 

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    @Flix6x
    Flix6x committed Dec 29, 2022
    Configuration menu
    Copy the full SHA
    91ddb93 View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2023

  1. Merge remote-tracking branch 'origin/main' into 7-ui-for-data-upload 

    Signed-off-by: F.N. Claessen <felix@seita.nl>

# Conflicts:
#	flexmeasures/api/v3_0/sensors.py
    @Flix6x
    Flix6x committed Jan 3, 2023
    Configuration menu
    Copy the full SHA
    2cea3c2 View commit details
    Browse the repository at this point in the history