Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional data to OpenSearch index. #766

Merged
merged 36 commits into from May 1, 2024

Commits on Jan 8, 2024

  1. Add data_abstract to opensearch index

    Adds the data_abstract (comment in yaml) to the opensearch indexer function.
    ItIsJordan committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    4d08a19 View commit details
    Browse the repository at this point in the history
  2. Fix test_submissions_csv in dashboard_test.py

    Fixes test_submissions_csv in dashboard_test.py to use regex when checking against URLs in text as the URL on the webpage does not always match what is contained in the Flask config.
    ItIsJordan committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    5c67dc8 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. Add data abstract search test

    Adds testing for data abstract searching
    ItIsJordan committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    37d1490 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2024

  1. Revert "Fix test_submissions_csv in dashboard_test.py"

    This reverts commit 5c67dc8.
    ItIsJordan committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    b2c1a2a View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. Add table data description to opensearch index

    Adds table data to the OpenSearch index for submission searching in the OpenSearch document enhancers.
    ItIsJordan committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    4cf908a View commit details
    Browse the repository at this point in the history
  2. Update search testing to include table description searching

    Updates test_search in search_test.py to include testing of table description text
    ItIsJordan committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    0f60eb1 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2024

  1. Revert "Add table data description to opensearch index"

    This reverts commit 4cf908a.
    ItIsJordan committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    f6437b2 View commit details
    Browse the repository at this point in the history
  2. Add additional resource descriptions to index

    Updates index to add file_description from additional resource objects within HEPSubmission/DataSubmission objects to the OpenSearch index.
    ItIsJordan committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    ea6e78f View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. Configuration menu
    Copy the full SHA
    040a4dd View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. Update test for resource searching in OpenSearch API

    Updates search_test OpenSearch testing to properly include some testing for searching of additional resources.
    ItIsJordan committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    c6e4920 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Update hard-coded/assumed testing variables

    Updates the hard-coded/assumed variables used as expected data within the tests to accommodate the existence of a new test submission in the identifiers dictionary within conftest.py
    ItIsJordan committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    5cd2256 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. Update broken test in test_dashboard.py

    Updates a broken test in test_dashboard.py, this test has issues running on my local system.
    ItIsJordan committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    7557c76 View commit details
    Browse the repository at this point in the history
  2. Fix unrendered LaTeX in table description

    Now triggers rendering of LaTeX in record table descriptions as this was bugged and only occurred when the full table is loaded. Mentioned in issue #756
    ItIsJordan committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    12214c7 View commit details
    Browse the repository at this point in the history
  3. Clear table quantity text on records page

    Fixes a bug where the table_options_region element ("Showing X of X values" and "Show All") on the records page would sometimes have the data of another table if loaded later.
    ItIsJordan committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    882d6a9 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. Refactor data table routes

    Refactors the route to get data tables at /data/ to separate the data table (yaml file) and the headers query code. Implements a second route to select only the table data.
    ItIsJordan committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    6445bd9 View commit details
    Browse the repository at this point in the history
  2. Update table renderer js

    Updates the table renderer JS to properly use new route to select only data table.
    ItIsJordan committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    4dc662a View commit details
    Browse the repository at this point in the history
  3. Update records_test.py

    Updates tests in response to previous commit.
    ItIsJordan committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    a9bd63b View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Fix button query bug

    Fixes an issue as a part of the large file loading bugs in #756 where the filesize threshold loading button would execute multiple queries. This occurred by multiple events being passed to the button. Now clears the buton events before.
    ItIsJordan committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    4afcb2f View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. Revert large file bugfixes to move to new branch (last 6 commits)

    Reverts a number of bugfixes/changes for the large file loading functionality that was intended for a separate branch.
    
    Reverts commits:
    
    12214c7
    4afcb2f
    a9bd63b
    4dc662a
    6445bd9
    882d6a9
    ItIsJordan committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e11033b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a07723f View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. Fix get json table data url

    Fixes and issue where the get_table_details route was not properly routing when load_all flag is not set. This resulted in failure to find the correct route.
    
    Also removes method, as it likely does not matter.
    ItIsJordan committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    40927b3 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. Add resource url generation function

    Adds a function to generate/return the correct web link for a DataResource object. Adds associated test.
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    0483fc1 View commit details
    Browse the repository at this point in the history
  2. Update record_mapping to include resources/data_abstract

    Update record_mapping to include resources/data_abstract at the record-level.
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    a60fad9 View commit details
    Browse the repository at this point in the history
  3. Add function to generate DataResource dictionary

    Adds function to generate a DataResource dictionary data from a DataResource object.
    Also adds testing.
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    f982f14 View commit details
    Browse the repository at this point in the history
  4. Add full resource data to OpenSearch index

    Adds the full resource data to the OpenSeach index for both records and table resource objects.
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    d897f87 View commit details
    Browse the repository at this point in the history
  5. Add full resource data to table output

    Adds full resource data to the output of the process_data_tables function so the full resource data shows on an individual record's JSON output
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    b6596cc View commit details
    Browse the repository at this point in the history
  6. Update search_test.py

    Add search testing for new search functionality
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    41b9b59 View commit details
    Browse the repository at this point in the history
  7. Update advanced search help

    Updates the "Advanced" search help tooltip in search_help.html to include new data_abstract searching, and searching by resource (type, description, url)
    ItIsJordan committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    bde6455 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    efad6bc View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Change use of SITE_URL

    Changes use of SITE_URL in the tests to retrieve from app.config.get as this is safer.
    ItIsJordan committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    f2d7ef1 View commit details
    Browse the repository at this point in the history
  2. Add shorthand for resource description searching

    Adds a shorthand for "resource.description" searching as "resources" in the OpenSearch query builder
    ItIsJordan committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    944638e View commit details
    Browse the repository at this point in the history
  3. Update search help text

    Updates search help text in search_help.html to standardise the text for data abstract searching. Also updated to mention "resources" shorthand for description searching.
    ItIsJordan committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    d26ca1e View commit details
    Browse the repository at this point in the history
  4. Fix broken test

    Add "app" fixture to broken tests that now seem to require it. (where app is used to get site url)
    ItIsJordan committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    6824749 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Add "resources" data to data table json output

    Adds resource informaton to the table_contents in get_table_details, which will be seen in the json output of an individual data table.
    ItIsJordan committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    9b1a5cc View commit details
    Browse the repository at this point in the history
  2. Update site_url use

    Replaces more use of SITE_URL importing directly from the config, which can be unsafe, to use the current_app value provided by flask.
    
    (adding missing imports)
    ItIsJordan committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    e24c6e6 View commit details
    Browse the repository at this point in the history
  3. Update search_help.html

    Updates the search help widget in search_help.html to remove reference to non-shorthand "resources.description". Now just references "resources" as the shorthand term for this.
    ItIsJordan committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    2846603 View commit details
    Browse the repository at this point in the history