Awards vocabularies: Invalid pagination parameters err.

[Samk13]

Package version (if known): 9.1

Describe the bug

After adding more than 10500 custom awards, when I click on the next page > in the awards modal or search for awards it works fine, but if I click the last page number or >> I get this error in the console with the message Oops! Something went wrong while fetching results.

Steps to Reproduce

1. add > 10000 awards
2. Go to upload page open add awards modal
3. click on >> or last result
4. See error Oops! Something went wrong while fetching results.
5. check the console:

Expected behavior

paginate to the last result

Screenshots (if applicable)

[2022-09-01 12:53:01,231] DEBUG in errors: A resource error handler caught the following exception:
Traceback (most recent call last):
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/resources.py", line 65, in view
    return view_meth()
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/content_negotiation.py", line 116, in inner_content_negotiation
    return f(*args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/responses.py", line 39, in inner
    res = f(*args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/resources/records/resource.py", line 77, in search
    hits = self.service.search(
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 223, in search
    search = self._search("search", identity, params, es_preference, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 198, in _search
    search = self.search_request(
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 173, in search_request
    search = interpreter_cls(search_opts).apply(identity, search, params)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/services/records/params/pagination.py", line 35, in apply
    raise QuerystringValidationError("Invalid pagination parameters.")
invenio_records_resources.services.errors.QuerystringValidationError: Invalid pagination parameters.
127.0.0.1 - - [01/Sep/2022 12:53:01] "GET /api/awards?q=&sort=bestmatch&page=10486&size=5 HTTP/1.1" 400 -

Additional context

I tried changing the max number to 10500 with no success
if change it to big number like 100000, then I will get different err:

[2022-09-01 13:00:34,964] DEBUG in errors: A resource error handler caught the following exception:
Traceback (most recent call last):
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/resources.py", line 65, in view
    return view_meth()
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/content_negotiation.py", line 116, in inner_content_negotiation
    return f(*args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/responses.py", line 39, in inner
    res = f(*args, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/resources/records/resource.py", line 77, in search
    hits = self.service.search(
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 224, in search
    search_result = search.execute()
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch_dsl/search.py", line 715, in execute
    self, es.search(index=self._index, body=self.to_dict(), **self._params)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/client/__init__.py", line 1670, in search
    return self.transport.perform_request(
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/transport.py", line 415, in perform_request
    raise e
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/transport.py", line 381, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 277, in perform_request
    self._raise_error(response.status, raw_data)
  File "/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/elasticsearch/connection/base.py", line 330, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'Result window is too large, from + size must be less than or equal to: [10000] but was [52430]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.')
/home/samk13/.virtualenvs/test_voc_venv/lib/python3.9/site-packages/flask_resources/errors.py:97: DeprecationWarning: 'JSONEncoder' is deprecated and will be removed in Flask 2.3. Use 'Flask.json' to provide an alternate JSON implementation instead.
  return json.dumps(body, cls=JSONEncoder)

127.0.0.1 - - [01/Sep/2022 13:00:34] "GET /api/awards?q=&sort=bestmatch&page=10486&size=5 HTTP/1.1" 500 -

[ppanero]

(tranfering back to records-resources, even if the issue happened with awards it would happen also with any other type or records).

Exception is raised here due to the fact that default_max_results is passed to the pagination (see here) and it's value is 10000, then the query is out of range.

Increasing the default would just postpone the problem. We need to make the pagination widget aware of this maximum.

[lnielsen]

The UI should behave properly and provide a valid error message and you should be able to click on >> to go to the last possible page - i.e e.g. 9950-9999. So probably there's an "off by one" error somewhere here.

That said, we won't be able to paginate beyond the configured max number of results due to how search is working. Basically the further back you get in a search results, the more memory your search engine needs to allocate on each node to hold the full sorted list in memory. And do that on every single HTTP request you execute to paginate the results.

Instead a scan REST API endpoint should be provided (the service layer already has some sort of support for it), but the REST API would need some sort of resumption token that it uses to keep a the context around. The OAI-PMH server for instance is already doing this job, and OAI-PMH is a way to harvest all records.