ComboBox does not lazy load items between from server to client

[heruan]

When using ComboBox with a huge backend data pool, I want the items to be lazily loaded based on the query string the user's typing, so that I avoid unnecessary consumption of memory/memory exhaustion.

ComboBox already implements HasDataProvider, but currently it loads the items eagerly:

itemsFromDataProvider = dataProvider.fetch(new Query<>())
                .collect(Collectors.toList());

The Web Component already support lazy loading:

Remote and local filtering

Lazy loading of data from any data source. Filter data locally or on a server.

— https://vaadin.com/components/vaadin-combo-box

Update The Web Component does not support lazy loading, at least not with a data provider like grid. I suppose then this will depend on support in the Web Component side first!

[lightoze]

Also with lazy loading it would make a lot of sense to add API to directly configure a link between user filter text and DataProvider filtering. So that the developer only has to provide a converter "filter string" -> "data provider filter".

[Legioth]

This is the way it has been implemented in FW8, and should probably be implemented in exactly the same way in Flow as well. Relevant API from FW8 are:

• HasFilterableDataProvider<T, String> instead of the regular HasDataProvider<T> interface which accepts a data provider regardless of the filter type.
• setDataProvider(ListDataProvider<T> listDataProvider)
• setDataProvider(CaptionFilter captionFilter, ListDataProvider<T> listDataProvider)
• setItems(CaptionFilter captionFilter, Collection<T> items)
• setItems(CaptionFilter captionFilter, T... items)
• setDataProvider(DataProvider<T, C> dataProvider, SerializableFunction<String, C> filterConverter)
• setDataProvider(FetchItemsCallback<T> fetchItems, SerializableToIntFunction<String> sizeCallback)

[heruan]

Getting a bit confused: does the ticket this one's waiting for already exist? In the Web Component issues I've found this ones: vaadin/vaadin-combo-box#26, vaadin/vaadin-combo-box#260 but they're both quite old and closed, so maybe they refer to a lazy-something else!

This is quite a blocking issue for porting FW8 ComboBoxes to Flow, as a simple field to search and select a user from a large database lags a lot.

[lightoze]

@heruan, there are two related issues:

• this ComboBox does not lazy load items between from server to client #72: loading items from the server page by page, currently not supported by web component, here is a comment from 2016 with a decision not to implement it at that point: Lazy loading of items vaadin-combo-box#260 (comment)
• Reintroduce the CaptionFilter concept in ComboBox #17: using server-side filter when user types something, this is supported by web component but Java side lacks DataProvider integration

[heruan]

Thank you @lightoze for the issue references. What I don't get is: does this feature (lazy loading) depend also on development in the Web Component side? Or is it just about Java implementation (including a connector)?

[lightoze]

@heruan, it depends on web component support.

Note: it is already possible to implement server-side filtering for user input and limit number of results visible to a user. This will prevent attempts to load thousands of items at once but it will be impossible to have an scroll through them like you do now in Vaadin 8. User will have to type in a sufficiently narrow filter text before being able to see target item.

[heruan]

Note: it is already possible to implement server-side filtering for user input […]

@lightoze you mean with DataProvider? I'm not able to, at least in the same way I'm doing for Grid: in Grid the first query is issued with a limit, in ComboBox the query is not limited and this currently generates huge load of data transfer (both backend-to-server and server-to-client).

[lightoze]

@heruan you cannot simply use DataProvider for that - #17 is covering that. But you should be able to use addFilterChangeListener and setFilteredItems to handle this manually.

[pleku]

The web component would need a similar data provider API as the vaadin-grid has. There is currently no issue for this, but it is mentioned in the "feature backlog" for the component here: vaadin/vaadin-combo-box#148

[heruan]

@pleku I've opened vaadin/vaadin-combo-box#657 to keep track on this on the web component side.

[heruan]

At the moment there is much confusion about this, since it's not an actual bug but more of a broken promise:

• the component page at vaadin.com states support for lazy loading;
• the component itself implements HasDataProvider which suggests to the developer the use of getDataProvider().refreshAll(), which does nothing.

If lazy loading isn't going to happen very soon, I'd strongly suggest to update the component page and temporarily do not implement HasDataProvider (or make it clear in the JavaDoc of setDataProvider that it will eagerly fetch all the items in a list not connected with the DP itself).

[Legioth]

getDataProvider().refreshAll() seems like a separate issue. It should work regardless of whether all previous items have already been non-lazily sent to the client. There are also other non-lazy implementations of HasDataProvider in both Vaadin 8 and Vaadin 10, e.g. RadioButtonGroup.

[kgeis]

@lightoze said

you should be able to use addFilterChangeListener and setFilteredItems to handle this manually.

I wish I could, but addFilterChangeListener(..) is protected.

[heruan]

Any update about this? Just a clarification: which ticket does the "awaits another ticket" label refer to?

[pleku]

Giovanni I think your issue is the one: vaadin/vaadin-combo-box#657

Since it is a major feature regression between V8 and V10, it should get priority and thus I'm optimistic about getting it done for and shipped in V12

[heruan]

Thank you @pleku! I asked because I created the issue after this was labelled so I wasn't sure if that was the right one 😄

I'm glad to hear this has hight priority, since the feature was taken for granted during 8-to-10 port analysis and it blasted out only during demos. I really hope we can get a prerelease before 12 🙂