This repository contains the Depaginator. The Depaginator is a tool for traversing all items presented by a paginated API: all pages are retrieved by independent goroutines, then each item in each page is iterated over, calling a HandleItem method. The index of the item in the list is also passed to HandleItem for the benefit of ordering-sensitive applications.
For full details, refer to the documentation. The basic concept is for the consuming application to create an object that implements GetPage, HandleItem, and Done methods, conforming to the depaginator.API interface. The GetPage method is passed a PageMeta object and a PageRequest, which bundles a PageIndex integer with an application-defined Request. The GetPage method must then retrieve the desired page of the results, add any relevant metadata--including requests for subsequent pages--to the PageMeta, and return a an array of items with the type used to call Depaginate. The Depaginator will then call HandleItem for each element in the Page. After all pages have been retrieved and all items handled, the consuming application calls Depaginator.Wait, which will call the Done method (passing it the final PageMeta) and return any page retrieval errors that were encountered.
Many server APIs that return lists of objects will "paginate" the response to avoid overwhelming the connection or the client--or the server or database. However, many clients consuming that API need to perform some operation on all the returned objects, such as displaying them to the user or applying additional filters to select specific items. Especially for large lists, this process can be quite slow; this may be fine for user-interactive clients, such as command line clients, but if some other operation is being performed, such as bulk modifications, this can be unacceptably slow. The Depaginator is intended to simplify the implementation of code that iterates over all the items in a list by allowing their retrieval as fast as the server API will permit.