Increased API Rate Limits and Search Functionality for Box SDK (java)

[pankil-4pp15]

This enhancement request proposes increasing specific rate limits and expanding search capabilities within the Box SDK. These changes aim to improve the efficiency and functionality of our system integration with Box.

Detailed Requests:

1. Increase Search Result Limit:

• Current Limitation: Search API results are capped at 200 responses.
• Request: We request an increase in the maximum number of retrievable results per search query.

2. API Rate Limit Adjustments:

a. General API Calls:

• Current Limits: 1000 API requests per minute, per user.
• Request: We would like to explore the possibility of raising these limits for our designated system account.

b. Search API:

• Current Limits:

6 searches per second, per user.
60 searches per minute, per user.
12 searches per second, per enterprise.

• Request: We require higher rate limits for all three search-related restrictions. Additionally, providing details on the two additional limits beyond the basic rate limit would be beneficial.

c. Enhanced Search Functionality:

Wildcard Search: We are interested in the ability to perform wildcard searches using the Box SDK. This would ideally allow us to:
- Search for all files of a specific type (e.g., query="*" followed by filtering based on managed metadata).
- Conduct more complex queries beyond the basic ".pdf" search.

d. Large File Downloads: We require efficient handling of large file downloads. Ideally, we'd like to explore functionalities such as:

• Chunked Downloads: The ability to download large files in chunks by seeking to specific offsets within the file.
• Parallelization: Any potential methods for parallelizing the download process for a single file would be beneficial.

Justification:

These enhancements would significantly improve our system's interaction with Box by:

• Retrieving More Search Results: Increased search result limits allow us to capture a more comprehensive dataset in a single query.
• Handling Higher Call Volumes: Raised API rate limits will accommodate our system's workload and prevent unnecessary throttling.
• Enabling More Powerful Searches: Advanced search functionalities like wildcard queries and more complex filtering options will streamline data retrieval.
• Optimizing Large File Downloads: Chunked downloads and parallelization techniques will improve download efficiency for large files.

We believe these changes will contribute to a smoother and more efficient integration between our system and the Box platform.

[mwwoda]

Hi @pankil-4pp15
Thanks for reaching out. I'll try to address some of your questions and suggestion.

1 The SDK cannot directly change any restrictions placed on it by the API. We act as a public client and can do as much with the API as anyone.
2a. Same as the previous point. The SDK cannot bypass any rate limits from the API directly.
2b. Same as in the previous point.
2c. If the API doesn't allow the use of wildcards in searches, I don't think we can offer much in our SDK. The solution would most likely be insufficient and highly unoptimized if done solely on the SDK side.
2d. The Java SDK should already support chunked uploads. https://github.com/box/box-java-sdk/blob/main/doc/files.md#upload-a-large-file-in-chunks. While our downloading of files is not parallel, uploading is. I'm not sure if parallel downloading would be possible because the API doesn't give us such options.

Hopefully, that clarifies a few things. For any suggestions/feedback or would like to know more about the limits of the API itself you could try to reach out to the https://forum.box.com/, reach out to support or give some product feedback on pulse https://pulse.box.com/forums/909778-product-feedback.

[mhagmajer]

@pankil-4pp15 did you find our answer helpful?