useSWRInfinite to fetch all pages

[RenaudAubert]

Hi!

I'm trying to fetch all the pages for a particular resource to display a Plotly graph.
Here's what I've done (this is a summary of my code below):

• fetch only the 1st page to determine the total number of pages
• Fetch the next 5 pages setSize(size + 5) if needed
• repeat until all pages have been fetched

const {data, size, setSize} = useSWRInfinite(getKey, {
        parallel: true,
        initialSize: 1,
        keepPreviousData: true,
        revalidateFirstPage: false,
    }
);

const count = data?.[0].count;
const fetchedDataLength = size * 5000; // 5000 is the page size

if (count && fetchedDataLength < count) {
    setSize(size + Math.max(5, ...)); // If less than 5 pages remaining only fetch those
}

This method is giving me an error

Cannot update a component (...) while rendering a different component (...)

I've used this section of the React doc to create this code but it looks like it only works if the value is a prop not a state from another hook.

I tried to move the setSize condition inside a useEffect but I'd have to use size inside the effect which whould cause too many renders and API calls if I provided the dependency array correctly. If I don't use size and only use a functional update with setSize((prevSize) => ...) then my effect would only run once.

// This causes too many requests
useEffect(() => {
    if (count && fetchedDataLength < count) {
        setSize(size + Math.max(5, ...)); // If less than 5 pages remaining only fetch those
    }
}, [count, size, setSize]);

// This would only run twice as it is not dependent on `size`
useEffect(() => {
    if (count) {
        setSize((prevSize) => fetchedDataLength < count
            ? prevSize + Math.max(5, ...)) // If less than 5 pages remaining only fetch those
            : prevSize
    }
}, [count, setSize]);

Please let me know if you need more details. Any help would be greatly appreciated.

Cheers,

Renaud