remotecache: remove cache manifest size limit check

[dejanceltra]

Removes arbitrary remote cache manifest limit of 1 MB, which was added here: 80d2f82#diff-43026ef44c9363551fae4714a9441a4fdc4e61ab4b315c07f29fd10f1199cfeaR80-R83

Resolves #4916

[jedevc]

No objections to removing the limit, but I am curious - how are you ending up with a 1MB cache manifest? That's quite large, are your builds just very large? Or is there some manifest manipulation going on?

[dejanceltra]

@jedevc Large, ~30 stage Dockerfile, with ~250 instructions.

[tonistiigi]

The limit isn't completely arbitrary. This content goes to JSON parsing so is read into memory. Additionally, registry has a maximum size limit for manifest (4MB iirc). That would apply to the outer manifest index, cache config could be bigger.

Large, ~30 stage Dockerfile, with ~250 instructions.

Still, that would be like 4KB per instruction.

I'm ok with raising it to 4MB but I would agree that likely there is some other issue in here.

[dejanceltra]

@tonistiigi It's seems there is a different issue here.

Other manifests for the same Dockerfile are around 80kB, but some of them are a bit more than 1 MB. One of the problematic manifest is here, if you want to take a look: https://gist.github.com/dejanceltra/65787845cd970fa2a71a6dd32c81bbf0

I can open a separate issue if needed, since I assume this is related to cache manifest generation.

[tonistiigi]

@dejanceltra Yes, if you see a manifest that is too big open an issue about it with steps of how to generate it.

[tonistiigi]

Closing this for now. Please open new issue if you have steps for generating big manifest.