Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shredded summary upload service fails on compressed binary summary blobs. #17438

Closed
milanro opened this issue Sep 22, 2023 · 3 comments
Closed
Assignees
Labels
bug Something isn't working status: stale

Comments

@milanro
Copy link
Contributor

milanro commented Sep 22, 2023

If the uploaded summary contains the binary compressed blob, the shredded summary upload fails. This happens, when Tinylicious server is used. In case of FRS, which uses whole summary upload, the use-case works. The failing code is in
packages/drivers/routerlicious-driver/src/summaryTreeUploadManager.ts
in the following method.

async writeSummaryBlob(content) {
        const { parsedContent, encoding } = typeof content === "string"
            ? { parsedContent: content, encoding: "utf-8" }
            : { parsedContent: Uint8ArrayToString(content, "base64"), encoding: "base64" };
        // The gitHashFile would return the same hash as returned by the server as blob.sha
        const hash = await gitHashFile(IsoBuffer.from(parsedContent, encoding));
        if (!this.blobsShaCache.has(hash)) {
            this.blobsShaCache.set(hash, "");
            const blob = (await this.manager.createBlob(parsedContent, encoding)).content;
            assert(hash === blob.sha, 0x0b6 /* "Blob.sha and hash do not match!!" */);
        }
        return hash;
    }

The upload fails on assert(hash === blob.sha, 0x0b6 /* "Blob.sha and hash do not match!!" */);, where the hashes do not match.

In order to reproduce the failure, the failing example was created. Following steps reproduce the problem

  • npx tinylicious@latest

  • git clone --single-branch --branch TinyBinarySummary git@github.com:milanro/FluidFramework.git FluidTinyBinSumDemo

  • cd FluidTinyBinSumDemo

  • pnpm i

  • npm run build:fast

  • cd examples/data-objects/miso_tinybinsummary/

  • npm start

  • in browser : http://localhost:8080/

  • Wait for summarization and check the console

Expected exception :

at assert (http://localhost:8080/main.073fc437a050ec3093e1.js:54590:15)    
at SummaryTreeUploadManager.writeSummaryBlob (http://localhost:8080/main.073fc437a050ec3093e1.js:62639:79)    
at async http://localhost:8080/main.073fc437a050ec3093e1.js:62597:32    
at async Promise.all (index 0)    
at async SummaryTreeUploadManager.writeSummaryTreeCore (http://localhost:8080/main.073fc437a050ec3093e1.js:62595:25)    
at async http://localhost:8080/main.073fc437a050ec3093e1.js:62597:32    at async Promise.all (index 0)    at async SummaryTreeUploadManager.writeSummaryTreeCore (http://localhost:8080/main.073fc437a050ec3093e1.js:62595:25)    
at async http://localhost:8080/main.073fc437a050ec3093e1.js:62597:32    
at async Promise.all (index 0) +0ms

@DLehenbauer @dstanesc @vladsud

@DLehenbauer
Copy link
Contributor

DLehenbauer commented Oct 20, 2023

@milanro - FYI - I'm still following up on this one. (AB#5815)

@DLehenbauer
Copy link
Contributor

DLehenbauer commented Oct 20, 2023

@milanro - we have a dev assigned this sprint to investigate the feasibility of switching Tinylicious to whole document summaries, but the sprint just started, so it may be mid-November before we hear back on their findings.

Copy link
Contributor

This issue has been automatically marked as stale because it has had no activity for 180 days. It will be closed if no further activity occurs within 8 days of this comment. Thank you for your contributions to Fluid Framework!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working status: stale
Projects
None yet
Development

No branches or pull requests

3 participants