knative 1.12 image push bug MANIFEST_BLOB_UNKNOWN

[Taylorzhao]

When using Func to compile functions, after the function is compiled, an error is reported when pushing it to a private registry, as shown in the figure. However, it can be pushed normally using the docker push command.

[drriguz]

I guess should be the same of #2181

[lkingland]

Thanks @drriguz for locating the previous issue.
@Taylorzhao this issue is resolved with an update to the underlying go-containerregistry library. If you build the main branch or go install knative.dev/func@main; that should give you the latest.

[matejvasek]

@drriguz @Taylorzhao can you confirm that this has been fixed in latter versions of function.

[drriguz]

@drriguz @Taylorzhao can you confirm that this has been fixed in latter versions of function.

v1.13.1 and v1.12.2 does not work.

➜  go-hello func version -v
Version: v0.40.1
Knative: =v1.13.1
Commit: bc40ec06
SocatImage: quay.io/boson/alpine-socat:1.7.4.3-r1-non-root
TarImage: quay.io/boson/alpine-socat:1.7.4.3-r1-non-root

➜  go-hello func deploy
function up-to-date. Force rebuild with --build
Pushing function image to the registry "localhost:5000" using the "" user credentials
Error: PUT http://localhost:5000/v2/go-hello/go-hello/manifests/latest: multiple errors returned: MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:a4d4471427378383add1d265db91e3e19c39f912b8b7411a81ae4e2f24a815ff; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:e1e9b23bc34db186541518ecf8c25f27cec5912c05f49120f606be361ba6c0dc; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:2ea7ad6f961ae6668e0f6fe434e9061194a5bdf600786f834ca24ab2d7ae8a3d; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:219779f600da11b69f882df7c9779b351367260ca44fbf6f390f1f4c0ea1ab2f; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:ac23843fe9774ae10cd54e11abc15a4ea90d82818d6ecda2994c9f45f024f22b; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:67b2702e72302bc27a3ac32dd9748d57af97c3242d2890e011c224d497483e21; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:92f1d3712a46e11688a8a106436d12fcb67b8382dade5ed6a9123f74b20f3369; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:ae6af002ee5672e39c09aa23ec8bdabf299703b26424864b93712d5ab402c40f; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:83d85471d9f8a3834b4e27cf701e3f0aef220cc816d9c173c7d32cd73909a590

➜  go-hello func version -v
Version: v0.39.1-1-g6a48e6b4
Knative: v1.12.1
Commit: 6a48e6b4
SocatImage: quay.io/boson/alpine-socat:1.7.4.3-r1-non-root
TarImage: quay.io/boson/alpine-socat:1.7.4.3-r1-non-root

➜  go-hello func deploy
function up-to-date. Force rebuild with --build
Pushing function image to the registry "localhost:5000" using the "" user credentials
Error: PUT http://localhost:5000/v2/go-hello/go-hello/manifests/latest: multiple errors returned: MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:a4d4471427378383add1d265db91e3e19c39f912b8b7411a81ae4e2f24a815ff; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:e1e9b23bc34db186541518ecf8c25f27cec5912c05f49120f606be361ba6c0dc; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:2ea7ad6f961ae6668e0f6fe434e9061194a5bdf600786f834ca24ab2d7ae8a3d; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:219779f600da11b69f882df7c9779b351367260ca44fbf6f390f1f4c0ea1ab2f; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:ac23843fe9774ae10cd54e11abc15a4ea90d82818d6ecda2994c9f45f024f22b; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:c6c679b0ead27e7b935cb8e58f6f47b32e8c60ccd4d2c941a5d59dba4f3a4a5e; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:92f1d3712a46e11688a8a106436d12fcb67b8382dade5ed6a9123f74b20f3369; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:ae6af002ee5672e39c09aa23ec8bdabf299703b26424864b93712d5ab402c40f; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:83d85471d9f8a3834b4e27cf701e3f0aef220cc816d9c173c7d32cd73909a590

custom build of commit e4010db0 and acae72d7 works:

➜  go-hello func version -v
Version: v0.40.0-76-ge4010db0
Knative: v1.13.0
Commit: e4010db0
SocatImage: ghcr.io/knative/func-utils:latest
TarImage: ghcr.io/knative/func-utils:latest

➜  go-hello func deploy
Building function image
Still building
Still building
🙌 Function built: localhost:5000/go-hello/go-hello:latest
Pushing function image to the registry "localhost:5000" using the "" user credentials
⬆️  Deploying function to the cluster
🎯 Creating Triggers on the cluster
✅ Function updated in namespace "default" and exposed at URL:
   http://go-hello.default.10.101.200.215.sslip.io

➜  go-hello func version -v
Version: v0.40.0-88-gacae72d7
Knative: v1.13.0
Commit: acae72d7
SocatImage: ghcr.io/knative/func-utils:latest
TarImage: ghcr.io/knative/func-utils:latest

# works too

[lkingland]

Thanks for checking @drriguz. I am gong to close this issue because it is fixed in main, and will be availalbe in the upcoming release. Please re-open if the issue resurfaces.