Kubernetes deployment issues #75
Comments
|
Okay I think I figured out the issue, when fetching the packages the authorization token is sent, but in my case the authorization token is very very large (about 7k) which is I believe exceeding nginx buffers resulting in the query not being parsed properly in nginx logs:
I've decoded the authorization token and it contains I'm doing this with the verdaccio-gitlab master Dockerfile, I simply changed to use the tagged 4 release which was released yesterday |
|
Finally got it working! By default I think the JWT token should be stripped to contain less information, is there any reasons for the duplicates in |
|
@simon-jouet Thanks for the update! GitLab differentiates between group name and group path, see also https://docs.gitlab.com/ce/api/groups.html |
|
@bufferoverflow thanks for that, make sense, I will have a deeper look into the group api once I get this up and running :). Regarding the duplicates in In the longer term, fixing the nginx config for me worked but I can quite easily imagine someone with significantly more repos resulting in a far too large header to be sensible. Do you think it would make sense to maybe just provide a token to the user and store the groups in memory (or possibly redis?). We can close this issue if you want, just detailed the symptoms and resolution in case anybody else run into a similar problem. |
|
Could you paste some small sample of the duplicate entries in the iirc the entries in the token depend on what we return from the |
|
Also, are you sure it's jwt what we're talking about? I thought that verdaccio 4.x required a new |
|
Thanks @dlouzan, Here is the anonymised content of the token, just renamed the projects and repos but kept it consistent
I don't have JWT explicitly enabled but i'm talking about the bearer token passed when querying the packages which highly looks like a JWT token EDIT: apologies for saying duplicate real_groups before, it's in |
|
The @juanpicado Does this ring a bell? any idea why we're seeing jwt tokens in this configuration? |
|
Ok, maybe the documentation is a bit misleading. According to the PR that introduced JWT, it's enabled by default on API calls, but not on web requests: The following post also documents the expected behaviour that groups are added as payload of the token:
So apart from the duplicated entries, I'm not sure we'll be able to solve that problem with the size directly. Additionally, since we expect to contact gitlab for authentication, we might need to document a recommended |
|
We're having this issue as well, even with these http2 settings the UI errors out after logging in (we're on verdaccio 4.0.0 due to #81): http2-max-field-size: 32k
http2-max-header-size: 64k
Submitted PR #81 to fix this. |
|
@dlouzan sorry late to the party @simon-jouet actually, here an snippet of the logic behind the real groups. async jwtEncrypt(user: RemoteUser, signOptions: JWTSignOptions): string {
const { real_groups, name, groups } = user;
const realGroupsValidated = _.isNil(real_groups) ? [] : real_groups;
const groupedGroups = _.isNil(groups) ? real_groups : groups.concat(realGroupsValidated);
const payload: RemoteUser = {
real_groups: realGroupsValidated,
name,
groups: groupedGroups,
};
const token: string = await signPayload(payload, this.secret, signOptions);
// $FlowFixMe
return token;
}
Why real group exist? I have NO CLUE 😆, many things were added to Sinopia with 0 backup, 0 context and 0 code review in the post Sinopia era and pre Verdaccio era. For reasons of backward compatibility were keep them all this time and I personally was more concerned about other topics. Perhaps they should be removed, I'll consider this for Verdaccio 5. The groups duplication looks suspicious and might be a bug, but I'm still puzzled about the jwt token authentication.
The logic above happens every time a token is singed, so @dlouzan (I removed the other comment it is a non sense, I was wrong) the same thoughts about the duplication, like I mentioned above, it was a legacy logic, might be a bug or just something intended, I'm not sure who/whom are related on this logic by far. |
|
@juanpicado I'll try to reserve some time tomorrow to take a detailed look at this, it might well be a bug from our end |
|
I updated my comment 👍 I was wrong. |
|
@juanpicado Is my assumption above right?
That would mean that by default the jwt token is the approach taken for all api calls, is this so? |
|
Nop, JWT is enabled in Web since someone developed that part by default, there is no legacy with the Web API, the CLI API is auth legacy token by default. |
|
Please bear in mind that while reducing the JWT size might resolve the issue for some/most users, this won't be a permanent fix. Could there be any other solution despite dropping everything into the JWT? Eg. configure a separate storage (defaults to JWT) for those information. |
|
I encountered this problem when using a cluster with Nginx Ingress Controller. But when I configured it on a K8s cluster that uses Traefik the listing worked ok at first but now I'm getting 400 (Bad Request) at every request . Other coworkers with access to almost the same repositories are not having the same issue. |
|
My header is over 18k with this plugin - our company has a group with a huge list of repositories (in subgroups as well). I might be abple to PR a solution by saving the details of the group in another way, like a database. |
|
Seems related verdaccio/verdaccio#2068 |
Hi,
(this isn't a follow up to #74 but quite a different question/issue)
So i've been trying to deploy verdaccio-gitlab in a kubernetes cluster that i'm setting for our next dev environment (migrating from docker swarm). I got most things working but i'm stumbling a bit on verdaccio-gitlab and was hoping to get some insights and maybe if i'm lucky some input from people who have a kubernetes deployments working properly.
I can get verdaccio to load properly, I can get the login to work but once I'm logged in I'm unable to fetch the packages. The issue is that once i'm logged in I get a ERR_CONNECTION_CLOSED/ERR_CONNECTION_RESET when fetching the packages
I've first tried to use the helm chart for verdaccio and simply change the the image to verdaccio-gitlab, this worked but I can't fetch the packages after login.
Expecting this to be possibly an issue with the helm chart, I've made my own deployment and also fetched the last copy from the repo to build a new image on top of verdaccio 4.0.0 beta10 instead of the beta3 version which is currently used for the latest tagged docker image. The symptoms are the same again.
Finally I've tried to simply deploy a verdaccio beta10 with just htpasswd and without the gitlab plugin and it looks to be working as it should. I might be missing obvious but I unfortunately can't see any errors in nginx ingress, gitlab or verdaccio
The text was updated successfully, but these errors were encountered: