Issue creating persistent storage for /var/lib/awx/projects and postgresql - using awx-operator 0.8.0 #260
Labels
type:bug
Something isn't working
Comments
|
@PugTheBlack it looks like a problem on the Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 7m42s (x244 over 9h) kubelet MountVolume.MountDevice failed for volume "pvc-ae007b9b-fe8d-42fe-9975-24ad1a79c67a" : rpc error: code = InvalidArgument desc = staging path /var/snap/microk8s/common/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ae007b9b-fe8d-42fe-9975-24ad1a79c67a/globalmount does not exist on nodeI would investigate the issue related with the |
|
Yea, it might be a rook-ceph problem more than awx-operator - the reason why I posted it here was mostly the part where the pvc is listed as bound. Will check it out with the rook-ceph guys and see if they can help :) |
|
@tchellomello like you said the problem was with the rook-ceph implementation. For some reason I had to create the rook operator with the full snap-path /var/snap/microk8s/common/var/lib/kubelet and not /var/lib/kubelet, then the awx-operator ran its course.... so now I have a different set of problems :) So basically - the containers can't find awx-postgres Also - I can't really see any PVC for postgres... |
|
On the logs provided, you did not show if the PostgreSQL statefulset is running. So what happens if you do: kubectl get pods -o wide awx-postgres-0Then if you take the IP address from that particular pod, and from your Furthermore, what happens if you deploy the latest |
So no it's not running. I will try to deploy a new awx instance with a different name using the 0.9.0 operator later today :) |
|
But I did a "wipe" of the old awx deployment Then cloned the new awx-operator from github - and now it all seems to be working much better :) So hopefully I will have a working deployment sometime today :) |
|
|
|
Still seems to be something fishy here though ... the regular "awx-" pod is not created, and the awx-operator |
|
I was able to reproduce the issue by creating the following spec: apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
name: awx-lb-annotations
namespace: default
spec:
kind: AWX
tower_admin_user: admin
tower_ingress_type: LoadBalancer <===== added thisHere is the error: [awx-operator-84694f9865-kvbc2] ok: [localhost] => (item=tower_persistent) => {"ansible_loop_var": "item", "changed": false, "item": "tower_persistent", "result": {"results": []}}
[awx-operator-84694f9865-kvbc2] fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'tower_loadbalancer_annotations' is undefined\n\nThe error appears to be in '/opt/ansible/roles/installer/tasks/resources_configuration.yml': line 20, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Apply Resources\n ^ here\n"}
awx-operator-84694f9865-kvbc2]
awx-operator-84694f9865-kvbc2] PLAY RECAP *********************************************************************
awx-operator-84694f9865-kvbc2] localhost : ok=29 changed=0 unreachable=0 failed=1 skipped=26 rescued=0 ignored=0 It's a bug that must be fixed. |
tchellomello
added
type:bug
This was referenced May 11, 2021
|
Changed the "my-awx.yaml" file to: And then the deployment went through as I expected :) Agree that it's a bug that needs fixing, but the workaround seems to be working fine, so I'm happy :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am currently testing out Kubernetes, and have set up a 4 node Microk8s Cluster on Ubuntu 20.04.2 LTS with Ceph (using the Rook operator branch 1.16)
4 virtual machines (4vCPU, 16GB RAM, 60GB OS disk + 200GB RAW disk for Ceph)
I installed the awx-operator without modifications and created a "my-awx.yaml" file to deploy a fairly basic install using rook-ceph-block storage for the PVCs
As you can see from the pod list the awx-86899bfb7b-pjjfs is stuck in ContainerCreating
So basically what it is saying is that the PVC does not exist on node - which means what exactly?
Like I said, I am completely green at this, so might be all manner of stuff wrong with my setup. Would be awesome if you had some pointers on where to look first though.
-Marius
The text was updated successfully, but these errors were encountered: