Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot start the pod-cpu-hog experiment. #580

Open
lzhang2015 opened this issue Dec 23, 2022 · 0 comments
Open

Cannot start the pod-cpu-hog experiment. #580

lzhang2015 opened this issue Dec 23, 2022 · 0 comments

Comments

@lzhang2015
Copy link

Cannot start the pod-cpu-hog experiment. Would you please help to have a look at the issue? The litmus version is 2.14.0.

C41K60Q0XJ:nginx$ k get pod -n litmus
NAME                                        READY   STATUS      RESTARTS   AGE
chaos-exporter-d767fcf5-hsbgq               1/1     Running     0          18h
chaos-operator-ce-7cf6cc79b4-xwh99          1/1     Running     0          18h
chaos-stress-cpu-1671764370-276557960       0/2     Completed   0          9m55s
chaos-stress-cpu-1671764370-670059491       0/2     Completed   0          10m
event-tracker-5ddb594676-9dhtj              1/1     Running     0          18h
litmusportal-auth-server-57899b796d-2tvvq   1/1     Running     0          18h
litmusportal-frontend-857dc974b7-n4jv4      1/1     Running     0          18h
litmusportal-server-59f79479db-92qmb        1/1     Running     0          78m
mongo-0                                     1/1     Running     0          18h
pod-cpu-hog-0hytghxz-runner                 0/1     Completed   0          9m51s
pod-cpu-hog-helper-xhsnlr                   0/1     Error       0          9m26s
pod-cpu-hog-mfbudy-skzzc                    0/1     Completed   0          9m46s
subscriber-5cbcb4df94-22cg7                 1/1     Running     0          18h
workflow-controller-8c548f686-zmk9c         1/1     Running     0          18h
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-helper-xhsnlr -n litmus
time="2022-12-23T03:01:21Z" level=info msg="Helper Name: stress-chaos"
time="2022-12-23T03:01:21Z" level=info msg="[PreReq]: Getting the ENV variables"
time="2022-12-23T03:01:21Z" level=info msg="container ID of nginx container, containerID: e15be7fdcc7442be032e1caf56d1e5bb41852bb07e395f2aaf88e5dd2588ff70"
time="2022-12-23T03:01:21Z" level=error msg="[docker]: **Failed to run docker inspect: []\nError: No such object: e15be7fdcc7442be032e1caf56d1e5bb41852bb07e395f2aaf88e5dd2588ff70\n"**
time="2022-12-23T03:01:21Z" level=fatal msg="helper pod failed, err: exit status 1"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-0hytghxz-runner -n litmus
W1223 03:00:45.714442       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2022-12-23T03:00:45Z" level=info msg="Experiments details are as follows" Experiments List="[pod-cpu-hog]" Engine Name=pod-cpu-hog-0hytghxz appLabels="app=nginx" appNs=nginx appKind=deployment Service Account Name=litmus-admin Engine Namespace=litmus
time="2022-12-23T03:00:45Z" level=info msg="Getting the ENV Variables"
time="2022-12-23T03:00:45Z" level=info msg="Preparing to run Chaos Experiment: pod-cpu-hog"
time="2022-12-23T03:00:46Z" level=info msg="Started Chaos Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:42Z" level=info msg="Chaos Pod Completed, Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:44Z" level=info msg="Chaos Engine has been updated with result, Experiment Name: pod-cpu-hog"
time="2022-12-23T03:01:44Z" level=info msg="[skip]: skipping the job deletion as jobCleanUpPolicy is set to {}"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-0hytghxz-runner -n litmus
W1223 03:00:45.714442       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2022-12-23T03:00:45Z" level=info msg="Experiments details are as follows" Experiments List="[pod-cpu-hog]" Engine Name=pod-cpu-hog-0hytghxz appLabels="app=nginx" appNs=nginx appKind=deployment Service Account Name=litmus-admin Engine Namespace=litmus
time="2022-12-23T03:00:45Z" level=info msg="Getting the ENV Variables"
time="2022-12-23T03:00:45Z" level=info msg="Preparing to run Chaos Experiment: pod-cpu-hog"
time="2022-12-23T03:00:46Z" level=info msg="Started Chaos Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:42Z" level=info msg="Chaos Pod Completed, Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:44Z" level=info msg="Chaos Engine has been updated with result, Experiment Name: pod-cpu-hog"
time="2022-12-23T03:01:44Z" level=info msg="[skip]: skipping the job deletion as jobCleanUpPolicy is set to {}"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-mfbudy-skzzc -n litmus
time="2022-12-23T03:00:58Z" level=info msg="Experiment Name: pod-cpu-hog"
time="2022-12-23T03:00:58Z" level=info msg="[PreReq]: Getting the ENV for the pod-cpu-hog experiment"
time="2022-12-23T03:01:00Z" level=info msg="[PreReq]: Updating the chaos result of pod-cpu-hog experiment (SOT)"
time="2022-12-23T03:01:02Z" level=info msg="The application information is as follows" Namespace=nginx Label="app=nginx" App Kind=deployment
time="2022-12-23T03:01:02Z" level=info msg="[Status]: Verify that the AUT (Application Under Test) is running (pre-chaos)"
time="2022-12-23T03:01:02Z" level=info msg="[Status]: Checking whether application containers are in ready state"
time="2022-12-23T03:01:02Z" level=info msg="[Status]: The Container status are as follows" Pod=nginx-deployment1-5bcb875959-gnxnl Readiness=true container=nginx
time="2022-12-23T03:01:04Z" level=info msg="[Status]: Checking whether application pods are in running state"
time="2022-12-23T03:01:04Z" level=info msg="[Status]: The status of Pods are as follows" Status=Running Pod=nginx-deployment1-5bcb875959-gnxnl
time="2022-12-23T03:01:06Z" level=info msg="[Info]: The chaos tunables are:" Sequence=parallel PodsAffectedPerc=0 CPU Core=1 CPU Load Percentage=100
time="2022-12-23T03:01:06Z" level=info msg="[Chaos]:Number of pods targeted: 1"
time="2022-12-23T03:01:06Z" level=info msg="[Info]: Target pods list for chaos, [nginx-deployment1-5bcb875959-gnxnl]"
time="2022-12-23T03:01:06Z" level=info msg="[Info]: Details of application under chaos injection" PodName=nginx-deployment1-5bcb875959-gnxnl NodeName=ip-10-180-6-87.eu-central-1.compute.internal ContainerName=nginx
time="2022-12-23T03:01:06Z" level=info msg="[Status]: Checking the status of the helper pods"
time="2022-12-23T03:01:24Z" level=info msg="pod-cpu-hog-helper-xhsnlr helper pod is in Running state"
time="2022-12-23T03:01:26Z" level=info msg="[Wait]: waiting till the completion of the helper pod"
time="2022-12-23T03:01:26Z" level=info msg="helper pod status: Running"
time="2022-12-23T03:01:26Z" level=info msg="[Status]: The running status of Pods are as follows" Pod=pod-cpu-hog-helper-xhsnlr Status=Failed
time="2022-12-23T03:01:27Z" level=error msg="[Error]: CPU hog failed, err: helper pod failed"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant