feat(pod-experiments): Add volume-fill experiment (to target pvc) #458
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Remi Ziolkowski <remi.ziolkowski-ext@pole-emploi.fr>
Vr00mm
requested review from
ispeakc0de,
ksatchit and
uditgaurav
as code owners
Signed-off-by: Remi Ziolkowski <remi.ziolkowski-ext@pole-emploi.fr>
Vr00mm
changed the title
ispeakc0de
reviewed
Oct 17, 2022
| } | ||
| } | ||
|
|
||
| //diskFill contains steps to inject disk-fill chaos |
There was a problem hiding this comment.
we can change diskfill -> volumefill at all places
ispeakc0de
reviewed
Oct 17, 2022
| volumes := container.VolumeMounts | ||
| for _, volume := range volumes { | ||
| if volume.Name == experimentsDetails.TargetVolume { | ||
| mountPath = volume.MountPath |
There was a problem hiding this comment.
we can handle the case when mountPath is empty - invalid target volume case
ispeakc0de
reviewed
Oct 17, 2022
| <th> Documentation Link </th> | ||
| </tr> | ||
| <tr> | ||
| <td> Disk Fill </td> |
There was a problem hiding this comment.
We can change the description as well
ispeakc0de
reviewed
Oct 17, 2022
| } | ||
|
|
||
| //PRE-CHAOS AUXILIARY APPLICATION STATUS CHECK | ||
| if experimentsDetails.AuxiliaryAppInfo != "" { |
There was a problem hiding this comment.
we can remove AuxiliaryApps status checks(from both pre/post checks) as it needs cluster role permissions
ispeakc0de
reviewed
Oct 17, 2022
| value: '' | ||
|
|
||
| - name: CONTAINER_PATH | ||
| value: '/var/lib/docker/containers' |
ispeakc0de
reviewed
Oct 17, 2022
| experimentDetails.AuxiliaryAppInfo = types.Getenv("AUXILIARY_APPINFO", "") | ||
| experimentDetails.TargetContainer = types.Getenv("TARGET_CONTAINER", "") | ||
| experimentDetails.TargetVolume = types.Getenv("TARGET_VOLUME", "") | ||
| experimentDetails.ContainerPath = types.Getenv("CONTAINER_PATH", "/var/lib/docker/containers") |
neelanjan00
requested changes
Oct 18, 2022
|
|
||
| fullPath := strings.TrimPrefix(volumePath, experimentsDetails.ContainerPath) | ||
|
|
||
| du := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $3}'", fullPath) |
There was a problem hiding this comment.
Can we add a comment on how this command is working?
| } | ||
| usedStorageSize, err := strconv.Atoi(strings.TrimSuffix(string(out), "\n")) | ||
|
|
||
| du2 := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $4}'", fullPath) |
There was a problem hiding this comment.
Can we add a comment on how this command is working?
| return err | ||
| } | ||
| } else { | ||
| log.Warn("No required free space found!, It's Housefull") |
There was a problem hiding this comment.
Suggested change
| log.Warn("No required free space found!, It's Housefull") | |
| log.Warn("No required free space found!") |
| return nil | ||
| } | ||
|
|
||
| // getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod |
There was a problem hiding this comment.
Suggested change
| // getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod | |
| // getVolumePath derives the ephemeral storage attributes from the target pod |
| return nil | ||
| } | ||
|
|
||
| //stopDockerContainer kill the application container |
There was a problem hiding this comment.
Suggested change
| //stopDockerContainer kill the application container | |
| // inspectDockerContainer returns the docker container mounts |
| // creating the helper pod to perform disk-fill chaos | ||
| for _, pod := range targetPodList.Items { | ||
| runID := common.GetRunID() | ||
| if err := createHelperPod(experimentsDetails, clients, chaosDetails, pod.Name, pod.Spec.NodeName, runID, labelSuffix); err != nil { |
There was a problem hiding this comment.
Can we add a log before this step?
| } | ||
|
|
||
| // creating the helper pod to perform disk-fill chaos | ||
| for _, pod := range targetPodList.Items { |
There was a problem hiding this comment.
Can we add a log before this step?
| <tr> | ||
| <td> Disk Fill </td> | ||
| <td> This experiment causes disk stress by filling up the ephemeral storage space of the pod and forces the pod to get evicted if the used space exceeds the set ephemeral storage limit. </td> | ||
| <td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td> |
There was a problem hiding this comment.
Suggested change
| <td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td> | |
| <td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/volume-fill/"> Here </a> </td> |
| log.Infof("[PreReq]: Updating the chaos result of %v experiment (SOT)", experimentsDetails.ExperimentName) | ||
| if err := result.ChaosResult(&chaosDetails, clients, &resultDetails, "SOT"); err != nil { | ||
| log.Errorf("Unable to Create the Chaos Result, err: %v", err) | ||
| failStep := "Updating the chaos result of disk-fill experiment (SOT)" |
There was a problem hiding this comment.
Can you update the failSteps so that the error string is also appended to them? You can check any other experiment file for reference, all the experiment files have been modified as such.
| labels: | ||
| app: litmus-experiment | ||
| spec: | ||
| serviceAccountName: disk-fill-sa |
There was a problem hiding this comment.
Suggested change
| serviceAccountName: disk-fill-sa | |
| serviceAccountName: volume-fill-sa |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Remi Ziolkowski remi.ziolkowski-ext@pole-emploi.fr
What this PR does / why we need it:
It add a new experiment into litmus lib that permit to target pvc into container
Which issue this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close that issue when PR gets merged): fixes NoSpecial notes for your reviewer:
No
Checklist:
breaking-changestagrequires-upgradetag