Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update zap.sh script to get memory usage in containers #8321

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

yannickvr
Copy link

Currently when hard limits are configured on memory in a containerized environment, zap.sh gets the host memory to calculate the memory available to Java, causing unexpected OOM errors.

Since all ZAP container images are given the environment var IS_CONTAINERIZED, i think this is currently the best method to detect if we are containerized, and thus use a different method of determining the memory available.

This has been tested in Docker Desktop and Amazon ECS Fargate and works properly. No limits or soft limits will return a linux default value ( 9223372036854771712 ), so there's a fallback to the current method.

Signed-off-by: Yannick <yannick@yannickvr.nl>
Copy link
Member

@ricekot ricekot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not work for the bare image (https://github.com/zaproxy/zaproxy/blob/main/docker/Dockerfile-bare), because the IS_CONTAINERIZED environment variable is not set there, but that can be changed easily.

The 9223372036854771712 value would not work for non 64-bit hosts, but I guess that should be okay since we only build our images for 64-bit architectures at the moment (amd64 and aarch64).

zap/src/main/dist/zap.sh Outdated Show resolved Hide resolved
zap/src/main/dist/zap.sh Outdated Show resolved Hide resolved
zap/src/main/dist/zap.sh Outdated Show resolved Hide resolved
@thc202 thc202 changed the title Get memory usage if containerized Update zap.sh script to get memory usage in containers Feb 9, 2024
@thc202
Copy link
Member

thc202 commented Apr 19, 2024

@yannickvr are you going to address the comments?

@yannickvr
Copy link
Author

@yannickvr are you going to address the comments?

My bad, totally forgot this issue. I'll take it up later this week.

fixed markup and remove duplication
Copy link

github-actions bot commented May 3, 2024

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@yannickvr
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

zapbot added a commit to zaproxy/cla that referenced this pull request May 3, 2024
zap/src/main/dist/zap.sh Outdated Show resolved Hide resolved
@yannickvr
Copy link
Author

I've updated this PR with a test to see if /sys/fs/cgroup/memory/memory.stat exists and then get the memory from there. This has been tested successfully on recent ubuntu/debian/amazonlinux environments. It does not work on MacOS, but as this fix is primarily for a production issue it should be fine.

zap/src/main/dist/zap.sh Outdated Show resolved Hide resolved
@thc202 thc202 added this to the 2.16.0 milestone May 20, 2024
if [ -f /sys/fs/cgroup/memory/memory.stat ]; then
# If we are running in a container, we need to use the cgroup memory limit
MEMLIMIT=$(grep hierarchical_memory_limit /sys/fs/cgroup/memory/memory.stat | cut -d' ' -f2)
MEM=$(expr $MEMLIMIT / 1024 / 1024)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: This can be moved to an else block of the following if condition, so that it's only evaluated if it's needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

3 participants