Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker daemon reports invalid memory address or nil pointer dereference #47654

Open
dreamcoin1998 opened this issue Apr 1, 2024 · 0 comments
Open

Docker daemon reports invalid memory address or nil pointer dereference #47654

dreamcoin1998 opened this issue Apr 1, 2024 · 0 comments
Labels
area/daemon area/volumes kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage version/26.0

Comments

@dreamcoin1998
Copy link

dreamcoin1998 commented Apr 1, 2024 •
edited by thaJeztah

Description

4月 01 11:13:08 tapsupertest7 dockerd[171183]: 2024/04/01 11:13:08 http: panic serving @: runtime error: invalid memory address or nil pointer dereference
4月 01 11:13:08 tapsupertest7 dockerd[171183]: goroutine 4693 [running]:
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.(*conn).serve.func1()
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:1868 +0xb9
4月 01 11:13:08 tapsupertest7 dockerd[171183]: panic({0x563fd8e9db00?, 0x563fda7de5b0?})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/runtime/panic.go:920 +0x270
4月 01 11:13:08 tapsupertest7 dockerd[171183]: go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End.func1()
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/vendor/go.opentelemetry.io/otel/sdk/trace/span.go:388 +0x25
4月 01 11:13:08 tapsupertest7 dockerd[171183]: go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End(0xc0017fe480, {0x0, 0x0, 0x40?})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/vendor/go.opentelemetry.io/otel/sdk/trace/span.go:426 +0xa3b
4月 01 11:13:08 tapsupertest7 dockerd[171183]: panic({0x563fd8e9db00?, 0x563fda7de5b0?})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/runtime/panic.go:914 +0x21f
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/pkg/plugins.(*Client).callWithRetry(0x0, {0x563fd8565f68, 0x10}, {0x563fd9324060, 0xc0022e6f00}, 0x1, {0xc001e0e7f0, 0x1, 0x20?})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/pkg/plugins/client.go:181 +0x14a
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/pkg/plugins.(*Client).CallWithOptions(0xc001b930c0?, {0x563fd8565f68, 0x10}, {0x563fd8f73de0, 0xc001b930f0}, {0x563fd8cd3700, 0xc0016dcb40}, {0xc001e0e7f0, 0x1, 0x1})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/pkg/plugins/client.go:134 +0x17a
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/drivers.(*volumeDriverProxy).Get(0xc001b930e0, {0xc002026900, 0x40})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/drivers/proxy.go:216 +0x128
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/drivers.(*volumeDriverAdapter).Get(0xc0022e6ed0, {0xc002026900?, 0xc002026900?})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/drivers/adapter.go:62 +0x2d
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/service.(*VolumeStore).getVolume(0xc000a24840, {0x563fd9352640, 0x563fda94a7c0}, {0xc002026900, 0x40}, {0x0, 0x0})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/service/store.go:747 +0x6f4
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/service.(*VolumeStore).create(0xc000a24840, {0x563fd9352640, 0x563fda94a7c0}, {0xc002026900, 0x40}, {0x0, 0x0}, 0x0, 0xc0022e6d50)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/service/store.go:612 +0x15e
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/service.(*VolumeStore).Create(0xc000a24840, {0x563fd9352640, 0x563fda94a7c0}, {0xc002026900, 0x40}, {0x0, 0x0}, {0xc001b93080, 0x2, 0x2})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/service/store.go:496 +0x1c5
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/volume/service.(*VolumesService).Create(0xc000d60360?, {0x563fd9352640?, 0x563fda94a7c0?}, {0x0?, 0x0?}, {0x0?, 0x0?}, {0xc00395d818, 0x1, 0x1})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/volume/service/service.go:78 +0x19b
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/daemon.(*Daemon).createContainerOSSpecificSettings(0xc0006b0780, {0x563fd93527e0, 0xc003e012f0}, 0xc00129f180, 0xc001e8b900, 0xc0033ca000)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/daemon/create_unix.go:66 +0x89f
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/daemon.(*Daemon).create(0xc0006b0780, {0x563fd93527e0?, 0xc003e012f0}, 0xc0001f4e70?, {{{0xc00278e123, 0x10}, 0xc001e8b900, 0xc0033ca000, 0xc001e0e580, 0x0, ...}, ...})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/daemon/create.go:224 +0x8a5
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/daemon.(*Daemon).containerCreate(0xc0006b0780, {0x563fd93527e0, 0xc003e012f0}, 0xc0007ac000, {{{0xc00278e123, 0x10}, 0xc001e8b900, 0xc0033ca000, 0xc001e0e580, 0x0, ...}, ...})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/daemon/create.go:116 +0x776
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/daemon.(*Daemon).ContainerCreate(0xc0017fe480?, {0x563fd93527e0?, 0xc003e012f0?}, {{0xc00278e123, 0x10}, 0xc001e8b900, 0xc0033ca000, 0xc001e0e580, 0x0, 0x0})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/daemon/create.go:45 +0x10d
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/api/server/router/container.(*containerRouter).postContainersCreate(0xc0010f2980, {0x563fd93527e0, 0xc003e012f0}, {0x563fd9348eb0, 0xc00313a1e0}, 0xc002241900, 0xc003e01140?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/api/server/router/container/container_routes.go:618 +0x97d
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/api/server/middleware.(*ExperimentalMiddleware).WrapHandler.ExperimentalMiddleware.WrapHandler.func1({0x563fd93527e0, 0xc003e012f0}, {0x563fd9348eb0?, 0xc00313a1e0?}, 0x563fd8d33f40?, 0xc001b92560?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/api/server/middleware/experimental.go:26 +0xb4
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/api/server/middleware.(*VersionMiddleware).WrapHandler.VersionMiddleware.WrapHandler.func1({0x563fd93527e0, 0xc003e012c0}, {0x563fd9348eb0, 0xc00313a1e0}, 0x40?, 0x40?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/api/server/middleware/version.go:84 +0x2ae
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1({0x563fd93527e0, 0xc003e012c0}, {0x563fd9348eb0?, 0xc00313a1e0?}, 0xc002241900, 0xc001b924e0?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/pkg/authorization/middleware.go:59 +0x683
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1({0x563fd9348eb0, 0xc00313a1e0}, 0xc002241800)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/api/server/server.go:56 +0x1ce
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.HandlerFunc.ServeHTTP(0x563fd93527e0?, {0x563fd9348eb0?, 0xc00313a1e0?}, 0x563fd8954bf8?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:2136 +0x29
4月 01 11:13:08 tapsupertest7 dockerd[171183]: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP(0xc0021f53f0, {0x563fd93407b0?, 0xc0036482a0}, 0xc002241600, {0x563fd9324180, 0xc001754108})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/handler.go:229 +0x1202
4月 01 11:13:08 tapsupertest7 dockerd[171183]: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1({0x563fd93407b0?, 0xc0036482a0?}, 0x563fd894da01?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/handler.go:81 +0x35
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.HandlerFunc.ServeHTTP(0x563fd93527e0?, {0x563fd93407b0?, 0xc0036482a0?}, 0x563fd894da30?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:2136 +0x29
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.HandlerFunc.ServeHTTP(0xc002241500?, {0x563fd93407b0?, 0xc0036482a0?}, 0x1?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:2136 +0x29
4月 01 11:13:08 tapsupertest7 dockerd[171183]: github.com/gorilla/mux.(*Router).ServeHTTP(0xc0016b5080, {0x563fd93407b0, 0xc0036482a0}, 0xc002241400)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /root/rpmbuild/BUILD/src/engine/vendor/github.com/gorilla/mux/mux.go:212 +0x1c5
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.serverHandler.ServeHTTP({0x563fd9334698?}, {0x563fd93407b0?, 0xc0036482a0?}, 0x6?)
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:2938 +0x8e
4月 01 11:13:08 tapsupertest7 dockerd[171183]: net/http.(*conn).serve(0xc003911830, {0x563fd93527e0, 0xc0038930e0})
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:2009 +0x5f4
4月 01 11:13:08 tapsupertest7 dockerd[171183]: created by net/http.(*Server).Serve in goroutine 4533
4月 01 11:13:08 tapsupertest7 dockerd[171183]:         /usr/local/go/src/net/http/server.go:3086 +0x5cb

Reproduce

The host machine's disk usage is full. After using docker compose down to clean up one of the services, the remaining disk space is now 40%. However, when using docker compose up -d to start another service, the above error is reported. This error still exists after restarting the docker daemon.

Expected behavior

No response

docker version

Client: Docker Engine - Community
 Version:           26.0.0
 API version:       1.45
 Go version:        go1.21.8
 Git commit:        2ae903e
 Built:             Wed Mar 20 15:19:04 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.0.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       8b79278
  Built:            Wed Mar 20 15:17:57 2024
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    26.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.25.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 58
  Running: 55
  Paused: 0
  Stopped: 3
 Images: 47
 Server Version: 26.0.0
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 4.18.0-348.7.1.el8_5.x86_64
 Operating System: CentOS Linux 8
 OSType: linux
 Architecture: x86_64
 CPUs: 64
 Total Memory: 251.5GiB
 Name: tapsupertest7
 ID: 6110621f-43d2-4d78-b47d-633c74842aec
 Docker Root Dir: /data/docker
 Debug Mode: false
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true

Additional Info

No response
@dreamcoin1998 dreamcoin1998 added kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage labels Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/daemon area/volumes kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage version/26.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@thaJeztah @vvoland @dreamcoin1998