Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo: startup failed #292

Open
yenesey opened this issue Jan 21, 2024 · 5 comments
Open

Demo: startup failed #292

yenesey opened this issue Jan 21, 2024 · 5 comments
Labels
help wanted 🆘 Extra attention is needed type/question 🙋 Further information is requested

Comments

@yenesey
Copy link

yenesey commented Jan 21, 2024

docker run
--name pgvecto-rs-demo
-e POSTGRES_PASSWORD=mysecretpassword
-p 5432:5432
-d tensorchord/pgvecto-rs:pg16-v0.1.14-beta

Startup is failed. The log is:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.


Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2024-01-21 18:50:56.269 UTC [48] FATAL:  called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
2024-01-21 18:50:56.269 UTC [48] LOG:  database system is shut down
pg_ctl: could not start server
Examine the log output.
 stopped waiting
@xieydd
Copy link
Member

xieydd commented Jan 23, 2024

cc @usamoi @cutecutecat

@gaocegege gaocegege added the type/question 🙋 Further information is requested label Jan 23, 2024
@cutecutecat
Copy link
Member

cutecutecat commented Jan 23, 2024

Thanks you for your report!

To debug it further, you could recreate the container with RUST_BACKTRACE=full environment to get more information and paste the logs again:

docker run --name pgvecto-rs-demo \
-e POSTGRES_PASSWORD=mysecretpassword \
-e RUST_BACKTRACE=full \
-p 5432:5432 -d tensorchord/pgvecto-rs:pg16-v0.1.14-beta

@yenesey
Copy link
Author

yenesey commented Jan 23, 2024

Here it is:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2024-01-23 19:31:07.015 UTC [48] FATAL:  called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
2024-01-23 19:31:07.015 UTC [48] DETAIL:
           0: pgrx_pg_sys::submodules::panic::register_pg_guard_panic_hook::{{closure}}::{{closure}}
                     at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pgrx-pg-sys-0.11.2/src/submodules/panic.rs:312:39
           1: std::thread::local::LocalKey<T>::try_with
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/thread/local.rs:270:16
           2: std::thread::local::LocalKey<T>::with
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/thread/local.rs:246:9
           3: pgrx_pg_sys::submodules::panic::register_pg_guard_panic_hook::{{closure}}
                     at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pgrx-pg-sys-0.11.2/src/submodules/panic.rs:309:24
           4: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/alloc/src/boxed.rs:2021:9
           5: std::panicking::rust_panic_with_hook
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panicking.rs:783:13
           6: std::panicking::begin_panic_handler::{{closure}}
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panicking.rs:657:13
           7: std::sys_common::backtrace::__rust_end_short_backtrace
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/sys_common/backtrace.rs:170:18
           8: rust_begin_unwind
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panicking.rs:645:5
           9: core::panicking::panic_fmt
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/core/src/panicking.rs:72:14
          10: core::result::unwrap_failed
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/core/src/result.rs:1649:5
          11: core::result::Result<T,E>::unwrap
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/core/src/result.rs:1073:23
          12: vectors::ipc::transport::mmap::init
                     at /home/runner/work/pgvecto.rs/pgvecto.rs/src/ipc/transport/mmap.rs:17:17
          13: vectors::ipc::init
                     at /home/runner/work/pgvecto.rs/pgvecto.rs/src/ipc/mod.rs:51:5
          14: vectors::_PG_init::_PG_init_inner
                     at /home/runner/work/pgvecto.rs/pgvecto.rs/src/lib.rs:30:9
          15: vectors::_PG_init::{{closure}}
                     at /home/runner/work/pgvecto.rs/pgvecto.rs/src/lib.rs:21:1
          16: std::panicking::try::do_call
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panicking.rs:552:40
          17: std::panicking::try
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panicking.rs:516:19
          18: std::panic::catch_unwind
                     at /rustc/dd430bc8c22f57992ec1457a87437d14283fdd65/library/std/src/panic.rs:142:14
          19: pgrx_pg_sys::submodules::panic::run_guarded
                     at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pgrx-pg-sys-0.11.2/src/submodules/panic.rs:408:11
          20: pgrx_pg_sys::submodules::panic::pgrx_extern_c_guard
                     at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pgrx-pg-sys-0.11.2/src/submodules/panic.rs:385:11
          21: _PG_init
                     at /home/runner/work/pgvecto.rs/pgvecto.rs/src/lib.rs:22:1
          22: <unknown>
          23: load_file
          24: <unknown>
          25: process_shared_preload_libraries
          26: PostmasterMain
          27: main
          28: <unknown>
          29: __libc_start_main
          30: _start

2024-01-23 19:31:07.015 UTC [48] LOG:  database system is shut down
pg_ctl: could not start server
 stopped waiting

What it does when:
fixing permissions on existing directory /var/lib/postgresql/data ... ok

Why volume permissions changed from root:root after start (maybe this is a cause):

/var/lib/docker/volumes/b2f55e2f25bfcf0702e237505fa6c8b426724cb73db84735fc24c382be3dade3/_data# ls -l
total 120
drwx------ 5 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 base
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 global
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_commit_ts
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_dynshmem
-rw------- 1 openmediavault-webgui openmediavault-config  5743 Jan 23 22:44 pg_hba.conf
-rw------- 1 openmediavault-webgui openmediavault-config  2640 Jan 23 22:44 pg_ident.conf
drwx------ 4 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_logical
drwx------ 4 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_multixact
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_notify
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_replslot
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_serial
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_snapshots
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_stat
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_stat_tmp
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_subtrans
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_tblspc
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_twophase
-rw------- 1 openmediavault-webgui openmediavault-config     3 Jan 23 22:44 PG_VERSION
drwx------ 3 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_wal
drwx------ 2 openmediavault-webgui openmediavault-config  4096 Jan 23 22:44 pg_xact
-rw------- 1 openmediavault-webgui openmediavault-config    88 Jan 23 22:44 postgresql.auto.conf
-rw------- 1 openmediavault-webgui openmediavault-config 29770 Jan 23 22:44 postgresql.conf

@cutecutecat
Copy link
Member

cutecutecat commented Jan 24, 2024

What it does when:
fixing permissions on existing directory /var/lib/postgresql/data ... ok

In short, this is a process of uid/gid mapping by postgres, it would use 999 as user postgres, which only shows up at the host as openmediavault-webgui/openmediavault-config.
This is expected behavior.
As a reference: https://stackoverflow.com/a/56188924


Back to the question, we could see that there is little discussion about it:
Failed to create UnixStream · dani-garcia/vaultwarden · Discussion #3800 (github.com)

It's due to UnixStream failed to create, and there could be many infra reasons for that. Since it is not easy to reproduce, things are more complicated.

We will do our best to dig it out, but success is not guaranteed. It would be much helpful if you could provide the same information as in the discussion above.

#Kernel/Host info:
uname -a

# CPU Info:
cat /proc/cpuinfo | grep -m1 -i "model name"

#Docker info:
docker info

# Storage space:
df -h

@yenesey
Copy link
Author

yenesey commented Jan 24, 2024

#Kernel/Host info:
Linux nas 5.15.131-2-pve #1 SMP PVE 5.15.131-3 (2023-12-01T13:42Z) x86_64 GNU/Linux
pve kernel for ZFS support on my NAS (If I remember correctly)

#CPU Info:
model name : Intel(R) Celeron(R) J4105 CPU @ 1.50GHz

#Docker info:

Client: Docker Engine - Community
 Version:    25.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 7
  Running: 1
  Paused: 0
  Stopped: 6
 Images: 7
 Server Version: 25.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: a1496014c916f9e62104b33d1bb5bd03b0858e59
 runc version: v1.1.11-0-g4bccb38
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.131-2-pve
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.567GiB
 Name: nas
 ID: SKTX:TBIW:KK2W:CDXR:SQJJ:RWC6:DIFQ:V53E:M4H2:EYXU:B4ZH:VFSU
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

#Storage space:

Filesystem                    Size  Used Avail Use% Mounted on
udev                          3.8G     0  3.8G   0% /dev
tmpfs                         775M  5.0M  770M   1% /run
/dev/nvme0n1p3                442G   32G  389G   8% /
tmpfs                         3.8G  4.0K  3.8G   1% /dev/shm
tmpfs                         5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p2                511M  288K  511M   1% /boot/efi
tmpfs                         3.8G   48K  3.8G   1% /tmp

@gaocegege gaocegege added the help wanted 🆘 Extra attention is needed label Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted 🆘 Extra attention is needed type/question 🙋 Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants