Does milvus support ipv6

[rajeevpadinharepattu]

We are currently setting up a Ipv6 environment, while installing the milvus-db, milvus standalone is failing with the below error.

eic@devgpu01:~> kubectl get pods -n milvus-db
NAME READY STATUS RESTARTS AGE
milvus-etcd-0 1/1 Running 0 3d4h
milvus-minio-7c65b8cd8c-js8cl 1/1 Running 0 3d4h
milvus-standalone-579cc9d946-kmpbz 0/1 CrashLoopBackOff 900 (56s ago) 3d4h

eic@devgpu01:~> kubectl logs -n milvus-db milvus-standalone-579cc9d946-kmpbz
2022/10/31 11:53:02 maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
[2022/10/31 11:53:02.551 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.551 +00:00] [INFO] [logutil/logutil.go:165] ["Log directory"] [configDir=]
[2022/10/31 11:53:02.551 +00:00] [INFO] [logutil/logutil.go:166] ["Set log file to "] [path=]

__  _________ _   ____  ______

/ |/ / / /| | / / / / / __/
/ /|/ // // /| |/ / // /\
// /////_/__/

Welcome to use Milvus!
Version: v2.1.2
Built: Fri Sep 16 06:40:43 UTC 2022
GitCommit: 3c3ba55
GoVersion: go version go1.16.9 linux/amd64

[2022/10/31 11:53:02.551 +00:00] [INFO] [roles/roles.go:358] ["starting running Milvus components"]
open pid file: /run/milvus/standalone.pid
lock pid file: /run/milvus/standalone.pid
[2022/10/31 11:53:02.552 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.552 +00:00] [DEBUG] [paramtable/component_param.go:769] ["init cacheSize"] ["cacheSize (GB)"=0]
[2022/10/31 11:53:02.554 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.554 +00:00] [DEBUG] [server/global_rmq.go:55] ["initializing global rmq"] [path=/var/lib/milvus/rdb_data]
[2022/10/31 11:53:02.554 +00:00] [DEBUG] [server/rocksmq_impl.go:157] ["Start rocksmq "] ["max proc"=8] [parallism=1] ["lru cache"=4042490511]
[2022/10/31 11:53:02.647 +00:00] [INFO] [tso/tso.go:121] ["sync and save timestamp"] [last=2022/10/31 11:48:04.599 +00:00] [save=2022/10/31 11:53:05.647 +00:00] [next=2022/10/31 11:53:02.647 +00:00]
[2022/10/31 11:53:02.647 +00:00] [DEBUG] [server/rocksmq_retention.go:99] ["Rocksmq retention goroutine start!"]
[2022/10/31 11:53:02.648 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.648 +00:00] [DEBUG] [paramtable/component_param.go:769] ["init cacheSize"] ["cacheSize (GB)"=0]
[2022/10/31 11:53:02.649 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.649 +00:00] [INFO] [paramtable/base_table.go:453] ["failed to parse pulsar config, assume pulsar not used"] [error="parse "": empty url"]
[2022/10/31 11:53:02.649 +00:00] [DEBUG] [paramtable/component_param.go:769] ["init cacheSize"] ["cacheSize (GB)"=0]
panic: runtime error: index out of range [0] with length 0

goroutine 169 [running]:
github.com/go-basic/ipv4.LocalIP(...)
/go/pkg/mod/github.com/go-basic/ipv4@v1.0.0/ip.go:26
github.com/milvus-io/milvus/internal/util/paramtable.(*grpcConfig).LoadFromEnv(...)
/go/src/github.com/milvus-io/milvus/internal/util/paramtable/grpc_param.go:85
github.com/milvus-io/milvus/internal/util/paramtable.(*grpcConfig).init(0x5a99e40, 0x400b731, 0x9)
/go/src/github.com/milvus-io/milvus/internal/util/paramtable/grpc_param.go:77 +0xd7
github.com/milvus-io/milvus/internal/util/paramtable.(*GrpcServerConfig).init(0x5a99e40, 0x400b731, 0x9)
/go/src/github.com/milvus-io/milvus/internal/util/paramtable/grpc_param.go:130 +0x45
github.com/milvus-io/milvus/internal/util/paramtable.(*GrpcServerConfig).InitOnce.func1()
/go/src/github.com/milvus-io/milvus/internal/util/paramtable/grpc_param.go:125 +0x3c
sync.(*Once).doSlow(0x5a9a100, 0xc00001dd18)
/usr/local/go/src/sync/once.go:68 +0xec
sync.(*Once).Do(...)
/usr/local/go/src/sync/once.go:59
github.com/milvus-io/milvus/internal/util/paramtable.(*GrpcServerConfig).InitOnce(0x5a99e40, 0x400b731, 0x9)
/go/src/github.com/milvus-io/milvus/internal/util/paramtable/grpc_param.go:124 +0x87
github.com/milvus-io/milvus/internal/distributed/rootcoord.(*Server).init(0xc00053c370, 0x8bdf3e, 0xc000b7d378)
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:154 +0x57
github.com/milvus-io/milvus/internal/distributed/rootcoord.(*Server).Run(0xc00053c370, 0xc00053c300, 0x0)
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:141 +0x2f
github.com/milvus-io/milvus/cmd/components.(*RootCoord).Run(0xc000d89fc0, 0xffffffffffffffff, 0x43c4c28)
/go/src/github.com/milvus-io/milvus/cmd/components/root_coord.go:55 +0x33
github.com/milvus-io/milvus/cmd/roles.(*MilvusRoles).runRootCoord.func1(0x1, 0x43b2830, 0xc000cad300, 0xc0005a2890, 0xc0005c8070, 0xc000b7d370)
/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:123 +0x125
created by github.com/milvus-io/milvus/cmd/roles.(*MilvusRoles).runRootCoord
/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:106 +0xba

Is there any specific cofiguration required milvus-db to running in ipv6 environment ?

Kuberentes version: 1.22.12
Milvus Version: 2.4.21

[yhmo]

We don't support IPv6 yet. The milvus components will only register its IPv4 address to etcd. For IPv6 only kubernetes cluster, this won't work.

[MR-GOYAL]

@yhmo if we want to register IPV6 only kubernetes cluster, what we have to do? If it requires any changes , What are those ?

[yhmo]

Currently, milvus doesn't support IPv6 address.
In the source code, we can see the util method returns IPv4 address:

milvus/pkg/util/funcutil/func.go

Line 63 in 75e6b65

func GetLocalIP() string {

Milvus uses gRPC as RPC lib, I believe the gRPC can support IPv6. To support IPv6 for milvus, we need to extend the util method to return IPv4 or IPv6 smartly, and double-check the address input matches the naming pattern of grpc:

milvus/internal/util/grpcclient/client.go

Line 264 in 75e6b65

addr,

You can add your comment in this thread: #6302 (comment)

[UnyieldingOrca]

Hi all, I opened #32960 which I think will allow milvus to run in IPv6 networks.

[xiaofan-luan]

Did you already test it on your environemnt?
IPv6 is support could be very complicated, we might need to change many third party party component as well?

[UnyieldingOrca]

I tested it locally using docker. I updated the network config in deployments/docker/dev/docker-compose-apple-silicon.yml to include:

networks:
  default:
    name: milvus_dev
    enable_ipv6: true
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 2001:3200:3200::/64
          gateway: 2001:3200:3200::1

In a builder container with port 19530 exposed I started milvus with scripts/start_cluster.sh and was able to connect to the cluster with pymilvus and run a command, I did not run the full e2e tests.

I tested this with the current code in that PR and another version of the code which only used ipv6 addresses because my test environment was dual stack. But my target environment is K8s with IPv6 which I have not tested against yet.

[UnyieldingOrca]

@xiaofan-luan Is that test sufficient? I believe the other tools in the stack already support ipv6.

[xiaofan-luan]

I'm not an expert, @LoveEachDay @zwd1208 what do you think?