High similarity response latency numbers for DISKANN index

[igmor]

Hello there!

We're exploring milvus for similarity search and found out some discrepancy between reported latency numbers and latencies we were able to observe.
We are testing image embeddings, 100K sample dataset with 512 dimensions.
We have ran this dataset on DiskANN internal tool
https://github.com/microsoft/DiskANN/blob/main/workflows/SSD_index.md

consistently getting numbers from 1.5 to 6 ms (L ranging from 10 to 100) of search time
DiskAnn index built for milvus puts as into 100-120ms search latency times almost two order of magnitude worse than the native DiskANN index.

Could you explain this discrepancy? Is there something we can optimize in milvus configuration to make things faster?
For the reference this is the milvus configuration we're using

        ---
        cluster:
          enabled: false

        etcd:
          enabled: true

        pulsar:
          enabled: false

        minio:
          enabled: false
          mode: standalone
          gcsgateway:
            enabled: false
          tls:
            enabled: false
        
        standalone:
          persistence:
            mountPath: "/data"
            enabled: true
            annotations:
              helm.sh/resource-policy: keep
            persistentVolumeClaim:
              existingClaim: ""
              storageClass: local-storage
              accessModes: ReadWriteOnce
              size: 410Gi
              subPath: "milvus"

        extraConfigFiles:
          user.yaml: |+
            common:
              storageType: local
            localStorage:
              path: /data/milvus/              

[xiaofan-luan]

Hello there!

We're exploring milvus for similarity search and found out some discrepancy between reported latency numbers and latencies we were able to observe. We are testing image embeddings, 100K sample dataset with 512 dimensions. We have ran this dataset on DiskANN internal tool https://github.com/microsoft/DiskANN/blob/main/workflows/SSD_index.md

consistently getting numbers from 1.5 to 6 ms (L ranging from 10 to 100) of search time DiskAnn index built for milvus puts as into 100-120ms search latency times almost two order of magnitude worse than the native DiskANN index.

Could you explain this discrepancy? Is there something we can optimize in milvus configuration to make things faster? For the reference this is the milvus configuration we're using

        ---
        cluster:
          enabled: false

        etcd:
          enabled: true

        pulsar:
          enabled: false

        minio:
          enabled: false
          mode: standalone
          gcsgateway:
            enabled: false
          tls:
            enabled: false
        
        standalone:
          persistence:
            mountPath: "/data"
            enabled: true
            annotations:
              helm.sh/resource-policy: keep
            persistentVolumeClaim:
              existingClaim: ""
              storageClass: local-storage
              accessModes: ReadWriteOnce
              size: 410Gi
              subPath: "milvus"

        extraConfigFiles:
          user.yaml: |+
            common:
              storageType: local
            localStorage:
              path: /data/milvus/              

1. You are testing on 100K data set right?
2. Only 1 concurrency, not parallism?
3. Same disk as u test diskann?
4. Using bounded staleness consistency?

If that's the case, I don't think 100ms or more is reasonable, you need to deploy monitorning and logs and do further analysis.

But ideally, diskann index is expected to be reponse at 100ms-200ms