What is the meaning of the distance of the result obtained from searching a Array of Struct?

[wh5938316]

If an entity's Array of Struct has multiple embeddings, the query result for an entity shows only one distance instead of multiple. How is this distance calculated—is it a weighted average of the distances from each embedding, or is it computed by averaging the multiple embeddings into a single vector to get one distance, or is it another method?

Is there a way to know the individual distances between each embedding in the Array of Struct and the query vector?

[yhmo]

For embedding list, MAX_SIM metric is used to calculate the similarity of N embeddings with M embeddings.
https://milvus.io/docs/array-of-structs.md

For example, we have a collection which contains a Struct field. Each value of the Struct field is a list of struct. The Struct field schema is:

struct_schema = milvus_client.create_struct_field_schema()
struct_schema.add_field("emb", DataType.FLOAT_VECTOR, dim=DIM)
collection_schema.add_field("st",datatype=DataType.ARRAY, element_type=DataType.STRUCT, struct_schema=struct_schema, max_capacity=10)

There are some rows in the collection. row_1 has 2 structs, row_2 has 3 structs.

row1 = {"id" 1, "st": [{"emb": vector_1_1}, {"emb": vector_1_2}]}
row2 = {"id" 2, "st": [{"emb": vector_2_1}, {"emb": vector_2_2}, {"emb": vector_2_3}]}

Now we search with an embedding list which contains 2 vectors:

embeddingList = EmbeddingList()
# Query with 2 vectors
embeddingList.add(q_vec_1)
embeddingList.add(q_vec_2)

res = milvus_client.search(collection_name, data=queries, limit=10, anns_field="st[emb]")

Assume the metric type is MAX_SIM_COSINE.
For the row1, the score is:

max(cosine_value_of(q_vec_1, vector_1_1), cosine_value_of(q_vec_1, vector_1_2)) + max(cosine_value_of(q_vec_2, vector_1_1), cosine_value_of(q_vec_2, vector_1_2)) 

For the row2, the score is:

max(cosine_value_of(q_vec_1, vector_2_1), cosine_value_of(q_vec_1, vector_2_2), cosine_value_of(q_vec_1, vector_2_3)) + max(cosine_value_of(q_vec_2, vector_2_1), cosine_value_of(q_vec_2, vector_2_2), cosine_value_of(q_vec_2, vector_2_3))

[wh5938316]

I understand that when performing a search on an array of structs, Milvus currently returns only the MAX_SIM distance for each entity, which represents the highest similarity score between the query vectors and the embeddings in the struct. However, since each embedding's distance is already computed during the search, I wanted to ask if it would be possible to return the individual distances between each query vector and the embeddings, rather than just the MAX_SIM distance.

In my case, I need to reorder the results based on the individual distances between the query vectors and each embedding, so having access to the distances for each embedding would be very useful. I checked the results in pymilvus, but I only found the MAX_SIM distance.

Is it possible to include the individual distances in the query results, or is there a way to access this information directly from Milvus?

Thank you for your help, and I look forward to your response!

[yhmo]

In proto level, the search result is defined as:
https://github.com/milvus-io/milvus-proto/blob/44bb43c7e07b36fc6dcecba3b902c0612c0463d0/proto/schema.proto#L251

message SearchResultData {
  int64 num_queries = 1;
  int64 top_k = 2;
  repeated FieldData fields_data = 3;
  repeated float scores = 4;
  IDs ids = 5;
  repeated int64 topks = 6;
  repeated string output_fields = 7;
  FieldData group_by_field_value = 8;
  int64 all_search_count = 9;
  repeated float distances = 10;
  optional SearchIteratorV2Results search_iterator_v2_results = 11;
  repeated float recalls = 12;
  string primary_field_name = 13;
}

"repeated float scores" is a one-dim float list, and split by "repeated int64 topks" to represent each topk items of query vectors.
I don't think it can represent "scores of N vectors with M vectors for K embedding lists".

[yhmo]

A proposal:
Add a new member to the SearchResultData

message SearchResultData {
  ......
  repeated FloatArray max_sim_scores = 14;

@alwayslove2013 is the feature owner of MAX_SIM, wait for his comment.

[alwayslove2013]

@wh5938316 @yhmo Thank you for sharing your requirements, and we will take them into serious consideration.

Calculating the distance between vectors is relatively straightforward, as it can be represented by a single numerical value. However, for structure-based matching, returning only an aggregated score is indeed insufficient to meet detailed verification needs.

I will discuss this design with our product management team. Our goal is to support more flexible and granular descriptions of structure matching results, thereby enhancing the reliability and credibility of the output.

We will keep you updated on any progress regarding this feature enhancement. Please feel free to reach out if you have additional suggestions or details to share.