Pinecone
类型：数据库
简介：实时且性能出色的向量数据库，专门针对大规模向量搜索进行优化。
语义搜索即从密集索引中搜索与查询在含义和上下文上最相似的记录，使用密集向量，而密集向量中的每个数字都对应于多维空间中的一个点，在该空间中彼此更接近的向量在语义上更相似。本文主要分享Pinecone向量数据库实现语义搜索的全流程。
一、使用文本搜索
要使用查询文本搜索密集索引，请使用带有以下参数的 search_records 操作：
1、查询的命名空间使用默认命名空间，请将命名空间设置为“__default__”。
2、query.inputs.text 参数，其中包含查询文本。Pinecone 使用与索引集成的嵌入模型自动将文本转换为密集向量。
3、query.top_k 参数，其中包含要返回的相似记录数。
4、可以选择指定要在响应中返回的字段。如果未指定，则响应将包含所有字段。
例如以下代码搜索与查询文本语义最相关的两条记录：
（1）Python：
from pinecone import Pinecone
pc = Pinecone(api_key="YOUR_API_KEY")
# To get the unique host for an index,
# see https://docs.pinecone.io/guides/manage-data/target-an-index
index = pc.Index(host="INDEX_HOST")
results = index.search(
namespace="example-namespace",
query={
"inputs": {"text": "Disease prevention"},
"top_k": 2
},
fields=["category", "chunk_text"]
)
print(results)
（2）JavaScript
import { Pinecone } from '@pinecone-database/pinecone'
const pc = new Pinecone({ apiKey: "YOUR_API_KEY" })
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
const namespace = pc.index("INDEX_NAME", "INDEX_HOST").namespace("example-namespace");
const response = await namespace.searchRecords({
query: {
topK: 2,
inputs: { text: 'Disease prevention' },
},
fields: ['chunk_text', 'category'],
});
console.log(response);
（3）Java
import io.pinecone.clients.Index;
import io.pinecone.configs.PineconeConfig;
import io.pinecone.configs.PineconeConnection;
import org.openapitools.db_data.client.ApiException;
import org.openapitools.db_data.client.model.SearchRecordsResponse;
import java.util.*;
public class SearchText {
public static void main(String[] args) throws ApiException {
PineconeConfig config = new PineconeConfig("YOUR_API_KEY");
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
config.setHost("INDEX_HOST");
PineconeConnection connection = new PineconeConnection(config);
Index index = new Index(config, connection, "integrated-dense-java");
String query = "Disease prevention";
List<String> fields = new ArrayList<>();
fields.add("category");
fields.add("chunk_text");
// Search the dense index
SearchRecordsResponse recordsResponse = index.searchRecordsByText(query, "example-namespace", fields, 2, null, null);
// Print the results
System.out.println(recordsResponse);
}
}
响应如下所示。每条记录都返回一个相似度得分，该得分表示其与查询向量的距离，根据索引的相似度度量计算得出。
（1）Python
{'result': {'hits': [{'_id': 'rec3',
'_score': 0.8204272389411926,
'fields': {'category': 'immune system',
'chunk_text': 'Rich in vitamin C and other '
'antioxidants, apples '
'contribute to immune health '
'and may reduce the risk of '
'chronic diseases.'}},
{'_id': 'rec1',
'_score': 0.7931625843048096,
'fields': {'category': 'digestive system',
'chunk_text': 'Apples are a great source of '
'dietary fiber, which supports '
'digestion and helps maintain a '
'healthy gut.'}}]},
'usage': {'embed_total_tokens': 8, 'read_units': 6}}
（2）JavaScript
{
result: {
hits: [
{
_id: 'rec3',
_score: 0.82042724,
fields: {
category: 'immune system',
chunk_text: 'Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases.'
}
},
{
_id: 'rec1',
_score: 0.7931626,
fields: {
category: 'digestive system',
chunk_text: 'Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut.'
}
}
]
},
usage: {
readUnits: 6,
embedTotalTokens: 8
}
}
（3）Java
class SearchRecordsResponse {
result: class SearchRecordsResponseResult {
hits: [class Hit {
id: rec3
score: 0.8204272389411926
fields: {category=immune system, chunk_text=Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases.}
additionalProperties: null
}, class Hit {
id: rec1
score: 0.7931625843048096
fields: {category=endocrine system, chunk_text=Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut.}
additionalProperties: null
}]
additionalProperties: null
}
usage: class SearchUsage {
readUnits: 6
embedTotalTokens: 13
}
additionalProperties: null
}
二、使用密集向量搜索
要使用查询的密集向量表示形式搜索密集索引，请使用以下参数的查询操作：
1、使用默认命名空间，请将命名空间设置为“__default__”。
2、包含表示查询的密集向量值的向量参数。
3、包含要返回的结果数的 top_k 参数。
4、可以选择将 include_values 和/或 include_metadata 设置为 true，以在响应中包含匹配记录的向量值和/或元数据。为了获得更好的性能，尤其是在 top_k 值较高的情况下，除非必要，否则请避免包含向量值。
例如，以下代码使用查询“疾病预防”的密集向量表示，在 example-namespaces 命名空间中搜索语义最相似的 3 条记录：
（1）Python
from pinecone.grpc import PineconeGRPC as Pinecone pc = Pinecone(api_key="YOUR_API_KEY") # To get the unique host for an index, # see https://docs.pinecone.io/guides/manage-data/target-an-index index = pc.Index(host="INDEX_HOST") index.query( namespace="example-namespace", vector=[0.0236663818359375,-0.032989501953125, ..., -0.01041412353515625,0.0086669921875], top_k=3, include_metadata=True, include_values=False )
（2）JavaScript
import { Pinecone } from '@pinecone-database/pinecone'
const pc = new Pinecone({ apiKey: "YOUR_API_KEY" })
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
const index = pc.index("INDEX_NAME", "INDEX_HOST")
const queryResponse = await index.namespace('example-namespace').query({
vector: [0.0236663818359375,-0.032989501953125,...,-0.01041412353515625,0.0086669921875],
topK: 3,
includeValues: false,
includeMetadata: true,
});
（3）Java
import io.pinecone.clients.Index;
import io.pinecone.configs.PineconeConfig;
import io.pinecone.configs.PineconeConnection;
import io.pinecone.unsigned_indices_model.QueryResponseWithUnsignedIndices;
import java.util.Arrays;
import java.util.List;
public class QueryExample {
public static void main(String[] args) {
PineconeConfig config = new PineconeConfig("YOUR_API_KEY");
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
config.setHost("INDEX_HOST");
PineconeConnection connection = new PineconeConnection(config);
Index index = new Index(connection, "INDEX_NAME");
List<Float> query = Arrays.asList(0.0236663818359375f, -0.032989501953125f, ..., -0.01041412353515625f, 0.0086669921875f);
QueryResponseWithUnsignedIndices queryResponse = index.query(3, query, null, null, null, "example-namespace", null, false, true);
System.out.println(queryResponse);
}
}
响应如下所示。每条记录都返回一个相似度得分，该得分表示其与查询向量的距离，根据索引的相似度度量计算得出。
（1）Python
{'matches': [{'id': 'rec3',
'metadata': {'category': 'immune system',
'chunk_text': 'Rich in vitamin C and other '
'antioxidants, apples contribute to '
'immune health and may reduce the '
'risk of chronic diseases.'},
'score': 0.82026422,
'values': []},
{'id': 'rec1',
'metadata': {'category': 'digestive system',
'chunk_text': 'Apples are a great source of '
'dietary fiber, which supports '
'digestion and helps maintain a '
'healthy gut.'},
'score': 0.793068111,
'values': []},
{'id': 'rec4',
'metadata': {'category': 'endocrine system',
'chunk_text': 'The high fiber content in apples '
'can also help regulate blood sugar '
'levels, making them a favorable '
'snack for people with diabetes.'},
'score': 0.780169606,
'values': []}],
'namespace': 'example-namespace',
'usage': {'read_units': 6}}
（2）JavaScript
{
matches: [
{
id: 'rec3',
score: 0.819709897,
values: [],
sparseValues: undefined,
metadata: [Object]
},
{
id: 'rec1',
score: 0.792900264,
values: [],
sparseValues: undefined,
metadata: [Object]
},
{
id: 'rec4',
score: 0.780068815,
values: [],
sparseValues: undefined,
metadata: [Object]
}
],
namespace: 'example-namespace',
usage: { readUnits: 6 }
}
（3）Java
class QueryResponseWithUnsignedIndices {
matches: [ScoredVectorWithUnsignedIndices {
score: 0.8197099
id: rec3
values: []
metadata: fields {
key: "category"
value {
string_value: "immune system"
}
}
fields {
key: "chunk_text"
value {
string_value: "Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases."
}
}
sparseValuesWithUnsignedIndices: SparseValuesWithUnsignedIndices {
indicesWithUnsigned32Int: []
values: []
}
}, ScoredVectorWithUnsignedIndices {
score: 0.79290026
id: rec1
values: []
metadata: fields {
key: "category"
value {
string_value: "digestive system"
}
}
fields {
key: "chunk_text"
value {
string_value: "Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut."
}
}
sparseValuesWithUnsignedIndices: SparseValuesWithUnsignedIndices {
indicesWithUnsigned32Int: []
values: []
}
}, ScoredVectorWithUnsignedIndices {
score: 0.7800688
id: rec4
values: []
metadata: fields {
key: "category"
value {
string_value: "endocrine system"
}
}
fields {
key: "chunk_text"
value {
string_value: "The high fiber content in apples can also help regulate blood sugar levels, making them a favorable snack for people with diabetes."
}
}
sparseValuesWithUnsignedIndices: SparseValuesWithUnsignedIndices {
indicesWithUnsigned32Int: []
values: []
}
}]
namespace: example-namespace
usage: read_units: 6
}
三、使用记录ID搜索
使用记录ID进行搜索时，Pinecone会使用与该记录关联的密集向量作为查询。要使用记录ID搜索密集索引，请使用以下参数的查询操作：
1、使用默认命名空间，请将命名空间设置为“__default__”。
2、id参数，其中包含要用作查询的向量的唯一记录 ID。
3、top_k 参数，指定要返回的结果数量。
4、或者可以选择将 include_values 和/或 include_metadata 设置为 true，以便在响应中包含匹配记录的向量值和/或元数据。为了获得更好的性能，尤其是在 top_k 值较高的情况下，除非必要，否则请避免包含向量值。
例如，以下代码使用 ID 在 example-namespace 命名空间中搜索与记录中的密集向量语义最相似的 3 条记录：
（1）Python
from pinecone.grpc import PineconeGRPC as Pinecone pc = Pinecone(api_key="YOUR_API_KEY") # To get the unique host for an index, # see https://docs.pinecone.io/guides/manage-data/target-an-index index = pc.Index(host="INDEX_HOST") index.query( namespace="example-namespace", id="rec2", top_k=3, include_metadata=True, include_values=False )
（2）JavaScript
import { Pinecone } from '@pinecone-database/pinecone'
const pc = new Pinecone({ apiKey: "YOUR_API_KEY" })
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
const index = pc.index("INDEX_NAME", "INDEX_HOST")
const queryResponse = await index.namespace('example-namespace').query({
id: 'rec2',
topK: 3,
includeValues: false,
includeMetadata: true,
});
（3）Java
import io.pinecone.clients.Index;
import io.pinecone.configs.PineconeConfig;
import io.pinecone.configs.PineconeConnection;
import io.pinecone.unsigned_indices_model.QueryResponseWithUnsignedIndices;
public class QueryExample {
public static void main(String[] args) {
PineconeConfig config = new PineconeConfig("YOUR_API_KEY");
// To get the unique host for an index,
// see https://docs.pinecone.io/guides/manage-data/target-an-index
config.setHost("INDEX_HOST");
PineconeConnection connection = new PineconeConnection(config);
Index index = new Index(connection, "INDEX_NAME");
QueryResponseWithUnsignedIndices queryRespone = index.queryByVectorId(3, "rec2", "example-namespace", null, false, true);
System.out.println(queryResponse);
}
}
四、并行查询
Python SDK v6.0.0 及更高版本提供了可用于 asyncio 的异步方法。异步支持让 Pinecone 可以与 FastAPI、Quart 和 Sanic 等现代异步 Web 框架配合使用，并能显著提高并行查询的效率。
