PANNS: Enhancing Graph-based Approximate Nearest Neighbor Search through Recency-aware Construction and Parameterized Search
Approximate Nearest-Neighbor Search (ANNS) has become the standard approach for querying in vector databases, especially with the recent surge in large-scale, high-dimensional data driven by LLM-based applications. Recently, graph-based ANNS has demonstrated enhanced throughput by constructing a graph from the dataset, where edges reflect the similarities between data points, and employing best-first search algorithms for query evaluation.
This work demonstrates that the design space for both graph construction and searching is under-explored. First, the construction phase overlooks the potential temporal correlation between input queries and final results. Second, in the search phase, the parameters of the best-first search have not been fully examined.
To address these limitations, we introduce the \texttt{PANNS} system. \texttt{PANNS} constructs a proximity graph that incorporates temporal information, revealing correlations between queries and results. It also features a fully parameterized search algorithm, allowing extensive performance tuning. \texttt{PANNS} can achieve up to a $3.7\times$ speedup in query throughput compared to the state-of-the-art graph-based ANNS solutions while maintaining the same recall levels. Moreover, the size of the constructed graph can be reduced by up to 30% without compromising query quality.
Tue 4 MarDisplayed time zone: Pacific Time (US & Canada) change
15:40 - 17:00 | Session 9: Concurrent Data Structures and Synchronization II (Session Chair: Pengfei Su)Main Conference at Acacia D | ||
15:40 20mTalk | PANNS: Enhancing Graph-based Approximate Nearest Neighbor Search through Recency-aware Construction and Parameterized Search Main Conference Xizhe Yin University of California, Riverside, Chao Gao University of California Riverside, Zhijia Zhao University of California at Riverside, Rajiv Gupta University of California at Riverside (UCR) | ||
16:00 20mTalk | Balanced Allocations over Efficient Queues: A Fast Relaxed FIFO Queue Main Conference Kåre von Geijer Chalmers University of Technology, Philippas Tsigas Chalmers University of Technology, Elias Johansson Chalmers University of Technology, Sebastian Hermansson Chalmers University of Technology | ||
16:20 20mTalk | LibRTS: A Spatial Indexing Library by Ray Tracing Main Conference | ||
16:40 20mTalk | Crystality: A Programming Model for Smart Contracts on Parallel EVMs Main Conference Hao Wang International Digital Economy Academy (IDEA), Shenzhen, China; and Fullnodes Labs, Minghao Pan International Digital Economy Academy (IDEA), Shenzhen, China; and Fullnodes Labs, Jiaping Wang International Digital Economy Academy (IDEA), Shenzhen, China; and Fullnodes Labs |