Re-ranking Text Matches in Semantic Search Queries

Rank candidate texts by how well they match a natural-language query and observe how re-ranking changes the order of search results

Semantic search can return multiple plausible matches for a query, but the most relevant result is not always the first one. The task is to take a query and a collection of candidate results, compute a relevance-based ordering, and inspect both the top match and the full ranked list.

Define a short list of candidate statements to be ranked against a query:

In[1]:=

info={"To perform symbolic integration, use the Integrate function in the Wolfram Language.","You can visualize data using the Plot function for 2D graphs and Plot3D for 3D graphs.","To solve differential equations, use DSolve for symbolic solutions and NDSolve for numerical solutions.","Access a wide range of curated data sets through the Wolfram Knowledgebase for your analysis needs.","To deploy a cloud-based application, utilize the CloudDeploy function in your Wolfram Language code.","For working with structured data, use the Wolfram Language's Tabular functions to manipulate and analyze tables."};

Return the single best-matching statement for the query:

In[2]:=

First@SemanticRanking[info,"API calls"]

Out[2]=

To deploy a cloud-based application, utilize the CloudDeploy function in your Wolfram Language code.

Compute and display the full ranking table for all candidates and sort it by rank:

In[3]:=

SemanticRanking[info,"explore large dataset",All]//SortBy[#Rank&]

Out[3]=

Tabular

Row count: 6

Column count: 3



The "Rank" column gives the ordering (1 is best match), and "Relevance" is the score used to produce that ordering. Next we will compare search results with and without reranking.

Build a small semantic index from a list of sentences:

In[4]:=

semse=CreateSemanticSearchIndex[{"Starfish regenerate lost body parts.","Cuttlefish change color for camouflage.","Silverfish infest homes, eating paper.","Salmon migrate upstream to spawn."}];

Get baseline search results without reranking:

In[5]:=

SemanticSearch[semse,"Fish",RerankingMethodNone]

Out[5]=

{Silverfish infest homes, eating paper.,Cuttlefish change color for camouflage.,Salmon migrate upstream to spawn.,Starfish regenerate lost body parts.}

Get re-ranked search results for the same query and compare the ordering:

In[6]:=

SemanticSearch[semse,"Fish",RerankingMethodAutomatic]

Out[6]=

{Salmon migrate upstream to spawn.,Cuttlefish change color for camouflage.,Starfish regenerate lost body parts.,Silverfish infest homes, eating paper.}

Reranking refines the initial retrieval order by rescoring the top candidates at the expense of possibly more computation time. Next we will scale this workflow to a large dictionary-derived index and compares different reranking settings.

Build a large semantic index that maps dictionary definitions to their corresponding words:

In[7]:=

index=CreateSemanticSearchIndex[Map[StringRiffle[WordDefinition[#]]#&,WordList[]]]

Out[7]=

SemanticSearchIndex

ID: SemSe-74a13892-19fe-4d47-ab8d-897d7f7e2302

Elements: 39184



Run and time a semantic search using a lightweight reranking model:

In[8]:=

SemanticSearch[index,"a very large sea mammal",RerankingMethod{"MiniLM",Tiny}]//AbsoluteTiming

Out[8]=

{201.096,{cetacean,blower,walrus,whale,fisher,seahorse,plumping,manatee,pachyderm,crocodile}}

Repeat the search with a larger reranking model to compare runtime and result ordering:

In[9]:=

SemanticSearch[index,"a very large sea mammal",RerankingMethod{"MiniLM",Large}]//AbsoluteTiming

Out[9]=

{526.415,{whale,walrus,cetacean,blower,plumping,seahorse,crocodile,dolphin,grampus,fisher}}

Without reranking, the top result reflects the approximate ordering from the initial retrieval. With reranking enabled, the strongest candidates are rescored and reordered, which can change which item appears first when several entries match the query meaning.

Publisher Information

Contributed by: Wolfram Staff

Wolfram Language Example Repository

Re-ranking Text Matches in Semantic Search Queries

See Also

Publisher Information

Re-ranking Text Matches in Semantic Search Queries

See Also

Related Symbols

Publisher Information