Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 LAIONSearch

Source Notebook

Search the LAION database of images with captions

Contributed by: Nikolay Murzin

ResourceFunction["LAIONSearch"]["query"]

searches the LAION database with the specified query.

ResourceFunction["LAIONSearch"][img]

searches based on an image.

ResourceFunction["LAIONSearch"][…,n]

gives up to n results.

Details and Options

LAION (Large-scale Artificial Intelligence Open Network) is a German non-profit which makes open-sourced artificial intelligence models and datasets. It is best known for releasing a number of large datasets of images and captions scraped from the web which have been used to train a number of high-profile text-to-image models, including Stable Diffusion and Imagen.
LAIONSearch uses an API to KNN search database, which is also can be used here: https://knn.laion.ai/
LAIONSearch takes the following options (also documented here: https://github.com/rom1504/clip-retrieval):
"Index""laion5B-L-14"the name of the index, either "laion5B-L-14", "laion5B-H-14" or "laion_400m"
"Deduplicate"Truewhether to deduplicate the result by image embedding
"AestheticScore"9the aesthetic score as rated by aesthetic detector
"AestheticWeight"0.5the weight of the aesthetic score
"UseViolenceDetector"Truewhether to remove images with violence
"UseSafetyModel"Truewhether to remove unsafe images
"UseMultilingualClip"Falsewhether to use a multi-lingual version of CLIP
"Modality""image"search over image or text in the index
LAIONSearch returns a Dataset of search results.

Examples

Basic Examples (2) 

Search with text:

In[1]:=
ResourceFunction[ "LAIONSearch", ResourceSystemBase -> "https://www.wolframcloud.com/obj/resourcesystem/api/1.0"]["cat chasing dog", 10]
Out[1]=

Search with an image:

In[2]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/8d1a7a3e-1dd1-4b3e-aedf-9989dccc4655"]
Out[2]=

Scope (2) 

Use an embedding by a CLIP Multi - domain Feature Extractor directly:

In[3]:=
image = First@WebImageSearch["party hat", 1]
Out[3]=
In[4]:=
imageEmbedding = NetModel[{"CLIP Multi-domain Feature Extractor", "InputDomain" -> "Image", "Architecture" -> "ViT-L/14"}][image];
In[5]:=
textEmbedding = NetModel[{"CLIP Multi-domain Feature Extractor", "InputDomain" -> "Text", "Architecture" -> "ViT-L/14"}]["cat"];

This textEmbedding is a vector of 768 real numbers:

In[6]:=
Dimensions[textEmbedding]
Out[6]=
In[7]:=
embedding = Mean@WeightedData[ Normalize /@ {textEmbedding, imageEmbedding}, {2, 1}];
In[8]:=
ResourceFunction[ "LAIONSearch", ResourceSystemBase -> "https://www.wolframcloud.com/obj/resourcesystem/api/1.0"][embedding, 5]
Out[8]=

Options (2) 

Index (1) 

Search in a different database index:

In[9]:=
ResourceFunction[ "LAIONSearch", ResourceSystemBase -> "https://www.wolframcloud.com/obj/resourcesystem/api/1.0"]["cat", 5, "Index" -> "laion_400m"]
Out[9]=

AestheticScore (1) 

Decrease AestheticScore to find more images:

In[10]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/27957058-6f28-4d04-a947-a9cf6d2f73ae"]
Out[10]=

Requirements

Wolfram Language 13.0 (December 2021) or above

Version History

  • 1.0.0 – 03 July 2023

Related Resources

License Information