Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 CaptionedEvaluate

Source Notebook

This functionality requires LLM Access »

Generate an audio caption for the given input

Contributed by: Arnoud Buzing

ResourceFunction["CaptionedEvaluate"][input]

evaluates input and generates an audio caption for the output.

Details and Options

The output of ResourceFunction["CaptionedEvaluate"] is a Button that will Speak the audio caption. The button also has a Tooltip which shows the output at 3x magnification.
ResourceFunction["CaptionedEvaluate"] uses LLMSynthesize to generate the audio caption. It requires an LLMEvaluator that supports text and images, such as "GPT-4 Omni". You can use the LLMEvaluator option to specify an LLM configuration.
ResourceFunction["CaptionedEvaluate"] can be useful to people with a visual impairment.

Examples

Basic Examples (1) 

Generate an audio caption for a plot:

In[1]:=
ResourceFunction["CaptionedEvaluate"][Plot[Sin[x], {x, 0, 10}]]
Out[1]=
In[2]:=
ResourceFunction["CaptionedEvaluate"][Histogram[RandomReal[1, 100]]]
Out[2]=
In[3]:=
ResourceFunction["CaptionedEvaluate"][PieChart[RandomReal[1, 7]]]
Out[3]=

Options (3) 

Change the magnification level of the tooltip:

In[4]:=
ResourceFunction["CaptionedEvaluate"][Plot[Sin[x], {x, 0, 10}], Magnification -> 3]
Out[4]=

Increase the font size of the text in the tooltip:

In[5]:=
ResourceFunction["CaptionedEvaluate"][Plot[Sin[x], {x, 0, 10}], FontSize -> 24]
Out[5]=

Use a different LLMEvaluator:

In[6]:=
ResourceFunction["CaptionedEvaluate"][Plot[Sin[x], {x, 0, 10}], LLMEvaluator -> LLMConfiguration[ "Model" -> <|"Service" -> "GoogleGemini", "Name" -> "gemini-1.5-pro"|>]]
Out[6]=

Publisher

Arnoud Buzing

Requirements

Wolfram Language 14.0 (January 2024) or above

Version History

  • 1.0.0 – 30 August 2024

License Information