Function Repository Resource:

LogSumExp

Source Notebook

Numerically stable implementation of addition on log-scaled numbers

Contributed by: Sjoerd Smit

ResourceFunction["LogSumExp"][list]

computes Log@Total[Exp[list]].

ResourceFunction["LogSumExp"][array, spec]

reduces array by aggregating the specified dimensions with the LogSumExp operation.

Details

The argument spec should be specified as described in the documentation for ArrayReduce.
ResourceFunction["LogSumExp"][array] is equivalent to ResourceFunction["LogSumExp"][array,1].
ResourceFunction["LogSumExp"][array,spec] is equivalent to ArrayReduce[ResourceFunction["LogSumExp"],array,spec].

Examples

Basic Examples (5) 

Compute the LogSumExp of a list of numbers:

In[1]:=
ResourceFunction["LogSumExp"][{1., 2., 3., 4.}]
Out[1]=

This is the same as:

In[2]:=
Log @ Total[Exp[{1., 2., 3., 4.}]]
Out[2]=

However, LogSumExp can be computed in a numerically stable way using MachinePrecision even when larger numbers are involved:

In[3]:=
ResourceFunction["LogSumExp"][{1., 2., 3995., 4000.}] // FullForm
Out[3]=

The naive implementation loses precision and switches to arbitrary precision to compensate:

In[4]:=
Log @ Total@Exp[{1., 2., 3995., 4000.}] // FullForm
Out[4]=

Compare with the exact result:

In[5]:=
N[Log @ Total@Exp[{1, 2, 3995, 4000}], 50]
Out[5]=

Scope (5) 

Compute the LogSumExp on symbolic data:

In[6]:=
ResourceFunction["LogSumExp"][{a, b, c}]
Out[6]=

Reduce a matrix by applying LogSumExp to its rows:

In[7]:=
matrix = N@Array[Plus, {3, 5}]
Out[7]=
In[8]:=
ResourceFunction["LogSumExp"][matrix]
Out[8]=

Apply to columns:

In[9]:=
ResourceFunction["LogSumExp"][matrix, 2]
Out[9]=

Aggregate both rows and columns:

In[10]:=
ResourceFunction["LogSumExp"][matrix, All]
Out[10]=

This is equivalent to:

In[11]:=
ResourceFunction["LogSumExp"][matrix, {1, 2}]
Out[11]=

Properties and Relations (3) 

The LogSumExp operation allows you to do addition outside of a Log in the same way that addition allows you to multiply outside of a Log:

In[12]:=
SeedRandom[1];
rand = RandomReal[1, 10]
Out[13]=

A Log transform turns multiplication into addition:

In[14]:=
Log[Times @@ rand] == Total[Log /@ rand]
Out[14]=

And it turns addition into LogSumExp:

In[15]:=
Log[Total@rand] == ResourceFunction["LogSumExp"][Log /@ rand]
Out[15]=

Publisher

Sjoerd Smit

Requirements

Wolfram Language 13.0 (December 2021) or above

Version History

  • 1.0.0 – 30 July 2024

Related Resources

License Information