Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 SparseAssociation

Source Notebook

Create a rectangular data structure that behaves like a SparseArray indexed by string keys

Contributed by: Sjoerd Smit

ResourceFunction["SparseAssociation"][array,{key1,key2,…}]

creates a data structure in which an element at position {i,j,…} can be accessed with the string key sequence {keyi, keyj, …}.

ResourceFunction["SparseAssociation"][array,{{key1,1,key1,2,…},{key2,1,…},…}]

creates a data stucture that uses the keys keyi,j at level j of the array.

ResourceFunction["SparseAssociation"][array,keys,default]

uses default as the default value for the underlying SparseArray.

ResourceFunction["SparseAssociation"][{keys1→val1,keys2→val2,…},Automatic,…]

specifies the data from array rules.

ResourceFunction["SparseAssociation"][{keys1,keys2,…}→ {val1,val2,…},Automatic,…]

can also be used.

ResourceFunction["SparseAssociation"][assoc,Automatic,…]

specifies the data from a nested Association.

Details and Options

ResourceFunction["SparseAssociation"] acts like a rectangular data structure of nested associations but is more memory efficient because the keys are not duplicated at each level. In addition, ResourceFunction["SparseAssociation"] offers the possibility to specify beforehand what default value should be returned when trying to access empty parts of the array, whereas Association returns Missing[…] in that situation.
ResourceFunction["SparseAssociation"] is a constructor-container function like SparseArray and Association. A ResourceFunction["SparseAssociation"] can be constructed from a regular array that satisfies ArrayQ, a nested Association or a list of rules of the form {{keyi,1,keyi,2,…} → valuei,…} in the same way as a SparseArray.
Only string keys are supported currently.
The underlying SparseArray can be extracted with Values. The keys used at different levels can be extracted with Keys. The function Normal converts the data to a nested Association.
A correctly constructed ResourceFunction["SparseAssociation"] matches ResourceFunction["SparseAssociation"][_Association?(#["ValidatedQ"]&)].
If a ResourceFunction["SparseAssociation"] is created from a nested Association, all Missing[…] values at any level will be dropped and replaced with the default value. The corresponding keys will still be included.
Like SparseArray, default is 0 if not specified.
A ResourceFunction["SparseAssociation"] can be converted to SparseArray with Values, to a list of rules with ArrayRules and to a Dataset of nested associations with Dataset.
Dataset[sparseAssoc, "IncludeDefaultValues"→False] can be used to create a ragged Association with all of the default values dropped. This also works with Normal.
A ResourceFunction["SparseAssociation"] can be queried like an Association using sparseAssoc[key1,key2,…] or using Part.
Accessing elements with Part can be done using string keys or integer indices.
Map is supported but only for operations that leave the dimensions of the array unchanged. For example, mapping Total over a matrix is currently not supported.

Examples

Basic Examples (14) 

Convert a regular array into a SparseAssociation:

In[1]:=
array = BlockRandom[ SeedRandom[1]; RandomInteger[2, {4, 3, 2}] ]
Out[1]=
In[2]:=
sparseAssoc = ResourceFunction["SparseAssociation"][array, {"a", "b", "c", "d"}]
Out[2]=

Extract elements from it:

In[3]:=
sparseAssoc["d", "b", "a"]
Out[3]=

This is equivalent to:

In[4]:=
array[[4, 2, 1]]
Out[4]=

Unlike Association, you cannot extract keys that haven’t been specified beforehand:

In[5]:=
sparseAssoc["e", "f", "g"]
Out[5]=

SparseAssociation works with Part:

In[6]:=
sparseAssoc[["d", "b", "a"]]
Out[6]=

Extracting multiple elements with Part yields another SparseAssociation of reduced size:

In[7]:=
sparseAssoc[[{"a", "c"}, All, "a"]]
Out[7]=

Indexing by position is also possible:

In[8]:=
sparseAssoc[[{1, 3}, All, 1]]
Out[8]=

Convert the SparseAssociation to a SparseArray:

In[9]:=
Values[sparseAssoc]
Out[9]=

Further convert the SparseArray to a nested list using Normal:

In[10]:=
Normal[%]
Out[10]=

Convert the SparseAssociation to array rules:

In[11]:=
ArrayRules[sparseAssoc]
Out[11]=

Convert the sparse association directly to a nested Association:

In[12]:=
Normal[sparseAssoc]
Out[12]=

Convert it to a ragged dataset with the default elements omitted:

In[13]:=
Dataset[sparseAssoc, "IncludeDefaultValues" -> False]
Out[13]=

Convert the sparse association to a nested Association with the default elements omitted:

In[14]:=
Normal[sparseAssoc, "IncludeDefaultValues" -> False]
Out[14]=

Reassemble the SparseAssociation from the nested Association (up to permutation of the keys):

In[15]:=
ResourceFunction["SparseAssociation"][%, Automatic]
Out[15]=

Scope (6) 

Construct a SparseAssociation from a SparseArray:

In[16]:=
ResourceFunction["SparseAssociation"][ SparseArray[{{1, 1} -> 1, {2, 2} -> 2, {3, 3} -> 3, {1, 3} -> 4}], {"a", "b", "c"} ]
Out[16]=

Define a SparseAssociation using an array of rules:

In[17]:=
rules = {{"a", "a"} -> 1, {"a", "c"} -> 4, {"b", "b"} -> 2, {"c", "c"} -> 3}; ResourceFunction["SparseAssociation"][rules, Automatic]
Out[18]=

Use Keys to extract the keys used at the different levels. The ordering of the keys depends on their appearance in the rules:

In[19]:=
Keys[%]
Out[19]=

Specify the ordering of the keys in advance:

In[20]:=
Keys@ResourceFunction["SparseAssociation"][rules, {"a", "b", "c"}]
Out[20]=

Specify different orderings at different levels:

In[21]:=
Keys@ResourceFunction["SparseAssociation"][ rules, {{"a", "b", "c"}, {"c", "b", "a"}}]
Out[21]=

Use a default element other than 0 and specify keys that do not appear in the list of rules:

In[22]:=
ResourceFunction[ "SparseAssociation"][{"a" -> 2}, {"a", "b", "c", "d"}, 1]
Out[22]=

Keys without a specified value will yield the default value:

In[23]:=
%["c"]
Out[23]=

Specify different keys at different levels:

In[24]:=
sparseAssoc = ResourceFunction[ "SparseAssociation"][{{"a", "f"} -> 2}, {{"a", "b", "c", "d"}, {"e", "f", "g"}}]
Out[24]=

The functions Length, Dimensions, ArrayDepth, MatrixQ, VectorQ and ArrayQ are supported:

In[25]:=
sparseAssoc = ResourceFunction[ "SparseAssociation"][{{"a", "d"} -> 2}, {"a", "b", "c", "d"}]
Out[25]=
In[26]:=
Through[{Length, Dimensions, ArrayDepth, MatrixQ, VectorQ, ArrayQ}@ sparseAssoc ]
Out[26]=

Map is supported for operations that preserve the dimensions of the array:

In[27]:=
Map[f, sparseAssoc, {2}]
Out[27]=
In[28]:=
Normal@Values[%]
Out[28]=

Construct a SparseAssociation from nested associations:

In[29]:=
ResourceFunction[ "SparseAssociation"][<|"a" -> <|"f" -> 1|>, "b" -> <|"f" -> 2, "g" -> 3|>|>, Automatic]
Out[29]=

Missing values at any level will be dropped, but the keys will be retained:

In[30]:=
ResourceFunction[ "SparseAssociation"][<|"a" -> <|"f" -> 1, "g" -> Missing[]|>, "b" -> <|"f" -> 2, "h" -> 3|>, "c" -> Missing[]|>, Automatic]
Out[30]=
In[31]:=
Keys[%]
Out[31]=

Applications (1) 

A SparseAssociation is more memory efficient than a nested Association because the keys do not have to be stored multiple times on each level:

In[32]:=
sparseAssoc = BlockRandom[SeedRandom[1]; ResourceFunction["SparseAssociation"][ RandomVariate[BernoulliDistribution[1/10], {26, 26, 26}], Alphabet[]] ]
Out[32]=
In[33]:=
{ ByteCount[sparseAssoc], ByteCount[ Normal@sparseAssoc], (* Full nested associations that store the default values as well *) ByteCount[ Normal[sparseAssoc, "IncludeDefaultValues" -> False]] (* Nested associations that only store the 1's *) }
Out[33]=

Properties and Relations (4) 

A SparseAssociation can be reconstructed from its array rules and its keys:

In[34]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ RandomInteger[5, {4, 2}], {{"a", "b", "c", "d"}, {"e", "f", "g"}}, 1]
Out[34]=
In[35]:=
ResourceFunction["SparseAssociation"][ArrayRules[sparseAssoc], Keys[sparseAssoc]] === sparseAssoc
Out[35]=

It can also be reconstructed from its underlying SparseArray and its keys:

In[36]:=
ResourceFunction["SparseAssociation"][Values[sparseAssoc], Keys[sparseAssoc]] === sparseAssoc
Out[36]=

It can be reconstructed from the corresponding nested Association if the correct default value is provided:

In[37]:=
ResourceFunction["SparseAssociation"][Normal[sparseAssoc], Keys[sparseAssoc], 1] === sparseAssoc
Out[37]=

Test if a SparseAssociation has been constructed correctly:

In[38]:=
MatchQ[sparseAssoc, ResourceFunction[ "SparseAssociation"][_Association?(#["ValidatedQ"] &)] ]
Out[38]=

Possible Issues (4) 

It is not possible to mix keys and position specifications at one level:

In[39]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ RandomInteger[5, {4, 3}], {"a", "b", "c", "d"}]; sparseAssoc[[{"a", 2}]]
Out[40]=

This is analogous to the same limitation of Association:

In[41]:=
<|"a" -> 1, "b" -> 2|>[[{"a", 2}]]
Out[41]=

Use only strings or only integers instead:

In[42]:=
sparseAssoc[[{"a", "b"}]]
Out[42]=
In[43]:=
sparseAssoc[[{1, 2}]]
Out[43]=

At different levels, different specifications can be used:

In[44]:=
sparseAssoc[[{1, 2}, {"a", "b"}]]
Out[44]=

Neat Examples (3) 

Excel files often have row and column headers. SparseAssociation provides an easy mechanism to create a lookup table from such data:

In[45]:=
sheet = Import["ExampleData/elements.xls", {"Data", 1}][[ All, {2, 1, 3, 4}]]; sheet // TableForm
Out[46]=
In[47]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ sheet[[2 ;;, 2 ;;]], { sheet[[2 ;;, 1]], (* row headers *) sheet[[1, 2 ;;]] (* column headers *) } ]
Out[47]=

The data can now be queried like an Association:

In[48]:=
sparseAssoc["He", "AtomicWeight"]
Out[48]=

Convert to a Dataset:

In[49]:=
Dataset@sparseAssoc[[{"H", "N"}, {"Name", "AtomicNumber"}]]
Out[49]=

Publisher

Sjoerd Smit

Version History

  • 3.0.0 – 01 October 2020
  • 2.0.0 – 24 August 2020
  • 1.0.0 – 26 March 2020

Related Resources

License Information