Function Repository Resource:

MapBatched

Source Notebook

Divide a Map operation into batches, performing an evaluation between batches

Contributed by: Joshua Schrier

ResourceFunction["MapBatched"][f,expr]

applies f to each element on the first level of expr,pausing for one second every five elements.

ResourceFunction["MapBatched"][f,expr,action]

applies f to each element on the first level of expr,performing action every five elements.

ResourceFunction["MapBatched"][f,expr,action,n]

uses batches of length n.

Details and Options

The default action is a one-second Pause and the default size for n is 5. This facilitates interacting with external web APIs that request limits on query frequency.

The Map operation occurs at level {1} only.

Examples

Basic Examples (2)

MapBatched applies a function to a list or association. By default, batches consist of 5 entries, and the action taken is to Pause for one second per batch:

In[1]:=

Out[1]=

The result using MapBatched identical to an ordinary Map except for inserting this Pause:

In[2]:=

Out[2]=

When the number of elements is less than the batch size, no Pause is inserted:

In[3]:=

Out[3]=

Scope (2)

The expr can be any expression over which the function f can be mapped (e.g., lists and associations). The action can be any expression:

In[4]:=

Out[4]=

The size n can be any integer:

In[5]:=

Out[5]=

Applications (3)

One intended usage is to access web APIs that restrict request rates. For example, manual access to PubChem via the PUG API:

In[6]:=

$retrieveName[myName_String] := With[ {pugrest = "https://pubchem.ncbi.nlm.nih.gov/rest/pug", pugoper = "property/CanonicalSMILES", pugout = "txt", pugin = "compound/name/" <> myName}, URLExecute@URLBuild[{pugrest, pugin, pugoper, pugout}]];$