Function Repository Resource:

LocalEvaluate

Source Notebook

Synchronously evaluate an expression in a separate kernel

Contributed by: Richard Hennigan (Wolfram Research)

ResourceFunction["LocalEvaluate"][expr]

evaluates expr in a separate kernel and returns the result.

ResourceFunction["LocalEvaluate"][expr,wrapper]

wraps the head wrapper around the result before returning it.

Details and Options

ResourceFunction["LocalEvaluate"] has attribute HoldFirst.
ResourceFunction["LocalEvaluate"][expr] automatically includes all definitions needed to evaluate expr, much like CloudEvaluate.
During the evaluation, the following events can be generated:
"TaskStarted"task is started
"FailureOccurred"task could not be started
"ScheduleTriggered"scheduled task evaluation is beginning
"MessageGenerated"evaluation generated a message
"PrintOutputGenerated"evaluation generated print output
"ResultReceived"evaluation generated a result
"ScheduleExecuted"scheduled task evaluation is executed
"TaskStatusChanged"task status changed
"TaskRemoved"task is being removed
With the specification HandlerFunctions<|,"eventi"fi,|>, fi[assoc] is evaluated whenever eventi is generated. The elements of assoc have keys specified by the setting for HandlerFunctionsKeys.
ResourceFunction["LocalEvaluate"] initiates a new subkernel whenever it is called.
The following options can be given:
HandlerFunctions<||>how to handle generated events
HandlerFunctionsKeysAutomaticparameters to supply to handler functions
IncludeDefinitionsTruewhether to include any dependent definitions
"ReturnDefinitions"Falsewhether to send dependent definitions from the subkernel back to the parent kernel
InitializationNonean expression to be evaluated as part of the kernel initialization
AsynchronousFalsewhether to return a TaskObject instead of waiting for results
ResourceFunction["LocalEvaluate"] uses WXF to transfer data between kernels, so the following BinarySerialize options can additionally be used to control serialization:
MethodAutomaticdetails of serialization methods to use
PerformanceGoalAutomaticaspects of performance to try to optimize
Possible settings for PerformanceGoal include:
"Speed"optimize for serialization and deserialization speed
"Size"optimize for size of data to be transferred
Automaticautomatically pick serialization strategy
Possible settings for Method include:
{typespec1enc1,}specify encodings for particular types
Automaticpick encodings automatically based on data
Possible forms for the typespeci include:
"PackedArrayIntegerType"integer packed arrays
"PackedArrayRealType"real-valued packed arrays
"PackedArrayComplexType"complex-valued packed arrays
Possible settings for "PackedArrayIntegerType" include "Integer8", "Integer16", "Integer32" and "Integer64" (64-bit systems only).
Possible settings for "PackedArrayRealType" include "Real32" and "Real64".
Possible settings for "PackedArrayComplexType" include "Complex64" and "Complex128".

Examples

Basic Examples (2) 

Run an evaluation in a separate kernel:

In[1]:=
ResourceFunction["LocalEvaluate"][$ProcessID]
Out[1]=

Compare to the current kernel:

In[2]:=
$ProcessID
Out[2]=

Supply a head to be applied to the results of a local evaluation:

In[3]:=
ResourceFunction["LocalEvaluate"][25!, wrapper]
Out[3]=

Options (14) 

IncludeDefinitions (2) 

By default, definitions are included:

In[4]:=
f[x_] := x + 1;
ResourceFunction["LocalEvaluate"][f[5], Hold]
Out[2]=

Do not include any definitions in the evaluation:

In[5]:=
ResourceFunction["LocalEvaluate"][f[5], Hold, IncludeDefinitions -> False]
Out[5]=

ReturnDefinitions (3) 

By default, definitions created on the subkernel are not returned to the current kernel session:

In[6]:=
ClearAll[g];
ResourceFunction["LocalEvaluate"][g[x_] := x + 1; Inactive[g][5]]
Out[2]=
In[7]:=
Activate[%]
Out[7]=

Return dependent definitions to the parent kernel for the output expression:

In[8]:=
ClearAll[g];
ResourceFunction["LocalEvaluate"][g[x_] := x + 1; Inactive[g][5], "ReturnDefinitions" -> True]
Out[2]=

Now the definition of g is returned from the subkernel:

In[9]:=
Activate[%]
Out[9]=

Only the definitions that the output depends on will be returned:

In[10]:=
ClearAll[f, g];
ResourceFunction["LocalEvaluate"][f[x_] := 2 x; g[x_] := x + 1; Inactive[g][5], "ReturnDefinitions" -> True]
Out[2]=

Since the output does not depend on f, its definition is not returned to the current session:

In[11]:=
f[5]
Out[11]=

Compare to g, which appears in the output literally:

In[12]:=
g[5]
Out[12]=

Initialization (2) 

Specify an initialization that should evaluate first:

In[13]:=
ResourceFunction["LocalEvaluate"][f[5], Initialization :> (f[x_] := 2 x)]
Out[13]=

The initialization only runs on the other kernel; the current kernel is not affected:

In[14]:=
f[5]
Out[14]=

Create a temporary init.m file and back up the original:

In[15]:=
init = FindFile["init.m"];
RenameFile[init, init <> ".backup"]
Out[16]=
In[17]:=
Put[Unevaluated[h[x_] := x!], init];
FilePrint[init]

Using Automatic for Initialization is equivalent to using Get[FindFile["init.m"]]:

In[18]:=
ResourceFunction["LocalEvaluate"][h[25], Initialization -> Automatic]
Out[18]=

The symbol h is undefined otherwise:

In[19]:=
ResourceFunction["LocalEvaluate"][h[25]]
Out[19]=

Restore the original init.m file:

In[20]:=
CopyFile[init <> ".backup", init, OverwriteTarget -> True]
Out[20]=

Method (5) 

Create a large packed array of integers:

In[21]:=
ints = RandomInteger[{-128, 127}, 100000000];

Pass a Method specification to the WXF serialization when sending expressions between kernels:

In[22]:=
AbsoluteTiming[
 ResourceFunction["LocalEvaluate"][Length[ints], Method -> "PackedArrayIntegerType" -> "Integer64"]]
Out[22]=

Use a smaller type to reduce transfer time:

In[23]:=
AbsoluteTiming[
 ResourceFunction["LocalEvaluate"][Length[ints], Method -> "PackedArrayIntegerType" -> "Integer16"]]
Out[23]=

By default, the smallest type that fits the data will be used:

In[24]:=
AbsoluteTiming[
 ResourceFunction["LocalEvaluate"][Length[ints], Method -> Automatic]]
Out[24]=

Compare the size difference by using BinarySerialize:

In[25]:=
BinarySerialize[ints, Method -> "PackedArrayIntegerType" -> "Integer64"]
Out[25]=
In[26]:=
BinarySerialize[ints, Method -> "PackedArrayIntegerType" -> "Integer8"]
Out[26]=

Asynchronous (2) 

By default, LocalEvaluate waits for results to be returned:

In[27]:=
AbsoluteTiming[ResourceFunction["LocalEvaluate"][Pause[5]; 1 + 1]]
Out[27]=

Set Asynchronous to True if you don't need to wait for results:

In[28]:=
AbsoluteTiming[
 ResourceFunction["LocalEvaluate"][Pause[5]; 1 + 1, Asynchronous -> True]]
Out[28]=

This is effectively equivalent to LocalSubmit when no definitions or initialization specifications are involved:

In[29]:=
AbsoluteTiming[LocalSubmit[Pause[5]]]
Out[29]=
In[30]:=
AbsoluteTiming[
 ResourceFunction["LocalEvaluate"][Pause[5], Asynchronous -> True, IncludeDefinitions -> False, Initialization -> None]]
Out[30]=

Applications (3) 

Discover symbols in a package without loading it in the current kernel:

In[31]:=
ResourceFunction["LocalEvaluate"][Needs["ComputerArithmetic`"]; Names["ComputerArithmetic`*"]]
Out[31]=

The current kernel is unaffected:

In[32]:=
Names["ComputerArithmetic`*"]
Out[32]=

Check how many symbols have been created in your current session that are not created by default:

In[33]:=
Length@Complement[Names[], ResourceFunction["LocalEvaluate"][Names[], Initialization -> Automatic]]
Out[33]=

Manipulate notebooks with a hidden front end:

In[34]:=
ResourceFunction["LocalEvaluate"][
 UsingFrontEnd[
  NotebookGet[
   NotebookPut[
    Notebook[{Cell["Testing", "Section"], Cell["Hello world", "Text"]}]]]]]
Out[34]=

Properties and Relations (3) 

Each evaluation of LocalEvaluate runs in a new kernel:

In[35]:=
ResourceFunction["LocalEvaluate"][{$ProcessID, SessionTime[]}]
Out[35]=
In[36]:=
ResourceFunction["LocalEvaluate"][{$ProcessID, SessionTime[]}]
Out[36]=

Compare to the current kernel:

In[37]:=
{$ProcessID, SessionTime[]}
Out[37]=

Similar behavior to LocalEvaluate can be obtained by using TaskWait with LocalSubmit:

In[38]:=
TaskWait@LocalSubmit[SessionTime[], HandlerFunctions -> <|
     "TaskFinished" -> (($result = #EvaluationResult) &)|>];
$result
Out[38]=

LocalEvaluate can often have better performance than LocalSubmit when sending/receiving large amounts of data:

In[39]:=
ints = RandomInteger[1000, 100000000];
ByteCount[ints]
Out[32]=
In[40]:=
{t1, r1} = With[{ints = ints}, AbsoluteTiming[
    TaskWait[
     LocalSubmit[Reverse[ints], HandlerFunctions -> <|
        "TaskFinished" -> (($result = #EvaluationResult) &)|>]]; $result]];
t1
Out[24]=
In[41]:=
{t2, r2} = With[{ints = ints}, AbsoluteTiming[ResourceFunction["LocalEvaluate"][Reverse[ints]]]];
t2
Out[26]=
In[42]:=
r1 === r2
Out[42]=

Possible Issues (2) 

Messages that are issued in the separate kernel are not printed in the current session:

In[43]:=
ResourceFunction["LocalEvaluate"][First[1], Hold]
Out[43]=
In[44]:=
First[1]
Out[44]=

Use HandlerFunctions to handle messages:

In[45]:=
ResourceFunction["LocalEvaluate"][First[1], Hold, HandlerFunctions -> <|
   "MessageGenerated" -> (Message[#MessageOutput] &)|>]
Out[45]=

Requirements

Wolfram Language 11.3 (March 2018) or above

Version History

  • 2.0.0 – 27 January 2020
  • 1.0.0 – 15 August 2018

Related Resources

License Information