Method (4) 

Suboptions "Folds" and "Runs" are support for the "KFold" method. Note how the total number of model fits is equal to "Folds" × "Runs":

In[33]:=

Out[34]=

Here is an example input using the "LeaveOneOut" method:

In[35]:=

Out[36]=

Use the "RandomSubSampling" method. The "SamplingFunction" is set in such a way that it is equivalent to the default option "SamplingFunction" → Automatic:

In[37]:=

Out[37]=

In this example, we use bootstrap sampling to estimate the error bars on the value of λ:

In[38]:=

Out[39]=

ValidationFunction (6) 

Specify a function that calculates the CDF fitted distribution at the points given by the validation data. We know that if the fitted distribution is the true distribution, then the obtained points should be distributed as UniformDistribution[0, 1], as can be illustrated with TransformedDistribution:

In[40]:=

Out[40]=

By calculating the CDF of the validation data, we can combine the results and use a QuantilePlot to judge if the validation data follows the same distribution as the fit. The quantile plot suggests that the ExponentialDistribution gives the best fit (as expected):

In[41]:=

Out[42]=

For fitting functions that return a FittedModel like NonlinearModelFit, you can specify the validation function as "ValidationFunction" → {Automatic,Function[…]}. In that case, the function will be applied to the fitted values and true values in the validation set:

In[43]:=

Out[44]=

For Predict and Classify, the form "ValidationFunction" → {Automatic,prop} can be used to extract specific properties from PredictorMeasurements or ClassifierMeasurements. As an example, we can generate multiple prediction comparison plots and show them together in a single graph:

In[45]:=

Out[46]=

For neural networks trained with NetTrain, the form "ValidationFunction" → {Automatic,args} can be used to supply arguments to NetMeasurements[trainedNet,validationData,args]:

In[47]:=

Out[48]=

With SpatialEstimate, the form "ValidationFunction" → {Automatic,fun} can be used to specify a function that computes the loss from the true values, the estimated values and the standard errors of the prediction:

In[49]:=

Out[52]=

With "ValidationFunction"→ None, no validation will be performed:

In[53]:=

Out[53]=

ParallelQ (1) 

Use parallelization to speed up the cross-validation when doing many validation runs. Sub-options for ParallelTable can specified as "ParallelQ" → {True,opts…}:

In[54]:=

Out[55]=