MikeYeh/ XGBPaclet

(1.0.0) current version: 1.8.0 »

This paclet utilizes XGBoost algorithm in Wolfram Language

Contributed by: Mike Yeh

In this paclet, we provide Wolfram Language functions for implementing XGBoost python functions, e.g., xgb.DMatrix(), xgb.train(), and predict(). So far we have XgbTrain[] to implement xgb.train() and XgbModelPredict[] to perform model prediction. More functions will be added later.

Installation Instructions

To install this paclet in your Wolfram Language environment, evaluate this code:
PacletInstall["MikeYeh/XGBPaclet"]

To load the code after installation, evaluate this code:
Needs["MikeYeh`XGBPaclet`"]

Details

The paclet now is in test version().

Paclet Guide

Examples

Basic Examples (4)

Create a XGBoost Python session

A XGBoost python session is required before using XgbTrain[]. The following code shows how to create a XGBoost python session:

In[1]:=

$ResourceFunction[ ResourceObject[<|"Name" -> "RegisterEnvironment", "UUID" -> "54433ce2-bf45-4364-959d-a88f5444a26c", "ResourceType" -> "Function", "Description" -> "Registers a new environment with the given Python package dependencies", "ResourceLocations" -> { CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/Resources/544/54433ce2-bf45-4364-959d-a88f5444a26c"]}, "Version" -> None, "Categories" -> {"External Interfaces & Connections", "Programming Utilities"}, "Compatibility" -> <|"WolframLanguageVersionRequired" -> "13.0+", "OperatingSystem" -> {"Windows", "MacOSX", "Unix"}, "Features" -> {}, "EvaluationEnvironment" -> {"Session", "WebEvaluation", "BatchJob", "Script", "WebAPI", "Subkernel", "Scheduled"}, "CloudSupport" -> True|>, "ContributorInformation" -> <|"ContributedBy" -> "Arnoud Buzing"|>, "DefinitionNotebook" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/RegisterEnvironment/RegisterEnvironment-DefinitionNotebook.nb"], "Documentation" -> <|"Usage" -> {<|"Usage" -> BoxData[ RowBox[{ StyleBox[ RowBox[{ StyleBox["ResourceFunction", "ResourceFunctionSymbol"], "[", StyleBox["\"RegisterEnvironment\"", "ResourceFunctionName"], "]"}], "ResourceFunctionHandle"], "[", RowBox[{ StyleBox["name", "TI"], "->", StyleBox["dependencies", "TI"]}], "]"}]], "Description" -> TextData[{"registers environment ", Append[ Cell[ BoxData[ StyleBox["name", "TI"]], "InlineFormula", FontFamily -> "Source Sans Pro"], FontFamily -> "Source Sans Pro"], " using the listed ", Append[ Cell[ BoxData[ StyleBox["dependencies", "TI"]], "InlineFormula", FontFamily -> "Source Sans Pro"], FontFamily -> "Source Sans Pro"], "."}]|>}|>, "DocumentationLink" -> URL[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/RegisterEnvironment"], "ExampleNotebook" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/RegisterEnvironment/RegisterEnvironment-ExampleNotebook.nb"], "ExampleNotebookData" -> Automatic, "FunctionLocation" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/Resources/544/54433ce2-bf45-4364-959d-a88f5444a26c/download/DefinitionData"], "PageHeaderClickToCopy" -> "ResourceObject[CloudObject[\"https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/RegisterEnvironment\"]]", "ShortName" -> "RegisterEnvironment", "SymbolName" -> "FunctionRepository`$54433ce2bf454364959da88f5444a26c`RegisterEnvironment"|>]][ "xgb" -> {"xgboost"}]$

Out[1]=

In[2]:=

$session = ResourceFunction[ ResourceObject[<|"Name" -> "StartEnvironment", "UUID" -> "eb4c6583-7ac8-404d-bd8d-e917e0ccef8b", "ResourceType" -> "Function", "Description" -> "Starts, or gives, the specified Python environment session", "ResourceLocations" -> { CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/Resources/eb4/eb4c6583-7ac8-404d-bd8d-e917e0ccef8b"]}, "Version" -> None, "Categories" -> {"External Interfaces & Connections", "Programming Utilities"}, "Compatibility" -> <|"WolframLanguageVersionRequired" -> "13.0+", "OperatingSystem" -> {"Windows", "MacOSX", "Unix"}, "Features" -> {}, "EvaluationEnvironment" -> {"Session", "WebEvaluation", "BatchJob", "Script", "WebAPI", "Subkernel", "Scheduled"}, "CloudSupport" -> True|>, "ContributorInformation" -> <|"ContributedBy" -> "Arnoud Buzing"|>, "DefinitionNotebook" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/StartEnvironment/StartEnvironment-DefinitionNotebook.nb"], "Documentation" -> <|"Usage" -> {<|"Usage" -> BoxData[ RowBox[{ StyleBox[ RowBox[{ StyleBox["ResourceFunction", "ResourceFunctionSymbol"], "[", StyleBox["\"StartEnvironment\"", "ResourceFunctionName"], "]"}], "ResourceFunctionHandle"], "[", "name", "]"}]], "Description" -> "starts or gives the specified Python environment session."|>}|>, "DocumentationLink" -> URL[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/StartEnvironment"], "ExampleNotebook" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/StartEnvironment/StartEnvironment-ExampleNotebook.nb"], "ExampleNotebookData" -> Automatic, "FunctionLocation" -> CloudObject[ "https://www.wolframcloud.com/obj/arnoudbuzing/Resources/eb4/eb4c6583-7ac8-404d-bd8d-e917e0ccef8b/download/DefinitionData"], "PageHeaderClickToCopy" -> "ResourceObject[CloudObject[\"https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/StartEnvironment\"]]", "ShortName" -> "StartEnvironment", "SymbolName" -> "FunctionRepository`$eb4c65837ac8404dbd8de917e0ccef8b`StartEnvironment"|>]]["xgb"]; ExternalEvaluate[session, "import xgboost as xgb"];$

The last line of code is to import XGBoost package in our python session. The imported name of xgboost must be xgb.

Prepare dataset for the following examples

Create training set and validation set for the following examples

The following code generates training set and validation set as pure numerical arrays or lists:

In[3]:=

(*Load "Titanic" data from ExampleData[]*)
traindataset = ExampleData[{"MachineLearning", "Titanic"}, "TrainingData"];
testdataset = ExampleData[{"MachineLearning", "Titanic"}, "TestData"];
traindataset = DeleteMissing[traindataset, 1, 2]; (*remove the Missing[]*)
testdataset = DeleteMissing[testdataset, 1, 2];(*remove the Missing[]*)

In[4]:=

$(*The maps coverting class labels to integers*) classMapping = <|"1st" -> 1, "2nd" -> 2, "3rd" -> 3, "crew" -> 4|>; genderMapping = <|"male" -> 0, "female" -> 1|>; statusMapping = <|"survived" -> 1, "died" -> 0|>; (* Function to convert data entry to integers using the mappings *) convertDataEntry[entry_] := {classMapping[entry[[1]]], entry[[2]], genderMapping[entry[[3]]]} (* Apply the conversion to data and label *) trainData = convertDataEntry /@ Keys@traindataset; trainLabel = statusMapping /@ Values@traindataset; testData = convertDataEntry /@ Keys@testdataset; testLabel = statusMapping /@ Values@testdataset;$

In[5]:=

How to use XgbTrain[] to train a model

Use XgbTrain[] to train a model with trainset and XGBoost python session, and store the trained model in the output session:

In[6]:=

$newsession = InterpretationBox[FrameBox[TagBox[TooltipBox[PaneBox[GridBox[List[List[GraphicsBox[List[Thickness[0.0025`], List[FaceForm[List[RGBColor[0.9607843137254902`, 0.5058823529411764`, 0.19607843137254902`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3]]], List[List[List[205.`, 22.863691329956055`], List[205.`, 212.31669425964355`], List[246.01799774169922`, 235.99870109558105`], List[369.0710144042969`, 307.0436840057373`], List[369.0710144042969`, 117.59068870544434`], List[205.`, 22.863691329956055`]], List[List[30.928985595703125`, 307.0436840057373`], List[153.98200225830078`, 235.99870109558105`], List[195.`, 212.31669425964355`], List[195.`, 22.863691329956055`], List[30.928985595703125`, 117.59068870544434`], List[30.928985595703125`, 307.0436840057373`]], List[List[200.`, 410.42970085144043`], List[364.0710144042969`, 315.7036876678467`], List[241.01799774169922`, 244.65868949890137`], List[200.`, 220.97669792175293`], List[158.98200225830078`, 244.65868949890137`], List[35.928985595703125`, 315.7036876678467`], List[200.`, 410.42970085144043`]], List[List[376.5710144042969`, 320.03370475769043`], List[202.5`, 420.53370475769043`], List[200.95300006866455`, 421.42667961120605`], List[199.04699993133545`, 421.42667961120605`], List[197.5`, 420.53370475769043`], List[23.428985595703125`, 320.03370475769043`], List[21.882003784179688`, 319.1406993865967`], List[20.928985595703125`, 317.4896984100342`], List[20.928985595703125`, 315.7036876678467`], List[20.928985595703125`, 114.70369529724121`], List[20.928985595703125`, 112.91769218444824`], List[21.882003784179688`, 111.26669120788574`], List[23.428985595703125`, 110.37369346618652`], List[197.5`, 9.87369155883789`], List[198.27300024032593`, 9.426692008972168`], List[199.13700008392334`, 9.203690528869629`], List[200.`, 9.203690528869629`], List[200.86299991607666`, 9.203690528869629`], List[201.72699999809265`, 9.426692008972168`], List[202.5`, 9.87369155883789`], List[376.5710144042969`, 110.37369346618652`], List[378.1179962158203`, 111.26669120788574`], List[379.0710144042969`, 112.91769218444824`], List[379.0710144042969`, 114.70369529724121`], List[379.0710144042969`, 315.7036876678467`], List[379.0710144042969`, 317.4896984100342`], List[378.1179962158203`, 319.1406993865967`], List[376.5710144042969`, 320.03370475769043`]]]]], List[FaceForm[List[RGBColor[0.5529411764705883`, 0.6745098039215687`, 0.8117647058823529`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[44.92900085449219`, 282.59088134765625`], List[181.00001525878906`, 204.0298843383789`], List[181.00001525878906`, 46.90887451171875`], List[44.92900085449219`, 125.46986389160156`], List[44.92900085449219`, 282.59088134765625`]]]]], List[FaceForm[List[RGBColor[0.6627450980392157`, 0.803921568627451`, 0.5686274509803921`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[355.0710144042969`, 282.59088134765625`], List[355.0710144042969`, 125.46986389160156`], List[219.`, 46.90887451171875`], List[219.`, 204.0298843383789`], List[355.0710144042969`, 282.59088134765625`]]]]], List[FaceForm[List[RGBColor[0.6901960784313725`, 0.5882352941176471`, 0.8117647058823529`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[200.`, 394.0606994628906`], List[336.0710144042969`, 315.4997024536133`], List[200.`, 236.93968200683594`], List[63.928985595703125`, 315.4997024536133`], List[200.`, 394.0606994628906`]]]]]], List[Rule[BaselinePosition, Scaled[0.15`]], Rule[ImageSize, 10], Rule[ImageSize, 15]]], StyleBox[RowBox[List["XgbTrain", " "]], Rule[ShowAutoStyles, False], Rule[ShowStringCharacters, False], Rule[FontSize, Times[0.9`, Inherited]], Rule[FontColor, GrayLevel[0.1`]]]]], Rule[GridBoxSpacings, List[Rule["Columns", List[List[0.25`]]]]]], Rule[Alignment, List[Left, Baseline]], Rule[BaselinePosition, Baseline], Rule[FrameMargins, List[List[3, 0], List[0, 0]]], Rule[BaseStyle, List[Rule[LineSpacing, List[0, 0]], Rule[LineBreakWithin, False]]]], RowBox[List["PacletSymbol", "[", RowBox[List["\"MikeYeh/XGBPaclet\"", ",", "\"MikeYeh`XGBPaclet`XgbTrain\""]], "]"]], Rule[TooltipStyle, List[Rule[ShowAutoStyles, True], Rule[ShowStringCharacters, True]]]], Function[Annotation[Slot[1], Style[Defer[PacletSymbol["MikeYeh/XGBPaclet", "MikeYeh`XGBPaclet`XgbTrain"]], Rule[ShowStringCharacters, True]], "Tooltip"]]], Rule[Background, RGBColor[0.968`, 0.976`, 0.984`]], Rule[BaselinePosition, Baseline], Rule[DefaultBaseStyle, List[]], Rule[FrameMargins, List[List[0, 0], List[1, 1]]], Rule[FrameStyle, RGBColor[0.831`, 0.847`, 0.85`]], Rule[RoundingRadius, 4]], PacletSymbol["MikeYeh/XGBPaclet", "MikeYeh`XGBPaclet`XgbTrain"], Rule[Selectable, False], Rule[SelectWithContents, True], Rule[BoxID, "PacletSymbolBox"]][trainset, session];$

How to use XgbModelPredict[] for the prediction.

XgbModelPredict[] use the given session and default model name "model" to predict the testData. Please note that the xbgoost python session should containing the trained "model":

In[7]:=

$InterpretationBox[FrameBox[TagBox[TooltipBox[PaneBox[GridBox[List[List[GraphicsBox[List[Thickness[0.0025`], List[FaceForm[List[RGBColor[0.9607843137254902`, 0.5058823529411764`, 0.19607843137254902`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]], List[List[0, 2, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3], List[0, 1, 0], List[1, 3, 3]]], List[List[List[205.`, 22.863691329956055`], List[205.`, 212.31669425964355`], List[246.01799774169922`, 235.99870109558105`], List[369.0710144042969`, 307.0436840057373`], List[369.0710144042969`, 117.59068870544434`], List[205.`, 22.863691329956055`]], List[List[30.928985595703125`, 307.0436840057373`], List[153.98200225830078`, 235.99870109558105`], List[195.`, 212.31669425964355`], List[195.`, 22.863691329956055`], List[30.928985595703125`, 117.59068870544434`], List[30.928985595703125`, 307.0436840057373`]], List[List[200.`, 410.42970085144043`], List[364.0710144042969`, 315.7036876678467`], List[241.01799774169922`, 244.65868949890137`], List[200.`, 220.97669792175293`], List[158.98200225830078`, 244.65868949890137`], List[35.928985595703125`, 315.7036876678467`], List[200.`, 410.42970085144043`]], List[List[376.5710144042969`, 320.03370475769043`], List[202.5`, 420.53370475769043`], List[200.95300006866455`, 421.42667961120605`], List[199.04699993133545`, 421.42667961120605`], List[197.5`, 420.53370475769043`], List[23.428985595703125`, 320.03370475769043`], List[21.882003784179688`, 319.1406993865967`], List[20.928985595703125`, 317.4896984100342`], List[20.928985595703125`, 315.7036876678467`], List[20.928985595703125`, 114.70369529724121`], List[20.928985595703125`, 112.91769218444824`], List[21.882003784179688`, 111.26669120788574`], List[23.428985595703125`, 110.37369346618652`], List[197.5`, 9.87369155883789`], List[198.27300024032593`, 9.426692008972168`], List[199.13700008392334`, 9.203690528869629`], List[200.`, 9.203690528869629`], List[200.86299991607666`, 9.203690528869629`], List[201.72699999809265`, 9.426692008972168`], List[202.5`, 9.87369155883789`], List[376.5710144042969`, 110.37369346618652`], List[378.1179962158203`, 111.26669120788574`], List[379.0710144042969`, 112.91769218444824`], List[379.0710144042969`, 114.70369529724121`], List[379.0710144042969`, 315.7036876678467`], List[379.0710144042969`, 317.4896984100342`], List[378.1179962158203`, 319.1406993865967`], List[376.5710144042969`, 320.03370475769043`]]]]], List[FaceForm[List[RGBColor[0.5529411764705883`, 0.6745098039215687`, 0.8117647058823529`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[44.92900085449219`, 282.59088134765625`], List[181.00001525878906`, 204.0298843383789`], List[181.00001525878906`, 46.90887451171875`], List[44.92900085449219`, 125.46986389160156`], List[44.92900085449219`, 282.59088134765625`]]]]], List[FaceForm[List[RGBColor[0.6627450980392157`, 0.803921568627451`, 0.5686274509803921`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[355.0710144042969`, 282.59088134765625`], List[355.0710144042969`, 125.46986389160156`], List[219.`, 46.90887451171875`], List[219.`, 204.0298843383789`], List[355.0710144042969`, 282.59088134765625`]]]]], List[FaceForm[List[RGBColor[0.6901960784313725`, 0.5882352941176471`, 0.8117647058823529`], Opacity[1.`]]], FilledCurveBox[List[List[List[0, 2, 0], List[0, 1, 0], List[0, 1, 0], List[0, 1, 0]]], List[List[List[200.`, 394.0606994628906`], List[336.0710144042969`, 315.4997024536133`], List[200.`, 236.93968200683594`], List[63.928985595703125`, 315.4997024536133`], List[200.`, 394.0606994628906`]]]]]], List[Rule[BaselinePosition, Scaled[0.15`]], Rule[ImageSize, 10], Rule[ImageSize, 15]]], StyleBox[RowBox[List["XgbModelPredict", " "]], Rule[ShowAutoStyles, False], Rule[ShowStringCharacters, False], Rule[FontSize, Times[0.9`, Inherited]], Rule[FontColor, GrayLevel[0.1`]]]]], Rule[GridBoxSpacings, List[Rule["Columns", List[List[0.25`]]]]]], Rule[Alignment, List[Left, Baseline]], Rule[BaselinePosition, Baseline], Rule[FrameMargins, List[List[3, 0], List[0, 0]]], Rule[BaseStyle, List[Rule[LineSpacing, List[0, 0]], Rule[LineBreakWithin, False]]]], RowBox[List["PacletSymbol", "[", RowBox[List["\"MikeYeh/XGBPaclet\"", ",", "\"MikeYeh`XGBPaclet`XgbModelPredict\""]], "]"]], Rule[TooltipStyle, List[Rule[ShowAutoStyles, True], Rule[ShowStringCharacters, True]]]], Function[Annotation[Slot[1], Style[Defer[PacletSymbol["MikeYeh/XGBPaclet", "MikeYeh`XGBPaclet`XgbModelPredict"]], Rule[ShowStringCharacters, True]], "Tooltip"]]], Rule[Background, RGBColor[0.968`, 0.976`, 0.984`]], Rule[BaselinePosition, Baseline], Rule[DefaultBaseStyle, List[]], Rule[FrameMargins, List[List[0, 0], List[1, 1]]], Rule[FrameStyle, RGBColor[0.831`, 0.847`, 0.85`]], Rule[RoundingRadius, 4]], PacletSymbol["MikeYeh/XGBPaclet", "MikeYeh`XGBPaclet`XgbModelPredict"], Rule[Selectable, False], Rule[SelectWithContents, True], Rule[BoxID, "PacletSymbolBox"]][testData, newsession]$