Wolfram Language Paclet Repository
Community-contributed installable additions to the Wolfram Language
Writing Mock Tests with Mockingbird
Fundamentals of Mock Evaluation | |
An Example Package-Under-Test: DeckOfCardsLink | |
Basic Mocking of an HTTP API Request | Further Notes |
Testing Failure Behavior | |
The Mockingbird package provides tools for locally overriding the definitions of Wolfram Language functions and symbols. This general functionality is useful for multiple purposes, including interactive development and debugging, but one particularly strong application is the writing of software tests involving a "mocking" or "stubbing" pattern.
Mocking and stubbing
"test double"
The original OOP sense of "mock/stub objects" does not readily lend itself towards use in the Wolfram Language, as the language does not offer a strict object-oriented paradigm. However, the Wolfram Language does offer powerful tools for locally modifying the behavior of function and symbol definitions. Mockingbird provides a wrapper around this functionality that can be used in a similar spirit, and to more or less the same end, when writing software tests in the Wolfram Language.
The distinction between mocking and stubbing is fairly subtle when viewed without the OOP lens. This rest of this document, and that of Mockingbird's documentation, will generally use the term "mock", although Mockingbird can also be used to implement Wolfram Language versions of stubs and other "test double"-like patterns.
Fundamentals of Mock Evaluation
This tutorial will focus mostly on the application of Mockingbird to the specific use case of software testing, but it is useful to first give a brief overview of the basic concepts and operation of the package.
Load the Mockingbird package.
In[1]:=
Needs["Wolfram`Mockingbird`"]
represents a delayed definition for use during mock evaluation | |
evaluate expr defs |
The basic mocking constructs.
is an inert container representing a definition for later use in mock evaluation.
In[2]:=
def=
[f[x_]:=Expand[(x/2)^3]]
 | MockDefinition |
Out[2]=
MockDefinitionf[x_]:=Expand
3
x
2
evaluates an expression after locally installing one or more mock definitions.
In[3]:=
| MockEvaluate |
Out[3]=
3
a
8
3b
2
a
8
3a
2
b
8
3
b
8
The "agricultural" mocking functions.
returns a list like , where is the result of evaluating , and is a list (or association) of expressions sown using .
{result,reaped}
result
expr
reaped
In[4]:=
| MockEvaluateReap |
| MockDefinition |
| MockSow |
Out[4]=
+++,
3
a
8
3b
2
a
8
3a
2
b
8
3
b
8
1
8
3
(a+b)
An Example Package-Under-Test: DeckOfCardsLink
The target of our mockery in this tutorial will be a simple toy package called DeckOfCardsLink. This package provides three functions for interacting with the Deck of Cards API (, an example HTTP API following RESTful design principles. The code for this package is below, followed by examples of the functions it defines.
https://deckofcardsapi.com/)
In[5]:=
BeginPackage["DeckOfCardsLink`"];ShuffleNewDeck[count_Integer : 1] := Enclose Replace Confirm@URLExecute "https://deckofcardsapi.com/api/deck/new/shuffle/", {"deck_count" -> count}, "RawJSON" , data : KeyValuePattern["success" -> True] :> data["deck_id"], data : KeyValuePattern["success" -> False] :> Failure["APIError", data["error"]] , "Expression"DrawCardsFromDeck[deck_String, count_Integer : 1] := Replace URLExecute "https://deckofcardsapi.com/api/deck/" <> deck <> "/draw/", {"count" -> count}, "RawJSON" , data : KeyValuePattern["success" -> True] :> Map <|"Value" -> #value, "Suit" -> #suit, "Image" -> Import[#image]|> &, data["cards"] , data : KeyValuePattern["success" -> False] :> Failure["APIError", data["error"]] ReturnCardsToDeck[deck_String, cards : {__String}] := Replace ImportByteArray URLRead HTTPRequest"https://deckofcardsapi.com/api/deck/" <> deck <> "/return/", <| "Query" -> {"cards" -> StringRiffle[cards, ","]} |> ["BodyByteArray"], "RawJSON" , data : KeyValuePattern["success" -> True] :> Success["ReturnedCards", <||>], data : KeyValuePattern["success" -> False] :> Failure["APIError", data["error"]] EndPackage[]
ShuffleNewDeck
In[10]:=
deckID=ShuffleNewDeck[]
Out[10]=
eib4sjpovsk7
In[11]:=
ShuffleNewDeck[400]
Out[11]=
Failure
|
|
draws the specified number of cards from a deck and imports each card's image.
In[12]:=
DrawCardsFromDeck[deckID,2]
Out[12]=
Value9,SuitDIAMONDS,Image
,Value8,SuitSPADES,Image
returns one or more drawn cards to a deck, with cards specified by abbreviated strings.
In[13]:=
(*3C=3ofclubs;JD=jackofdiamonds*)ReturnCardsToDeck[deckID,{"3C","JD"}]
Out[13]=
Success
✓ |
|
Basic Mocking of an HTTP API Request
This tutorial will focus on mocking the behavior of an external service () accessible via an HTTP API. Bear in mind, though, that Mockingbird is not limited to such applications – mocking I/O operations such as database queries or filesystem access is also feasible. Indeed, the behavior of most any built-in or custom Wolfram Language function can be mocked.
deckofcardsapi.com
For simplicity, the DeckOfCardsLink package uses the and functions directly to make HTTP requests, so our tests will in turn mock the behavior of those functions directly. A more complex package would be likely to instead delegate HTTP requests to a dedicated, custom utility function that itself calls an appropriate built-in function (, , etc.), in addition to handling common steps such as request serialization, authentication, and response parsing. In such a case, the test author could choose whether to test their higher-level functions by mocking the utility function or by mocking the underlying built-in function.
The mocked version of the API call is far faster than sending a real HTTP request.
Validating Request Parameters
Validating Request Parameters
Confirmation functions.
Mock the /deck/new/shuffle API endpoint, asserting that the request includes the expected deck count as an HTTP query parameter.
Testing Failure Behavior
Testing that code behaves as expected during failure cases is an important yet oft-neglected element of software testing. Mockingbird makes it easy to simulate failure modes in practically any part of an application or an external service that it depends on.
Constructs for validating properties of HTTPRequest and other "object"-like expressions.
Further Notes
This tutorial has covered an example application of Mockingbird to testing a Wolfram Language client for an HTTP API. As noted in the introduction, Mockingbird is a general tool for creating and reusing local definitions, and is not limited to the particular use patterns demonstrated in this document.
This section will describe some specific features, patterns, and potential use cases beyond those demonstrated previously.
Recursion
Recursion
Contrary to the standard behavior of Wolfram Language definitions, Mockingbird mock definitions normally do not recurse – i.e. a given definition can only appear once on the evaluation stack. This behavior is useful for "spying" patterns that involve inspecting the arguments passed to a function while still ultimately calling the original or built-in implementation of the function.
Catching Fall-Through Evaluations
Catching Fall-Through Evaluations
Mock definitions precede, but do not supplant, existing definitions for a given symbol. If an invocation of a mocked function does not match any mock definition patterns, the function's original definition will be called.
The mock definition intended for sendRequest does not catch URLExecute called with two arguments, as done in sendRequestWithParameter.
Saving and Reusing Mock Definitions
Saving and Reusing Mock Definitions
Maintaining State in Mock Definitions
Maintaining State in Mock Definitions
It is permissible for mock definitions to access and modify global state, or state otherwise outside of their local scope.
It is probably advisable to exercise discretion in applying this pattern so as to avoid writing tests that are excessive in imitating application behavior irrelevant to the part of the system under test. In particular, mock definitions should use global state with extreme care.