Wolfram Language Paclet Repository
Community-contributed installable additions to the Wolfram Language
Wolfram`Parser`
SpannedToken  |
|
| | ||||
▪
SpannedToken
ParsePosition[]~~token~~ParsePosition[]~~ws
setSource
{start,end}
▪
token is the parser for the leaf's text (often a or ); ws is the trailing-whitespace parser to consume after the span is closed (so the span covers the token only); build is the leaf constructor, e.g. .
ParseRegex
LeafNode["Integer",#,]&
▪
The captured span is a raw pair of 1-based character offsets, where end is one past the last character token matched - is the token's length. The trailing whitespace ws is eaten outside the two s, so it never widens the span.
{start,end}
end-start
▪
A non-node build result passes through unstamped: when the grammar runs over a semantic algebra, build returns a value (a number, say) rather than a node, and there is nothing to carry , so returns it unchanged.
Source
SpannedToken
▪
The offsets here are not yet line/column. is the finalizer that spans the composites and converts every offset span to a pair against the source.
{start,end}
{{startLine,startColumn},{endLine,endColumn}}
Examples
(3)
Basic Examples
(1)
Match an integer token, skip trailing whitespace, and build a carrying its span - is the character offsets of :
Source->{1,3}
"42"
In[1]:=
 | Parse |
 | SpannedToken |
 | ParseRegex |
| ParseMany |
 | ParseCharacter |
| LeafNode |
Out[1]=
LeafNode[Integer,42,Source{1,3}]
The span is with end one past the last character, so is the token length - here , the two digits of .
{start,end}
end-start
3-1=2
"42"
The trailing whitespace ws is consumed outside the span, so it never widens it - parsing still yields the span :
"42 "
{1,3}
In[2]:=
| Parse |
| SpannedToken |
| ParseRegex |
| ParseMany |
| ParseCharacter |
| LeafNode |
Out[2]=
LeafNode[Integer,42,Source{1,3}]
Scope
(1)
Properties & Relations
(1)
 |
|
 |
|
""