FaizonZaman/ LexicalCases

Extract lexical patterns from text

Contributed by: Faizon Zaman

Search text for lexical patterns. While similar to StringCases, TextCases etc., content types can be used freely in a StringExpression (as opposed to expressing complex lexical patterns in a Containing wrapper). Files, search index objects, and Wikipedia queries are supported. Results are returned in a summary object which supports several subvalues. Consult the documentation for usage.

Installation Instructions

To install this paclet in your Wolfram Language environment, evaluate this code:
PacletInstall["FaizonZaman/LexicalCases"]

To load the code after installation, evaluate this code:
Needs["FaizonZaman`LexicalCases`"]

Details

This functionality came about when I needed to find adjectives before certain phrases. I initially gathered article text from Wikipedia, then used TextCases and TextContents with the Containing wrapper, (Containing["AdjectivePhrase",Verbatim["music"]] for example), but it turned out to be slow. So I implemented some lexical pattern objects for use in StringExpression's. This way I can search for the string pattern with StringCases.

TextCases is used internally to extract examples of a content type from the source text. These examples then replace the TypeToken in the StringExpression. You can see the lexical pattern generated from a piece of text with ExpandPattern. Note that this means, for example, if you have TypeToken["Adjective"] in your lexical pattern, the token will be replaced by all text snippets identified as adjectives in the source text.

I'd also like to say thanks to Swastik Banerjee for helping me improve SearchIndexObject support!