Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 SentenceComplexityMeasures

Source Notebook

Obtain a set of complexity measures for an English sentence

Contributed by: Mark Greenberg

ResourceFunction["SentenceComplexityMeasures"][s]

gives a report listing five measures of the complexity of sentence s plus an overall complexity score.

Details and Options

ResourceFunction["SentenceComplexityMeasures"][s] only works for English s.
ResourceFunction["SentenceComplexityMeasures"][s] combines five separately rescaled measurements of the graph and tree of s:
"leaf count"number of surface-level structures (words, contracted words)
"vertex count"number of grammatical structures (words, phrases, clauses, etc.)
"tree depth"maximum depth of nested grammatical structures
"graph diameter"longest distance between grammatical structures
"distance"average distance between grammatical structures
The author determined the scaling ranges from empirical tests.
The five measurements are rescaled and summed to produce a single empirically-defined metric.
ResourceFunction["SentenceComplexityMeasures"][s] uses this formula to determine the overall complexity score: (Rescale[leafCt,{2,120}]+Rescale[vertCt,{6,360}]+Rescale[depth,{3,15}]+Rescale[diam,{6,22}]+Rescale[dist,{2.5,10}])/5
The "Format" option gives different kinds of output and can take the following values:
"Association"an association for all five measures and their complexity scores
"Number"a real number from 0 (simplest possible sentence) to 1 (extremely complex sentence)
"Dataset"(default) an easily read report with the same information as "Association"

Examples

Basic Examples (2) 

Get complexity measures for a sentence:

In[1]:=
ResourceFunction[ "SentenceComplexityMeasures"]["Here is a simple sentence contrived just now for this example."]
Out[1]=

Measure a more complex sentence:

In[2]:=
ResourceFunction[ "SentenceComplexityMeasures"]["This sentence, which has a nested clause, scores higher."]
Out[2]=

Options (2) 

Get the complexity measure as a real number:

In[3]:=
orwell1984 = "It was a bright cold day in April, and the clocks were striking thirteen."; ResourceFunction["SentenceComplexityMeasures"][orwell1984, Format -> "Number"]
Out[4]=

Get an association with more detail for the same sentence:

In[5]:=
ResourceFunction["SentenceComplexityMeasures"][orwell1984, Format -> "Association"]
Out[5]=

Applications (1) 

Demonstrate the effects of different sentence enhancements:

In[6]:=
s1 = "Joseph is a postman."; s2 = "Joseph is a dedicated postman."; s3 = "Joseph is a postman who is dedicated."; s4 = "Joseph is a postman who is dedicated to his work."; complexities = ResourceFunction["SentenceComplexityMeasures"][#, Format -> "Number"] & /@ {s1, s2, s3, s4}; BarChart[complexities]
Out[11]=

Neat Examples (2) 

Measure the sentence complexity mean and standard deviation of an entire piece of literature. (This took 110 seconds on the development computer.):

In[12]:=
sentences = TextSentences[ExampleData[{"Text", "AliceInWonderland"}]]; aliceComp = (ResourceFunction["SentenceComplexityMeasures"][#, Format -> "Number"] &) /@ sentences; {Mean[aliceComp], StandardDeviation[aliceComp]}
Out[14]=

Demonstrate the difference between a run-on sentence and one with nested clauses:

In[15]:=
s1 = "She painted landscapes, but sometimes the landscapes had floating objects, and the objects might be skulls or flowers."; s2 = "She painted landscapes that sometimes had floating objects, which could be skulls or flowers."; Column[ResourceFunction["SentenceComplexityMeasures"] /@ {s1, s2}, Spacer[12]]
Out[17]=

Possible Issues (1) 

There is no limit to how long and complex an English sentence might be. The parser fails on extremely complex sentences. In those cases, SentenceComplexityMeasures will return a Failure object:

In[18]:=
ResourceFunction[ "SentenceComplexityMeasures"]["In the event that the Purchaser defaults in the payment of any instalment of purchase price, taxes, insurance, interest, or the annual charge described elsewhere herein, or shall default in the performance of any other obligations set forth in this Contract, the Seller may: at his option: (a) Declare immediately due and payable the entire unpaid balance of purchase price, with accrued interest, taxes, and annual charge, and demand full payment thereof, and enforce conveyance of the land by termination of the contract or according to the terms hereof, in which case the Purchaser shall also be liable to the Seller for reasonable attorney's fees for services rendered by any attorney on behalf of the Seller, or (b) sell said land and premises or any part thereof at public auction, in such manner, at such time and place, upon such terms and conditions, and upon such public notice as the Seller may deem best for the interest of all concerned, consisting of advertisement in a newspaper of general circulation in the county or city in which the security property is located at least once a week for Three (3) successive weeks or for such period as applicable law may require and, in case of default of any purchaser, to re-sell with such postponement of sale or resale and upon such public notice thereof as the Seller may determine, and upon compliance by the Purchaser with the terms of sale, and upon judicial approval as may be required by law, convey said land and premises in fee simple to and at the cost of the Purchaser, who shall not be liable to see to the application of the purchase money; and from the proceeds of the sale: First to pay all proper costs and charges, including but not limited to court costs, advertising expenses, auctioneer's allowance, the expenses, if any required to correct any irregularity in the title, premium for Seller's bond, auditor's fee, attorney's fee, and all other expenses of sale occurred in and about the protection and execution of this contract, and all moneys advanced for taxes, assessments, insurance, and with interest thereon as provided herein, and all taxes due upon said land and premises at time of sale, and to retain as compensation a commission of five percent (5%) on the amount of said sale or sales; SECOND, to pay the whole amount then remaining unpaid of the principal of said contract, and interest thereon to date of payment, whether the same shall be due or not, it being understood and agreed that upon such sale before maturity of the contract the balance thereof shall be immediately due and payable; THIRD, to pay liens of record against the security property according to their priority of lien and to the extent that funds remaining in the hands of the Seller are available; and LAST, to pay the remainder of said proceeds, if any, to the vendor, his heirs, personals representatives, successors or assigns upon the delivery and surrender to the vendee of possession of the land and premises, less costs and excess of obtaining possession."]
Out[18]=

Publisher

Mark Greenberg

Requirements

Wolfram Language 13.0 (December 2021) or above

Version History

  • 1.0.0 – 22 January 2024

Related Resources

License Information