Wolfram Function Repository
Instant-use add-on functions for the Wolfram Language
Function Repository Resource:
Evaluate a function applied to each record in a file
ResourceFunction["ReadScan"][f,file] evaluates f applied to each expression read from file in turn. | |
ResourceFunction["ReadScan"][f,file,type] evaluates fapplied to each object of the specified type read from file in turn. | |
ResourceFunction["ReadScan"][f,file,{type1,type2,…}] evaluates fapplied to each list of the specified types read from file in turn. |
Print expressions from a file:
In[1]:= | ![]() |
Out[3]= | ![]() |
Print lines from a file:
In[4]:= | ![]() |
Out[4]= | ![]() |
Process groups of records in bulk from a file:
In[5]:= | ![]() |
Out[5]= | ![]() |
Print records from a file:
In[6]:= | ![]() |
Out[6]= | ![]() |
Specify record separators:
In[7]:= | ![]() |
Out[7]= | ![]() |
ReadScan[f,file,type] is comparable to Scan[f,ReadList[file,type]], but it may use less memory:
In[9]:= | ![]() |
Out[9]= | ![]() |
In[10]:= | ![]() |
Out[10]= | ![]() |
ReadScan used with Reap and Sow can replicate behavior of ReadList:
In[11]:= | ![]() |
Out[11]= | ![]() |
Processing a large file having a simple structure can use a lot of memory if done with Import:
In[12]:= | ![]() |
Out[12]= | ![]() |
ReadScan can achieve similar results with significantly less memory:
In[13]:= | ![]() |
Out[13]= | ![]() |
If the lines of the file are to be processed, but not stored in the current session, substantially less memory is required:
In[14]:= | ![]() |
Out[14]= | ![]() |
Download and extract a somewhat large, "nearly-flat" JSON file from USDA FoodData Central:
In[18]:= | ![]() |
Out[18]= | ![]() |
Import requires a large amount memory:
In[19]:= | ![]() |
Out[19]= | ![]() |
Processing the file line-by-line requires much less memory, and allows for fine-grained control of how each line is processed:
In[20]:= | ![]() |
Out[21]= | ![]() |
Clean up by deleting the extracted file:
In[22]:= | ![]() |
This work is licensed under a Creative Commons Attribution 4.0 International License