Ingesting Smart Building System Data

Import and analyze a Kaggle dataset collected from sensors deployed in an office building

This Kaggle dataset contains 255 sensor time series collected from 51 rooms across four floors of Sutardja Dai Hall at UC Berkeley. It records physical room conditions and occupancy-related signals, making it suitable for smart building analytics, Internet of Things applications, sensor fusion studies and time-series analysis.

Connect to the Kaggle dataset using

DataConnectionObject

In[1]:=

conn=DataConnectionObject["ConnectionName""Kaggle","Location""ranakrc/smart-building-system"]

Out[1]=

DataConnectionObject

Connection name: Kaggle

Location: ranakrc/smart-building-system



The remote file is compressed:

In[2]:=

conn//FileFormat

Out[2]=

ZIP

Extract the file names from the zip file:

In[3]:=

dataElements=DeleteCases[Import[conn,"FileNames"],_?(StringContainsQ["README"][#]&)];dataElements//Short

Out[3]//Short=

{KETI/413/co2.csv,KETI/413/humidity.csv,KETI/413/light.csv,KETI/413/pir.csv,KETI/413/temperature.csv,KETI/415/co2.csv,KETI/415/humidity.csv,241,KETI/754/pir.csv,KETI/754/temperature.csv,KETI/776/co2.csv,KETI/776/humidity.csv,KETI/776/light.csv,KETI/776/pir.csv,KETI/776/temperature.csv}

Extract Id's of rooms and column keys from the dataset:

In[4]:=

roomIDs=Union[FileNameSplit[#]〚2〛&/@dataElements];colKey={"co2","humidity","light","pir","temperature"};

Convert the data to

Tabular

In[5]:=

TabGen[file_]:=With[{type=StringDrop[FileNameSplit[file]〚-1〛,-4]},ToTabular[Import[conn,file],"Rows","ColumnKeys"{"UnixEpochTime",type}]]

In[6]:=

TabGenRoom[roomID_]:=With[{tabs=TabGen[FileNameJoin[{"KETI",roomID,#<>".csv"}]]&/@colKey},{tab2=Fold[JoinAcross[#1,#2,"UnixEpochTime","Outer"]&,tabs〚1〛,Rest[tabs]]},roomIDInsertColumns[tab2,"room"ConstantArray[roomID,Length[tab2]]]]

In[7]:=

df=Association[TabGenRoom/@roomIDs];allData=Join@@Values[df]

Out[7]=

Reveal the structure of the

Tabular

data:

In[8]:=

TabularStructure[allData]

Out[8]=

Plot a smooth kernel histogram for the carbon dioxide data from the dataset:

In[9]:=

Show[Histogram[allData"co2",{10},"PDF"],SmoothHistogram[allData"co2",{"Scott","Gaussian"},"PDF"]]

Out[9]=

External Links

Smart building system on Kaggle

Source Metadata

Citation:
- Dezhi Hong, Quanquan Gu, Kamin Whitehouse. High-dimensional Time Series Clustering via Cross-Predictability. In AISTATS'17

Publisher Information

Contributed by: Shenghui Yang

License Information

This work is licensed under a Creative Commons Attribution 4.0 International License

Wolfram Language Example Repository

Ingesting Smart Building System Data

External Links

Source Metadata

See Also

Publisher Information

License Information

Ingesting Smart Building System Data

External Links

Source Metadata

See Also

Related Symbols

Publisher Information

License Information