Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 HampelFilter

Source Notebook

Remove outliers from a signal using a Hampel window

Contributed by: Jon McLoone

ResourceFunction["HampelFilter"][data]

removes outliers from data.

ResourceFunction["HampelFilter"][data,w]

uses a moving window of size w.

ResourceFunction["HampelFilter"][data,w,d]

removes outliers that are more than d standard deviations away from the expectation.

Details

ResourceFunction["HampelFilter"] replaces outliers with the median of a moving window. By default, three points on each side are used.
A point is considered an outlier if its value is more than one standard deviation from the moving median.
The distance from the moving median is estimated using MeanDeviation.

Examples

Basic Examples (2) 

Signal data with outliers:

In[1]:=
data = {0., 0.07467452157637092, 0.035908394979099804`, 0.03298286712280798, 0.028575047661953956`, 0.09431071164572534, 0.17131118710785453`, 0.22335837716353057`, 0.2273107380522667, 0.020795348947157653`, -0.01184281402239689, 0.10433406873330527`, 0.09351487010494579, 0.0543645379253127, 0.05101452415040744, -0.15800119819010483`, -0.14692005598108446`, -0.10953787931410232`, -0.09725202225682003, -0.015771578558602294`, -0.07637344823954687, -0.07157199703831428, -0.11825725599394134`, -0.09377902591150483, 0.06719742370351492, 0.04606156646017129, 0.23639663235921515`, 0.0807870930965896, 0.07976094593773095, 0.09180044109802266, 0.22323209978204278`, 0.25698147085120904`, 0.3931212065536937, 0.3446683897108681, 0.3446339958762545, 0.4260727944166894, 0.47254355683860993`, 0.43699252779886716`, 0.15583506027111121`, 1, 0.1276413191220588, 0.043339612994388335`, -0.08977202346201933, -0.13016659079924736`, -0.16894987909140294`, -0.19583728500934255`, -0.2470182418920058, -0.1961952145043373, -0.20444580328486514`, -0.17556372476584284`, -0.15969105821679613`, -0.025559591039181675`, 0.05581641174257388, 0.054036383695374966`, -0.0964614815841567, -0.008511894596689176, -0.011733830166795047`, 0.0223924218251653, -0.10488638902583693`, -0.12916391298095964`, -0.010433713089892965`, 0.19923723356826742`, 0.1859027586618538, 0.07277973738953583, 0.1662143325277546, 0.11302344823122765`, 0.21117227528134383`, 0.2930259848660719, 0.31770272204004424`, 0.2158279179228266, 0.2634331032441018, 0.41578895003607275`, 0.4268564297313227, 0.39822200607899677`, 0.3652724156144793, 0.405292100653545, 0.3114632998099347, 0.18011580702602661`, 0.2371665371807742, 0.30816644505363444`, 0.269742043561081, 0.43661874566059944`, 0.10184886381709998`, -0.014826136682917357`, -0.031128925752836562`, -0.16648959986907366`, -0.17479948398739845`, -0.10124260081546775`, -0.1579667937787252, -0.040375318837511345`, -0.09577406308306102, -0.0506842503170067, -0.10043720875941697`, -0.20196098532317444`, -0.1891207021532106, -0.08838201982081814, -0.13323189450446105`, 0.019918656280844528`, 0.024586120237373298`, -0.02187695045976697, 0.};
In[2]:=
ListLinePlot[data, PlotRange -> {-0.3, 1}]
Out[2]=

Remove outliers from the data:

In[3]:=
ListLinePlot[ResourceFunction["HampelFilter"][data], PlotRange -> {-0.3, 1}]
Out[3]=

Scope (4) 

Reduce the window size used to decide if a point is an outlier:

In[4]:=
data = {0., 0.07467452157637092, 0.035908394979099804`, 0.03298286712280798, 0.028575047661953956`, 0.09431071164572534, 0.17131118710785453`, 0.22335837716353057`, 0.2273107380522667, 0.020795348947157653`, -0.01184281402239689, 0.10433406873330527`, 0.09351487010494579, 0.0543645379253127, 0.05101452415040744, -0.15800119819010483`, -0.14692005598108446`, -0.10953787931410232`, -0.09725202225682003, -0.015771578558602294`, -0.07637344823954687, -0.07157199703831428, -0.11825725599394134`, -0.09377902591150483, 0.06719742370351492, 0.04606156646017129, 0.23639663235921515`, 0.0807870930965896, 0.07976094593773095, 0.09180044109802266, 0.22323209978204278`, 0.25698147085120904`, 0.3931212065536937, 0.3446683897108681, 0.3446339958762545, 0.4260727944166894, 0.47254355683860993`, 0.43699252779886716`, 0.15583506027111121`, 1, 0.1276413191220588, 0.043339612994388335`, -0.08977202346201933, -0.13016659079924736`, -0.16894987909140294`, -0.19583728500934255`, -0.2470182418920058, -0.1961952145043373, -0.20444580328486514`, -0.17556372476584284`, -0.15969105821679613`, -0.025559591039181675`, 0.05581641174257388, 0.054036383695374966`, -0.0964614815841567, -0.008511894596689176, -0.011733830166795047`, 0.0223924218251653, -0.10488638902583693`, -0.12916391298095964`, -0.010433713089892965`, 0.19923723356826742`, 0.1859027586618538, 0.07277973738953583, 0.1662143325277546, 0.11302344823122765`, 0.21117227528134383`, 0.2930259848660719, 0.31770272204004424`, 0.2158279179228266, 0.2634331032441018, 0.41578895003607275`, 0.4268564297313227, 0.39822200607899677`, 0.3652724156144793, 0.405292100653545, 0.3114632998099347, 0.18011580702602661`, 0.2371665371807742, 0.30816644505363444`, 0.269742043561081, 0.43661874566059944`, 0.10184886381709998`, -0.014826136682917357`, -0.031128925752836562`, -0.16648959986907366`, -0.17479948398739845`, -0.10124260081546775`, -0.1579667937787252, -0.040375318837511345`, -0.09577406308306102, -0.0506842503170067, -0.10043720875941697`, -0.20196098532317444`, -0.1891207021532106, -0.08838201982081814, -0.13323189450446105`, 0.019918656280844528`, 0.024586120237373298`, -0.02187695045976697, 0.};
In[5]:=
ListLinePlot[ResourceFunction["HampelFilter"][data, 1], PlotRange -> {-0.3, 1}]
Out[5]=

The window size can be also be increased:

In[6]:=
ListLinePlot[ResourceFunction["HampelFilter"][data, 10], PlotRange -> {-0.3, 1}]
Out[6]=

Reduce the outlier threshold to make HampelFilter more sensitive:

In[7]:=
ListLinePlot[ResourceFunction["HampelFilter"][data, 3, 0.5], PlotRange -> {-0.3, 1}]
Out[7]=

Increasing the threshold will adjust fewer points:

In[8]:=
ListLinePlot[ResourceFunction["HampelFilter"][data, 3, 2], PlotRange -> {-0.3, 1}]
Out[8]=

Publisher

Jon McLoone

Version History

  • 1.0.0 – 22 March 2021

License Information