How To Filter Noise With A Low Pass Filter - Python - by Neha Jirafe - Analytics Vidhya - Medium

17/12/2021 19:43 How to filter noise with a low pass filter — Python | by Neha Jirafe | Analytics Vidhya | Medium
You have 2 free member-only stories left this month. Sign up for Medium and get an extra one
How to filter noise with a low pass

filter — Python
Neha Jirafe Follow
Dec 27, 2019 · 5 min read
Recently while I was working on processing a very high frequency signal of

12.5 Khz , i.e. 12500 samples per second or a sample every 80 microsecond.
What was more interesting is that I had to derive various data points into
this data set.
This is how my data in a single cycle looked like
https://medium.com/analytics-vidhya/how-to-filter-noise-with-a-low-pass-filter-python-885223e5e9b7 1/13
You can see the noise when I zoom in the data
So now consider, if had to determine the point where the curve starts it rise.
With so much of noise there is a very high probability of getting false

positive data point. Also imagine the performance of the algorithm with so
much fluctuation in the data.
I would start with some signal processing basics , which are essential to
understand before we jump into code.
Basics : Band Pass Filters

The four common filters.
1. Low-pass filter, passes signals with a frequency lower than a certain

cutoff frequency and attenuates signals with frequencies higher than the
cutoff frequency.
2. High-pass filter, passes signals with a frequency higher than a certain

cutoff frequency and attenuates signals with frequencies lower than the
cutoff frequency.
3. A band-pass filter can be formed by cascading a high-pass filter and a

low-pass filter.
4. A band-reject filter is a parallel combination of low-pass and high-pass

filters.
Now lets see a sample data ,which would be ideal to work with
As you can see the distortion caused by a lot of noise has deformed actual
data which is a sin wave data.
Sample Period — 5 sec (t)
Sampling Freq — 30 samples / s , i.e 30 Hz (fs)
Total Samples — (fs x t) = 150
Signal Freq = 6 signal / 5 sec = 1.2 Hz
This means we need a filter that would pass the signal with at most
frequency of 1.2 Hz , However in real life the signal frequency may fluctuate
, hence it would be good if we choose a slightly higher number than the
ideally calculated frequency.
You can also try using FFT (Fast Fourier Transform) to find investigate the
frequencies and amplitudes of the Signal vs the noise components, more
details along with code can be found here
Butterworth Filter
The frequency response of the Butterworth filter is maximally flat (i.e. has
no ripples) in the passband and rolls off towards zero in the stopband,
hence its one of the most popular low pass filter.
Nyquist Frequency
The term Nyquist is often used to describe the Nyquist sampling rate or the
Nyquist frequency.
The Nyquist rate or frequency is the minimum rate at which a finite

bandwidth signal needs to be sampled to retain all of the information. If a
time series is sampled at regular time intervals dt, then the Nyquist rate is
just 1/(2 dt ).
Lets jump into Code

Step 1 : Define the filter requirements
Sample Period — 5 sec (t)
Sampling Freq — 30 samples / s , i.e 30 Hz (fs)
Total Samples — (fs x t) = 150
Signal Freq = 6 signal / 5 sec = 1.2 Hz
Nyquist Frequency = 0.5 * fs
order = Polynomial order of the signal
import numpy as np
from scipy.signal import butter,filtfilt
# Filter requirements.
T = 5.0 # Sample Period
fs = 30.0 # sample rate, Hz
cutoff = 2 # desired cutoff frequency of the filter, Hz ,

slightly higher than actual 1.2 Hz
nyq = 0.5 * fs # Nyquist Frequency
order = 2 # sin wave can be approx represented as quadratic
n = int(T * fs) # total number of samples
Step 2 : Create some sample data with noise
# sin wave
sig = np.sin(1.2*2*np.pi*t)
# Lets add some noise
noise = 1.5*np.cos(9*2*np.pi*t) + 0.5*np.sin(12.0*2*np.pi*t)
data = sig + noise
Step 3 : Filter implementation using scipy
def butter_lowpass_filter(data, cutoff, fs, order):
normal_cutoff = cutoff / nyq
# Get the filter coefficients
b, a = butter(order, normal_cutoff, btype='low', analog=False)
y = filtfilt(b, a, data)
return y
Step 4 : Filter and plot the data
# Filter the data, and plot both the original and filtered signals.
y = butter_lowpass_filter(data, cutoff, fs, order)
fig = go.Figure()
fig.add_trace(go.Scatter(
y = data,
line = dict(shape = 'spline' ),
name = 'signal with noise'
))
fig.add_trace(go.Scatter(
y = y,
line = dict(shape = 'spline' ),
name = 'filtered signal'
))
fig.show()
It’s surprising how smoothly the filtered signal aligns to the data, feels like
‘butter’.
And that’s it for today ! Happy Filtering
Check out my other articles
How to make Jupyter Notebooks Extensible and

Reusable
All those who use Jupyter notebooks for data analysis or machine
learning workloads , know the pain of the “copy —…
medium.com
Data Modeling — Designing Facebook style

“comments” with SQL
Let’s agree , we all have used the “comments” section of some or the
other social networking site.
medium.com
Using FFT to analyse and cleanse time series data

Often when dealing with IOT data , the biggest challenge is
encountering unexpected noise, this noise can be very…
medium.com
Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya
Latest news from Analytics Vidhya on our Hackathons and some of our best
articles! Take a look.
Your email Get this newsletter
By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information
about our privacy practices.
Signal Processing Python Scipy Low Pass Filter Noise
Learn more. Make Medium yours. Write a story on Medium.

Medium is an open platform where 170 million Follow the writers, publications, and topics that If you have a story to tell, knowledge to share, or
readers come to find insightful and dynamic matter to you, and you’ll see them on your a perspective to offer — welcome home. It’s
thinking. Here, expert and undiscovered voices homepage and in your inbox. Explore easy and free to post your thinking on any topic.
alike dive into the heart of any topic and bring Start a blog
new ideas to the surface. Learn more
About Write Help Legal

How To Filter Noise With A Low Pass Filter - Python - by Neha Jirafe - Analytics Vidhya - Medium

Uploaded by

Copyright:

Available Formats

You might also like

How To Filter Noise With A Low Pass Filter - Python - by Neha Jirafe - Analytics Vidhya - Medium

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How To Filter Noise With A Low Pass Filter - Python - by Neha Jirafe - Analytics Vidhya - Medium

Uploaded by

Copyright:

Available Formats

17/12/2021 19:43 How to filter noise with a low pass filter — Python | by Neha Jirafe | Analytics Vidhya | Medium

How to filter noise with a low pass

Dec 27, 2019 · 5 min read

Recently while I was working on processing a very high frequency signal of

This is how my data in a single cycle looked like

You can see the noise when I zoom in the data

With so much of noise there is a very high probability of getting false

Basics : Band Pass Filters

1. Low-pass filter, passes signals with a frequency lower than a certain

2. High-pass filter, passes signals with a frequency higher than a certain

3. A band-pass filter can be formed by cascading a high-pass filter and a

4. A band-reject filter is a parallel combination of low-pass and high-pass

Sample Period — 5 sec (t)

Sampling Freq — 30 samples / s , i.e 30 Hz (fs)

Total Samples — (fs x t) = 150

Signal Freq = 6 signal / 5 sec = 1.2 Hz

The Nyquist rate or frequency is the minimum rate at which a finite

Lets jump into Code

Sample Period — 5 sec (t)

Sampling Freq — 30 samples / s , i.e 30 Hz (fs)

Total Samples — (fs x t) = 150

Signal Freq = 6 signal / 5 sec = 1.2 Hz

Nyquist Frequency = 0.5 * fs

order = Polynomial order of the signal

from scipy.signal import butter,filtfilt

T = 5.0 # Sample Period

fs = 30.0 # sample rate, Hz

cutoff = 2 # desired cutoff frequency of the filter, Hz ,

nyq = 0.5 * fs # Nyquist Frequency

order = 2 # sin wave can be approx represented as quadratic

n = int(T * fs) # total number of samples

Step 2 : Create some sample data with noise

# Lets add some noise

noise = 1.5*np.cos(9*2*np.pi*t) + 0.5*np.sin(12.0*2*np.pi*t)

data = sig + noise

Step 3 : Filter implementation using scipy

def butter_lowpass_filter(data, cutoff, fs, order):

normal_cutoff = cutoff / nyq

# Get the filter coefficients

b, a = butter(order, normal_cutoff, btype='low', analog=False)

Step 4 : Filter and plot the data

y = butter_lowpass_filter(data, cutoff, fs, order)

line = dict(shape = 'spline' ),

name = 'signal with noise'

line = dict(shape = 'spline' ),

name = 'filtered signal'

And that’s it for today ! Happy Filtering

Check out my other articles

How to make Jupyter Notebooks Extensible and

Data Modeling — Designing Facebook style

Using FFT to analyse and cleanse time series data

Sign up for Analytics Vidhya News Bytes

Your email Get this newsletter

Signal Processing Python Scipy Low Pass Filter Noise

Learn more. Make Medium yours. Write a story on Medium.

About Write Help Legal

You might also like

noise = 1.5np.cos(92np.pit) + 0.5np.sin(12.02np.pit)