IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO.
5, MAY 2018
Efficient Scalable Median Filtering Using
Histogram-Based Operations
Oded Green
Abstract— Median filtering is a smoothing technique for noise
removal in images. While there are various implementations
of median filtering for a single-core CPU, there are few
implementations for accelerators and multi-core systems. Many
parallel implementations of median filtering use a sorting
algorithm for rearranging the values within a filtering window
and taking the median of the sorted value. While using sorting
algorithms allows for simple parallel implementations, the cost
of the sorting becomes prohibitive as the filtering windows
grow. This makes such algorithms, sequential and parallel alike,
inefficient. In this work, we introduce the first software parallel
median filtering that is non-sorting-based. The new algorithm
uses efficient histogram-based operations. These reduce the
computational requirements of the new algorithm while also
accessing the image fewer times. We show an implementation of
our algorithm for both the CPU and NVIDIA’s CUDA supported
graphics processing unit (GPU). The new algorithm is compared
with several other leading CPU and GPU implementations. The
CPU implementation has near perfect linear scaling with a
3.7× speedup on a quad-core system. The GPU implementation
is several orders of magnitude faster than the other GPU
implementations for mid-size median filters. For small kernels,
3 × 3 and 5 × 5, comparison-based approaches are preferable
as fewer operations are required. Lastly, the new algorithm is
open-source and can be found in the OpenCV library.
Index Terms— Median filtering, parallel algorithms.
I. I NTRODUCTION
M EDIAN filtering is a key building block used by
numerous image processing application for denoising
an image, smoothing, or removing unwanted artifacts [1].
In addition to image processing, median filtering can be
found in signal processing [2], computer vision based appli-
cations [3], [4], and video processing [5], [6].
Median filtering requires a single parameter, k, which rep-
resents the size of the filtering windows (sometimes referred
to as the kernel). Median filtering works as follows, for each
pixel in the image, a k × k window is placed around that pixel
and the median value of theses pixels is taken as the output
value. Hence, the name median filtering. How that median
value is found is algorithm dependent. One popular method
for finding the median value is to sort all the k 2 values and
then simply take the value at the middle of the outputted array.
Manuscript received December 6, 2016; revised August 16, 2017 and
November 8, 2017; accepted December 1, 2017. Date of publication
December 8, 2017; date of current version February 9, 2018. The associate
editor coordinating the review of this manuscript and approving it for
publication was Dr. Christos Bouganis.
The author is with the College of Computing, Georgia Institute of
Technology, Atlanta, GA 30332 USA (e-mail: ogreen@gatech.edu).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIP.2017.2781375
1057-7149 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
2217
The complexity of the sort is dependent on the sorting function
used and can range from O(n) for a radix sort or bucket sort,
O(n·log(n)) for merge sort or quick sort, or O(n 2 ) for bubble-
sort where n is the number of elements that will be sorted.
For median filtering, n = k 2 which means that the cost of
sorting increases quadratically with the window size. For small
kernels, the performance of the sorting algorithm is dictated
by several factors: the data itself, the storage requirements,
amount of data movement required by the algorithm, and the
parallel scalability of the algorithm.
Huang et al. [7] showed a fast implementation for median
filtering that was based on histogram operations rather
than sorting. They showed how it is possible to “add” and
“subtract” whole histograms. Specifically, given a single
master histogram and using a sliding window technique, the
histogram of the left-most column in the kernel is removed
and the histogram of the column to the right is added. In this
approach, for each pixel two computationally expensive his-
togram operations are required. Further, each pixel is accessed
O(k 2 ) times.
Perreault and Hébert [8] recently showed an algorithm
that cuts the number of times that each pixel is accessed to
O(1) times. Thus, the operation is no longer dependent on the
kernel size and the number of pixel accesses is greatly reduced.
Their algorithm extends that of Huang et al. [7]. Unlike [7],
they show that the output of the median filter for two con-
secutive rows in an image contains a relatively large number
of overlapping pixels. They introduce a new auxiliary data
structure that stores a histogram for every column in an image
and through additional histogram manipulative instructions
they show an efficient algorithm for median filtering. A similar
window movement pattern is used in [9] to efficiently compute
the estimated covariance matrix - reducing the computational
complexity by several orders of magnitude.
Our new algorithm, extends the work of Perreault and
Hébert [8] and presents to the best of our knowledge the
first software parallel algorithm for median filtering that uses
histogram based operations and accesses each pixel O(1)
times. The main contribution of this paper is to show how
to partition the work required by the median filtering to a
many-core system. This algorithm is appropriate both for a
standard multi-core CPU system and for modern day graphics
processing unit (GPU) systems.
In the experiment section of this paper, Section V,
the parallel algorithm is evaluated on both the CPU and
the GPU. The CPU implementation is based off the algorithm
Perreault and Hébert [8] found in OpenCV and the GPU imple-
mentation was built afresh especially for NVIDIA’s CUDA
 2218
platform. Of these implementations, the GPU is the more
challenging of these platforms for algorithm parallelization as
it requires more optimizations to achieve good performance.
The new algorithm is benchmarked against several leading
libraries and implementations, both CPU and GPU libraries.
All these implementations use sorting operations except for
the CPU implementation by Perreault and Hébert [8] (which
is also sequential). The new GPU algorithm outperforms all
the sorting-based implementations for mid-size filters (such as
an 11 × 11 window) and large filters. In some cases, the new
algorithm is over a 100× faster than other implementations.
For smaller windows such as 3×3 and 5×5 the new algorithm
is not the fastest and is out-performed by several finely tuned
implementations. Lastly, our implementation is open-source
and has been tested and integrated in the OpenCV project [10]
and can be found at https://github.com/opencv/opencv.
II. R ELATED W ORK
Median filtering is a widely used technique for cleaning
images. Other widely used filters include: biliteral [11], neigh-
borhood filtering [12], mode filtering [13], and Gaussian filter-
ing [14]. These filtering methods smooth images making them
visually more appealing while removing unwanted artifacts.
Median filtering is popular because of its conceptual sim-
plicity which also leads to straightforward implementations,
both in software and hardware. Median filtering can be
found in a large number of software libraries, including:
OpenCV [10], NVIDIA’s NPP [15], ArrayFire [16],
SciPy [17], and the Halide [18] image processing compiler.
These algorithms vary in their implementation based on the
system that they target (CPU, GPU, or some specific accel-
erator) and on the method used for finding the median value
(histogram based operations or sorting of the kernel window).
The simplicity and the determinism of the sorting oper-
ations, specifically sorting networks, allows for imple-
menting power efficient hardware median filters. This
includes numerous FPGA implementations of median filtering
[19]–[21]. Sorting networks implemented in hardware typi-
cally focus on small and mid-size kernels due to the cost of
the hardware logic. Lastly, sorting networks can be based of
the different sorting algorithms, including Batcher’s sorting
algorithm [22] (which was originally designed for hardware).
A. GPU Algorithms
GPU implementations for median filtering include the
BVM algorithm [23], [24], PCMF algorithm [25], [26], and
PRMF [27]. While these algorithms are very different from
each other, they share a common trait - they use compar-
isons and sorting operations for finding the median value.
BVM [23], [24] avoids using branch instructions. This was
shown to be extremely costly for merging and sorting based
operations [28]. BVM algorithm is implemented for both the
CPU and the GPU. On the CPU, BVM utilizes SIMD instruc-
tions to further increase parallelism. PCMF [25], [26] uses a
sorting algorithm based on the Complementary Cumulative
Distribution (CCD) function [29]. CCD benefits from low
computational costs and a highly parallelizable data structure.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018
PCMF allows for near constant processing time for each pixel.
While the time complexity of PCMF is similar to the one
presented in this paper, the approaches are very different.
The time complexity of PCMF is dependent on the value
distributions giving it an amortized execution time. In contrast,
the new algorithm presented in this paper has a deterministic
execution time that is independent of pixel values. Lastly,
PRMF [27] shows how to implement a median filter through
a comparison network designed for a specific kernel size.
PRMF includes a tool for creating this network. In each
step of the PRMF algorithm, elements that are no longer
needed are pruned out of the comparison network (and are not
used). From a performance standpoint, BVM is outperformed
both by PRMF and by PCMF. For smaller kernel sizes,
PRMF outperforms PCMF. However, for larger kernels PCMF
outperforms PRMF due to a reduced number of operations
per pixel. PRMF does not scale to large kernel sizes as the
amount of computation and memory grows quadratically with
the kernel size.1 Nonetheless, for extremely small kernel sizes,
the PRMF algorithms can outperform the algorithm presented
in this paper.
B. Historgram Based Algorithms
Sorting based median filtering implementations are not
always the most efficient, especially for larger kernels due
to a quadratic increase in the computational complexity.
Recently, several papers showed efficient median filtering that
use histogram-based operations rather than sorting. These
operations, if sequenced in the correct order and with the right
auxiliary data structures, avoid doing many redundant compu-
tations. Consider two adjacent pixels in an image, the filtering
windows for both these pixels contain significant overlapping
values, yet due to the nature of the sorting algorithms the
elements in both windows are sorted from scratch. The works
by Perreault and Hébert [8] and Weiss [30] avoid these waste-
ful computations by using the overlapping sections and by
accessing the pixels in a desired order. The algorithm by Weiss
uses a hierarchy of histograms for finding the median value.
III. M EDIAN F ILTERING IN C ONSTANT T IME [8]
Perreault and Hébert [8] present a median filtering algo-
rithm with a reduced time complexity that uses histogram-
based operation and accesses each pixel a constant number
of times, O(1). This approach is quite different from other
sorting based median filtering approaches [18], [31], [32] and
also tends to reduce the number of memory accesses. Perreault
and Hébert [8] use an array of histograms for counting pixel
value appearances. This is different than the approach taken
in the classic paper by Huang et al. [7] which uses a single
histogram for storing the state of the kernel. In fact, the column
histograms helps to greatly reduce the number of accesses to
the individual pixels.
Perreault and Hébert [8] point out that each pixel is needed
for O(k 2 ) unique kernels (except for pixels on the image bor-
1 From a practical standpoint the generated code requires a very large number
of registers, and limits its scalability on the GPU.
 GREEN: EFFICIENT SCALABLE MEDIAN FILTERING USING HISTOGRAM-BASED OPERATIONS
Algorithm 1 Pseudo Code of the Serial Algorithm by Perreault
and Hébert [8]
ders that are also used by fewer kernels). By maintaining a his-
togram for each column of the image, the number of times that
a pixel is accessed is reduced to O(k), similar to the bounds
given by Huang et al. [7]. Further, they show that due to
overlap of adjacent kernels, the number of times that the pixel
needs to accessed can be reduced to O(1) times. This is espe-
cially important for mid-size and large kernels where k > 7.
From a theoretical standpoint, the pixels of an image can
be filtered in a random order. However, by using a prede-
fined access pattern, optimizations such as the ones given by
Huang et al. [7] and by Perreault and Hébert [8] become
viable. Additional benefits of this access pattern is memory
pre-fetching, cache reuse, and deterministic execution. The
following describes the access pattern. The algorithm starts
at the top-left corner of the image, where the first filtering
kernel (window) is placed. This kernel is then moved one
column at a time to the right until it reaches the last column.
After the last column is processed, the kernel is then moved
back at the left most column and moves one row down.
This is repeated until all the rows have been accessed in a
contiguous fashion. Simplified pseudo-code of Perreault and
Hébert’s median filtering algorithm can be found in Alg. 1.
Table I depicts the symbols used in the pseudo-code and in
the remainder of the paper.
A. Data Structure
Perreault and Hébert [8] suggest the following data struc-
tures for their fast implementation. For each column in the
input image, there is a histogram for the pixels of that given
column. This is maintained only for one row at a time in
the images. In addition to the column-histogram, there is one
master histogram, denoted as H , that is a summation of all the
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
2219
TABLE I
S YMBOLS AND T ERMINOLOGY U SED IN T HIS W ORK
histograms in the kernel and is used for finding the median
value of a given pixel.
1) Storage Complexity: Given an input image with c
columns and histogram of depth h, a total of (c+1)·h memory
bytes are required to store both the column histograms and
the master histograms. The histogram depth, h, is dictated by
the number of bytes needed to represent an image. Typically,
h = 2b where b = log2 k 2 , is the log of the maximal times a
number can appear in a bin. In comparison to Huang et al. [7],
the memory requirements increases by a factor of c. However,
this storage increase is affordable on most systems.
B. Algorithmic Details
With the above data-structures, Perreault and Hébert [8]
show how to reduce the number of access each pixel from
O(k) to O(1) using histogram based operations. In this paper,
we only offer the intuition on why this is correct and refer the
reader to [8] for formal proofs and additional details.
When a kernel moves to the right by one position from its
current position, it needs to remove the left most column from
the master histogram as it is no longer needed. In addition, one
column histogram needs to be added to the master histogram.
Note that this column histogram already contains k values
from the previous row. Of these, k − 1 values are relevant for
the current kernel. Thus, by removing the value of the top most
pixel in the column from the column histogram and adding one
pixel, which will be the bottom most pixel in the column, the
column histogram is modified such that it represents correctly
the right most column of the kernel.
C. Histogram-Based Operations and
Additional Optimizations
Perreault and Hébert [8] show that their algorithm bene-
fits from vector instructions. They use Intel’s SSE [33] and
AVX [34] vector instruction set. These vector operations are
ideal for speeding up the histogram-based operations. The
specific instructions that can be used from the SSE or AVX
instruction set depends on the depth of the image (typically
8-bits per pixel) and the size of the kernel. Given a kernel of
size k, the same value can be counted upto k 2 times. Thus, the
SIMD instructions need to take this into account. Perreault and
Hébert [8] use 16 bits to count instances. As such, the SIMD
instruction that they use are 8-elements wide.
Further, Perreault and Hébert [8] show an additional opti-
mization in their OpenCV implementation - a two level
 2220 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018

histogram hierarchy for efficient median value lookup. This Algorithm 2 Pseudo Code of the Partitioning and Initialization
simplifies the search for the median value in the master of the New Parallel Algorithm
histogram making it more efficient and time effective. The
two level histogram also benefits from vectorized operations.
We too have adopted this optimization for both the CPU and
GPU implementations.

IV. PARALLEL A LGORITHM

In this section, the new parallel algorithm for efficient
median filtering is given. This algorithm consists of two
main phases: 1) the work-load partitioning phase and 2) the
parallel median filtering phase. Any discussion of threads,
cores, or processors in this section is entirely independent
of the executing platform. Rather these terms are used to
highlight the parallel granularity of the algorithm. These two
parallel phases are important for both the CPU and GPU
implementations.

A. Workload Partitioning
Our parallel median filtering algorithm divides the image
into blocks of contiguous rows. This allows each processor to
work on a different section of the image. This is a simple par-
titioning scheme - divide the input image into non-overlapping
row segments. The segments are all roughly the same size. To
be precise, the number of rows in the input image is divided 
by P processors. Each processors receives either rows
 rows P or
P + 1 rows. This partitioning ensures that each processor
will do a near equal amount work, also ensuring good load-
balancing. There are two extreme cases worth noting, 1) P = 1
which leads to a sequential execution and 2) P = r ows which
means that is processor is responsible for processing one row.
The later might seem ideal as each processor will be able to
finish quickly due to little work assigned to it. In practice, due
to overheads this leads to low utilization.
Alg. 2 depicts the pseudo-code for partitioning and data- Algorithm 3 Parallel Median Filtering
structure initialization of our new algorithm. To ensure both
good
 rows workload balancing, each processor is given at least
P rows and at most rows
P + 1 rows. Each of these
row-segments is represented by two numbers, the star t Rowi
and the stop Rowi . star t Rowi represents the first row to be
processed, stop Rowi represents the last row to be processed,
and i represents the processor/core (i ∈ {0, .., P − 1}). This
partitioning ensures that all the rows are accounted for.

B. Initialization
Unlike the sequential algorithm of Perreault and Hébert [8]
where there is only one column histogram for the single
processor; our algorithm requires P data structures - one
for each processor. In the sequential algorithm, this column
histogram is initialized for the first row. In the parallel version,
each processor is responsible for initializing its own set
of column histograms. The real challenge with the parallel
initialization, is due to the pixels at the border of the image,
namely the first k/2 rows of the image. For these rows, the
kernel exceeds the boundary of the image. To overcome this
problem, the pixels in the first row of the input image are
mirrored. The exact number of mirrored values is dependent
on the kernel size and the starting row of the row-segment
given to the processor. For pixels in the first row, each pixel
is mirrored k/2 + 2 times, as stated in [8]. For pixels in
the second row, the number of mirrored values from the first
row k/2 + 1. For the third row, this goes down to k/2.
And so forth, all the way to the k-th row. In the sequential
algorithm, the mirroring is only done for the first row. Beyond,

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
 GREEN: EFFICIENT SCALABLE MEDIAN FILTERING USING HISTOGRAM-BASED OPERATIONS
Fig. 1. The algorithm initialization depends on three partitioning regions,
which are determined by the first row of the row-segments. The first row,
denoted by the blue box, is also the first region. The following region, denoted
by the green box,is from the second row and upto the k + 1 row. The number
of processors starting within this region depends on the number of processors
and the kernel size. The last region starts at the end of the green box and
goes to the end of the image.
the k-th row, mirroring is no longer necessary and the mirrored
results have been entirely removed from both the column
histograms and the master histogram.
In the initialization phase of the parallel algorithm, the
star t Row value is used to do decide on how to initialize
the column-histogram. The star t Row falls into one of these
zones, Fig. 1:
• star t Row = 0: similar to the sequential implementation.
Values of the pixels in first row are mirrored k + 2 times for
each of the column histograms.2 This is followed by scanning
k − 1 rows and updating the column histograms. Denoted by
the blue box at top Fig. 1.
• 1 ≤ star t Row ≤ k + 1: similar to the previous
scenario, except pixels are replicated fewer times and requiring
additional rows to be scanned. See Alg. 2 for the exact number
of replicates and scanned rows. Denoted by the green box in
Fig. 1.
• k + 2 ≤ star t Row: the entire kernel is within the bounds
of the image. No pixel mirroring is necessary, 2 · k + 1 rows
are scanned for the initialization. star t Row is below the green
box in Fig. 1.
C. Parallel Filtering
The filtering phase of the parallel algorithm is nearly
identical to that of the sequential implementation of
Perreault and Hébert [8]. The key difference between the
parallel and serial algorithms is that in the parallel algorithm
the processors are responsible for a smaller number of rows,
see Alg. 3. In practice, this can be problematic. Using a large
number of processors can lead to the fact that initialization
phase is more demanding than the filtering. Using a small
number of processors, to avoid the overhead, can lead to under-
utilization of the system. This is discussed in addition detail in
the Section VII. Lastly, we note that there are addition schemes
for parallelizing the filtering such that histogram operations for
2 See Perreault and Hébert [8] for additional details.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
2221
a single partition are also parallelized. For example, on the
GPU after the image is partitioned to the multi-processors,
the histogram operation is also parallelized using 32 threads
for each operation.
D. Complexity Analysis
1) Storage Complexity: Similar to the sequential algorithm,
we use h to refer to refer to the depth of the histogram and
c refers to the number of columns in the input. The storage
complexity of the parallel algorithm has one additional para-
meter, P the number of processor. Each processors requires
the same amount of memory as the sequential version (c+1)·h
bytes. As there are P processors, the total amount of storage
require by the parallel algorithm is P · (c + 1) · h bytes. Thus,
our approach requires P× more memory than the sequential
algorithm. Given the relatively large memory available in most
modern systems, the increase in the storage requirements is
more than reasonable. However, there are some cases where
this increase might be to much: 1) when number of processors
is extremely high, 2) the number of columns is extremely large,
or 3) the system has limited memory (typically ultra low-end
processors).
2) Time Complexity: The following notation denotes the
time complexity of the two main work-intensive phases, initial-
ization and filtering, Tinit and T f ilt . For the work complexity
we denote these as Winit and W f ilt . The time required by the
partitioning phase is minuscule and can be ignored. We note
the following:
• The time spent for the initialization can increase by
upto a factor of two. Whereas, the sequential algorithm
only needs k/2 − 1 rows for the initialization process,
the parallel algorithms requires k rows for the initialization.
Seq
As such, Tinit ≤ Tinit Par ≤ 2 · T Seq . As each processor is
init
responsible for initializing its own column histogram, the
amount of work required by the initialization is increased by
a factor of P and the two times overhead of the initialization,
Seq
Par
Winit = 2 · P · Winit .
• The work executed in the filtering phase is identical to
that of the sequential algorithm. This means that the work
complexity of this phase is identical to that of the sequential
Seq
ilt = W f ilt . As each processor is responsible for
algorithm, W fPar
a smaller set of rows, this time per processor is also reduced
Seq
T
ilt = P .
to T fPar
f ilt
Given the above, we offer the following theoretical time
speedup of the parallel algorithm over the sequential:
Seq Seq Seq Seq
Tinit + T f ilt Tinit + T f ilt
Speedup = Par + T Par
= Seq Seq
(1)
Tinit 2 · Tinit + T f ilt /P
f ilt
Note, that for the parallel algorithm’s initialization time, we
use the worst case scenario where the initialization takes twice
as long as it does for the sequential algorithm.
3) Theortical Perfomance Analysis: Using Eq. (1) we offer
the following insights on the scalability of our algorithm,
assuming large enough images3 :
3 For simplicity consider images of size 1024 × 1024
 2222
TABLE II
GPU AND CPU S YSTEMS U SED IN E XPERIMENTS
a) For a small number of processors P ≈ O(1) and a small
number kernel k, the speedup of our algorithm is almost linear.
Given that P is small, rowsp  k, means that most of the time
is spent filtering and a relatively small amount of time is spent
in the initialization.
b) For a small number of processors P ≈ O(1) and a mid-
size to large kernel k (such as k = 81), the speedup of our
algorithm is somewhat reduced. This leads to a situation where
p ≈ k. In all likelihood more time is spent in the filtering
rows
phase, however, the initialization phase becomes larger and is
no longer negligible.
c) For a large enough number of processors, P, the initial-
ization phase becomes a larger overhead. This also means that
for the large kernels, more time is spent in the initialization
phase than the time spent in the filtering phase.
Most modern day CPU processors fall into the first two sce-
narios where the number of processors is small. Accelerators
such as the GPU and Intel Xeon PHI, which have a have a
large processor count, fall into the third scenario.
V. E XPERIMENTAL S ETUP
A. Systems
The new algorithm is implemented for both the CPU and for
NVIDIA’s CUDA platform. The analysis of the new GPU algo-
rithm and the benchmarking of all the remaining GPU algo-
rithms are tested on an NVIDIA 750 GTX GPU - a low-end
GPU found in most desktops. The CPU implementations and
the other CPU based algorithms are tested on an Intel system.
Table II has additional details of both the CPU and GPU sys-
tems used in the experiments. The following sections discuss
in greater detail the parameter selection for getting good per-
formance for median filtering. This is especially important for
the GPU architecture which has a lot of parallelism available.
B. CUDA Supported GPUs
A single NVIDIA CUDA GPU consists of multiple stream-
ing multiprocessors known as SMs. Each SM is made up of a
larger number of streaming processors (SP). These are respon-
sible for executing a large number of threads concurrently.
Threads are executed in groups known as thread warps. Warps
are the smallest execution unit on the GPU. At the time of
writing, the warp size in current GPU technology is 32 threads.
While fewer than 32 threads can be allocated for a thread block
by the programmer, the hardware will still allocate 32 threads
and will not use the remaining threads. As such, it usually
preferable to find a way to use all 32 threads.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018
C. Library Comparison
We compare the performance of our new algorithm with
several of the most widely used implementations available:
NVIDIA’s Performance Primitives (NPP) [15], ArrayFire [16],
SciPy [17], PRMF [27], and Perreault and Hébert [8] algorithm
(part of the OpenCV [10] project). NPP is a closed library and
as such its source code is not available. The ArrayFire library
is one of the comprehensive open-source image processing
libraries available for the GPU. The CPU implementations
include the implementation found in SciPy [17] and that of
Perreault and Hébert [8] taken from the OpenCV [10] library.
Of the aforementioned implementations, only that by Perreault
and Hébert [8] uses histogram-based operations while the
others use sorting and comparisons to find the median. PRMF
[27] is a highly tuned median filtering algorithm for the
GPU that uses a comparison network. In PRMF, unnecessary
comparisons are pruned in each iteration of the comparison
network. While highly efficient for small networks, for mid-
size networks of 11×11 and above this network does not scale
due to a mix of computational requirements, excessive use of
registers, and increase size of its code base. The OpenCV
library contains two sequential implementations of Perreault
and Hébert [8] algorithm: a sequential version and a vectorized
version that uses Intel’s SSE instruction set. We use the vector-
ized algorithm as it is better performing. Table III summarizes
the libraries on which system that libraries implementation was
executed.
Our experiments used varying sizes images and kernels.
This includes the images found in Fig. 2: Barbara, Lena,
Cameraman, and Peppers. We used square images from
128 × 128 and upto 4096 × 4096 for performance analysis.
VI. CPU P ERFORMANCE A NALYSIS
The CPU implementation in OpenCV in uses two different
median filtering algorithms. For small window (3 × 3 and
5 × 5), OpenCV uses a sorting based implementation. For
mid-size and larger filter kernels, OpenCV uses the histogram
based algorithm of Perreault and Hébert [8]. These histogram-
based operations are implemented using Intel’s SSE instruction
set. In Fig. 3, the OpenCV implementation is denoted by the
C PU − Sequenti al − OpenC V curve. The remaining curves
denote the implementation of the new algorithm for the CPU.
The main difference between the new parallel implementation
and the sequential implementation are two new code sections:
1) the workload partitioning and 2) the initialization phases
which initializes the master histogram for each thread.
Fig. 3 depicts the performance of the new algorithm for
multiple image sizes. In each row of the figure different
performance metrics are presented. The top rows depicts the
 GREEN: EFFICIENT SCALABLE MEDIAN FILTERING USING HISTOGRAM-BASED OPERATIONS
TABLE III
L IBRARIES AND I MPLEMENTATIONS U SED IN O UR E XPERIMENTS
Fig. 2. Images used in experiments: (a) Barbara, (b) Lena, (c) Cameraman,
and (d) Peppers.
execution time, the middle row depicts the throughput rate of
the algorithm (in pixels per second), and lastly the bottom
row depicts the speedup of the algorithm in comparison to
a sequential execution. On the CPU, the execution time for
a given thread count, does not increase significantly with the
growth of the kernel size. This is true for all the thread counts
tested and matches the results of Perreault and Hébert [8]. For
these thread counts, the majority of the execution time is spent
in the filtering phase and not in the initialization phase (as will
be seen the next section).
As the the number of threads is increased on the CPU, the
performance increases in a nearly perfect linear fashion. For
two threads, the average speedup is 1.95×, for threads threads
it is 2.8×, and for four threads it is 3.7×. This low decrease in
performance is in part due to the overhead of the initialization
stage which. In comparison to the overhead of the initialization
on the GPU, this is not as high.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
2223
VII. GPU P ERFORMANCE A NALYSIS
Our new GPU algorithm has been integrated into OpenCV.
As such, the implementation adheres to the OpenCV standards
and data-types. Specifically, at the time of writing, OpenCV
supports two value-types of arrays for the GPU, 8-bit and
32 bit. An 8-bit value is limited to a max value of 255. Our
goal is to support larger kernel sizes, including kernels that are
greater in size than 17 × 17,4 thus the larger 32-bit data type
is used. This increase from 8-bits to 32-bits quadruples the
memory requirements and data movement. This also results
in a slower a slower execution time. On a positive note, this
implementation can be extended to support 16-bit grayscale
images (with some additional optimizations).
A. Performance Analysis of Different Implementations
Fig. 4 depicts the execution time (ordinate) of the various
implementations for different sizes of the Barbara image. The
abscissa represents the kernel size. Lower values imply better
performance. Fig. 7 compares the performance of the GPU
algorithm with the CPU algorithm. Further, this figure also
depicts the performance of the PRMF algorithm [27], one of
the best known GPU median filtering algorithms.
These implementations can be classified into main two
groups: 1) sorting and comparisons based implementations and
2) histogram-based operations implementations. The sorting
based implementations are: ArrayFire, NVIDIA’s NPP, SciPy,
and PRMF. The non-sorting based implementations are those
by Perreault and Hébert [8] and our new one. As the kernel
size increases, the execution time increases in a polyno-
mial fashion for the sorting algorithms whilst the execution
time of the histogram-based implementations is near constant
(i.e. independent of the kernel size). As such, the histogram-
based implementations can support larger kernels than the
sorting based implementations.
PRMF had some scalability issues and could not be tested
for all the kernel sizes. The PRMF algorithm, uses an explicit
laid out software comparison comparison network. This net-
works requires a quadratic number of registers and does not
scale beyond a relatively small kernel size. For an 11×11 ker-
nel size, compiling PRMF took over 5 hours.5 For the smaller
kernel sizes PRMF performs well. To capture the performance
of PRMF, we have placed execution time in Fig. 7. Note,
that we include two curves for PRMF: 1) execution time of
4 For this size, the maximal times that a pixel-value can appear is 172 . This
cannot be represented by an 8-bit integer.
5 The compilation was stopped after 5 hours.
 2224 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018

Fig. 3. Execution time (top row), throughput (middle row), and speedup (bottom row) of the histogram-based implementations on the CPU (within the
OpenCV library). Each subplot depicts a different image size, from 1024 × 1024 (on the left) and upto 4096 × 4096 (on the right). Note that the speedup is
nearly linear with number of threads. (a) Execution time - lower is better. (b) Throughput - higher is better. (c) Parallel Scalability - higher is better.

only the median filtering and 2) execution time that includes
overheads such as initialization, memory allocation, and data
transfer. The throughput measured in PRMF [27] measures
only the median filtering time. Our analysis will include both
these times. The motivation for separating these two curves is
discussed below. We also note the following:
• As the kernel size grows, the sorting based implementa-
tions do not perform well. The histogram-based implemen-
tations can be hundreds of times faster than sorting based
implementations. In addition, the memory requirements of the
sorting based implementations also hinder scalability as the
memory requirements increase in a polynomial fashion.
• For really small kernel sizes, such 3 × 3 and 5 × 5
the sorting based GPU implementations outperform both the
algorithm by Perreault and Hébert [8] and our new algorithm.
These tend to have improved cache performance and memory
requirements and lower computational requirements.
• Table IV breaks down the execution time of the new
algorithm and PRMF. As the kernel size increases for PRMF,
the execution time increases quadratically (as expected) and

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
 GREEN: EFFICIENT SCALABLE MEDIAN FILTERING USING HISTOGRAM-BASED OPERATIONS 2225

Fig. 4. Execution time of the various implementations for the Barbara image. The abscissa for each subplot is the kernel size and the ordinate is the execution
time. Each subplot shows the execution time for a different sized image, starting at 1024 × 1024 in the left and up-to to 4096 × 4096 in the bottom right
corner. Lower execution time is better. PRMF is not included due its limited scalability.

TABLE IV
E XECUTION T IME ( IN Milli-Seconds) B REAKDOWN OF THE N EW GPU A LGORITHM AND FOR THE PRMF A LGORITHM . N OTE T HAT THE T OTAL
E XECUTION T IME FOR PRMF IS D OMINATED BY THE A LLOCATION AND D ATA T RANSFER T IME . A LSO , N OTE T HAT THE T IME S PENT
F ILTERING FOR PRMF I NCREASES Q UADRATICALLY W ITH THE K ERNEL S IZE . I T IS N EAR C ONSTANT FOR THE
N EW GPU A LGORITHM . F OR E ACH A LGORITHM , W E S HOW THE T OTAL E XECUTION T IME AS W ELL THE T IME FOR
E ACH P HASE . N OTE , THE PRMF [27] H AS AN A DDITIONAL I NITIALIZATION P HASE FOR A DDING B ORDERS

more time is spent in executing the filtering versus the other
phases. Also note, the kernel-only execution time curve slowly
converges to the total execution time curve as the kernel grows.
For an 11 × 11 filter, PRMF would perform on the same
order of magnitude as the new algorithm. Lastly, note that
the allocation and initialization time for PRMF is slower than
the new algorithm because PRMF needs to setup boundaries
to the input image to stay within bounds. This time is included
in the overall execution time as it is a necessary step of the
algorithm.
B. GPU Performance Analysis
Lastly, we discuss the impact of GPU parameter selection
for the overall performance of the algorithm. The program-
ming model for NVIDIA’s CUDA [35] supported GPUs typ-
ically requires having a large number of threads available
for execution. Usually, the number of software threads is
about six to tens times the number of hardware threads (SPs).
In hardware, threads are grouped into units of 32 threads
known as a warp. The GPU used in our experiment has 4 SMs
and 512 SPs, which means that it can execute 16 warps
concurrently. Our algorithm uses thread blocks of a warp
size. Each of these thread-blocks can be thought of as a
processor in the CPU execution. Instead of using an exact
number of processors, as we did in the CPU implementation,
we overstate the exact number of processors and instead use
virtual processors, P̂, such that P̂ = ĉ · P. P in this case is
the maximal number of thread-blocks executed concurrently.
Using the aforementioned rule of thumb, the best selection of
ĉ will be in {6, 8, 10}. Perhaps not surprisingly, we see that
the best performance is for P̂ = 8 · 16 = 128.
Fig. 5 depicts the execution time of our algorithm for
different values of P̂ for different sizes of the Barbara image.
Each curve depicts a different number of partitions that the
image is divided into. Recall from Eq. 1 - there are two
main components that dictate the execution time: initialization
time and filtering time. On the one hand, a small number of
partitions risks the chance that the GPU will be underutilized.
On the other hand, a large number of partitions ensures that the
GPU has enough work units to dispatch to the SM processors;
however, this can cause the initialization to become a dominant
factor - leading to an increased execution time. Both of these
partitioning side effects can be seen in Fig. 5. Note that for the
smallest number of partitions, P̂ = 32, the execution is always
the slowest as the GPU is underutilized. When the number of
partitions is increased to P̂ = 224 the overhead dominates the
total execution time. For most of the images, image sizes, and
kernel sizes we saw that dividing the image to 128 partitions
offered the best performance for the NVIDIA GTX 750. This
offers the best balance between low system utilization and the
cost of a long initialization.
C. Additional Images
Fig. 6 depicts the execution time of our new algorithm
for the various images used in our benchmark, see Fig. 2.
There is a roughly 10% different in execution time of the

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
 2226 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018

Fig. 5. Execution time for the new algorithm as a function of the number of partitions used for the parallelization scheme. The abscissa for each subplot
is the kernel size while the ordinate is the execution time. Each subplot depicts a different image size, from 1024 × 1024 and upto 4096 × 4096. Each curve
represents a different number of partitions (or processors) used by our new algorithm.

Fig. 6. Execution time of the histogram-based implementations for different images. The abscissa for each subplot is the kernel size while the ordinate is
the execution time. Each subplot depicts a different image size, from 1024 × 1024 and upto 4096 × 4096.

Fig. 7. Execution time (top row) and throughput (bottom row) of the histogram-based implementations for the Barbara image. The abscissa for each subplot
is the kernel size while the ordinate is the execution time. Each subplot depicts a different image size, from 1024 × 1024 and upto 4096 × 4096.

various images. This can be accounted to the different memory images can cause memory hot-spots as multiple threads try
access patterns, which can also change due the pixel values. to update the same column histogram. Overall, this is a small
For example, the difference in the pixels of the different performance deviation and is expected. Lastly, note that the

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
 GREEN: EFFICIENT SCALABLE MEDIAN FILTERING USING HISTOGRAM-BASED OPERATIONS
execution time as a function of the kernel size grows at
a relatively moderate rate and the difference between the
fastest execution time (for the smallest kernel) and the slowest
execution time (for the largest kernel) as roughly 25%. This
increase is due to the overhead of the initialization phase,
which requires more work as kernel size grows.
D. Comparison of CPU and GPU Histogram-Based
Implementations
Fig. 7 depicts the performance of the histogram-based
implementations, Perreault and Hébert [8] sequential algorithm
for the CPU and our new GPU algorithm. Also in this plot
are two curves for the PRMF algorithm: 1) execution time
for the entire algorithm including the memory allocation,
initialization, and data transfer, and 2) the execution time
for only the filtering. For the sake of brevity we do not
repeat the analysis of PRMF and refer the reader back to
VII-A; though we do remind the reader that the current
implementation of PRMF has some overhead in the initial-
ization phase due to some padding of the image. The top row
shows the execution time for these algorithms as a function
of the kernel size and the bottom row shows the throughput
of these algorithms. Note for the histogram operation based
implementations (CPU and GPU), the execution time increases
with the kernel size. The increase is slight for the CPU
algorithm, while it is more apparent for the new parallel
GPU implementation where a large kernel size can become
a significant part of the execution, especially for large thread
counts. This is due to additional overhead introduced by the
parallel algorithm. Note, the sequential CPU implementation
is quite efficient and in fact outperforms the GPU algorithm.
It is also worth noting that the CPU has a significantly larger
cache. This allows the CPU to store larger parts of the image
as well as most of the auxiliary data structures. The GPUs
smaller L1 and L2 caches are not able to hold all the entire
auxiliary data structures, especially when the number of virtual
processors is set to 128. While the GPU offers a lot of parallel
scalability, it is clear that the algorithms overheads are limiting
the performance. Nonetheless, the new GPU implementation
is significantly faster than several leading implementations and
is the first histogram-based algorithm for the GPU.
VIII. C ONCLUSIONS
This paper showed the first software parallel and scalable
algorithm for median filtering that is based on histogram oper-
ations. Unlike sorting based and comparison based algorithms
(whose computation tends to grow at a quadratic rate with the
window size), the new algorithm is almost entirely indepen-
dent of the window. This results in an efficient algorithm that
is high performing for mid-size windows and upwards. The
new algorithm is implemented both for the CPU and GPU.
Both implementations use a multi-level histogram hierarchy
for the data management - enabling efficient implementations.
We show that these histogram based operations can also be
implemented efficiently on the GPU using NVIDIA’s CUDA
programming model. Our GPU implementation is open-source
and be found in OpenCV (https://github.com/opencv/opencv).
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
2227
For mid-size to large filtering windows the new algorithm is
well over an order of magnitude faster than several leading
implementations. For example, for an 11x11 filtering window,
the new algorithms is over 60x faster then the fastest GPU
implementation (found in NVIDIA’s NPP library) and hun-
dreds of times faster than other sorting based implementations
(CPU and GPU). Lastly, a parallel CPU implementation of
the new algorithm was presented. This implementation extends
one of the fastest known sequential algorithms for the CPU,
namely the algorithm by Perreault and Hébert [8]. The new
parallel implementation for the CPU shows good scalability,
achieving a 3.7× speedup on a quad-core system.
R EFERENCES
[1] T. Chen, K.-K. Ma, and L.-H. Chen, “Tri-state median filter for image
denoising,” IEEE Trans. Image Process., vol. 8, no. 12, pp. 1834–1838,
Dec. 1999.
[2] J. Fitch, E. Coyle, and N. Gallagher, “Median filtering by thresh-
old decomposition,” IEEE Trans. Acoust., Speech, Signal Process.,
vol. ASSP-32, no. 6, pp. 1183–1188, Dec. 1984.
[3] A. C. Bovik and D. C. Munson, Jr., “Edge detection using median
comparisons,” Comput. Vis., Graph., Image Process., vol. 33, no. 3,
pp. 377–389, 1986.
[4] A. C. Bovik, T. S. Huang, and D. C. Munson, Jr., “The effect of median
filtering on edge estimation and detection,” IEEE Trans. Pattern Anal.
Mach. Intell., vol. PAMI-9, no. 2, pp. 181–194, Mar. 1987.
[5] S.-C. S. Cheung and C. Kamath, “Robust techniques for
background subtraction in urban traffic video,” Proc. SPIE,
vol. 5308, pp. 881–892, Jan. 2004. [Online]. Available:
https://www.spiedigitallibrary.org/conference-proceedings-of-
spie/5308/1/Robust-techniques-for-background-subtraction-in-urban-
traffic-video/10.1117/12.526886.pdf?SSO=1
[6] H. Rantanen, M. Karlsson, P. Pohjala, and S. Kalli, “Color video signal
processing with median filters,” IEEE Trans. Consum. Electron., vol. 38,
no. 3, pp. 157–161, Aug. 1992.
[7] T. Huang, G. Yang, and G. Tang, “A fast two-dimensional median
filtering algorithm,” IEEE Trans. Acoust., Speech, Signal Process.,
vol. ASSP-27, no. 1, pp. 13–18, Feb. 1979.
[8] S. Perreault and P. Hébert, “Median filtering in constant time,” IEEE
Trans. Image Process., vol. 16, no. 9, pp. 2389–2394, Sep. 2007.
[9] O. Green and Y. Birk, “A computationally efficient algorithm for the 2D
covariance method,” in Proc. Int. Conf. High Perform. Comput., Netw.,
Storage Anal. (SC), 2013, Art. no. 89.
[10] G. Bradski, “The OpenCV library,” Doctor Doobs J., vol. 25, no. 11,
pp. 120–126, 2000.
[11] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color
images,” in Proc. 6th Int. Conf. Comput. Vis., 1998, pp. 839–846.
[12] M. Mahmoudi and G. Sapiro, “Fast image and video denoising via
nonlocal means of similar neighborhoods,” IEEE Signal Process. Lett.,
vol. 12, no. 12, pp. 839–842, Dec. 2005.
[13] J. van de Weijer and R. van den Boomgaard, “Local mode filtering,” in
Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR),
vol. 2. Dec. 2001, pp. II-428–II-433.
[14] K. Ito and K. Xiong, “Gaussian filters for nonlinear filtering prob-
lems,” IEEE Trans. Autom. Control, vol. 45, no. 5, pp. 910–927,
May 2000.
[15] NVIDIA Performance Primitives (NPP), NPP NVIDIA, Santa Clara, CA,
USA, Feb. 2011.
[16] J. Malcolm, P. Yalamanchili, C. McClanahan, V. Venugopalakrishnan,
K. Patel, and J. Melonakos, “ArrayFire: A GPU acceleration plat-
form,” Proc. SPIE, vol. 8403, pp. 8403-1–8403-8, 2012. doi:
10.1117/12.921122.
[17] E. Jones, T. Oliphant, and P. Peterson, “SciPy: Open source scientific
tools for Python,” Tech. Rep., 2014.
[18] J. Ragan-Kelley, C. Barnes, A. Adams, S. Paris, F. Durand, and
S. Amarasinghe, “Halide: A language and compiler for optimizing
parallelism, locality, and recomputation in image processing pipelines,”
ACM SIGPLAN Notices, vol. 48, no. 6, pp. 519–530, 2013.
[19] K. Benkrid, D. Crookes, and A. Benkrid, “Design and implementation
of a novel algorithm for general purpose median filtering on fpgas,”
in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), vol. 4. May 2002,
pp. IV-425–IV-428.
 2228
[20] Y. Hu and H. Ji, “Research on image median filtering algorithm and its
FPGA implementation,” in Proc. WRI Global Congr. Intell. Syst. (GCIS),
vol. 3. May 2009, pp. 226–230.
[21] S. A. Fahmy, P. Y. K. Cheung, and W. Luk, “Novel FPGA-based
implementation of median and weighted median filters for image
processing,” in Proc. Int. Conf. Field Programm. Logic Appl., Aug. 2005,
pp. 142–147.
[22] K. Batcher, “Sorting Networks and Their Applications,” in Proc. Spring
Joint Comput. Conf., 1968, pp. 307–314.
[23] W. Chen, M. Beister, Y. Kyriakou, and M. Kachelrieß, “High perfor-
mance median filtering using commodity graphics hardware,” in Proc.
IEEE Nucl. Sci. Symp. Conf. Rec. (NSS/MIC), Oct. 2009, pp. 4142–4147.
[24] M. Kachelrieß, “Branchless vectorized median filtering,” in Proc. IEEE
Nucl. Sci. Symp. Conf. Rec. (NSS/MIC), Oct. 2009, pp. 4099–4105.
[25] R. M. Sánchez and P. A. Rodríguez, “Bidimensional median filter for
parallel computing architectures,” in Proc. IEEE Int. Conf. Acoust.,
Speech Signal Process. (ICASSP), Mar. 2012, pp. 1549–1552.
[26] R. M. Sánchez and P. A. Rodríguez, “Highly parallelable bidimensional
median filter for modern parallel programming models,” J. Signal
Process. Syst., vol. 71, no. 3, pp. 221–235, 2013.
[27] G. Perrot, S. Domas, and R. Couturier, “Fine-tuned high-speed imple-
mentation of a GPU-based median filter,” J. Signal Process. Syst.,
vol. 75, no. 3, pp. 185–190, 2014.
[28] O. Green, “When merging and branch predictors collide,” in Proc. IEEE
4th Workshop Irregular Appl., Archit. Algorithms, Nov. 2014, pp. 33–40.
[29] T. P. Ryan, Modern Engineering Statistics. Hoboken, NJ, USA: Wiley,
2007.
[30] B. Weiss, “Fast median and bilateral filtering,” ACM Trans. Graph.,
vol. 25, no. 3, pp. 519–526, 2006.
[31] S. van der Walt, S. C. Colbert, and G. Varoquaux, “The NumPy array:
A structure for efficient numerical computation,” Comput. Sci. Eng.,
vol. 13, no. 2, pp. 22–30, 2011.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY CALICUT. Downloaded on March 09,2020 at 04:52:02 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 5, MAY 2018
[32] J. Gil and M. Werman, “Computing 2-D min, median, and max filters,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 5, pp. 504–507,
May 1993.
[33] A. Klimovitski, “Using SSE and SSE2: Misconceptions and reality,”
Intel Develop. Update Mag., pp. 3–8, 2001.
[34] N. Firasta, M. Buxton, P. Jinbo, K. Nasri, and S. Kuo, “Intel AVX: New
frontiers in performance improvements and energy efficiency,” Intel,
Santa Clara, CA, USA, White Paper 19, 2008, p. 20.
[35] NVIDIA CUDA Programming Guide, NVIDIA, Santa Clara, CA, USA,
2011.
Oded Green received the B.Sc. degree in com-
puter engineering and the M.Sc. degree in electrical
engineering from the Technion–Israel Institute of
Technology, and the Ph.D. degree from the School
of Computational Science and Engineering, Georgia
Institute of Technology. He is currently a Research
Scientist with the School of Computational Science
and Engineering, Georgia Institute of Technology.
His research focuses on improving performance and
increasing scalability for large-scale data analytics
using a wide range of high-performance computing
platforms.