Human pose estimation
Using Part affinity field and confidence maps
Abstract ----Here we represent the approach
of 2D human pose estimation by using part
affinity field and confidence map. Part
eedling
affinity is a parameter representation which
means to learn to associate body parts of
individuals from given images.
Using greedy bottom-up parsing step we
maintain high accuracy while achieving real
-me performance.we design a architecture
which jointly learn part locations and their
association via two branches of the same
sequential prediction process irrespective of
the number of people in the image.
I.INTRODUCTION
Human pose estimation is a problem of
localizing human body parts of an
individual. Finding out body parts of an
individual in a image full of other human
beings is a great challenge to overcome.Here
we divide the problems into three category.
First , each image may contain an unknown
number of people that can occur at any
position or scale. Second, interactions
between people induce complex spatial
interference, due to contact, occlusion, and
limb articulations, making association of
parts difficult. Third, run-time complexity
tends to grow with the number of people in
image,making real-time performance a
challenge.
One of the common approaches is to use
body detector and perform single person
pose estimation from each detection. In this
we use top down approach. This approach is
only useful if there is only person or, in
multi-person case, people are in certain
distance apart. This method will give error
while identifying the body parts
individually. Top-down approaches is
proportional to the number of people: for
each detection, a single-person pose
estimator is run, and the more people there
are, the greater the computational cost.
However, Bottom up approach seems more
suitable for pose estimation of several
people in a single image. This approach is
used because it offer robustness to early
commitment and have the potential to
decouple runtime complexity from the
number of people in the image.
In this project, we present the first bottom-
up representation of association scores via
Part Affinity Fields (PAFs), a set of 2D
vector fields that encode the location and
orientation of limbs over the image domain.
We demonstrate that simultaneously
inferring these bottom-up representations of
detection and association encode global
context sufficiently well to allow a greedy
parse to achieve high-quality results, at a
fraction of the computational cost. We use
the code from the Github repository .
 III. METHODS
A. Datasets
We used COCO dataset for object
detection, segmentation, person key points
detection, stuff segmentation, and caption
generation.
The system takes, as input, a color image of
size w × h and produces, as output, the 2D
locations of anatomical keypoints for each
person in the image.
The above image shows Architecture of the
two-branch multi-stage CNN. Each stage in
the first branch predicts confidence maps St,
and each stage in the second branch predicts
PAFs Lt. After each stage, the predictions
from the two branches, along with the image
features, are concatenated for next stage.
. affinity shows the orientation of the joints. For
Graph matching. (a) Original image with part
detections (b) K-partite graph (c) Tree structure (d) A
set of bipartite graphs
We first obtain a set of body part detection
candidates for multiple people. These part
detection candidates still need to be
associated with other parts from the same
person—in other words, we need to find the
pairs of part detections that are in fact
connected limbs. We define a variable to
indicate whether two detection candidates
are connected, and the goal is to find the
optimal assignment for the set of all possible
connections. If we consider a single pair of
parts j1 and j2 (e.g., neck and right hip) for
the c-th limb, finding the optimal association
reduces to a maximum weight bipartite
graph matching problem.
The above flow diagram shows the whole
process of the system to identify key points
of a person. The image is the input of this
project. The image (size w*h) is first analyzed by
a convolution network (initialized by the first 10
layers of VGG-19 model) and produces a set of
feature map. These feature maps are the input
of confidence map and part affinity fields.
Confidence map shows the joints where as part
each joints of each person we have single
confidence map and part affinity field. Part
Confidence Maps, a set of 2D confidence
maps S for body part locations. Each joint
location has a map. Part Affinity Fields (PAFs), a
set of 2D vector fields L which encodes the
degree of association between parts. Finally,
the Confidence Maps and Part Affinity Fields are
processed by a greedy algorithm to obtain the
poses for each person in the image.
B. The Architecture
The architecture of the network we used
for training is depicted in Figure 4. Our
model consists of only seven learned layers
– five convolutional layers and followed by
two fully connected.
Figure 2. (Left side) N-joint rigid kinematic
skeleton model.(Right side) Shape-based body
model.
C. Implementation
Figure 2 depicts the design of the setup
we employed for training. For coming up
with the deep neural network prototype,
images are resized into 64 x 64 x 1
GRAYSCALE images. It comprises seven
learned layers – five convolutions and two
fully connected.
network classifier model is then used for
validation.
V. CONCLUSION
The project has high runtime and when the
collision of the people is maximum then it
create an issue to separate the joints from
individual. For instance, in the dataset of a
couple dancing salsa style the collision of body
parts are maximum which means it will be
difficult to calculate the part affinity fields as
the limbs are almost closed to limbs of other
person.
To reduce the high runtime problem we use
Greedy bottom up parsing of multiple people.
And to overcome the second issues, i.e.
maximum collision problem, we can use several
cameras to get accurate co ordinates of the
joints and limbs from all the possible direction.
Greedy algorithm makes the optimal choice at
each step as it attempts to find the overall
optimal way to solve the entire problem. In this,
after the part affinity fields is identified, Greedy
algorithm uses optimal way to join the joint so
that the joints are of same person. Here bottom
up parse helps to first join the joints and at the
end combine all the joints to form a complete
body.

REFERENCES
1. Realtime Multi-Person 2D Pose
Estimation using Part Affinity
Fields.
https://openaccess.thecvf.com/conten
t_cvpr_2017/papers/Cao_Realtime_
Multi-
Person_2D_CVPR_2017_paper.pdf
2. http://image-
net.org/challenges/talks/2016/Multi-
person%20pose%20estimation-
CMU.pdf
3. Githubrepository:
https://github.com/ZheC/Realtime_
Multi-Person_Pose_Estimation
4. https://journals.sagepub.com/doi/abs/
10.1177/1747954119879350
5. https://ieeexplore.ieee.org/abstract/do
cument/6691503
12. https://openaccess.thecvf.com/content
_cvpr_2013/papers/Dantone_Human_P
ose_Estimation_2013_CVPR_paper.pdf
13. https://sci-
hub.tw/https://link.springer.com/chapt
er/10.1007/978-3-319-46475-6_16
14. http://arno.uvt.nl/show.cgi?fid=148438
15. https://sci-
hub.tw/https://www.tandfonline.com/
doi/abs/10.1080/02640414.2018.15217
69
16. https://github.com/ZheC/Realtime_Mul
ti-Person_Pose_Estimation.git
17. https://towardsdatascience.com/cvpr-
2017-openpose-realtime-multi-person-
2d-pose-estimation-using-part-affinity-
fields-f2ce18d720e8

6. https://sci-hub.tw/10.1007/978-3-030-
17274-9

7. https://arxiv.org/abs/1611.08050

8. https://www.xyonix.com/blog/using-ai-
to-improve-sports-performance-amp-
achieve-better-running-efficiency

9. https://mc.ai/psg-sports-analytics-
challenge-how-deep-learning-can-be-
used-in-a-football-context/

10. https://sci-
hub.tw/10.1109/ICAIIT.2019.8834602

11. https://link.springer.com/chapter/10.1
007/978-3-319-46478-7_44