M.A.

M College of Enginnering And Technology

Siruganur,Trichy-621105

Draw In Air Using Machine Learning

Submitted by
Om Prakash P - 812021205035
Pandi Mugundesh A - 812021205036
Prabu K L - 812021205038

ANNA UNIVERSITY
CHENNAI

FEBRUARY 2023

CS3491 – Artificial Intelligence and Machine Learning

1
 CONTENT
Chapter 1
1.1 Abstract …………………………………………………………….. 03
Chapter 2
2.1 Introduction …………………………………………………………. 04
Chapter 3
3.1 Proposed System ……………………………………………………. 06
3.2 System Requirements ……………………………………………...... 07
3.3 System Architecture ……………………………………………….... 08
Chapter 4
4.1 Implementation ……………………………………………………... 09
4.1.1 NumPy ………………………………………………………. 11
4.1.2 OpenCV ……………………………………………………... 12
4.1.3 Features …………………………………………………….... 13
4.1.4 Advantages …………………………………………………... 14
Chapter 5
5.1 Algorithm …………………………………………………………… 15
5.1.1 Applications …………………………………………………. 16
Chapter 6
6.1 Conclusion .…………………………………………………………. 18
6.1.1 Future enhancement ………………………………………..... 18
Chapter 7
7.1 Source Code ……………………………………………………….... 19
7.2 Output screenshots ………………………………………………….. 22
Chapter 8
8.1 References …………………………………………………………... 23

2
 CHAPTER 1
1.1 Abstract
Air Canvas is developed for communicating a concept or presenting an
idea in the real-world space. Air canvas makes all easy to the user. With AIR-
CANVAS we present the thought of the increased reality material for data
representation. This is past the customary void (white),rectangular, and level
dimensional material seen in customary data representation. The venture
proposed in a Python 3.6 and it utilizes the exceptionally well known OpenCV
library. The Draw in Air project is an exciting and innovative technology that
brings a new dimension to the art of drawing. Using a digital pen or stylus,
users are able to create drawings in the air that are displayed in a 3D space,
creating an immersive and interactive experience for both the artist and the
viewer.

The Draw in Air project also offers a unique and engaging viewing
experience for audiences.. By bridging the gap between the digital and physical
worlds, it offers a new form of artistic expression that is not limited by
traditional materials or techniques. This has the potential to inspire artists to
think outside the box and push the boundaries of what is possible in the world
of art.The Draw in Air project is an exciting and innovative technology that has
the potential to transform the art world and change the way we think about
drawing and creativity. By bridging the gap between the digital and physical
world, the Draw in Air project offers a new form of artistic expression that has
the potential to revolutionize the way we think about drawing and art.

3
 CHAPTER 2
2.1 Introduction
At any point needed to draw your creative mind simply by deferring your
finger in air. PC Vision is an interdisciplinary logical field that arrangements
with how PCs can be made to acquire significant level comprehension by
utilization of numerous computerized ways. In this undertaking we will
fabricate an Air Canvas which can draw anything on it simply by catching the
movement of a finger with Webcam. We will utilize the PC vision procedures
of OpenCV to fabricate this task. The favored language is python because of its
comprehensive libraries and simple to utilize linguistic structure however
understanding the fundamentals it very well may be carried out in any OpenCV
upheld language. Here Color Detection and following is utilized to accomplish
the target.

A. Air Canvas:- Air Canvas is a without hands computerized drawing material

that uses, a camera and OpenCV to perceive and plan hand signals onto a
screen.

B. Contour:- Shapes may be clarified simply as a bend becoming a member of

all of the consistent points(alongside the limit), having equal tone or power. The
forms are a valuable instrument for shape analysis and item location and
granting.

The Draw in Air project is a revolutionary new technology that brings

the art of drawing to a new dimension. With the use of advanced augmented
reality and computer vision technologies, this project allows users to create
drawings in mid-air using a digital pen or stylus. The result is a unique and
engaging experience that is sure to captivate audiences of all ages.The
technology behind the Draw in Air project is designed to be intuitive and easy
to use. By tracking the movement of the digital pen in real-time, the device
4
translates the artist's movements into a digital representation that is displayed in
a 3D space. This creates an immersive experience for both the artist and the
viewer, as the artwork is displayed in a real-world environment and can be
explored from multiple angles and perspectives.

In addition to its entertainment value, the Draw in Air project has the
potential to change the way we think about art and creativity. By breaking down
the traditional boundaries of what is possible with a traditional pen and paper
setup, the Draw in Air project offers a new form of artistic expression that is not
limited by traditional materials or techniques. This has the potential to inspire
artists to think outside the box and push the boundaries of what is possible in
the world of art.

Whether you are an artist looking for a new platform for expression or a
viewer seeking a unique and engaging experience, the Draw in Air project is
sure to captivate and inspire. With its cutting-edge technology and limitless
potential for artistic expression, the Draw in Air project is a project that is sure
to have a profound impact on the world of art for many years to come.

The Draw in Air project is not just about creating art in a new way, but
also about exploring new forms of digital interaction and communication. With
the ability to create drawings in mid-air, the project opens up exciting new
possibilities for collaboration, play, and education.

5
 CHAPTER 3
3.1 Proposed system
In this proposed framework, going to utilize webcam and show
unit(monitor screen). Here, will be utilizing pen or hand for drawing attractive
images in front of the camera then we will attract those images, it will be shown
on the presentation unit. Our mounted framework is suit for decoding time-
collection pace boom alerts into extensive thing vectors. Users can make use of
the pen to compose digits or make hand motions and so on may be proven at the
presentation unit.

6
3.2 System Requirements

Hardware Requirement

7
Laptop or PC
1. Webcam: A webcam is a video cam that accepts or transfers a picture or
video progressively to or through a PC organization, like the Internet.

2. Display Unit (Screen): Means "Visual Display Unit." A VDU shows

pictures created by a PC or other electronic gadget. The term VDU is
regularly utilized interchangeably with "screen," however it can likewise
allude to one more kind of show, like a computerized projector.

Software Requirement

Laptop or PC
 Open Cv libraries
 Numpy
 Python

3.3 System Architecture

8
CHAPTER 4

9
4.1 Implementation

The implementation of a draw in air project typically involves the use of

several modules, including image processing, data analysis, user interface
design, hardware communication, and algorithm development. The following
are some of the most common modules used in a draw in air project.

Key Words – OpenCV, Contour, Tracking, NumPy

Image Processing: Image processing is a crucial component in a draw in

air project, as it is used to process the images captured by the system. The
images are processed to extract features and information that can be used to
perform gesture recognition and other tasks. Some of the most commonly used
image processing libraries in Python include OpenCV and scikit-image. These
libraries provide functions for tasks such as image filtering, thresholding, and
edge detection.

Data Analysis: Data analysis is used to analyze the data generated by the draw
in air system, such as the movement patterns of the user's hand. This
information can be used to gain insights into how users interact with the system,
and to identify areas for improvement. The most commonly used data analysis
library in Python is Pandas, which provides functions for working with data
frames, as well as for performing tasks such as data manipulation, aggregation,
and visualization.

User Interface Design: A user-friendly interface is essential for a draw in air

project, as it allows users to interact with the system in an intuitive and user-
friendly way. Some of the most commonly used libraries for user interface
design in Python include PyQt and Tkinter, which provide functions for creating
graphical user interfaces. These libraries can be used to create a variety of user
interfaces, from simple command line interfaces to more complex graphical
interfaces.

10
Hardware Communication: In a draw in air project, it is necessary to
communicate with the sensors and actuators used to capture images and provide
feedback to the user. Libraries such as PySerial can be used to communicate
with hardware devices in Python, allowing the system to receive data from the
sensors and control the actuators. This enables the system to interact with the
physical world, allowing users to experience a more immersive and interactive
experience.

Algorithm Development: Algorithm development is an important component

of a draw in air project, as it is used to develop algorithms for tasks such as
gesture recognition, image classification, and image generation. Some of the
most commonly used machine learning libraries in Python include TensorFlow
and SciPy, which provide functions for tasks such as linear regression, decision
trees, and neural networks. These algorithms can be used to provide more
advanced functionality to the draw in air system, such as recognizing specific
gestures or generating images based on the user's input.

A draw in air project requires the use of several modules, including

image processing, data analysis, user interface design, hardware
communication, and algorithm development. By using these modules in
combination, it is possible to create a draw in air system that is capable of
processing images, recognizing gestures, providing feedback to the user, and
much more. Whether you're developing a simple prototype or a more complex
system, these modules can help you get the job done.

11
4.1.1 Numpy
NumPy is a powerful module for scientific computing in Python, which
can be useful for "draw in air" projects. Here are a few ways NumPy could be
used:

Signal processing: NumPy includes functions for signal processing, which

could be useful for filtering and analysing sensor data from "draw in air"
devices.

Mathematical operations: NumPy includes a wide variety of mathematical

functions, such as trigonometric functions and linear algebra operations. These
could be useful for performing calculations on sensor data or for manipulating
digital drawings.

Array manipulation: NumPy provides powerful tools for manipulating arrays

of data. This could be useful for creating and manipulating digital images or for
transforming sensor data into useful formats.

Overall, while NumPy may not be a core component of a "draw in air" project,
it can be a useful tool for data analysis and manipulation that can help enhance
the capabilities of the project.

12
4.1.2 Open CV
OpenCV is an open-source computer vision library that can be useful for
"draw in air" projects. Here are a few ways OpenCV could be used:

Object detection: OpenCV includes algorithms for object detection, which

could be useful for detecting hand movements and gestures in "draw in air"
projects.

Image processing: OpenCV includes a wide variety of image processing

functions, such as filtering, thresholding, and edge detection. These could be
useful for processing sensor data from "draw in air" devices or for manipulating
digital images.

Contour detection: OpenCV includes functions for detecting contours in

digital images, which could be useful for detecting and tracking hand
movements in "draw in air" projects.

Optical flow: OpenCV includes algorithms for computing optical flow, which
could be useful for tracking the motion of a hand in real-time and using it to
create more accurate digital drawings.

Augmented reality: OpenCV can be used for augmented reality applications,

such as overlaying digital images onto real-world objects. This could be useful
for creating interactive "draw in air" experiences.

Overall, OpenCV can be a powerful tool for enhancing the capabilities of

"draw in air" projects, especially when it comes to processing sensor data and
creating digital images. By incorporating OpenCV into a "draw in air" project,
developers can enable more accurate and precise hand tracking, create more
advanced drawing tools and interfaces, and even integrate augmented reality
features.

13
4.1.3 Features

Hand gesture recognition: The system is designed to recognize specific hand

gestures, such as those used to draw lines, circles, or text. This enables users to
create precise and accurate drawings.

Real-time feedback: As the user draws in mid-air, the system provides real-
time feedback to help them adjust their hand movements and create the desired
drawing.

Customization options: The system may include various customization

options, such as the ability to change the color, thickness, or shape of the
drawing tool.

Multi-user support: The system may be designed to support multiple users

simultaneously, allowing them to collaborate on a drawing or create their own
individual drawings.

Digital sharing: Once the drawing is complete, the system may enable users to
share it digitally, either through social media or other online platforms.

14
4.1.4 Advantages

Intuitive and engaging: The project provides an intuitive and engaging way for
users to create digital drawings, which can be particularly appealing for people
who may find traditional drawing tools or software too complex or challenging.
Innovative and unique: The project offers a unique and innovative way to
create digital drawings, which can set it apart from other drawing tools or
software.
Flexibility and customization: The project may include various customization
options, such as the ability to change the color, thickness, or shape of the
drawing tool, which can provide users with more flexibility and control over
their drawings.
Real-time feedback: The project provides real-time feedback as the user draws
in mid-air, which can help them adjust their hand movements and create the
desired drawing more accurately.
Collaboration and sharing: The project can be designed to support multiple
users simultaneously, allowing them to collaborate on a drawing or create their
own individual drawings. Once the drawing is complete, the system may enable
users to share it digitally, either through social media or other online platforms.

15
 CHAPTER 5

5.1 Algorithm

The algorithm for a draw in air project typically involves several stages,
including image capture, pre-processing, feature extraction, classification, and
post-processing. The following is a detailed explanation of each stage in the
algorithm.

1. Image capture
2. Pre – processing
3. Feature extraction
4. Classification
5. Post processing

1. Image Capture: The first stage in the algorithm is to capture images of

the user's hand movements. This is typically done using a camera or other
image capturing device, such as a depth sensor. The images are captured
at a high frame rate, typically around 30-60 frames per second, to ensure
that the hand movements are captured accurately.

2. Pre-processing: Once the images have been captured, they are pre-
processed to remove any noise or artifacts that may affect the accuracy of
the algorithm. This may involve tasks such as image denoising,
normalization, and thresholding. Pre-processing is an important step, as it

16
 helps to ensure that the images are of high quality and contain only the
relevant information needed for the subsequent stages of the algorithm.

3. Feature Extraction: The next step is to extract relevant features from the
pre-processed images. This may involve tasks such as edge detection,
blob detection, and contour detection. The features extracted from the
images are used to represent the hand movements, and are typically
stored as numerical data that can be easily processed by the algorithm.

4. Classification: The next stage is to classify the hand movements based

on the features extracted from the images. This is typically done using a
machine learning algorithm, such as a decision tree, support vector
machine, or neural network. The algorithm is trained on a dataset of hand
movements, and is used to make predictions about the user's hand
movements based on the features extracted from the images.

5. Post-processing: The final stage of the algorithm is post-processing,

which involves using the results of the classification stage to perform
tasks such as gesture recognition, object tracking, or feedback generation.
This may involve tasks such as smoothing the results to reduce any
jaggedness or noise, or calculating the distance between the hand
movements and the target location.

17
5.1.1 Applications

Art and design: The project can be used by artists and designers to create
digital drawings and prototypes quickly and easily. It can be particularly useful
for creating 3D sketches or models.

Education: The project can be used in educational settings, such as classrooms

or museums, to engage students in learning about art, design, or technology.

Entertainment: The project can be used in entertainment settings, such as

amusement parks or interactive exhibits, to create interactive and immersive
experiences for visitors.

Advertising and marketing: The project can be used by advertisers and

marketers to create interactive and engaging advertisements, such as interactive
billboards or social media campaigns.

Medical and rehabilitation: The project can be used in medical and

rehabilitation settings, such as physical therapy or occupational therapy, to help
patients improve their hand-eye coordination and fine motor skills.

18
 CHAPTER 6

6.1 Conclusion
In conclusion, the draw in air project is a promising and exciting
technology that has the potential to revolutionize the way we interact with
digital devices. By allowing users to control and manipulate digital objects
simply by moving their hands in the air, the draw in air project offers a more
intuitive and natural way to interact with technology. This technology has
numerous potential applications, including in gaming, virtual and augmented
reality, and even medical and rehabilitation applications.

Despite its potential, there are also several challenges that need to be
addressed before the draw in air project can be widely adopted. One of the main
challenges is the accuracy and reliability of the technology, as even small
inaccuracies in hand tracking can result in significant problems in the final
output. Additionally, there is also the challenge of making the technology
accessible and usable for people with disabilities, who may have difficulty using
traditional computer interfaces.

6.1.1 Future enhancement

i. More advanced sensors

ii. Improved gesture recognition
iii. Haptic feedback
iv. Virtual reality integration
v. Collaboration features

19
 vi. Integration with other devices and software

CHAPTER 7
7.1 Source Code:

import cv2
import numpy as np
ix, iy, k = 200, 200, -1
def mouse(event, x, y, flags, param):
global ix, iy, k
if event == cv2.EVENT_LBUTTONDOWN:
ix, iy = x, y
k=1
cv2.namedWindow(“draw”)
cv2.setMouseCallback(“draw”, mouse)
cap = cv2.VideoCapture(0)
while True:
_, frm = cap.read()
frm = cv2.flip(frm, 1)
cv2.imshow(“draw”, frm)
if cv2.waitKey(1) == 27 or k == 1:
old_gray = cv2.cvtColor(frm, cv2.COLOR_BGR2GRAY)
mask = np.zeros_like(frm)
break
cv2.destroyAllWindows()
20
old_pts = np.array([[ix, iy]], dtype=np.float32).reshape(-1,1,2)
color = (0,255,0)
c=0
while True:
_, new_frm = cap.read()
new_frm = cv2.flip(new_frm, 1)
new_gray = cv2.cvtColor(new_frm ,cv2.COLOR_BGR2GRAY)
new_pts,status,err = cv2.calcOpticalFlowPyrLK(old_gray,
new_gray,
old_pts,
None, maxLevel=1,
criteria=(cv2.TERM_CRITERIA_EPS |
cv2.TERM_CRITERIA_COUNT, 15, 0.08))
key = cv2.waitKey(1)
if key == ord(‘e’):
mask = np.zeros_like(new_frm)
elif key == ord(‘c’):
color = (0,0,0)
lst = list(color)
c+=1
lst[c%3] = 255
color = tuple(lst)
elif key == ord(‘g’):
pass
else:
for i, j in zip(old_pts, new_pts):
x,y = j.ravel()
a, b = i.ravel()

21
 cv2.line(mask, (int(a),int(b)), (int(x), int(y)), color, 15)
cv2.circle(new_frm, (int(x),int(y)), 3, (255,255,0), 2)
new_frm = cv2.addWeighted(new_frm ,0.8, mask, 0.2, 0.1)
cv2.imshow(“”, new_frm)
cv2.imshow(“drawing”, mask)
old_gray = new_gray.copy()
old_pts = new_pts.reshape(-1,1,2)
if key == 27:
break
cv2.destroyAllWindows()
cap.release()

22
7.2 Output Screenshots

23
 CHAPTER 8

8.1 References

1. L. A. Gatys, A. S. Ecker, and M. Bethge. “Image Style Transfer Using

Convolutional Neural Networks”. June 2016.

2. Mark Gross and Ellen Do. “Ambiguous Intentions: a Paperlike

Interface”. In: (May 2019).

3. K. Kim et al., “Real Time Foreground-Background Segmentation Using

Code Book Model,” Real-Time Imaging, vol. 11, no.3, 2005, pp. 172–
185

4. https://www.microsoft.com/en-us/research/publication/air-sketch/

5. https://makezine.com/2017/09/28/draw-air-arduino-powered-wireless-
3d-sensing-pen/

6. https://makezine.com/2017/09/28/draw-air-arduino-powered-wireless-
3d-sensing-pen/

24