A PROJECT REPORT ON

“SUSPICIOUS ACTIVITY DETECTING USING VIDEO

SURVEILLANCE”
SUBMITTED TO THE MSBTE, MUMBAI

IN PARTIAL FULFILLMENT OF THE

REQUIREMENTS FOR THE AWARD

DIPLOMA IN COMPUTER TECHNOLOGY

By
Patil Shravani Dhiraj
Parjane Shruti Dattatray
Painjane Shruti Sunil
Parakh Tanishka Rupesh

Under the guidance of

Mr. V. S. Dhande

DEPARTMENT OF COMPUTER TECHNOLOGY,

SANJIVANI RURAL EDUCATION SOCIETY
S. K. P. POLYTECHNIC, KOPARGAON
DIST. AHMEDNAGAR-423601
 (AC. YR. 2022-23)
Department of Computer Technology

S. K. B. P. Polytechnic, Kopargaon

CERTIFICATE
This Is to Certify That Project Entitled

“SUSPICIOUS ACTIVITY DETECTING USING

VIDEO SURVEILLANCE”
Submitted By
Patil Shravani Dhiraj
Parjane Shruti Dattatray
Painjane Shruti Sunil
Parakh Tanishka Rupesh

Is a bonafide work carried out by above students under the supervision of Prof. G. N.
Jorvekar and it is submitted towards the partial fulfilment of the requirement of
MSBTE, Mumbai for the award of Diploma in Computer Technology.

Prof. V. S. Dhande Prof. V. A. Parjane

Project Guide Project Coordinator

Prof. G. N. Jorvekar Prof. A. R. Mirikar

H.O.D. Principal
Date:
Place:

MAHARASHTRA STATE BOARD OF TECHNICAL EDUCATION,

MUMBAI

EXAMINERS CERTIFICATE

“SUSPICIOUS ACTIVITY DETECTING USING VIDEO

SURVEILLANCE”

Submitted By
Patil Shravani Dhiraj
Parjane Shruti Dattatray
Painjane Shruti Sunil
Parakh Tanishka Rupesh

for the partial fulfillment of the requirement for the Diploma in Computer Technology is
examined affiliated to M.S.B.T.E.

Internal Examiner External Examiner

( ) ( )
 ACKNOWLEDGEMENT

I would like to acknowledge all the people who have motivated and helped me
throughout my dissertation. We are thankful to the college, S. K. B. P.
Polytechnic, Kopargaon, for allowing us to perform the final year project as a part
of fulfillment for the Diploma in Computer Technology. We would like to thank
our internal Guide Mr. V. S. Dhande for providing his valuable assistance
throughout the project. We would also like to thank Prof. G.N. Jorvekar (HOD
comp Department) for providing facilities and resources for the implementation
of the project. Finally, we would like to thank our colleagues and lab assistants
who encouraged and supported us in developing the idea and approach to the
implementation of our project.

Miss. Patil Shravani Dhiraj

Miss. Parjane Shruti Dattatray
Miss. Painjane Shruti Sunil
Miss. Parakh Tanishka Rupesh
ABSTRACT
With the rise in criminal activity in urban and suburban areas, it is more important
than ever to prevent them and detect them. Because it is nearly impossible for
humans to keep an eye on these surveillance cameras all the time. It necessitates
a workforce and their constant attention in order to determine whether the
captured activities are unusual or suspicious. The most recent research aims to
incorporate computer vision, image processing, and artificial intelligence into
video surveillance applications. Even though there have been many advances in
the acquisition of datasets, methods, and frameworks, there are few papers that
provide a comprehensive picture of the current state of video surveillance
systems. This paper provides a Deep learning algorithm of suspicious activity
detection. This algorithm displays which frame and part of unusual activity,
which aids in the quicker assessment of that unusual action as unusual or
suspicious. The goal is to detect signs of aggression and violence in real time,
allowing irregularities to be distinguished from normal patterns by utilizing deep
learning models in identification or classification high movement frames, where
we can set off a detection alert in the event of a threat, alerting us to suspicious
activity at a specific point in time.

KEYWORDS: video surveillance, CNN (convolutional neural network),

LSTM (long short term memory network), suspicious activity, feature
extraction.
CONTENTS
1. Introduction
1.1Problem Statement
1.2Goals and Objectives
1.3Scope of the work
1.4Outcomes
2. Requirement Analysis
2.1System Requirement
2.2Software Requirement
2.3Hardware Requirement
3. Literature Survey
3.1Existing Methods/Techniques
3.2Literature Survey
4. Proposed System
4.1System Architecture
4.2Data Flow Diagram
4.3Activity Diagram
5. Testing and Analysis
5.1Introduction
5.2
6. Applications and Source Code
6.1Applications
6.2Source Code
7. Result & Discussion
7.1Results
7.2Discussion
8. Conclusion & Future Work
8.1Conclusion
8.2Future Work
9. Appendix
10.References and Bibliography
10.1References
10.2Bibliography

List of Figures
4.1 System Architecture
4.2 Data Flow Diagram
4.3 Activity Diagram
CHAPTER 1
INTRODUCTION
Now a day’s human behaviour and activity pattern research are more
important in surveillance. Detection and tracking the object of behaviour is
important factor in video surveillance system. Over a last decade it has been seen
the rapid growth and an extraordinary improvement in real time video analysis.
Main goal of video analytics is to identify the potential threaten events with less
or no human intervention. Video surveillance is a prominent area of research
which includes recognition of human activities and categorization of them into
usual, unusual, or suspicious activities. The primary goal is to find unexpected
events in videos using a surveillance system that might be human, semi-
automatic, or fully automated. Humans are completely reliant on the manual
surveillance system. Analyzing behaviour or distinguishing abnormal from
regular conduct required physical labour. Semiautomatic systems require less
human interaction, whereas fully automatic video surveillance systems do not
require human intervention to make decisions. Face recognition is another
approach for detecting intrusions. A criminal dataset is built and saved in the
system. Python's OpenCV package is used to recognize faces. This recognition
procedure involves internal picture processing and deep learning. The system
becomes more accurate as a result of such advanced technology.

1.1 PROBLEM STATEMENT

For activity-based analysis, activity detection is a critical component of video
surveillance systems. These operators keep an eye on numerous displays at once,
looking for unusual activity. This is a tedious and ineffective method of
monitoring. Because humans are likely to make mistakes, this method is
inefficient. Multiple screens cannot be monitored by a human operator at the same
time. As a result, obtaining timely and reliable activity data becomes extremely
challenging. This is why we require an automated method, and the suspicious
activity detection system has provided us with the ideal option. Suspicious
activity detection systems based on video can either replace or assist human
operators in monitoring odd behaviour. They get a quick and precise answer from
the system.
1.2 GOALS & OBJECTIVES
• The main objective of this project is to design and implement suspicious
activity detection using video surveillance.
• To design a strong security system for public places.
• The primary goal is to find unexpected events in videos using a
surveillance system.
• The objective of the system is prevention of theft, trespass, criminal
damage, anti-social behaviour or many other crime-related activities.
• The video detected by the system will be useful as strong evidence in future
for case study.

1.3 SCOPE OF WORK

• The aim of the project is to build a strong & effective security system.

1.4 OUTCOMES
• No Human intervention.
CHAPTER 2
REQUIREMENT ANALYSIS
2.1 SYSTEM REQUIREMENT
▪ Laptop
▪ Web Browser
▪ Internet Connection

2.2 SOFTWARE REQUIREMENT

▪ Python (Jupyter Notebook, Numpy)
▪ Visual Studio
▪ Django
▪ HTML (Javascript)

2.3 HARDWARE REQUIREMENT

▪ 8GB RAM
▪ 20GB Storage
CHAPTER 3
LITERATURE SURVEY
3.1 EXISTING METHODS/TECHNIQUES
The earliest system is not so fast enough to track the suspicious activity. In
existing system the victim has to press the alarm button to alert the staff and
surrounding people, then the police would arrive at the investigation place. The
people used to get a frightened to press the alarm button. For recognizing human
behavior from movies, the majority of the publications described above used
computer vision employing various algorithms or neural networks. To extract the
motion pattern and interpret the evolution of characteristics in a video series,
computer vision technologies require a lot of pre-processing. In addition,
background removal is based on a static backdrop hypothesis that is frequently
inapplicable in real-world circumstances. The majority of problems in the actual
world arise in crowds. When it comes to crowd management, the approaches
mentioned above are inefficient. Based on the literature review, a deep
architecture for suspicious activity prediction can be modelled using Camera,
improving the system's accuracy. In deep learning approach, most of the papers
detect only the suspicious activity. So, an efficient mechanism is needed to alert
the security in the case of any suspicious behaviour.

3.2 LITERATURE SURVEY

1. We studied a paper by Piyush Kakkar, Mr. Vibhor Sharma, International
Journal of Advance Research in Computer Engineering (IJARCCE): Criminal
Identification System Using Face Detection and Recognition in which they
proposed a model that can detect guns in still frames(images), the demerit was
the model was less effective in low light conditions and angel of the face not
properly inclined towards the camera.
2. We also studied a paper by Rajesh Kumar Tripati, Anand Singh Dalal, Int. J.
of Innovative Science and Eng. Feb 2017: Suspicious Human Activity
Recognition. The objective of that paper was to provide the literature review
of six different suspicious activity recognition systems with its general
framework to the researchers of this field.
CHAPTER 4
PROPOSED SYSTEM
4.1 SYSTEM ARCHITECTURE

In our project we proposed a system that is video based framework that identifies
normal and abnormal activities and tells our security system if the detected
activity is suspicious. We proposed a video based framework that identifies
criminal activities happening in the surrounding such as ATMs, banks, jewellery
shops, etc. A video consists of certain images, that are called as frames, a video
consists of multiple frames. We split the video into its images i.e., frames and we
pass each image to an image classifier and then we get the prediction of the
activity detected in the video. So, it checks the each frame activity in the video.
In this system we are using a combination of CNN (Convolutional Neural
Network) and LSTM (Long Short Term Memory network). Convolutional neural
networks are great are working with image data and image classification. LSTM
are great at working with sequences of data. But when we combine both of them
together, we can do video classification. Long short term memory network is a
sequence based data. We take a bunch of input samples that are connected through
a time. If we consider a video containing 30 frames, we input a sequence of data
and generate output sequence data. If we take video, split it into frames and use
a convolutional neural network to get an extra visual features out of it and fill
those visual features into long short term memory network and then get
predictions out of it. The prediction is in the form of percentage of normal or
abnormal activities and confidence about the activity detected.
4.2 DATA FLOW DIAGRAM

Input Data Pre-

Segmentation
Processing
(Video)

Background/
Foreground
Detection

Feature
Training and
Output Extraction
Testing

4.3 ACTIVITY DIAGRAM

CHAPTER 5
TESTING & ANALYSIS
CHAPTER 6
APPLICATIONS & SOURCE CODE
6.1 APPLICATIONS

6.2 SOURCE CODE

index.html
{% block content %}
<style>
body{
background-image: url('https://wallpaperaccess.com/full/1627619.jpg');
background-repeat: no-repeat;
background-size: cover;
color: white;
display: flex;
justify-content: center;
align-items: center;
}
.main{
display: flex;
flex-direction: column;
padding: 10px;
width: 50%;
height: 80vh;
background-color: rgb(6 29 55 / 58%);
border-radius: 10px;
}
h1{
text-align: center;
}
.form_data{
display: flex;
flex-direction: column;
align-items: center;
height: 65vh;
width: 100%;
}
p{
display: flex;
align-items: center;
width: 90%;
padding: 10px;
margin: 20px;
background-color: rgb(47 47 143);
border-radius: 5px;
}
.video-container{
margin: 10px;
width: 70% ;
height: 40vh;
border: 5px solid;
color: #000000c2;;
background-color: #ffffff96;
border-radius: 5px;
}
button{
padding: 10px;
width: 300px;
height: 50px;
background-color: #05091b;
color: white;
border-radius: 5px;
}
button:hover{
background-color:#4f0e2fcb ;
}
.loading {
opacity: 0.5;
pointer-events: none;
}
</style>
<body>
<div class="main">
<h1 class="header">Suspecious Activity Detection</h1>
<form id="my-form" class="form_data" method="post" enctype="multipart/form-data">
{% csrf_token %}
{{ form.as_p }}
<div id="video-container" class="video-container"></div>
<button type="submit">Analyse</button>
<div id="loading-indicator" style="display:none;">Analysing...</div>
</form>
</div>
</body>
<script>
const inputElement = document.querySelector('input[type="file"]');
inputElement.addEventListener('change', handleFiles, false);
function handleFiles() {
const fileList = this.files;
if (!fileList.length) return;
const file = fileList[0];
const video = document.createElement('video');
video.src = URL.createObjectURL(file);
video.controls = true;
video.style.width = '100%'; // set width to 100%
video.style.height = '100%'; // keep aspect ratio
const container = document.querySelector('#video-container');
container.appendChild(video);
const form = document.querySelector('#my-form');
const loadingIndicator = document.querySelector('#loading-indicator');
form.addEventListener('submit', function() {
// Show the loading indicator
loadingIndicator.style.display = 'block';
// Disable the form while loading
form.classList.add('loading');
});
}
</script>
{% endblock %}

result.html
<style>
body{
color: white;
display: flex;
flex-direction: row;
background-image: url('https://wallpaperaccess.com/full/1627626.jpg');
background-repeat: no-repeat;
background-size: cover;
align-items: center;
justify-content: center;
}
.vidtab{
display: flex;
flex-direction: column;
justify-content: center;
width: 50%;
background-color: #12102982;
height: 70vh;
border-radius: 10px;
margin: 20px;
padding: 10px;
align-items: center;
}
.main{
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
margin: 10px;
padding: 10px;
width: 30%;
height: 75vh;
}
.player{
height: 80%;
width: 80%;
margin: 10px;
padding: 20px;
}
.pre{
width: 100%;
height: 50%;
margin: 10px;
padding: 10px;
display: flex;
flex-direction: column;
border-radius: 10px;
align-items: center;
justify-content: center;
background-color: #21080885;
}
</style>
<body>
<div class="vidtab">
<h1>Video Detection</h1>
<video class="player" controls>
<source src="{{video_path}}post.mp4" type="video/mp4"/>
</video><br>
</div>
<div class="main">
<span class="pre"><h3>Activity Detection</h3><h1>{{Prediction}}</h1></span>
<span class="pre"><h3>Confidence</h3><h1>{{Confidence}}</h1></span>
</div>
</body>

apps.py
from django.apps import AppConfig
class SuspeciousActivityConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "suspecious_activity"

forms.py
from django import forms
from.models import Video
class VideoForm(forms.ModelForm):
class Meta:
model = Video
fields = ('video_file',)

model_detection.py
from keras.models import load_model
import numpy as np
import cv2
from collections import deque
LRCN_model = load_model(r'C:\Users\DELL\OneDrive\Documents\Suspecious activity
detection\app\suspecious_activity\model\LRCN_model_Accuracy_0.8659999966621399.h5')
IMAGE_HEIGHT , IMAGE_WIDTH = 64, 64
CLASSES_LIST=['Normal','Suspecious']
def predict_on_video(video_file_path, output_file_path, SEQUENCE_LENGTH):'''
This function will perform action recognition on a video using the LRCN model.
Args:
video_file_path: The path of the video stored in the disk on which the action recognition is to be performed.
output_file_path: The path where the ouput video with the predicted action being performed overlayed will be
stored.
SEQUENCE_LENGTH: The fixed number of frames of a video that can be passed to the model as one sequence.'''
#Initialize the VideoCapture object to read from the video file.
video_reader = cv2.VideoCapture(video_file_path)
#Get the width and height of the video.
original_video_width = int(video_reader.get(cv2.CAP_PROP_FRAME_WIDTH))
original_video_height = int(video_reader.get(cv2.CAP_PROP_FRAME_HEIGHT))
#Initialize the VideoWriter Object to store the output video in the disk.
video_writer = cv2.VideoWriter(output_file_path, cv2.VideoWriter_fourcc(*'H264'),
video_reader.get(cv2.CAP_PROP_FPS), (original_video_width, original_video_height))
#Declare a queue to store video frames.
frames_queue = deque(maxlen = SEQUENCE_LENGTH)
#Initialize a variable to store the predicted action being performed in the video.
predicted_class_name = ''
#Iterate until the video is accessed successfully.
while video_reader.isOpened():
#Read the frame.
ok, frame = video_reader.read()
#Check if frame is not read properly then break the loop.
if not ok:
break
#Resize the Frame to fixed Dimensions.
resized_frame = cv2.resize(frame, (IMAGE_HEIGHT, IMAGE_WIDTH))
#Normalize the resized frame by dividing it with 255 so that each pixel value then lies between 0 and 1.
normalized_frame = resized_frame / 255
#Appending the pre-processed frame into the frames list.
frames_queue.append(normalized_frame)
#Check if the number of frames in the queue are equal to the fixed sequence length.
if len(frames_queue) == SEQUENCE_LENGTH:
#Pass the normalized frames to the model and get the predicted probabilities.
predicted_labels_probabilities = LRCN_model.predict(np.expand_dims(frames_queue, axis = 0))[0]
#Get the index of class with highest probability.
predicted_label = np.argmax(predicted_labels_probabilities)
#Get the class name using the retrieved index.
predicted_class_name = CLASSES_LIST[predicted_label]
#Write predicted class name on top of the frame.
cv2.putText(frame, predicted_class_name, (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
#Write The frame into the disk using the VideoWriter Object.
video_writer.write(frame)
#Release the VideoCapture and VideoWriter objects.
video_reader.release()
video_writer.release()
def predict_single_action(video_file_path, SEQUENCE_LENGTH):
#Initialize the VideoCapture object to read from the video file.
video_reader = cv2.VideoCapture(video_file_path)
# Get the width and height of the video.
original_video_width = int(video_reader.get(cv2.CAP_PROP_FRAME_WIDTH))
original_video_height = int(video_reader.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Declare a list to store video frames we will extract.
frames_list = []
# Initialize a variable to store the predicted action being performed in the video.
predicted_class_name = ''
# Get the number of frames in the video.
video_frames_count = int(video_reader.get(cv2.CAP_PROP_FRAME_COUNT))
# Calculate the interval after which frames will be added to the list.
skip_frames_window = max(int(video_frames_count/SEQUENCE_LENGTH),1)
# Iterating the number of times equal to the fixed length of sequence.
for frame_counter in range(SEQUENCE_LENGTH):
# Set the current frame position of the video.
video_reader.set(cv2.CAP_PROP_POS_FRAMES, frame_counter * skip_frames_window)
# Read a frame.
success, frame = video_reader.read()
# Check if frame is not read properly then break the loop.
if not success:
break
# Resize the Frame to fixed Dimensions.
resized_frame = cv2.resize(frame, (IMAGE_HEIGHT, IMAGE_WIDTH))
# Normalize the resized frame by dividing it with 255 so that each pixel value then lies between 0 and 1.
normalized_frame = resized_frame / 255
# Appending the pre-processed frame into the frames list
frames_list.append(normalized_frame)
# Passing the pre-processed frames to the model and get the predicted probabilities.
predicted_labels_probabilities = LRCN_model.predict(np.expand_dims(frames_list, axis = 0))[0]
# Get the index of class with highest probability.
predicted_label = np.argmax(predicted_labels_probabilities)
# Get the class name using the retrieved index.
predicted_class_name = CLASSES_LIST[predicted_label]
# Display the predicted action along with the prediction confidence.
#print(f'Action Predicted: {predicted_class_name}\nConfidence:
{predicted_labels_probabilities[predicted_label]}')
# Release the VideoCapture object.
video_reader.release()
return predicted_labels_probabilities,predicted_label,predicted_class_name

models.py
from django.db import models
# Create your models here.
class Video(models.Model):
video_file = models.FileField(upload_to='videos/')
tests.py
from django.test import TestCase
# Create your tests here.

urls.py
from django.contrib import admin
from django.urls import path
from.import views
from django.conf import settings
from django.conf.urls.static import static
urlpatterns = [
path('',views.upload_video,name='index'),
]
if settings.DEBUG: urlpatterns+=static(settings.MEDIA_URL,document_root=settings.MEDIA_ROOT)

views.py
from django.shortcuts import render, redirect
from .models import Video
from .forms import VideoForm
from .model_detection import predict_single_action,predict_on_video
from django.conf import settings
SEQUENCE_LENGTH=20
output_path=r'C:\Users\DELL\OneDrive\Documents\Suspecious activity detection\app\media\post.mp4'
outputs="\outputs\post.mp4"
def upload_video(request):
if request.method == 'POST':
form = VideoForm(request.POST, request.FILES)
if form.is_valid():
video = form.save()
input_video = video.video_file.path
plp,pl,pcn=predict_single_action(input_video,SEQUENCE_LENGTH)
videos=predict_on_video(input_video,output_path,SEQUENCE_LENGTH)
print(settings.MEDIA_URL)
param={'Confidence':round(plp[pl]*100,2),
'Prediction':pcn,
'video_path':settings.MEDIA_URL,}
return render(request, 'result.html',param)
else:
form = VideoForm()
context={'form':form}
return render(request, 'index.html', context)

manage.py
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "app.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if _name_ == "_main_":
main()
CHAPTER 7
RESULTS & DISCUSSION
7.1 Results

7.2 Discussion
CHAPTER 8
CONCLUSION & FUTURE WORK
8.1 CONCLUSION
The research suggests employing a convolutional neural network for feature
extraction and a discriminative deep belief network for action classification to
detect suspicious behaviour from surveillance video. By using a deep-learning-
based model, the suggested approach achieves better categorization than earlier
efforts. To begin, we divided video into frame segments and used CNN to extract
features from the background and foreground. The output is then input into a
trained DDBN, which classifies the recognized behaviours as normal or
suspicious. The deep learning model guarantees more precision and fewer false
positives.

8.2 FUTURE SCOPE

As part of the project’s future work, our research advises experimenting with
other structures and comparing them in order to enhance speedier detections. Due
to a lack of time and resources, we were only able to complete the research to the
level described in this report, allowing for more research into how to improve the
identification of suspicious activities in real time. Other features, other than
surveillance films, could be used to improve real-time detections. The method to
detect suspicious activities described in this paper does not address the issue of
explosives detection. As a result, more research on this topic can be done to
advance the field.
CHAPTER 9
APPENDIX
CERTIFICATES
CHAPTER 10
REFERENCES & BIBLIGRAPHY
10.1 REFERENCES
https://www.electronicsforu.com
https://www.researchgate.net

10.2 BIBLIGRAPHY