B.N.M.

Institute of Technology
Department of Computer Science and Engineering

AN END-TO-END TRAINING BASED APPROACH FOR CAPTIONING COMPLEX

IMAGES USING NEURAL NETWORKS

Team Members :
1. Shreyas V Kashyap - 1BG13CS097
2. Swaroop Gupta B.A - 1BG13CS113
3. Spandana Rao B.A - 1BG13CS103

Guide - Smt. Jebah Jaykumar

Description :
Our ability to effortlessly describe all aspects of an image relies on a strong
semantic understanding of a visual scene and all of its elements. However, despite
numerous potential applications, this ability remains a challenge for our state of the art
visual recognition systems.

To address this problem, we introduce the complex captioning task, which

requires a computer vision system to both localize and describe salient regions in
images in natural language. For localization, the Dense Captioning task generalizes
prediction of object detection when the description consist of a single word, and the
Image Captioning describes the image when one whole predicted region covers the full
image.

This will be accomplished by using an architecture is composed of a

Convolutional Network, an efficient dense localization layer, and Recurrent Neural
Network language model that generates the label sequences for the complex images.

Guide Signature