FACE RECOGNITION SYSTEM

Human face recognition has drawn considerable attention from the researchers in recent years. An automatic face recognition system will find many applications in areas such as humancomputer interfaces, model-based video coding and security control systems. In addition, face recognition has the potential of being a non-intrusive form of biometric identification. The difficulties of face recognition lie in the inherent variability arising from face characteristics (age, gender and race), geometry (distance and viewpoint), image quality (resolution, illumination, signal to noise ratio), and image content (background, occlusion and disguise). Because of such complexity, most face recognition systems to date assume a well-controlled environment and recognise only near frontal faces. However, these constraints need to be relaxed in practice. Also, in applications such as video database search, a persons face can appear in arbitrary backgrounds with unknown size and orientation. Thus there is a need for robust face recognition systems to handle these uncertainties. In the seminar report we shall see different techniques for face recognition and algorithms followed by these technologies.

Also we shall see their advantages, disadvantages and solutions to ameliorate the problems, applications and different vendors. CH. 1 INTRODUCTION

People have amazing ability to recognize and remember thousands of faces. Face is an important part of who you are and how people identify you. While humans have had the innate ability to recognize and distinguish faces for millions of years, computers are just catching up. Face recognition is a fascinating problem with important commercial applications such as mug shot matching, crowd surveillance & witness face reconstruction. In computer vision most of the popular face recognition algorithms have been biologically motivated. Using these models researchers can quantify the similarity between faces, images whose projections are close in face space are likely to be from the same individual. Compare results of these models with human perception to determine whether distance in face space corresponds to the human notion of facial similarity. Biometrics is used for that purpose.

1.1

What is biometrics?

2. It is non intrusive and requires no physical interaction on behalf of the user. A biometric is a unique,measureable characteristic of a human being that can be used to automatically recognize an indiviual or verify an individuals identity. Biometrics can measure both physiological and behavioral characteristics. Physiological biometrics (based on measurements and data derived from direct measurement of a part of human) include :: 1. finger scan 2. facial recognition 3. iris scan 4. retina scan 5. hand scan. Behavioral biometrics (based on measurements and data derived from an action ) include:: 1. voice scan 2. signature scan 1.3 What is face recognition system? 3. keystrokes scan. A biometric system refers to the integrated hardware and software used to conduct biometric identification and verification. In clear terms face recognition system is a system, which turns your face to computer code so that it can be compared with thousands of faces. Inorder for face recognition system to work it has to know what a basic face looks like. Face recognition system is based on ability to first recognize faces ,which is a 7. It is the only biometric that allows to perform passive identification in 1 to many environment (eg: identifying a terrorist in a busy airport terminal). 8. In some cases one may also want to look at deploying multilayered biometric .Example of multi layered biometric would be the use of both finger print scans and facial scans in one unified access control devices. 3. It is accurate and allows for high enrollment and verification rates. 4. It does not require an expert to interpret the comparisons. 5. It can use your existing hardware infrastructure, existing cameras and image capture devices will work with no problem. 6. You can use existing images without having to re-enroll every user . (e.g.: passports, id cards, drivers licenses etc..)

Why choose face recognition over other biometrics?

technological feat in itself and then measure the various features of each face. If you look into mirror you can see that your face has certain distinguishable landmarks sometimes called as nodal points. There are about 80 nodal points on human face like 1. distance between eyes 2. width of nose 3. depth of eye sockets 4. cheekbones 5. jaw line 6. chin etc.. These nodal points are used to create numerical code, a string of numbers that represents the face in database(called face print). Only 14-22 nodal points are needed to complete the recognition process.

4. representation 5. matching define the process flow of facial scan technology.

1. sample capture - When the system is attached to a video surveillance system, the recognition software searches the field of view of a video camera for faces. If there is a face in the view, it is detected within a fraction of a second. A multi-scale algorithm is used to search for faces in low resolution. The system switches to a high-resolution search only after a head-like shape is detected.

2. Alignment - Once a face is detected, the system determines the head's position, size and pose. A face needs to be turned at least 35 degrees toward the camera for the system to register it.

CH .2 Facial scan process flow

3. Normalization -The image of the head is scaled and rotated so that it can be registered and mapped into an appropriate size and pose. Normalization is performed regardless of the head's location and distance from the camera. Light does not impact the normalization process.

With all biometrics 5-steps 1. sample capture 2. alignment 3. normalization 4. Representation - The system translates the facial data into a unique code also called as template. This coding process allows for easier comparison of the newly acquired facial data to stored facial data the template is much smaller than the image from which

it is drawn whereas quality facial images generally require 150-300 kb, the templates are approx. 1300 bytes or less than 1/100th of original.

There are 4 primary methods employed for facial scanning to identify and recognize and verify subjects.

1. eigenfaces 5. Matching - The newly acquired facial data is compared to the stored data and (ideally) linked to at least one stored facial representation. 2. feature analysis 3. neural network 4. automatic face processing.

The degree of similarity required for verification also known as threshold can be adjusted for different personnels, pcs, time of the day and other factors. 2.1 Technologies used for face recognition

2.1.1 Eigen face

There are various methods by which facial scan technology recognizes people. All share certain commonalities such as emphasizing those sections of the face, which are less susceptible to alteration, including the upper outlines of the eye sockets; areas surrounding ones cheek bones and side of mouth. Most technologies are resistant to moderate changes in hairstyle, as they do not utilize areas of the face located near hairline. All primary technologies are designed to be robust enough to conduct one to many searches i.e. to locate single face out of a database of thousands of faces.

Eigen face roughly translated, as ones own face is a technology which utilizes 2d global grayscale images representing distinctive characteristics of a facial image. Variations of eigen faces are frequently used as the basis of other face recognition methods. As suggested by graphic, distinctive characteristics of the entire face are highlighted for use in future authentication. The vast majority of faces can be reconstructed by combining features of approximately 100-125 eigen faces. Upon enrollment, the subjects eigen face is mapped to a series of numbers (coefficients).

 For one to one authentication, in which the image is being used to verify a claimed identity ones live template is compared against the enrolled template to determine coefficient variation. The degree of variance from template will determine acceptance or rejection. For one to many identification, the same principle applies but with a much larger comparison set. Like all facial recognition technology, eigenface technology is best utilized in welllit, frontal image capture situation. 2.1.2 Feature analysis Feature analysis is the most widely used facial recognition technology. It is related to eigenface but is more capable of accommodating changes in appearance or facial aspect (smiling vs. frowning). It uses LFA (local feature analysis), which can be summarized as an irreducible set of building elements. LFA utilizes dozens of features from different regions of the face and also incorporates the relative location of the features. The extracted (very small) features are building blocks and both types of blocks and their arrangement are used to identify or verify. It anticipates that the slight movement of feature located near ones mouth will be accompanied by relatively similar movement of adjacent features.

E.G. if a person is smiling then the changes that occur in the features adjacent to mouth are also taken into consideration. Since feature analysis is not global representation of the face, it can accommodate angle up to approximately 25 degree in horizontal plane, approximately 15 degree in vertical plane. Of course a straightforward video image from a distance of 3 feet will be the most accurate. Feature analysis is robust enough to perform one to one or one to many searches.

2.2.3. Neural network mapping technology In this technology features from both the faces i.e. the enrollment and verification face-vote on whether there is a match. Neural networks employ an algorithm to determine the similarity of unique global features of live vs. enrolled or reference faces using as much of facial images as possible. An incorrect match (false match) prompts the matching algorithm to modify the weight it gives to certain facial features. This method theoretically leads to an increased ability to identify faces in difficult conditions. As with all primary technologies neural network facial recognition can do one to one or one to many.

2.2.4. Automatic face processing It is more rudimentary technology using distances and distance ratios between easily acquired features such as eyes, end of nose and corners of mouth. Though overall not as robust as eigen face, feature analysis or neural network automatic face processing may be more effective in dimly lit, frontal image capture situations. Also it uses the combined approach of above three techniques. CH.3 ALGORITMS FOR FACE RECOGNITION Eigen faces for recognition many researchers focused on detecting individual facial features and categorizing different faces by the position, size, and relationship of these features. But now they take an informatics theoretic approach to the problem and derive their classification features with principle component analysis (PCA), which extracts the optimal linear encoding of the datasets.

2. eigen vector method rely on being able to classify images based solely on the coefficients of the top k eigen vectors. 3. These eigen vectors can be calculated using an matrix in which M is the number of pixels in the image. so the technique of pre multiplying the covariance matrix is computationally more efficient.

3.1.1 Following is the technique used to initialize the system 1. After acquiring an initial training set of images, they calculate the eigen faces (eigen vectors)for the matrix and use those to define a face space. 2. Each individual is represented in this face space by averaging the coefficient extracted from labeled exemplars of his/her face. 3. Once system is initialized ,images can be classified by projecting them into eigenvector basis using only top k vectors with the highest eigen value. 4. The projection is simply achieved by taking dot product of the image with each eigen vector. This computation can be performed with a 3-layer network in which weights are the eigen vectors and the hidden layers output are the coefficients. 5. The final output of the network is an image quite similar to the original image that has been reconstructed from the information stored in the coefficients of the top k eigen vectors. 6. Non-face images are not represented well by this basis set so the final image will be noticeably different from the original image, whereas a face passed through the network

3.1 Algorithm for eigen face

1. In a cluster of image vectors, the dimension with the greatest variance is captured by the eigen vector with highest eigen value, thus when projecting a point into the eigen vector basis and examining the value of its coefficients, the most information about point can be gained from coefficients of the top eigen vector.

will have almost identical inputs and outputs. 3.1.4. Solutions 7. After non-face images are discarded the test image is assigned the label of the closest class mean .If the distance between the test image and the closest mean is above a certain threshold it is classified as an unknown face which can be added to the dataset later. 8. New faces can be used to modify the eigen faces which unfortunately involves recalculating all eigen vectors and coefficient means. A multi resolution method is used where faces are compared to eigen faces of varying sizes to compute the best match. Also image background can have a significant effect on performance, which they minimize by multiplying input images with 2D gaussian to diminish the contribution of the background and highlight the central facial features. 3.2. Feature based recognition

3.1.2. Advantages This system performs face recognition in real time and also use this method along with motion cues to segment faces out of images by discarding squares that are classified as non-face images. It is a fast, simple and practical algorithm. The other dominant trend in face recognition is feature matching i.e. deriving distances and position features from the placement of the internal facial elements. One of the most earliest algorithm based on automatic feature detection were by localizing the corners of eyes, nostrils etc. in frontal views, the system computed parameters for each face, which compared (using Euclidean metric) against the parameters of known face. A most recent feature based system based on elastic bunch graph matching system detects features using gabor kernels, which are believed to simulate output of simple cells in the v1 cortical area. This is a general class recognition method for classifying members of a known class of objects.

3.1.3. Disadvantages It is limited because optimal performance requires high degree of correlation between pixel intensities of training set and set images.(this limitation has been addressed by using extensive preprocessing to normalize the images). This algorithm creates problems for faces scaled larger or smaller than the original dataset. Raw gray scale images are sensitive to noise and lighting conditions.

3.2.1. Algorithm for feature based recognition

1. Faces are represented as graphs with nodes positioned at fiducial points and edges labeled with 2D distance vectors. 2. Each node contains a set of 40 complex gabor wavelet coefficients including both phase and magnitude known as jet. 3. Wavelet coefficients are extracted using a family of gabor kernels with five different spatial frequencies and eight orientations .All kernels are normalized to be zero mean. 4. To identify a new face , an object adapted graph has to be positioned on the face using elastic bunch graph matching. 5. This can be performed automatically if the face bunch graph (FBG) has already been initialized with enough faces(approx. 70). 6. A FBG consists of collection of indiviual face model graphs combined into a stack like structure in which each node contains the jet of all previously initialized faces from databases. 7. To position the grid on a new face ,the graph similarity between the image graph and the existing FBG is maximized. 8. Graph similarity is defined by an average of the best possible jet match between the new image and any face restored within the FBG minus a topography term , which accounts distortion between the image grid and the FBG. 9. After the grid has been positioned on the new face , the face is identified by comparing the similarity between that face and every face stored in the FBG , ignoring the phase term.

3.2.2. Advantages

Initial grid matching requires 30 sec and the actual recognition process completes in less than 1 sec. On FERET (face recognition technology) dataset , the algorithm performs impressively well on frontal images, correctly ranking the image first 98% For half rotated and profile images, performance degrades to 57% and 84% correct , however since this is a difficult case for face detection recognition system these results are comparatively good.

3.2.3. Disadvantages

The main disadvantage is that the tedious grid placement must be performed manually for the first 70 images before the automatic graph matching becomes sufficiently reliable. Also this algorithm does not work well for faces with variable illumination and background.

3.3 Neural network based face recognition

 The hidden layers Neural network approach is a better approach to recognize faces in unsupervised manner It collects typical faces from each individual, projects them onto the eigen spaces and neural networks learn how to classify them with the new face descriptor as input. Neural network is used to create the face database and recognize the face. A separate network is built for each person. The input face is projected onto the eigenface space first and get a new descriptor. The new descriptor is used as network input and applied to each person's network. The one with maximum output is selected and reported as the host if it passes predefined recognition threshold.

In order to make the training of neural network easier one neural net is created for each person. Each neural net identifies whether the input face is the networks hosts or not. The recognition algorithm selects the network with max output If the output of the selected network passes a predefined threshold it will be reported as host of the input face, otherwise the input face is rejected. After neural structure is set training examples are prepared. At the start of training select a number of face images from each person that are well aligned frontal view. These faces will be used as positive examples for their own networks and negative examples for other networks After basic neural networks are created and if face is reported it will be applied face recognition code. Face recognition results are checked to find more faces for training.

3.3.1 Algorithm for neural network

The neural network has general back propagation structure with three layers . The input layer has 100 nodes from face descriptors. The hidden layer has 10 nodes. The output unit gives a result from 0.0 to 1.0 telling how much input can be thought as networks host.

There are 4 categories in which faces fall:

1. Faces have high output on their own networks and low output on other networks, no action here. 2. Faces have high output on both networks, these faces used as negative examples for other networks.

3. Faces are well aligned frontal view and have low output on their own networks, these faces will be used as both positive examples for other networks. 4. Faces are not well cut and have low output on their own networks. If they have high output on other networks they will be included as negative examples for other networks else ignored.

3.4 An Automatic System for Detection, Recognition & Coding of Faces The system diagram above shows automatic system for detection, recognition and modelbased coding of faces for potential applications such as video telephony, database image compression, and automatic face recognition.

The system consists of a two-stage ; Once new faces are generated they are added to training examples and retrain the neural networks. If original face descriptors from the training examples are used as neural network input, it will be difficult to make the network converge. So make the average of training set zero. object detection and alignment stage, a contrast normalization stage, and an eigenspace based feature extraction stage whose output is used for both recognition and coding.

This leads to a compact representation of the face that can be used for both recognition as well as image compression.

3.3.2 Advantages 3.4.1 Detection and Alignment original input image This algorithm extracts features automatically such as eyebrow thickness and vertical position, nose vertical position and width, chin shape and zygomatic breadth. Algorithm performs recognition based on single feature and features are sorted by decreasing performance as eyes, mouth and whole face template. Thos algorithm also finds and recognizes face in real time .

Estimated Head Location & Scale

Head-Centered Image

Estimated Facial Feature Locations

Warped & Masked Facial Region

Once the image is suitably normalized with respect to individual geometry and contrast, it is projected onto a set of normalized eigenfaces. The figure above shows the first few eigenfaces obtained from a KL expansion on an ensemble of 500 normalized faces. In our system, the projection coefficients are used to index through a database to perform identity verification and recognition using a nearest-neighbor search.

The process of face detection and alignment consists of a two-stage object detection and alignment stage, a contrast normalization stage, and a feature extraction stage whose output is used for both recognition and coding. Pictures above illustrate the operation of the detection and alignment stage on a natural test image containing a human face. The first step in this process is illustrated in "Estimated Head Position and Scale" where the ML estimate of the position and scale of the face are indicated by the crosshairs and bounding box. Once these regions have been identified, the estimated scale and position are used to normalize for translation and scale, yielding a standard ``head-in-the-box'' format image. A second feature detection stage operates at this fixed scale to estimate the position of 4 facial features: the left and right eyes, the tip of the nose and the center of the mouth. Once the facial features have been detected, the face image is warped to align the geometry and shape of the face with that of a canonical model. Then the facial region is extracted (by applying a fixed mask) and subsequently normalized for contrast.

Here, the geometrically aligned and normalized image is projected onto a custom set of eigenfaces to obtain a feature vector which is used for recognition purposes as well as facial image coding.

3.4.3 Advantages

Good-quality facial images are automatically generated using approximately 100-bytes worth of encoded data. The system has been successfully tested on a database of nearly 2000 facial photographs from the FERET database with a detection rate of 97%. Recognition rates as high as 99% have been obtained on a subset of the FERET database consisting of 2 frontal views of 155 individuals.

3.4.2 Recognition and Coding CH. 4 HARDWARE REQUIREMENTS

 Video capture device requires 320 x 420 resolution and at least 3-5 frames/sec.

children, particularly as it relates to child pornography. 5.1.1 CABS:

More frames/sec leads to better performance.

Store all offence-related detain one easyto-use system -- data is entered once and only once. Integrate with any database -- including other detachments and other applications (RMS, CAD, Jail Management systems, and "most-wanted" databases) . Link victims to offenders -- to aid in criminal analysis and investigations

For slightly greater distances there is strong correlation between camera quality and system capabilities.

An adequate video card and processor speed are key components.

Capture and store digital images of the offender -- encode all mug shots, marks, tattoos, and scars Perform rapid and accurate searches -- on all data and image fields for crime statistics and reporting Produce digital lineups -- using any stored image in minutes Identify previous offenders -- preintegrated with advanced biometric face recognition software.

Face recognition system works well with standard off the shelf pc.

Ch. 5 Applications of face recognition system

5.1 Law enforcement and justice solutions: 5.1.2 Childbase protection: Today's law enforcement agencies are looking for innovative technologies to help them stay one step ahead of the world's ever-advancing criminals. As such, FRS is committed to developing technologies that can make the jobs of the law enforcement officer easier. This includes acclaimed CABS-computerized arrest and booking system and the childbase protection, a software solution for global law enforcement agencies to help protect and recover missing and sexually exploited ChildBase is an application that helps protect and recover missing and sexuallyexploited children, particularly those children victimized through child abuse images. 5.2 Identification solutions: With regards to primary identification documents, (Passports, Driver's licenses, and ID Cards), the use of face recognition for identification programs has several

advantages over other biometric technologies. Leverage your existing identification infrastructure. This includes, using existing photo databases and the existing enrollment technology (e.g. cameras and capture stations); and Increase the public's cooperation by using a process (taking a picture of one's face) that is already accepted and expected; Integrate with terrorist watch lists, including regional, national, and international "most-wanted" databases. 5.3 Homeland defence: Since the terrorist events of September 11, 2001, the world has paid much more attention to the idea of Homeland Defense, and both governments and private industries alike are committed to the cause of national defense. This includes everything from preventing terrorists from boarding aircraft, to protecting critical infrastructure from attack or tampering (e.g. dams, bridges, water resevoirs, energy plants, etc.), to the identification of known terrorists. 5.4 Airport security: Airport and other transportation terminal security is not a new thing. People have long had to pass through metal detectors before they boarded a plane, been subject to questioning by security personnel, and restricted from entering "secure" areas. What has changed, is the vigilance in which these security efforts are being applied. The use of biometric identification, can enhance security efforts already underway at

most airports and other major transportation hubs (seaports, train stations, etc.). This includes the identification of known terrorists before they get onto an airplane or into a secure location. 5.5 Immigration: Most countries do not want to be perceived as being a "weak link" when it comes to accepting immigrants and refugees, particularly if that individual uses the new country as a staging ground for multinational criminal and terrorist activities. Consequently, governments around the world are examining their immigration policies and procedures. Biometric technology, particularly face recognition software, can enhance the effectiveness of immigration and customs personnel. After all, to the human eye it is often difficult to determine a person's identity by looking at a photo, especially if the person has aged, is of a different ethnic background, has altered their hair style, shaved their beard, etc. FRS does not have this difficulty. 5.6 Access control: The use of biometric technology, particularly face recognition software (either independently or as one part of a multilayered biometric solution), can enhance your security efforts considerably. Biometric identification ensures that a person is who they claim to be, eliminating any worry of someone using illicitly obtained keys or access cards. 5.7 Financial services:

 The financial services industry revolves around the concept of security. Yet for the most part, security within the industry is limited to a simple personal identification number (PIN) or password. Biometrics, particularly face recognition software, can improve the security of the financial services industry, saving the institution time and money both through a reduction of fraud cases and the administration expenses of dealing with forgotten passwords. Furthermore, biometric-based access control units can safeguard vaults, teller areas, and safety deposit boxes to protect against theft. The use of biometrics can also ensure that confidential information remains confidential while deterring identity theft, particularly as it relates to ATM terminals and card-not-present e-commerce transactions.

spend hours, days, or weeks monitoring video streams (i.e. examining a bank's security in a criminal investigation).

5.8 Scene analysis and surveillance solutions: This includes the ability to extract, categorize, and search non-facial imagery. For example, within the law enforcement application it allows you to capture, archive, and retrieve such identifying characteristics as tattoos, marks, or scars. It can also analyse scenes from either streaming or archived video, "looking" for out-of-the-ordinary occurrences, the presence of certain vehicles, specific faces, etc. This is beneficial and can save significant time and money to those individuals who