CHAPTER-1 INTRODUCTION

Some neurological conditions, such as amyotrophic lateral sclerosis (ALS), polyneuritis with Guillan-BarrSyndrom, or brain-stem infarct, can lead to severe motor disability. Such patients are referred to as lockedin because they are often completely paralyzed and not able to communicate with caregivers or to express their needs and feelings. In addition to tetraplegia, a lack of control over head and eye movements as well as facial muscles appears in some cases. Even when small muscle twitches are possible, they are often unreliable. Many patients cannot use conventional devices made for patients with severe mo- tor disability, because these require consistent control over at least one muscle group. Recent studies indicate that humans can learn to control certain components of their EEG and can use them as a new communication channel. Unlike conventional alternative communication methods, EEG-based communication requires no neuromuscular control. Therefore, computer mediated communication may be a possibility for locked-in patients Birbaumer and coworkers demonstrated that healthy subjects can attain reliable control over their slow cortical potential (SCP) amplitude at the vertex (Cz, according to the international 10-20 system), if it is reinforced for the brain response. Compared with other EEG signals, SCPs have two important advantages: first, they are present in the brain activity of every per- son, whereas rhythmic EEG components (such as alpha- or -rhythm) cannot be recorded in some subjects. Second, the physiological function of SCP shifts from Some neurological conditions, such as amyotrophic lateral sclerosis (ALS), polyneuritis with Guillan-Barr- Syndrom, or brainstem infarct, can lead to severe motor disability. Such patients are referred to as locked-in because they are often completely paralyzed and not able to communicate with caregivers or to express their needs and feelings. In addition to tetraplegia, a lack of control over head and eye movements as well as facial muscles appears in some cases (Harvey et al. 1979; Hayashi and Kato 1989). Even when small muscle twitches are possible, they are often unreliable. Many patients cannot use conventional devices made for patients with severe mo- tor disability, because these require consistent control over at least one muscle group. Recent studies indicate that humans can learn to control certain components of their EEG and can use them as a new communication channel. Unlike conventional alternative communication methods, EEG-based communication requires no neuromuscular control. Therefore, computer mediated communication may be a possibility for locked-in patients. Birbaumer and coworkers demonstrated that healthy subjects can attain reliable control over their slow cortical potential (SCP) amplitude at the vertex (Cz, according to the international 10-20 system), if it is reinforced for the brain response. Compared with other EEG signals, SCPs have two important advantages: first, they are present in the brain activity of every person, whereas rhythmic EEG components (such as alpha- or -rhythm) cannot be recorded in some subjects. Second, the physiological function of SCP shifts from ate a multi-level menu, using one dimension for forward and backward movements within the current level and using the other dimension for proceeding onto another level, i.e., enter and escape functions. Subjects learn to self-regulate their SCP activity within a feedback paradigm. A feedback device provides prompt measurement of cortical activity and presents it in a simple, easy-toperceive manner (e.g., ball movement on a screen) (Miller 1978). The training procedure is based on the operant-conditioning paradigm (Ferster and Skinner 1957). A feedback of the correct response serves as a

reward. The desired reaction is systematically shaped. Shaping is a widely used training technique, in which a desired motor, physiological, or behavioral objective is approached in small steps by successive approximations. The technique has been used extensively in humans with mental retardation, self- injurious behavior, depression, motor deficits, children with autism, and for a variety of other conditions. Be- cause different individuals have different characteristics of EEG activity and baseline performance, the shaping procedure must be adapted to each individual separately. For communication through brain activity, three major problems must be solved. First, the learning ability of completely paralyzed patients must be proven. It cannot be ruled out that healthy controls as well as neurological patients with intact motor abilities (e.g., epilepsy: Rockstroh et al. 1993) regulate their SCP by using peripheral strategies, for instance varying muscle tension to modulate the sensory input to the cortex. A paralyzed patient is obviously unable to employ such a strategy. Second, the EEG must be analyzed on-line for immediate reinforcement. The on-line analysis must include all the transformations necessary to extract the to-be-controlled signal from the background noise (e.g., filtering, spectral analysis, baseline correction, elimination or correction of artifacts, or both, etc.). The cortical process must be converted on-line into a signal (e.g., cursor movement) that can be observed by the user. Third, the software has to be flexible in order to adjust it to the individual shaping procedure. The study presented here proposes a solution to these problems.

CHAPTER-2 Brain-Computer Interface (BCI)

A brain-computer interface (BCI) is a communication channel connecting the brain to a computer or another electronic device. The intrinsic feature of a BCI is that it does not depend on the brains normal output pathways of peripheral nerves and muscles. Two basic requirements are met for a communication channel between the brain and computer:

1) Features that are useful to distinguish several kinds of brain state, 2) Methods for the detection and classification of such features implemented in real time.

Various techniques are now available to monitor brain function, e.g. electroencephalography (EEG), magneto encephalography, functional magnetic resonance imaging, and positron emission tomography. The latter three techniques are technically demanding and expensive. At present, EEG is the optimal choice for BCI implementation. These systems differ greatly in their inputs, feature extraction and translation algorithms, outputs, and other characteristics such as speed, accuracy, and appropriate user population. Typical input signals of BCIs include slow cortical potentials, or rhythms recorded over sensorimotor cortex EEG patterns associated with different mental tasks, and visual evoked potentials (VEPs). Electrodes can be placed either on the scalp or on the cortex. Typical BCI applications involve cursor movement, letter or icon selection, or device control. Currently, BCIs are mainly used as augmentative communication technology for individuals with motor impairments, such as amyotrophic lateral sclerosis (ALS) or cerebral palsy. Several issues are crucial to further development and expanded utilization of the BCI technology. The first issue is the information transfer rate. The second issue is the training time for users to develop competence. The third issue is medical invasiveness. The less invasive the technique the more likely it can be used in a wide range of applications. Based on the above considerations, our interests concentrate on the high transfer rate, minimal training, and noninvasiveness. Steady-state visual evoked potentials (SSVEPs) recorded from the occipital scalp are used as the input of our BCI system. The system has the advantage of focusing on EEG activity that occurs at a specific frequency. This feature simplifies the feature extraction methods, and users require little or no training. SSVEP-based BCIs belong to dependent BCIs. An intact visual system is necessary, and it will be wholly devoted to EEG-based communication. We have applied SSVEP-based BCI to control cursor movements. In that work, four buttons illuminated at different frequencies represented four directions; users could move the cursor to different directions simply by looking at the corresponding buttons. In this paper, we

will introduce a new application of SSVEP-based BCI to show the systems ability to provide high transfer rates.

2.1 BCI COMPONENTS A typical BCI device consists of several components. These include electrode cap, EEG amplifiers, computer and subjects screen. A critical issue is how the users commands, i.e., the changes in the EEG are converted to actions on the feedback screen or the application. This process can be divided into five stages: 1) Measurement of EEG It is done by using the electrodes. Many BCIs use a special electrode cap, in which the electrodes are already in the right places, typically according to the international 10-20 systems. It saves time because the electrodes do not have to be attached one by one. Typically, less than 10 electrodes are used in online BCIs with sampling rates of 100-400 Hz. 2) Preprocessing This includes amplification, initial filtering of EEG signal and possible artifact removal. Also A/D conversion is made, i.e. the analog EEG signal is digitized.

3) Feature extraction In this stage, certain features are extracted fromthe preprocessed and digitized EEG signal. In the simplest form a certain frequency range is selected and the amplitude relative to some reference level measured. Typically the features are certain frequency bands of a power spectrum. If the feature sets representing mental tasks overlap each other too much, it is very difficult to classify mental tasks, no matter how good a classifier is used. On the other hand, if the feature sets are distinct enough, any classifier can classify them. 4) Classification The features extracted in the previous stage are the input for the classifier. Different BCIs can classify different number of classes, typically 2 to 5 classes. The classifier can be anything from a simple linear model to a complex nonlinear neural network that can be trained to recognize different mental tasks. With the exception of a simple threshold detection, the classifier can calculate the probabilities for the input belonging to each class. Usually the class with the highest probability is chosen. However, in some BCI protocols none of the classes may be chosen, if the classification probability does not exceed some predefined level. This kind of classification result can be called nothing or reject. 5) Device control The classifiers output is the input for the device control. The device control simply transforms the classification to a particular action. The action can be, e.g., an up or down movement of a cursor on the feedback screen or a selection of a letter in a writing application. However, if the classification was nothing or reject, no action is performed, although the user may be informed about the rejection. 2.2 FEEDBACK Feedback is an important factor in BCIs. In the BCIs based on the operant conditioning approach, feedback training is essential for the user to acquire the control of his or her EEG response. The BCIs based on the pattern recognition approach and using mental tasks do not definitely require feedback training. However, feedback can speed up the learning process and improve performance. Cursor control has been the most popular type of feedback in BCIs. Feedback can have many different effects, some of them beneficial and some harmful. Feedback used in BCIs has similarities with biofeedback, especially EEG biofeedback. EEG MECHANISM Electroencephalography (EEG) is a method used in measuring the electrical activity of the brain. This activity is generated by billions of nerve cells, called neurons. Each neuron is connected to thousands of other neurons. Some of the connections are excitatory while others are inhibitory. The signals from other neurons sum up in the receiving neuron. When this sum exceeds a certain potential level called a threshold, the neuron fires nerve impulse. The electrical activity of a single neuron cannot be measured with scalp EEG. However, EEG can measure the combined electrical activity of millions of neurons. The temporal resolution of EEG is very good: millisecond or even better. However, the spatial resolution is poor. It depends on the number of electrodes, but the maximum resolution in centimeter range whereas, for example, in MEG,

PET or fMRI it is in millimeter range. The ongoing EEG is characterized by amplitude and frequency. The amplitudes of the EEG signals typically vary between 10 and 100 _V (in adults more commonly between 10 and 50 _V). The electrical activity goes on continuously in every living humans brain. We may sleep one third of our life times, but the brain never rests. Even when one is unconscious the brain remains active. Much of the time, the brain waves are irregular and no general pattern can be observed. There exist various properties in EEG, which can be used as a basis for a BCI: 1. Rhythmic brain activity 2. Event-related potentials (ERPs) 3. Event-related de synchronization (ERD) and event-related synchronization (ERS).

Depending on the level of consciousness, normal peoples brain waves show different rhythmic activity. For instance, the different sleep stages can be seen in EEG. Different rhythmic waves also occur during the waking state. These rhythms are affected by different actions and thoughts, for example the planning of a movement can block or attenuate a particular rhythm. The fact that mere thoughts affect the brain rhythms can be used as the basis for the BCI. The EEG can be divided into several frequency ranges as displayed in the above table. These ranges set the limits in which the different brain rhythms can be observed. 2.2.1 FEEDBACK IN BCI In most BCIs some kind of feedback is provided to the user. The most popular form of feedback has been the cursor control. In a typical trial, the user tries to move the cursor to the target, which is located on one side of the screen by using two commands (i.e., up & down or left & right). At the start of the trial the cursor is at the middle of the screen. The trial ends when the cursor hits either the target or the opposite end of the screen. If the target side of the screen is hit, the target can be flashed to indicate the trial outcome. One trial typically lasts a few seconds. After the cursor has hit the target, it blinks and a smiley face saying very good appears

as a positive reinforcement. Why has the cursor control been such a popular type of feedback in BCIs? One reason may be that the goal of many BCI research groups is to give the user, a disabled person, an opportunity to operate an ordinary personal computer by thoughts. In addition to this, there may be other reasons. The cursor control is an example of continuous feedback.

Fig: Experimental Procedure of TTD as an SCP driven BCI System

CHAPTER-3 METHODS OF TTD

Scientific Background VEPs reflect the electrophysiological mechanisms underlying the processing of visual in the brain. The signals are always in response to changes in the stimulus. A static stimulus in the visual field does not appear to effect any significant alterations in EEG activity. The signals evoked by changes in the visual input have been shown to reflect certain properties of the stimulus. A distinction is made between transient VEP and SSVEP based on the stimulation frequency. The former arises when the stimulation frequency is less than 2 Hz. If the repetition rate of the stimulus is higher than 6 Hz, however, a periodic response called the SSVEP will result. It is composed of a series of components whose frequencies are exact integer multiples of the repetition frequency. The amplitude and phase of the SSVEP are highly sensitive to stimulus parameters such as repetition rate, contrast or modulation depth, and spatial frequency. The SSVEP was found to be strongly dependent on spatial attention, being substantially enlarged in response to a flickering stimulus at an attended versus an unattended location. The increased SSVEP amplitudes reflect an enhancement of neural responses to a stimulus that falls within the range of spatial attention. This observation shows that SSVEP may provide an on-line method to identify the attentional target among a group of stimuli. Hardware and Software Figure shows the block diagram of the SSVEP based BCI system. The system was designed to help users to express his thoughts. Twenty-eight buttons that flickered on and off at different frequencies were displayed on a computer monitor. The on-off duty cycles were 50/50 for all frequencies. The 4 X 7 stimulus matrixes constituted a virtual keypad, representing the alphabets, BACKSPACE, and ENTER. Users could input alphabets and correct input errors by gazing at these buttons. A beep was sent out from the loudspeaker of the computer after each selection, and the result was displayed on the monitor so that users could know whether the selection was correct.

If the selection was wrong, users could delete it by gazing at the button BACKSPACE. When ENTER was selected, the input alphabet would be displayed. The button ON/OFF was designed to control the start and stop of the other stimuli. It would be flickered all the time. If the stimulus matrix was static, the system only needed to detect the frequency of the button ON/OFF. A rigorous detection criterion was applied to this button to reduce the occurrence of false positives. The buttons were widely spaced on the screen. Each button was a 2 cm 2.7 cm rectangle. Two-channel EEG signals were recorded from O1 and O2 according to the international 1020 system and referred to the left and right ear lobes respectively. The electronic circuits provided signal amplification, A/D conversion (sampling rate 200 Hz), and signal transmission. A wireless transmitter was adopted in the system to give users more freedom. Users could move their heads freely as long as their eyes were fixed on the desired buttons. A receiver connected to the serial port provided input data to the computer. The data were filtered with a band pass of 435 Hz.

11

CHAPTER-5 VRD DEVELOPMENT

Using seed funds from the Washington Technology Center the first VRD prototype was developed in the HITL by Dr. Tom Furness, Joel Kollin, and Bob Burstein. The project's initial goal was to prove the viability of forming an image on the retina using a scanned laser. This development work began in November 1993. 1. Prototype 1 The original prototype had very low effective resolution, a small field of view, limited gray scale, and was difficult to align with the eye. One objective of the current development effort was to quickly produce a bench-mounted system with improved performance. This new system is Prototype 1.

Figure 5.1 Prototype 1

12 Prototype 1 uses a directly modulated red laser diode at a wave length of 635 nanometers as the light source. The required horizontal scanning rate of 73,728 Hertz could not be accomplished with a simple galvanometer or similar commercially available moving mirror scanner. The use of a rotating polygon was deemed impractical because of the polygon size and rotational velocity required. It was thus decided to perform the horizontal scan with an acousto-optical scanner. The vertical scanning rate of 72 Hertz is within the range of commercially available moving mirrors and is accomplished with a galvanometer. The use of the acousto optical scanner comes with a number of drawbacks: * It requires optics to shape the input beam for deflection and then additional optics to reform the output beam to the desired shape. Total optical path length for this system is 45 centimeters. * It requires complex drive electronics that operate at frequencies between 1.2 GHz and 1.8 GHz 2. Prototype 2 To overcome the limitations of the acousto-optical scanner, HITL engineers have developed a miniature mechanical resonant scanner. This scanner, in conjunction with a conventional galvanometer, provides both horizontal and vertical scanning with large scan angles, in a compact package. The estimated recurring cost of this scanner will allow the VRD system to be priced competitively with other displays. Prototype #2 of the VRD uses the mechanical resonant scanner. The VGA resolution images produced are sharp and spatially stable. A schematic of the optical path of Prototype 2 is shown in Figure 5.2 The total optical path length for this system is 8 centimeters. The mechanical resonant scanner has many unique features. Foremost among these is the fact that the device has neither a moving magnet nor a moving coil. Instead, it uses a flux circuit whose only moving part is

the torsional spring/mirror combination. Eliminating moving coils or magnets greatly lowers the rotational inertia of the device, thus raising the potential operating frequency.

13

Figure 5.2 Prototype 2

3. Prototype 3 The third prototype system developed uses the same scanning hardware as Prototype 2 but uses three light sources to produce a full color image. In addition the eyepiece optics have been modified to allow for see through operation. In the see through mode the image produced by the VRD is overlaid on the external world.

4. Present Scenario

In the current version, a wireless computer with a touch-pad control is worn on the belt. Such units are largely used by the production units of many industries, most of them automobile manufacturers. Like a high-tech

monocle, a clear, flat window angled in front of the technician's eye reflects scanned laser light to the eye. That lets the user view automobile diagnostics, as well as repair, service, and assembly instructions superimposed onto the field of vision. The information that the device displays comes from an automaker's service-information Web site through a computer running Microsoft Windows Server 2003 in the dealership or repair shop. The data gets to the display via an ordinary IEEE 802.11b Wi-Fi network, and all the technicians in the service center are able to access different information simultaneously from one server. 14 Typical MEMS scanner today measures about 5 mm across, with a 1.5-mm- diameter scan mirror capable of motion on two scan axes simultaneously Using MEMS allows us to integrate the scanner, coil windings, and angle-sensor functions all on one chip. Such a scanner provides SVGA (800-by-600) equivalent resolution at a 60-hertz refresh rate and is now in production and in products. In addition, multiple scanners could provide higher-resolution images by each providing full detail in a tiled subarea. Eventually, costs will become low enough to make this practical, allowing the scanned- beam approach to surpass the equivalent pixel count of any other display technology. With green laser diodes, it will be possible to build bright, full-color see-through displays. Microvision uses laser light sources in many of its see-through products because our customers' applications demand display performances with color-gamut and brightness levels far exceeding the capabilities of flat panel displays, notebook displays, and even higher-end desktop displays. For today's commercial products, only red laser diodes are small enough, efficient enough, and cheap enough to use in such see-through mobile devices as Nomad. Blue and green diode-pumped solid-state lasers are still too expensive for bright, full-color, headup or projection displays for mainstream markets, but that could change soon. In the mid-1990s Shuji Nakamura of Nichia Chemical Industries Ltd. (now Nichia Corp., Tokushima, Japan) demonstrated efficient blue and green LEDs, and then blue laser diodes made of gallium nitride. When these designs and materials are extended to green laser diodes, it will be possible to build bright, full-color see-through displays. . As an alternative, small green laser are now being produced which use a crystal to frequency double a neodymium YAG laser. These devices are larger than desired and are not directly modulable at the required frequency. They do however, offer a short term solution. In the HITL researchers are investigating a number of alternatives to blue and green laser diodes. One frequency doubling technique being researched uses rare earth doped fibers as the doubling medium. A second technique uses wave guides placed in a lithium niobate substrate for the doubling.

The above methods all utilize a laser as the light source. Additional work is directed at using nonlazing, light-emitting diodes (LEDs) as the light source. In order for this to be successful two primary issues are being addressed. The first issue is how to focus the LED output to the desired spot size. The second issue is the development of fabrication techniques that will allow us to directly modulate the LEDs at the desired frequency. Enter the edge-emitting LED. Unlike conventional LEDs, which emit light from the surface of the chip, an edge-emitting LED has a sandwich-like physical structure similar to that of an injection-laser diode, but it operates below the lasing threshold. These LEDs emit incoherent beams of light that, while not so fine 15 as a laser's beam, provide a tenfold increase in brightness. We also use multiple inexpensive surface emitting LEDs, each contributing a portion of the overall power, to achieve high brightness. Further performance improvements of LED materials driven by huge investments aimed at general lighting applications will increase the brightness and range of applications for scanned-beam displays based on green and blue gallium nitride devices and aluminum gallium indium phosphide red LEDs. In addition to displaying images, the scanned-beam technology can capture them. In a display, the data channel through a digital-to-analog converter controls the light source to paint a picture on a blank canvas. In image capture, the light source is steadily on, and the data channel looks at the reflections from the object through an analog-to- digital converter connected to a photodiode. The light source, beam optics, and scanner are essentially the same in both applications.

16

CHAPTER-6 SAFETY ANALYSIS

In order to make the VRD a safer industrial product its safety analysis was performed. Maximum Permissible Exposures (MPE) was calculated for the VRD in both normal viewing and possible failure modes. The MPE is a level of exposure or irradiance of laser light which can be thought of as the theoretical border between safe and potentially harmful. The MPE power levels were compared to the measured power that enters the eye while viewing images with the VRD. For calculation of MPE different laser sources were assume and after that most conservative values were chosen. The authors done the analysis for color VGA system with 640 X 480 configuration, by considering the following parameters of systems, sweep time for each pixel was considered 40 sec and system scanning frequency was considered equal to 60Hz. An 8 hour exposure was assumed based on a working day for a user who would be wearing and viewing the display continuously. Authors used ANSI Z136.1 standard for there worst case analysis for laser exposure in visible range within the range of 400-550 nanometer wavelength while for calculation of range 500 to 700 nanometer the calculation done for the 400-550nanometer was multiplied with an correction factor CB having value greater than one. The MPE values calculated for different sources are summarized in Table 6.1 all values listed are in watt. The output power of the traditional VRD is in range of 100-300 watt.

Table 6.1 MPE for Different Sources As it can be clearly seen that all MPE's values calculated for different laser sources are well above the range of power values calculated for the VRD light source, which makes VRD a safe device to use. The most worst

case considered is that when both the horizontal and vertical beam controllers got failed, than one spot of the retina will be exposed to the whole output of the laser system. For this case only laser having the continuous wave output was considered. The power calculated for this case is 0.98103watts. The MPE's values shown in the Table 6.1 are calculated when viewer's retina is exposed to laser with continuous wave for 2.78 hours after the scanner assembly failure. Once again the power value calculated for VRD is well above those (MPE's) values which are presented in the Table 6.2 as a function of wavelength.

17

Table 6.2 MPE Values as a Function of Wavelength in case of Scanner Failure

18

CHAPTER-7 ADVANTAGES OF THE VRD

The VRD is able to provide several major advantages over current display technologies: color range, resolution, luminance and viewing modes, contrast ratio, power consumption and cost. 1. Color Range The range of hues that can be produced by the VRD is significantly greater as compared to those which can be produced by CRTs and FPDs(Flat Panel Displays).CRTs and miniature FPDs are able to reflect only a portion of the total palette of colors visible to the human eye, and are limited in the degree of saturation they can achieve. Because red (645 nm), green(513 nm) and blue(457 nm) light sources used in VRD technology emit highly saturated, pure color, the VRD can produce a range of possible colors and color fidelity superior to other available electronic display technologies.
.9 .8 .7 .6 .5 .4 .3 .2 .1 480 0 0
470 400 520 540

VRD
500

560 580 600

CRT
700

.1

.2

.3

.4

.5 .6 .7 .8

Figure 7.1 Color Gamut of VRD vs. CRT

2. Resolution The current VRD can produces 800X600 SVGA resolution (monochrome) images. Its resolution is limited only by diffraction and optical aberrations in the light source. It is not constrained as in the case of FPDs by how physically small one can make an individual pixel element. 3. Luminance and Viewing Modes The amount of energy incident at the corneal surface can be varied in the range of 60nW to 300nW. This flexibility in the range of intensity allows to produce images in varying environments as compared to conventional electronic displays which do not emit (or transmit) substantial amounts of light energy. As a result, they are primarily used in controlled lighting environments, and it is difficult to see them under bright 19

ambient light conditions such as exist outdoors. Generally, the VRD can be used in two viewing modes,

Figure 7.2 Comparison of illumination of the retina by a CRT versus the VRD. occluded or augmented. In the occluded mode, the outside environment is not visible and only images generated by the VRD can be seen. In the augmented mode, the VRD can overlay an image on the real world allowing both to be seen at the same time. In the augmented mode, the VRD luminance can be controlled to allow the user to see an image even under outside ambient light conditions. 4. Contrast Ratio The brightness of the VRD can be increased to very high levels or decreased to minimal levels as already explained. As a result, its contrast ratio is inherently high and far greater than that of standard front panel displays or even conventional CRT monitors. As a result of this greater range, the VRD's contrast ratio is inherently higher. 5. Power Consumption Conventional displays do not efficiently convert electrical energy into light energy. Both backlit FPDs and CRTs draw substantial power to produce radiant energy. As a result because most of their energy input is 20

wasted their brightness is relatively low. In addition, they are among the biggest battery consumers in portable devices that use them. VRD technology, by contrast, conveys virtually all of its generated light onto the retina, allowing a brighter display with minimum power requirements (based on laser emitting diode). 6. New Viewing Experiences

-Giving a speech while information is streamed to your eyeglasses in real time. -Presenting to a crowd with your notes secretly stowed away in your eye glasses. -Extending your everyday computing environment to include your eyewear display -Walking down the street, seeing your favorite friends show up "on screen" 2 blocks and 1 cafe away -Receiving turn by turn directions as you walk toward your destination -Reviewing messages while on the go -Creating your personal big screen experience from mobile TV and video (occluded or escape mode) -Seeing building schematics and locations of others (especially useful for security or firefighters) - Viewing virtual recipes while cooking without losing your rhythm.
7. Yield One limiting aspect in the manufacture of liquid crystal array image generators is the yield and reliability of the hundreds of thousands of individual liquid crystal cells present in these displays. For a liquid crystal array display to function properly at all times, each picture element must function properly. The Virtual Retinal Display requires only constant functionality from the light sources and the scanners. As resolution increases in virtual image displays, liquid crystal arrays will contain more and more individual liquid crystal cells. The Virtual Retinal Display will gain an increasing advantage over liquid crystal array image generators in terms of yield as resolution demands increase in the future. 8. Size The theoretical size for horizontal and vertical scanners plus light sources for the VRD is smaller than the size of conventional liquid crystal array and CRT image sources. A typical size for a liquid crystal array image generator for helmet mounted display applications is one inch by one inch. The Mechanical Resonant Scanner used in this project was approximately 1 [cm] by 2 [cm]. Furthermore, the problem of scanner size has not been directly addressed. Further size reduction is certainly possible. It should be noted that light sources for a smaller, usable full color VRD must be much smaller than the sources used in this project. The potential size of light emitting diodes and diode lasers indicate that these sources show greatest promise for future systems in terms of size. Moreover, it will be quite surprising to know that the original stereographic display, or the three dimensional view as the eye means it, can be accomplished only by an imaging system like the one proposed above. 9. Field of View The field of view of the VRD is controlled by the scan angle of the primary scanner and the power of the optical system. Initial inclusive systems with greater than 60 degree horizontal fields of view have been demonstrated. Inclusive systems with 100 degree fields of view are feasible. See through systems will have somewhat smaller fields of view. Current see through systems with over 40 degree horizontal fields of view have been demonstrated. 21

CHAPTER-8 APPLICATIONS
The range of applications which can be addressed by the VRD are of diversified nature. Among a large number of application few has been described in this part. 1. Head Mounted Displays (HMDs) Currently the retinal based displays are manufactured by the Micro Vision and marketed with the name of NOMAD. Two types of HMDs are being made one for being used in the industrial applications while other is manufactured for the usage of army. The two types have their own specifications and requirements but both HMDs have some common characteristics which are listed in Table 6.1 Both systems utilize

Table 8.1 Some Common Characteristics of VRD based HMDs the same see-through display and control functionality i.e. The system will project the images in a "see through" fashion the user will still be able to see the background scene, but has the option of focusing in on the presented information. 2. Interactive VRD Currently the research work is aimed at designing and developing of an interactive VRD for army pilots. As mentioned VRD is the only display technology that has sufficient luminance to be used as an augmented display over the pilot's real world view in bright sunlight. The goal is to develop the augmented image as interactive display (Figure 8.1). Goals of the IVRD project are as follows: _ Superimpose high luminance color images over the pilot's real world view, _ Measure instantaneous location of the augmented image in the real world view, _ Measure gaze direction, eye, and possibly hand position within the augmented image, _ Display sensor data from 360 degrees within a limited field-or-view, augmented image,

22

Figure 8.1 Augmented image over the real world view

3. VRD as Low Vision Aid Possibly the most exciting result has been the identification of the VRD as a potential low vision aid. Initially it was noticed that people working on the development of the VRD, who had myopia (near-sightedness) did not need their glasses to see the images being produced. This initial discovery was followed by the analysis of improvements brought by using VRD by persons having low vision. Usually these individuals are only correctable to 20/100 to 20/200 (higher digit shows greater visual acuity). But with the VRD and their correction removed, researchers were able to improve their vision to approximately 20/60. Two prototypes have already been developed to use VRD as wearable low vision aid. In both the prototypes VGA camera is used to get input from the environment, after that some machine vision algorithms are applied to identify potential obstacles and then final images with enhanced information are scanned on viewer's eye and in this way the person is warned about different obstacles present in his environment. There are also other large number of potential applications of VRD which includes image guided surgery, head mounted display for the vehicles, consumer products etc.

23

Figure 8.2 Interactive VRD and its components

4. Radiology One examination performed by radiologists is the fluoroscopic examination. During a fluoroscopic examination, the radiologist observes the patient with real-time video x-rays. The radiologist must continually adjust the patient and the examination table until the patient is in a desired position. When the patient is in a desired position, the radiologist takes a film copy of the x-ray image. The positioning process can be difficult and cumbersome because the radiologist must visually keep track of a patient, a video monitor, and an examination table simultaneously. Because the VRD can operate in a see-through mode at high luminance levels, it is an ideal display to replace the bulky video monitor in a fluoroscopic examining room. The radiologist could see through the x-ray display and see the patient as well. Other features such as a display luminance control or on/off switch could easily be included for this application.

24

Figure 8.3 Prototype of Wearable Low Vision Aid

5. Surgery Surgery to remove a cancerous growth requires knowledge of the growth's location. Computed tomographic or magnetic resonant images can locate a tumor inside a patient. A high luminance see-through display, such as the VRD, in conjunction with head tracking, could indicate visually where a tumor lies in the body cavity. In the case that a tumor lies hidden behind, say, an organ, the tumor location and a depth indicator could be visually laid over the obstructing organ. An application in surgery for any display would clearly require accurate and reliable head tracking. 6. Manufacturing 25

The same characteristics that make the VRD suitable for medical applications, high luminance and high resolution, make it also very suitable for a manufacturing environment. In similar fashion to a surgery, a factory worker can use a high luminance display, in conjunction with head tracking, to obtain visual information on part or placement locations. Drawings and blueprints could also be more easily brought to a factory floor if done electronically to a Virtual Retinal Display (with the option of see-through mode). Operator interface terminals on factory floors relay information about machines and processes to workers and engineers. Thermocouple temperatures, alarms, and valve positions are just a few examples of the kind of information displayed on operator interface terminals. Eyeglass type see-through Virtual Retinal Displays could replace operator interface terminals. A high luminance eyeglass display would make the factory workers and engineers more mobile on the factory floor as they could be independent of the interface terminal location. 7. Communications The compact and light weight nature of the mechanical resonant scanner (MRS) make an MRS based VRD an excellent display for personal communication. A hand held monochrome VRD could serve as a personal video pager or as a video FAX device. The display could potentially couple to a telephone. The combination of telephone services and video capability would constitute a full service personal communication device. 8. Virtual Reality The traditional helmet display is an integral part of virtual reality today. The VRD will be adapted for this application. It can then be used for educational and architectural applications in virtual reality as well as long distance virtual conference communications. Indeed it can be utilized in all applications of virtual reality. The theoretical limits of the display, which are essentially the limits of the eye, make it a promising technology for the future in virtual reality HMD's.

26

CHAPTER-9 FUTURE SCOPE

When cost of production falls further, we will see VRDs fulfilling many functions and applications, and may perhaps see a time where they become ubiquitous in the more distant future. Future systems will be even more compact with the advent of MEMS (Micro Electro Mechanical System) scanners, miniature laser diodes and application specific IC technology.

27

CHAPTER-10 CONCLUSION
In this small report, i have tried to summarized the research work which span over last 20 years and which continue to discover new bounds in the thrust of more improvements in the VRD. Certainly it can be said that in the long term the VRD can pose strong challenges to its rival displays, but still large amount of work is left to be done e.g. until today only monochrome VRD devices have been produced commercially which also have their limitations in term of resolution. The color VRD has been successfully tested in the labs but still it hasn't appear in the market because of limitation of its size. These days research is directed towards to develop a MEMS (Mechanical Electro Mechanical System) based bidirectional beam scanner having the capability of scanning in both horizontal and vertical direction along with very high scan rate. Also some research part is directed to find algorithms and techniques for interactive VRD which includes head tracking system, stereo system for VRD to have 3D images. Still it requires more detailed analysis to check the safety bounds of VRD, because long term usage of the VRD can be fatal for eyes.

28

References
[1] http://en.wikipedia.org/wiki/Virtual_retinal_display [2] http://www.hitl.washington.edu/projects/vrd/ [3] http://www.wisegeek.com/what-is-a-virtual-retinal-display.htm [4] http://virtualretinaldisplay.blogspot.in/2008/12/advantages-using-virtual-retinal.html [5] http://www.brother.com/en/news/2011/airscouter/index.htm

29