Introduction and Digital Image Fundamentals Introduction and Digital Image Fundamentals : The origins of Digital Image Processing, Examples of Fields that Use Digital Image Processing, Fundamentals Steps in Image Processing, Elements of Digital Image Processing Systems, Image Sampling and Quantization, Some basic relationships like Neighbours, Connectivity Distance Measures between pixels, Linear and Non Linear Operations. Introduction: Motivation and Perspective = Various Definitions Related to Digital Image Processing (DIP) Evolution of Digital Image Processing} (IP) Motivation and Perspective ‘Scenes and Images Applications ‘Components of an Image Processing| System The Terminology of Digital Image| Processing Some Philosophical Considerations Digital Image Processing in Practice} A Simple Image Formation Model! Introduction to Sampling and Quantization Digitizing images Characteristics of an Image Digitizer Types of Image Digit Image Digitizing Components Introduction to Digital Image Display] Introduction Display Technologies Digital image processing, the manipulation of images by computer, is a relatively recent development in terms of man’s ancient fascination with visuals. In its short history, digital image processing has been applied to practically every type of imagery, with varying degree of success. Several factors combine to indicate a lively future for digital image processing. A major factor is the declining cost of computer equipment. Both processing units and bulk storage devices are becoming less expensive year-by-year. A second factor is the increasing availability of equipment for image digitizing and display. There are indications that its cost will continue to decline. Several new technological trends promise to further promote digital image processing. These include parallel processing made practical by low-cost microprocessors and the use of charge-coupled devices (CCDs) for digitizing, storage during processing and display and large, low cost image storage always. Another impetus for development in this field stems from some exciting new application on the horizon. Certain types of medical diagnosis, including differential blood cell counts and chromosome analysis, are nearing a state of practicality by digital techniques. The remote sensing programs are well suited for digital image processing techniques. Thus, with increasing 3Harry Nyquist (2/7/1889-4/4/1976) Harry Nyquist received his Ph.D. from Yale in 1917. From 1917 to 1934, he was employed by Bell Labs where he worked on transmitting pictures using telegraphy and on voice transmission, Hewas the first to quantitatively explain thermal noise. He invented the vestigial sideband transmission technique still widely used in the transmission of television signals, He also invented the Nyquist diagram for determining the stability of feedback systems. availability of reasonably inexpensive hardware and some very important applications 9 the horizon, one can expect digital image processing to continue its phenomenal growth are to play an important role in the future, 1.2. VARIOUS DEFINITIONS RELATED TO DIGITAL IMAGE PROCESSING (DIP) Image An image may be defined as a two-dimensional functional g(x, y). Here x and y ar spatial (plane) coordinates. Gray level or Intensity With respect to a two dimensional function g(x, y), the amplitude of g at any pair « coordinates (x, y) is known as the intensity or gray level of the image at that point. Digital Image When g(x,y) and the amplitude levels of g are all finite, discrete quantities, the image i defined as a digital image. Pixels A digital image is composed of a finite number of elements, each of which has a particular location and value. These elements are referred to as picture elements, image elements, pels and pixels, In fact pixel is a term which is most widely used to denote the elements of a digital image. Digital Image Processing (DIP) The digital image processing (DIP) refers to processing digital images by means of a digital computer. The digital image processing encompasses a wide and various field of applications, We can also say that this field of digital image processing encompasses processes whose inputs and outputs are images and in addition encompasses processes that extract attributes from images including the recognition of the individual objects. 1.3 EVOLUTION OF DIGITAL IMAGE PROCESSING (DIP) The very first application of digital images was in the newspaper industry. In this application the pictures were sent by submarine cable between the two cities, Invention of the Bartlane11 Introduction and Digital Image Fundamentals m @ cable picture transmission system in the early 1920s reduced the time required to send a picture across the Atlantic from more than a week to less than three hours. Specialized printing equipment coded pictures for cable transmission and then reconstructed them at the receiving end. The early Bartlane systems were capable of coding images in five distinct levels of gray. This capability was increased to 15 levels in 1929. As a matter of fact the history of digital image processing is intimately tied to the development of the digital computer. In fact, digital images require so much storage and computational power that progress in the field of digital image processing has been dependent on the development of digital computers and of supporting technologies that include data storage, display and transmission. The first computer powerful enough to carry out meaningful image processing tasks appeared in the early 1960s. In parallel with space applications, digital image processing techniques began in the late 1960s and early 1970s to be used in medical imaging, remote earth resources observations and astronomy. The invention in the early 1970s of computerised axial tomography (CAT), also called computerized tomography (CT) for short is one of the most important events in the application of image processing in medical diagnosis. From the 1960s until the present, the field of image processing has grown vigorously. In addition to applications in medicine and the space program, digital image processing techniques now are used in a broad range of applications. Computer procedures are used to enhance the contrast or code the intensity levels into color for easier interpretation of X-rays and other images used in industry, medicine, and the biological sciences. Similarly succe | applications of image processing concepts can be found in astronomy, biology, nuclear medicine, law enforcement, defence, and industrial applications. The continuing decline in the ratio of computer price to performance and the expansion of networking and communication | _ bandwidth via the world wide web and the Internet have created unprecedented opportunities for continued growth of digital image processing. 1.4 MOTIVATION AND PERSPECTIVE As a matter of fact, the birth of digital computer has introduced to the society of a machine which is much more powerful than human beings in numerical computation. In a computer, the human capability of processing non-numerical information received trom the environment as well as society can be incorporated with equal or more etficien a new subject called artifical intelligence, which has a large area of common internet and motivation with another subject known as pattern recognition. A major portion of information received by a human from the environment is visual. Hence, visual information by computer has been drawing a very significant attention of the researchers over the last few decades. In fact, the process of receiving and analyzing visual information by the human species is called as sight, perception or understanding. In a similar way, the process of receiving and analyzing visual information by digital computer is called digital image processing and scene | analysis. The term ‘image’ rather than ‘picture’ is used here, because the computer stores and | processes numerical image of a scene. Although ‘pattern recognition’ and ‘image processing’ have a lot in common, yet they were developed as separate disciplines. Two broad classes of techniques, iv, ‘processing’ and analysis have evolved in the field of digital image processing and analysis. Processing of an image includes improvement in its appearance and efficient | representation. So, the field consists of not only feature extraction, analysis and recognition of images, but also coding, filtering, enhancement and restoration. The entire process of -y. This led to evolution of rro 1 Digital Image Processing image processing and analysis starting from the receiving of visual information to the giving out of description of the scene, may be divided into three major stages which are also considered as major sub-areas and are given below : (i) Discretization and Representation It converts visual information into a discrete form which is suitable for computer processing. It also approximates visual information to save storage space as well as time requirement in subsequence processing. (ii Processing Processing improves image quality by filtering etc., compressing data to save storage and channel capacity during transmission. (iii) Analysis Analysis extracts image features, quantifying shapes, registration and recognition. In the initial stage, the input is a scene (je, visual information) and the output is corresponding digital image. In the secondary stage, both the input and the output are images where the output is an improved version of the input. Also, in the final stage the input is still an image but the output is a description of the contents of that image. Figure 11 shows a schematic diagram of different stages. 7 Transmission <—§ Lagat | Compression Image | 1 sensing Digitization = eee | Enhancement [*—} Segmentation + 1 = Restoration be le Feature extraction Statstica/structural analysis Primitive and relation extraction T Description and interpretation Fig. 4.4. Different stages of image processing and analysis scheme. Information flow along the outgoing link from each block, “Asuccessful mans =David Brinkley1m Introduction and Digital Image Fundamentals = 4.5 PRACTICAL ASPECTS OF DIGITAL IMAGE PROCESSING Human beings are primarily visual creatures who depend on their eyes to gather information around them. OF the five senses that human beings have, sight is what we depend upon the most. Not many animals depend on their visual systems; the way human beings do Bats use high frequency sound waves. They emit sound waves which reflect back when they encounter some obstruction. Cats have poor vision but an excellent sense of smell Snakes locate prey by heat emission and fish have organs that sense electrical fields aves 1.5.1 What Happens when we Look at an object? The eye records the scene and sends signals to the brain. These signals get processed in the brain and some meaningful information is obtained. Let us take a simple example when we see fire, we immediately identify it as something hot. Two thi ( The scene has been recorded by the eye. (ii) The brain processed this scene and gave out a warning signal This is image processing. pened here: We start processing images from the day we are born. Hi integral part of us and we continue to process images till the dav we die. So, even if this subject seems to be new, we have been sub-consciously do: human eye brain mechanism represents the ultimate imaging svs we have another important trait that is common to all information, analyse it, discuss it with others and try to better i for the rapid development of the human race. ese vears. The Early human beings strove to record their world by carving crude diagrams on stone. All the drawings that we see in old caves is just that; storing images seen, trying to analyse them and discussing it with others in the tribe. Let us consider figure 1.2. This art developed through the ages by way of materials and skill. By the mid-nineteenth century, photography was well established. Image processing that we study starts from this era. Though it was stated earlier that the human eye-brain mechanism represents the ultimate imaging system, image processing as a subject involves processing images obtained by a camera. With the advent of computers, image processing as a subject grew rapidly. Images from a camera are fed into a computer where algorithms are written to process these images. Here, the camera replaces the human eye and the computer does the processing, Hence, image processing as an engineering subject is basically manipulation of images by a compute Tife, which does not go into action, i6 = Digital Image Processing & Fig. 1.3 4.6 IMAGES (GGSIPU, Delhi, Sem. Exam; 2008-09) (10 Mark Let us now define what we mean by an image. The world around us is 3-dimensional, whi? images obtained through a camera are 2-dimensional. Hence, an image can be defined as 2-dimensional representation of a 3-dimensional world. Consider the image shown ! figure 14. y 30 World Camera 2-D image Fig. 14 In moving from the 3-dimensional world to the 2-dimensional image, we loose! dimension. Depth of information is lost. As can be seen from the figure 1.4, the handle of" tea cup is missing, All family pictures, photographs on identity cards etc, are 2-dimensio If this statement is not clear, let us take a simple example. 4.64 Example ofa 1-Dimensional and a Weed 2-Dimensional Signal Consider a voltage signal shown in figure 1.5. Weare all familiar with a signal ofthis kind. Here the voltage is varying with respect to time. This is 7 a typical dimensional signal. If we want to locate a dot on the wave, all we need to know is its corresponding time, Fig. 1.5Let us see why images are 2-dimensional functions. Consider the image shown in figure 1.6. y [ee] | Fig. 1.6 Fig. 4.7 | In this case, to locate the dot shown, we need to know its position in two directions (x and y) in figure 1.7. Hence, all images that we see are 2-dimensional functions. A typical « image is represented as shown in figure 1.7. Here, (x,, y,) are the spatial coordinates and f 4 is the gray level (colour in the case of colour image) at that point. Hence, gray level f varies 1, with respect to the x-and y coordinates. ht is actually a very small section of the electromagnetic energy spectrum. The entire spectrum is shown in figure L8. 18 Introduction and Digital Image Fundamentals m= o The optical spectrum optical spectnu 10 200 300 390 455 492 577 597 622 770 1800 ecco 410° 10° Voilet Blue Green Yellow Orange Red | Near Visible light In tedium Far Extreme fra-red J Cosmic rays X-rays mmyaves Radio waves Aucio frequencies LPL ee LP FS | 10401010710 101010710710" 1 10 10° 10° 10* 10° 10° 10” 10° 10° 10°10"'10"710" 10" ty Fig. 1.8 Electromagnetic energy, as the name suggests, exists in the simultaneous form of electricity and magnetism. These two forms of the same energy are transmitted together as 1 electromagnetic radiation. One cannot exist without the other. A flow of electric current always produces magnetism, and magnetism is used to produce electricity. Electromagnetic1 Digital Image Processing m radiation is propagated outwards from its source at a velocity of 300,00,0000 meter second (3 x 108 m/sec). Although our natural source of electromagnetic radiation iS the there are also a number of man-made sources which among many others, include tun filament lamps, gas discharge lamps and lasers. Light is a band of electromagnetic rad} mediated by the human eye and is limited to a spectrum extending from 380 nm to 76 Most of the images that we encounter in our day to day life are taken from cameras are sensitive to this range of the electromagnetic spectrum (380-760 nm). We must not f, though, that there are cameras which are capable of detecting infrared, ultraviolet ° Xerays and radio waves too. ‘The electromagnetic spectrum can be expressed in terms of wavelength and frequ_ The wavelength (i) and the frequency (f) are related by the expression. e 7 Here c is the speed of light = 3 x 108 m/sec. EXAMPLE, 1.1. Calculate the frequency of oscillation of green light. Solution : It has been known that green light has a wavelength of approximately 50¢ (600 x 10° m) Its frequency of oscillations can be calculated using Equation (1.1). Salo =f _ 3x10 m/sec P93 Sono m f 6x 104 Hz i, the frequency of green light is 600,000,00,000,000 cycles/sec. Hence, its more convenient to discuss electromagnetic in terms of wavelengths (y rather than frequencies (Hz). [18 UNITS OF INTENSITY ‘We know that for an object to be seen, some amount of light has to fall on it. The unit of luminous intensity (I) is candela (cd) (61 unit) which by definition, is (1/é of a square centimeter of the surface of a black body radiator at the absolute temperatur 2045 K. Important Point: A black body radiator is one that absorbs all the radiation incident upon it a ts no reflecting per. The rdation from a heated Back body source is called black bs radiation and is always quoted in Kelvin. ‘The unit is called candela because initially it was defined as a standard candle burt a specific amount of wax per hour. There is another important unit apart from candela we encounter. This unit is called lux. 1.9\ THE HUMAN VISUAL SYSTEM It is important for designers and users of image processing to understand the characterisIntroduction and Digital Image Fundamentals = @ | of the human vision system. For efficient design of algorithms whose output is a photograph or a display viewed by a human observer, it is beneficial to have an understanding of the mechanism of human vision. Many interesting studies have been carried out and the subject of visual perception has grown over the years. We begin with the structure of the human eye. Figure 1.9 shows the horizontal as well as the vertical cross-section of a human eye ball. i, The front of the eye is covered by a transparent surface called cornea. The remaining outer cover, sclera, is composed of a fibrous coat that surrounds the choroid, a layer containing blood capillaries. Sclera is the white portion of the eye. Inside the cholorid, is the retina which is composed of two types of photoreceptors. These photoreceptors are specialised to transduce light rays into receptor potentials. These 4, photoreceptors are called rods and cones (named due to their shape). Rods are long and slender while cones are shorter and thicker. Nerves connecting the retina leave the eyeball through the optic nerve bundle. Light entering the cornea is focussed on the retina surface by a lens that changes shape under muscular control to perform proper focussing of near and distant objects. The iris acts as a diaphragm to control the amount of light entering the eye. Visual axs Sclera Nasal sido Optic nerve Fig. 1.9 Each retina has about 6 million cones and 120 million rods. Rods are most important for black and white vision in dim light. They also allow us to discriminate between different shades of dark-and light and permit us to see shapes and movements. At low levels of illumination, the rods provide a visual response called Scotopic Vision. The fact that we can see in the dark is due to this scotopic vision. \ Cones provide colour vision and high visual acquity (sharpness of vision) in bright light. Cones respond to higher levels of illumination, their response is called Photopic Vision. In the intermediate range of illumination, both rods and cones are active and provide Mesopic Vision, In moon light, we cannot see colour because only the rods are functioning. The distribution of rods and cones in the retina are shown in figure 1.10.= Digilal Image Processing m 60} Numbor of rods and cones ‘per mm’ 107 3 5 60° ce 20° 0 20° 40° 60° ‘80° Temporal on retina Nasal Fig. 1.10 Permetrc angle, degree Ata point near the optic nerve, called fovea, the density of the cones is the greatest. Dus to this, itis the region of sharpest photopic vision. The photoreceptors are connected to th; nerves and these nerves combined together, leave the eyeball as the optic nerve. 4.9.4 Image Formation In Eye The main difference between the lens of the eye and an ordinary optical lens is that the former is flexible. The shape of the lens is controlled by the tension of the ciliary muscles These muscles help in focussing of objects near or far. The qistance between the centre of th lens and the retina (focal length) varies from 14 mm to about 17 mm, When the eye focuse: on an object far away (approx. > 3 m) the lens exhibits its lowest refractive power (Foca length = 17 mm). When the eye focuses on near by objects, the lens of the eye is strongly refractive (Focal length = 14 mm), Let us take an example: EXAMPLE 1.2. Consider an observer looking at a lamp-post which is at a distance of 50 m, Ifthe height of the lamp post is 10 m, find the size of the image formed in the retina (Retinm image) Solution : Since the image is far away, the focal length is approximately 17 mm. We use th similarity of triangles. Let the retinal image be r1m Introduction and Digital Image Fundamentals = ® 1 =34 mm «. The height of the retinal image is 3.4 mm If the same image is observed at a distance of 100 m, we get A Ae 100 17 r= 17mm 4.9.2 Visual Phenomena (GGSIPU, Delhi, Sem. Exam., 2008-09)(10 marks) The human eye is complex system and the images that we perceive are also equally complex. We shall now explain the visual phenomena What the human eye senses are in general intensity images-Intensity, Brightness and Contrast are three different phenomena. Intensity of a light source depends on the total amount of light emitted by it. Hence, intensity is a physical property and can be measured. Brightness on the other hand is a psycho-visual concept and hence is actually a sensation to light intensity. Contrast may be defined as the difference in perceived brightness. 1. Contrast Sensitivity ‘The response of the human eye to changes in the intensity of illumination is known to be non-linear. Consider the simple experiment : aL * ey Fig. 1.42 We take a diffuser (opaque) glass plate and illuminate it from the bottom with constant illumination I. The observer is asked to look at this glass plate, from the top. At the centre of this glass : plate, the intensity of illumination is increased from I to | + MI. The observer is now asked to observe whether he or she can detect this increase. If the observer cannot detect the change, the intensity is further increased by another increment of Al. This procedure is continued till the observer detects the difference. This is known as the just Noticeable Difference (JND). This ratio of ao is called the Weber ratio Weber ratio = “fe Here, I is constant and J, is the incremented value. A low Weber ratio implies that even a small variation (Al) was detected by the observer. A higher Weber ratio implies that large variations (AI + Al + ...) were required for the observer t© 1 Digital image Processing Hence, a low Weber ratio means sl that the observer has good discernible" vision, At proper illuminations, the Weber ratio of a group of people in more or less constant at 0.02, Weber ratio also depends on the illumination I. As can be seen from the graph, the weber ratio is large at very low as well A as very high levels of illumination. This should not come as a surprise. Our discrimination quality reduces when we Fig. 4.14 are ina room that is not well lit. At the same time out discrimination quality also reduces when there is too much light. 2 Brightness Adaptation ‘The range of light intensity levels to which the human visual system can adapt is enorm, of the order of 10" from the scotopic threshold to the glare limit. It simply means we see things in the dark and also when there is a lot of illumination. It has been shown the intensity of light perceived by the human visual system (subjective brightness) logarithmic function of the light intensity incident on the human eye. The curve in the figure 1.15 represents the range of intensities that the human viy system can adapt When illumination is low, it is the scotopic vision that plays a domiz role while for high intensities of illumination, it is photopic vision which is dominant As can he seen from the figure, the transition from scotopic vision is gradual an certain levels of illuminations, both of them play a role. To cut a long story short, dynamic renge of the human eye is enormous. But there is a catch here. The eye ca, operate over the entire range simultaneously, ie, at a given point of time, the eye can observe a small range of illuminations. This phenomena is known as Brightness adapta ‘The fact that our eyes can operate only on a small range, can be proved by a simple experi» on ourselves. Inten consequences.8 Introduction and Digital Image Fundamentals a @ Stare at the sun for a couple of seconds, the eye adapts itself to the upper range of the intensity values. Now, look away from the sun and you will find that you cannot see anything for sometime. This is because the eye takes a finite time to adapt itself to this new Tange. A similar phenomenon is observed when the power supply of our homes is cut-off in the night. Everything seems to be pitch dark and nothing can be seen for sometime. But gradually our eyes adjust to this low level of illumination and then things start getting visible even in the dark. 3. Acquity and Contour (Mach Bands) Mach band are named after the Austrian physicst Ernst Mach. (1838-1916). Consider a set of gray scale strips shown in figure 1.16. Fig. 1.16 4 Each of the eight strips have uniform intensities within the strip. This can be seen if we ® place our hand on all the other strips and observe only one strip at a me. But when we look at all the eight strips together, visual appearance is that each strip looks darker at its right side than its left. This is called the Mach band effect. ‘The actual and the perceived intensity charts are shown in figure 1.17 (a) and (). 1 (@) Actual intensity (0) Perceived intensity Fig. 1.47 The intensities tend to overshoot at the right hand edges. The overshoot is a consequence of the spatial frequency response of the eye. The human eye possesses a lower sensitivity to high and low spatial frequencies than to mid range frequencies. The implication of this is that perfect edge contours in an image can be sacrificed to a certain extent as the human eye has an imperfect response to high spatial frequency brightness transitions. 4, Simultaneous Contrast Let us consider the image shown in figure 1.18. Each of the small circles have the same intensity, but because the surrounding gray level of each of the circles is different, the circles do not appear equally bright. Hence, the intensity that we perceive are not actually the absolute values. The happiness of every country depends upon the character of its people rather than the form of is government, — Thomas C. Haliburton)1m Digital image e+ #="+ Fig. 1.18 5, Integration Another important property of our eye is that it integrates the scene as a whole. This i. a property that facilitates programming. When we look at a picture, we do not look at eat point, but the image as a whole. Let us take an example, read the following lines as fast as you can. “According to a recheearch at an Elingsh University, it doesn’t matter in what order th letters in a word are the only important thing is that the first and the last letter is at the right) place. The rest can be total mess and you can still read it without a problem. This is becausg, we do not read every letter by itself but the word as a whole.” You could 1ead i, inspite of the spelling mistakes only because ot the integration properts of the eye. Before, we end one issue needs io be sorted out. A very common question that people ask is what is the difference between image pro-zscing, computer graphics and computer vision? Image processing deals with manipulation «i 1ni=es. A inpul image is-moditied into » new image. Computer graphics deals with creation 0! images. In computer graphics, models (2D or 3D) are created using mathematical functions (Descriptors). Computer vision Whidh is also known as Machine vision deals wtih the analysis of image corttent. Computer vision is used to automate a process. Table 1.1 -— Output | Image | Description Input _—! (Mathematical function) Image TP. cv. Description CG. 2 (Mathematical function) From the Table 1.1 Image processing > Input (Image) - Output (Image) Computer graphics - Input (Description) - Output (Image) Computer vision Input (Image) - Output (Description) indira Gand)8 Introduction and Digital Image Fundamentals = 1.10 SCENES (@ Scene The term ‘scene’ stands for a land-scape or a view as seen by a spectator. The scene, indoor or outdoor, we see around us is, in general, three-dimensional. i¢., each point in the scene may have different depth from the observer. However there are scenes which can be considered as two-dimensional, i¢., points comprising the scene have equal depth. Some examples of two dimensional scenes are as under (a) Flat terrain seen from satellite situated at a very high altitude, (®) The shadow of objects created by X-rays, (0) Flat polished surface of objects or thin slices seen through microscope. (@ Documents printed on paper. 1. APPLICATIONS ‘As a matter of fact, digital image processing and analysis techniques are used today in a variety of problems. Many application oriented image analyzers are available and are working, satisfactorily in real environment. The following are a few major application areas 1. Office Automation This includes optical character recognition, document processing, cursive script recognition, logo and icon recognition, identification of address area on envelops etc 2. Industrial Automation Industrial automation area includes automatic inspection syste ting, automatic assembling process related to VLSI manufacturing, PCB checking, robotics, oil and natural gas exploration, seismography, process control applications etc 3. Bio-Medical Bio-medical area includes various fields such as ECG, EEG, EMG analysis, cytological, histological and stereological applications, automated radiology and pathology, X-ray image analysis, mass screening of medical images such as chromosome slides tor detection of various diseases, mammograms, cancer smears, CAT, MRI, PET, SPECT, USG, and other tomographic images, routine screening of plant samples, +D reconstruction and analysis etc. ., non-destructive t 4, Remote Sensing It includes natural resources survey and management, estimation related to agriculture, hydrology, forestry, mineralogy, urban planning, environment and pollution control, cartography, registration of satellite images with terrain maps, monitoring traffic along roads, docks and airfields ete. 5, Scientific Applications Scientific applications include high energy physics, bubble chamber and other forms of track analysis ete. 6. Criminology It includes finger print identification, human face registration and matching, forensic investigation etc. CC,1 Digital mage FREe+= ~~ 7. Astronomy and Space Applications It includes restoration of images suffering from geometric and photometric disty ‘computing close-up picture of planetary surfaces etc. 8. Meteorology This particular are includes short term weather forecasting, long term climatic ¢ detection from satellite and other remote sensing data, cloud pattern analysis etc. 4, information Technology This includes fascmile image transmission, videotex, video-conferencing and videos etc. 10. Entertainment and Consumer Electronics This includes HDTV, multimedia and video-editing etc. 11. Printing and Graphic Arts This includes colour fidelity in desktop publishing, art conversation and dissemi etc. 12 Military Applications This includes missile guidance and detection, target identification, navigation of pil vehicle, reconnaissance and range finding etc. va Important Point: Hit may be noted tha in addition to the above mentioned areas, ar portant application of mage procesing techniques is improvement of quality or appearars 4 gioen image, 4.12 COMPONENTS OF AN IMAGE PROCESSING SYSTEM Digital image processing is basically modification of images on a computer. The & componenis of an image processing system are shown below. 1. Image Acquisition 2 Image Storage Tes 3, Image Processing storage 4. Display 5. Transmission (if We ee each Digital eae aaa conputet rorsoaing iP Trans™ 1. image Acquistion and software Image acquisition is the | first step in any image processing system, The Display general aim of image 00 acquisition is to transform Fig. 119, A general-purpose image processing system.11 Introduction and Digital Image Fundamentals a @ > an optical image (real world data) into an array of numerical data which could be later manipulated on a computer. Image acquisition is achieved by suitable cameras ‘We use different cameras for different applications. If we need an X-ray image, we use a camera (film) that is sensitive to X-rays. If we want an infra-red image, we use cameras which are sensitive to infra-red radiations. For normal images (family pictures etc.) we use cameras which are sensitive to the visual spectrum. In this book we shall discuss cameras (sensors) which are sensitive only to the _ visual range. Quantum detectors Quantum detection is the most important mechanism of image sensing and acquisition. It relies upon the energy of absorbed photons being used to promote electrons from their stable state to a higher state above an energy threshold. Whenever this occurs, the properties of that material get altered in some measurable way. Planck/Einstein came up with a . relationship between the wavelength of the incident photon and the energy that it carries. e-# (12) where —_e = Energy carried by a photon h = Planck's constant, 6.626 x 10 Js c =Speed of light, 3 x 10° m/sec 2. = Wavelength of the incident radiation On collision, the photon transfers all or none of this quantum of energy to the electron. The equation (1.2) is very important as it tells us t!-.t the maximum wav to which the quantum detector will respond is determined 2; the energy threshold and hence by the material selection. "Of the several modes of operation of qu. .zm detectors, the most important ones are (@) Photoconductive () Photovoltaic (@) Photoconductive The resistance of photoconductive materials drop in the presence of light due to the generation of free charge carriers. An external bias is applied across the material to measure this change. The principle of photoconductivity is used in Vidicon imaging tubes. Vidicon imaging tubes Vidicon is a vacuum tube used for image acquisition. Figure 1.20 is the schematic diagram of the vidicon tube image sensor, The light image is focused on a transparent signal plate coated on the inside with a photoconductive target material. The front face of the signal plate is coated with SnO, (tin oxide) which forms the conducting layer. This plate is biased to a Positive potential of about 50 V. The target coating is maintained at approximately 0 V by a mesh gtid which is placed behind it.6 1 Digital Image procoselly Glass face plate Electron beam 1 Target Sn0> ' Tayer, =) Mesh grid —o} (Target connection Focus and deflection cols + (sas) Load resistor 1 cer Voltage signal output Fig. 1.20 Due to this the signal plate and target behave as a continuous array of capacitors conti, a photoconductive dielectric. When there is no light incident on the target, its resistan very high due to which the charge across the capacitor is maintained. However, when| strikes the target, the dielectric material becomes leaky and the target side voltage rises. target is now scanned by a low velocity electron beam. When this beam strikes the dischar capacitor, current flow <0 restore the target side to its original voltage of 0 V. This cw also flows in the exten al bias circuit, which produces a voltage drop in the load resist. In order to produce the output necessary for broadcast video applications, the eley beam is scanned across the surface of the target by electromagnetic deflection coils. (Defle coils were encountered when you studied CRO, (cathode ray oscilloscopes), in your k semesters). The original vidicons use antimony-trisulphide, which is a photocondu material, a their target coatings. Other forms of vidicon are made using more complex k. materials. The silicon target vidicon achieves improved sensitivity and a spectral resp similar to that of the human eye. ‘The biggest drawback of the vidicon tube is that it uses the concept of deflection to position the electron beam. These deflection coils make the vidicon bulky and are respor for major power drain. Since the coils are magnetic, the vidicon also becomes sensiti external fields. (b) Photovoltaic Photovoltaic devices consist of semiconductor junctions. They are solid state % composed of discrete silicon imaging elements known as Photosites. Photovoltaic device: a voltage output signal that is proportional to the intensity of the incident light. No ext bias is required as was in the case of photoconductive devices, The technology used in solid-state imaging sensors is based principally on charge coe devices, commonly known as CCDs. Hence, the imaging sensors are called CCD ser’ The solid state array (CCD) can be arranged in two different configurations. (bl) Line array CCD (b.2) Area array CCD.1 Introduction and Digital Image Fundamentals (ba) Line Arrays The line array represents the simplest form of CCD imager and has been employed since the early 1970s. Line arrays consist of a one-dimensional array of photosites. Imaging pixels ZZ | SS | Video output Output register Fig. 1.2 A single line of CCD pixels are clocked out into the parallel output register as shown in figure 1.21. The amplifier outputs a voltage signal proportional to the lass plate contents of the row of photosites. One thing to note is that line array CCD scans only one line (hence is one dimensional). In order to produce a two-dimensional image, the line array CCD imager has to be used as a scanning device by moving this array over the object by some mechanical activity. This technique is used in flat bed scanners (the scanners that you come across in your laboratory or in acyber cafe). A line array CCD can have anything from a few elements to upto 6000 or more. (6.2) Area Arrays ‘The problem with line arrays is that it scans only one line. To get a two-dimensional image, we need to mechanically move the array over the entire image. Area arrays or matrix arrays consist of a two- dimensional pa of photosites. They make it possible S568) ine array to investigate static real world scenes without any Fig. 1.22 mechanical scanning, Thus, much more information can be deduced from a single real time glance than would be possible with line arrays. Area arrays can be seen in all the digital cameras that we use for video imaging, The area arrays are move versatile than the line arrays, but there is a price to be paid for this, Area arrays are higher on cost and complexity. ‘Area sensors come in different ranges, ie., 256 * 256, 490 x 380, 640 x 480, 780 x 575. CCD arrays are typically packaged as TV cameras. A significant advantage of solid state array sensors is that they can be shuttered at very high speeds (1/10,000 series). This makes them ideal for applications in which freezing motion is required. ‘dream: not only plan, but also believe. — Anatole France ‘o accomplish great things, we must not only act, bulB w Digital Image rtuesors LN Sy LAH yy i VY LU L CL L Lp>— Fig. 1.23 2, Image Storage All video signal are essentially in analog form, i, electrical signals convey luminaney and colour with continuously variable voltage. The cameras are interfaced to a compute where the processing algorithms are written. This is done by a frame grabber card. Usualh, a frame gabber card is a printed circuit board (PCB) fitted to the host computer with ite analog entrance port matching the impedance of the incoming video signal. The A/D converte: translates the video signals into digital values and a digital image is constructed. Fram: grabber cards usually have a A/D card with resolution of 8-12-bits (256 to 4096 gray levels} Hence, a frame grabber card is an interface between the camera and the computer. ‘Computer bus (Sloi Data (6-bits) Ye on | (ET Camera1 Introduction and Digital Image Fundamentals = eS Frame grabber card has a block of memory, separate from the computers own memory, large enough to hold any image. This is kown as the frame buffer memory. Image data usually in the form of bytes (8 bits), is written into the frame grabber memory under the computer control, usually by DMA transfers. The contents of the memory are continuously read out at video rate (30 frames/sec.), passed through the D/A converter and displayed on the monitor. The output has a colour map or a colour Look Up Table (LUT) to permit pseudo colouring. The table consists of 8-bit values for each of the red, green and blue guns of the monitor. The commercially available frame grabber cards have additional features such as ability to zoom regions of the image and pan the images. The frame buffer memory can be upto 5 MB. Images being 2-dimensional functions, occupy a lot of space and hence the storage space on the host computer has to be large. Let us try to find out as to how much space is actually taken by images. Each image is stored as a matrix, where every value of the matrix represents the grey level at that point. Suppose the image (matrix) is of size A x B and suppose we recquite C bits to represent each element of the matrix. Memory space required to store this image is approximately equal to A x B x C bits. Suppose we have D such images then total number of bits = Ax Bx Cx D__ ..(1.3) Fig, 1.25 Fig. 1.26 Thus the formula = ABCD. gives us a fairly good idea of the amount of space required. ASapoe e+ CAPD Number of bits Low quality video phone oe | + 6 = 0.2 * 10° bits Digital broadcast TV 7 | T 3 10? bits HDTV 120 | ny 30 8 = $83 < 10" bits With the advent of the PCI bus, the host computers memory could now be used as the frame buffer memory. This is because the PCI bus faciliates fast communication. Hence, the image could now directly be stored on the computer RAM. Due to this, the frame buffer memory has become more or less obselete. Only the very high end frame grabber card still have a small frame grabber memory embedded on them. Images being two-dimensional functions, cccupy a lot of space and hence it is advisable to have a computer with a sizeable hard disk and also a good RAM. Proceesed images could be stored on magnetic floppies or CDs. Archival data are stored on magnetic tapes. 3, Image Processing Systems ranging from microcomputers to general purpose large computers are used in 1g, Dedicated image processing systems connected to host computers are veryBo 1 Digital Image Probessus popular nowadays. Processing of digital images involve procedures that are usually expres in algorithmic form due to which most image processing functions are implementej software. The only reason for specialized image processing hardware is the need for sp, in some applications or to overcome some fundamental computer limitations. The tre though is to merge general purpose small computers with image processing hardware, stated in the earlier section, frame grabber cards play the important role of merging t image processing hardware with the host computer. One thingsthttt we should remember image processing is characterized by specific solutions. Hence, tliere is no one way to prow images. A technique that works well in one area can be totally useless in some oj} applications. The image processing software that is used in this book is MATLAB. 4, Display A display device produces and shows a visual form of numerical values stored in computer as an image array. Principle display devices are printers, TV monitors and CRi Any erasable raster graphics display can be used as a display unit with an image processi system, However, monochrome and colour TV monitors are the principal display devix used in modern image processing systems. These raster devices convert image data into a video frame. One major problem is, must refresh the screen af a fate,of about 30 frames per second to avoid flicker. since sor computers are rnahlao transfer data at such high speeds, it is a good idea to buy a hig, end frame grabherard which has frame buffer memory on it. If you want fo build a image processing the system a home, you should have a Vi Graphics Card (VGA) attached to a computer and a monitor that supports that VGA car One could then use a simple camera that comes along with the system. 5. Transmission There are a lot of applications where we need to transmit images. The stages in & transmission .{ an image over a channel or network are shown in figure 1.27. tuninaton Display Rellecied Camera Traisission light . channel — ——A\ coder | ----- +| Decoder Fig. 4.27 The image sequence from the camera is coded into as concise a representation as possi for transmission over the channel. Most transmission in broadcast television are still in analog form and analog cod" methods are used to make it as efficient as possible. NTSE. PAL and SECAM are the 3 nv} coding systems used in various parts of the world (USA uses NTSE, while India uses PA! Digital image coding is a high activity area. In image processing; it is concerned with effi transmission of images over digital communication channels. A variety of indigenous it have been developed in recent years. Coding is influenced by the type of the channel t! to carry the image signals, Several different types of transmission channels are encount™ in practice including cables, terrestrial radio, satellites, and optical fibres.1 Introductior and Digital Image Fundamentals = @ Gamma In the image transmission chain, there are many elements which exhibit a non-linear behaviour. If x is the input and y is the output, then y =ext c = Constant = Gamma of the device The camera, the display device and eye all have non-unity gammas (all are non-lifiear). Hence, to make sure that the perceived gray scale in the displayed image is correct, it is necessary to insert an additional compensating, non-linear device called the gamma corrector. We have discussed a lot of things so far and it is advisable at this stage to have a branch diagram of these topics. Basic elements of an image processing system | 7 | | fd image . Image image Display Transmssion acquisition storage processing | ‘Quantum defectors ; Gamma Frame grabber correction care Photoconductive Photovoltaic Frame butter : memory Vidicon cop Imaging tubes Linear Area array array : Fig. 1.28 |_IMAGE TYPES It was stated earlier that images are 2-dimensional functions, refer figure 1.29. Images can be classified as follows: 1. Monochrome images (Binary images) 2. Grey scale images 3. Colour (24-bit) images 4. Half-toned images 1. Monochrome image In this, each pixel is stored as a single bit (0 or 1). Here, 0 represents black while 1 represents white. It is a black and white image in the strictest sense. These images are also called bit mapped images. In such images, we have only black and white pixels and no other shades of gray. 2. Gray Scale Image : : Here, each pixel is usually stored as a byte (S-bits). Due to this, each pixel can have ues ranging from 0 (black) to 255 (white). Gray scale images, as the name suggests have . 1 ayhije and various shades of gray presents in the image.1 Digital Image Processing & 3, Colour Image (24-bit) Colour images are based on the fact that a variety of colours can be generated by r the three primary colours viz. Red, Green, and Blue, in proper proportions. In colour ij each pixel is composed of RGB values and each of these colours require 8-bits (one by, its representation. Hence, each pixel is represented by 24-bits [R(8-bits), G(8-bits), B(® A 24-bit colour image supports 16, 777, 216 different combination of colours. Colour images can be easily converted to gray scale images using the equation X =030R +059 G +011 B An easier formula that could achieve similar results is X= R+G+B "3 4, Half Toning It is obvious that a gray scale image definitely looks better than the monochrome asit utilizes more gray levels, But there is problem in hand. Most of the printers that v (inkjet, lasers, dot matrix) are all bi- level devices, ic, they have only a black cartridge and can only produce two levels (black on a white back- ground). In fact, most of the printing jobs are done using bi-level devices. You have all read newspapers at some point of time (hopefully). The mages do look like gray level images. But if you look closely, all the images generated are basically using black colour. Even the images that you see in most of the books (including this one) are gene using black colour on a white background. Inspite of this we do get an illusion of seeingy levels. The technique to achieve an illusion of gray levels from only black and white is called half-toning. The human eye integrates the scene that it sees. Consider a simple example. Cor two squares of say 0.03 x 0,03 sq. inch. One of these squares contain a lot of black dots the other square contains fewer black dots. When we look at these squares from a dis, the two squares give us a perception of 2 different gray levels. This integration prope- the eye is the basis for half toning. In this, we take a matrix of a fixed size and depey on the gray level required, we fill the matrix with black pixels. Let us take’an example. Consider a 3 x 3 matrix. This matrix can generate an illusion of 10 different gray when viewed from a distance. Here, the first block represents white, the last block represents black and all the blocks represent intermediate values of gray. So, in the case, while printing, if we enc® gray level 0, we plot 9 pixels which are all white, If we encounter gray level 1, we | pixels of which only one pixel is black and so on. Fig. 1.30.@ Introduction and Digital Image Fundamentals = White Fig. 1.31. As is evident a 3 x 3 matrix will generate 10 differént gray levels. The dots that are placed in the 3 x 3 matrix example can be in any order. But we need to follow two rules. 1. Dots should be arranged in such a manner so that they don’t form straight lines. Lines are very obvious-to the viewer and hence should be avoided. Example, suppose in an image we have 4 consecutive gray levels of value 3, and if we use a code a shown in figure 1.32 the half toned image would look like figure 1.33. Fig. 1.32 Fig. 1.33 It can be seen that it does not look like a gray level. It looks like a straight black line. Hence, lines should be avoided while defining the matrices. 2Ifa pixel ina particular matrix is black for gray level i, it should be black, for all further levels j > i. This reduce false contouring. Half toning gives excellent results and we do perceive a gray level image just by using black pixels on a white background. The logic to implement a half-toned image from a gray level image is given below. 1. Define the size of the half-toned matrices based on the number of gray levels the original image has. 2. Generate the matrices starting from all white pixels to all black pixels. 3, Read the original image. For every gray level value read, plot the corresponding matrix. \ OE1 Digital Image Processing Remember, the physical size of the half-toned image will always be bigger than original images as for every single gray level value, we output an entire matrix. 1. Image Sensor An image sensor intercepts the radiant energy propagating from the scene and transfor, it to produce an intensity image. Following two different technologies are most common : (i) Photochemical (ii) Photoclectronic Photochemical methods have the advantage of combining image formation and recordin on a single entity called ‘photographic film’. Photoelectronic methods, on the other han; separate the recording process from image formation and detection. However, in the secon method, recorded image can be converted more easily to a form which is suitable to compute processing. Devices used as Image Sensors The devices used as image sensors range from still camera to multispectral scanner oy board in satellites, Television cameras are widely used and are easy to operate. There arez many as five different systems used in television imaging as under : () Image orthicon tube (ii) Vidicon tube (ii) Iconoscope tube (ia) Image dissector tube (0) Charge transfer devices Charge-Coupled Devices Most popular amongst the charge transfer devices is charge-coupled devices which ara commercially called as CCD camera. They behave similar to an array of metal-oxids semiconductors field-effect transistors (MOSFET) in a sense that both have a source, a drain and a depletion region, When an image is formed on a CCD array, the photons incident om the semiconductor accumulate charges in the depletion region. Hence, during image scanning the CCD image plane can be considered as a monolithic array of charged capacitors. Now these charges are transferred to output by applying a series of clock pulses between the source and the drain, This transfer can be made either one line at a time or a complete frame at a time. In the second case, a frame buffer would be required to avoid data loss. Multispectral Scanners Multispectral scanners also use the same technology except some filters corresponding to different wavelength bands are added and a full size image is recorded for each spectral band. Apart from there optical image sensors image can also be captured in devices like, X-Ray, scanner, radar, ultrasonic ranger, magnetic resonance imaging (MRI) system and so on. 2. Digitizer _A digitizer produces digital image composed of discrete intensity values at discrete Position. In other words, a digitizer is a device for converting the output of the physical sensing device into digital form. For instance, in a digital video camera, the sensors produce