This full-text paper was peer-reviewed and accepted to be presented at the IEEE ICCSP 2015 conference. Image Fusion Using Adaptive Thresholding and Cross Filtering Ninad Dileep Mehendale and Snehal Ajit Shah continuum producing a range of shades of grey. A colourimage Abstract-this paper presents an algorithm for multifocus consists of three grayscale images namely R, G and B which image fusion in spatial domain using adaptive thresholding and stand for Red, Green and Blue. cross filtering. The basic idea is to gather edge information from the source images and then segment the source images into blocks using the soft blending technique instead of cutting it into simple blocks. The differences between the edge information from both the source images is computed and the mean of these differences is set as the adaptive threshold. The differences obtained from each block are compared with this adaptive threshold and only those blocks for which the difference exceeds the threshold are chosen and incorporated into the final fused image. A further enhancement is achieved by making this process iterative. In every next iteration, the image is divided such that each block is subdivided by twice the number of divisions used in the last iteration. Also the number of iterations is fixed to 100. The performance of this method has been tested on many pairs of multifocus images and has shown promising results on comparison with existing methods. Fig. 1. Photograph clicked by a phone camera Keywords- Image Fusion, Multifocus Images, Spatial Domain, Edge Information, Soft Blending T. INTRODUCTION What is an image? It is a graphical representation of the external form of a person or a thing in art. In simple words, it is a photograph taken by a camera, our 'smart' phones. However, the problem with the images taken with such cameras is that the focus of the image is concentrated only at a Colour image Black and white Grayscale particular region of the image. This is because the optical lenses of the imaging sensor in our camera with long focal Fig. 2. Representation of black & white, Grayscale and length have a limited depth of field. We do not get all the colour images desired information we need from a single image captured by Image fusion fmds applications in the field of medicine our phone camera. The information which is desired is that if where it is used to fuse the CT scan images with the MRI scan our image be that of a 3D scene, all the objects of the image be images. These scans individually do not give significant in focus. But such "all in focus" images cannot be obtained details, but when fused, they do. The CT scan individually from the image capturing devices used nowadays. provides us with information of the geometry of our brain. The There are many types of images like black and white images, MRI, on the other hand provides us with information of the location of tumors present in our brain. Fusing these two scans grayscale images, colour images etc. a black and white image provide us with crucial information of the location of a consists of only two colours viz., black and white (like a particular tumor in a particular part of the brain. This chessboard). A grayscale image combines black and white in a technique started in 1992 and it is called MS Fusing. Image fusion is also used for face recognition, satellite imaging, auto exposure correction etc. Image fusion is the technique in which Ninad Dileep Mehendale is with the Indian Institute of Technology, mUltiple images are combined to form a single image. This Mumbai.India. (e-mail:ninad.mehendale@gmail.com) Snehal Ajit Shah was with Mumbai University, Mumbai, India. She is now single fused image provides more information than the source with Ninad's Research Lab, Thane, Mumbai, India (e-mail: images. Multifocus images: Generally, when a photo is snehalshah115@gmail.com) 978-1-4799-8081-9/15/$3l.00 © 2015 IEEE 0144 This full-text paper was peer-reviewed and accepted to be presented at the IEEE ICCSP 2015 conference. clicked, the objects which are to be captured are distanced weighted average of source coefficients as proposed by [1], from each other and the focal length of the camera can focus [2]. Analogous to other forms of information fusion, image only on one point which becomes a problem. This problem fusion is usually performed at any of the three processing becomes even more significant while taking satellite images. If levels: signal, feature and decision. Signal-level image fusion, there is a layer of clouds overlooking the earth, then the also known as pixel-level image fusion, defmes the process of images clicked focus more on the cloud layer hence providing fusing visual information associated with each pixel from a insufficient data. Fig. 3a and 3b are examples of multifocus number of registered images into a single fused image, images. representing a fusion at lowest level. As the pixel-level fusion is part of the much broader subject of multifocus and multisensor information fusion, it has attracted many researchers in the last two decades [3-6].In the last two decades, a lot of research has been carried out in the area of multifocus and multispectral image fusion. Multispectral image fusion based on intensity hue saturation method is described in Carper et al. [7] and that based on Laplacian pyramid mergers in Toet [8]. The multifocus image fusion proposed in Haeberli [9] uses the fact that the focused area of the image will have highest intensity compared to that of unfocused areas. Fig. 3(a). Image with foreground in focus. Fig. 3(b). Image with background in focus III. IMAGE FUSION There are two methods in which image fusion can be done: As explained before, image fusion is the process of merging Spatial domain Image fusion and Spectral Domain Image two photos with different objects in focus to give a fused fusion. In spatial domain image fusion, the source images are image which provides more information than both images cut in spatial domain and these parts are considered for the combined. Image fusion is an extremely important emerging fusion process. In spectral domain fusion, frequency of the technology. Fusion of images improves image quality by images is extracted. Not colour, not position and not even the reducing the amount of data like noise and removal of blurred space of the image is considered. The most popular method of data thereby retaining important information. Also it creates a image fusion used is the method of spectral wavelet transform. whole new image which is easy for human perception. Image In this method, images of high resolution and low resolution fusion is of many types, namely multiview image fusion, are merged and decomposed into components called wavelet multimodal image fusion and multifocus image fusion. components. The low resolution DC components are padded up with the high resolution images to get focused images. We have used the spatial domain image fusion because high speed is achieved as no complex components are encountered, we . ,-, , are not transforming actual images and as a result, 100% : t • I , ' originality of images is maintained. In this paper, we have T o proposed an algorithm which fuses multifocus images in I , spatial domain to give a smooth and a perfectly fused fmal I image. The software which we have used is MATLAB. It is a program which uses high level language and the image Foreground in focus Background in focus Fused image processing toolbox of this software is very powerful. Section II gives a summary of the existing methods of image fusion. An Fig. 4. The first image depicts a photograph having its overview of image fusion is given in section III of this foreground in focus. The second image is the same photograph technical note followed by the proposed method in section IV. taken with the background in focus. The third image is the Results are stated in section V and a comparison of results in final fused image. done in section VI followed by the conclusion section. In multifocus image fusion, images are taken with different focal lengths. The focal distance between the two lenses of the II. LITERATURE SURVEY camera is adjusted so that focus can be directed towards two different points. Most of the cameras available in today's times Image fusion can be as simple as taking pixel-by-pixel have automated focus and hence we get multifocus images. average of the source images, but that often leads to The optic model in Fig. 5 shows how multifocus images are undesirable side effects such as reduced contrast. Fusion can formed. broadly be classified as, fusion in frequency domain and in spatial domain. It can be implemented using various fusion rules e.g. 'mean' or 'max' where fused coefficient is average or maximum of source coefficients respectively. One can also take 'weighted average' instead, where fused coefficient is 0145 This full-text paper was peer-reviewed and accepted to be presented at the IEEE ICCSP 2015 conference. Procure the edge information of these images using the canny lens Film plate edge detector. The images are then subtracted row wise and column wise. As a result, we get a matrix consisting of the row B A c values and another matrix consisting of the column values. Selecting the minima of both the row and the column matrices, the image is cut into a 2*2 block. In the next iteration, the minima Circle of selected in the first iteration is replaced with a maxima and a new confusion minima is found out. This process of cutting the image into blocks is called as soft blending. 2 II 0 4 1 2 11 a o 1 o II 10 II Fig. 5. There are 3 objects at points A, B and C. when 4 1 I 2 7 2 2 capturing an image, only object A can be in focus and others appear blurred. Similarly, when object B is kept in focus, objects A and C will appear out of focus. This happens because of the circle on confusion as shown in the image. The circle of confusion is responsible for defocusing of the objects. Fig. 6. The resultant obtained after performing row and column Image fusion can be broadly classified in the categories of subtractions is shown. pixel/data level image fusion, feature level image fusion and In the left matrix, out of all the 4 values obtained i.e. 9, 4, 12, 11 of column number 1,2,3,4 respectively, we get the minima decision level image fusion. Of these, the pixel level image value (4) for column 2, so we select column number 2. Similarly fusion belongs to the spatial domain whereas the other two the transpose of the matrix in the left is computed and then we categories belong to frequency domain image fusion. Spatial obtain the row values by performing the summing operation for domain and frequency domain are the two domains of image the column matrix as in the matrix at the right. Out of all the 4 fusion. We have used spatial domain mainly because it is fast. values obtained i.e.15, 7,1,13 we get the minima value of (1) for In frequency domain, Gibb's phenomena is involved which column number 3. This marks the end of iteration one. The gives ringing effects to images. There are no ideal filters as a unused blocks are then sent to iteration 2 where again using result of which unwanted frequency and noise gets passed to the minima technique the blocks are divided. For every images. following iteration, each block is subdivided into a number IV. PROPOSED METHOD which is 4 times greater than the previous iteration. This process is called adaptive segmentation. Edge detection is the fundamental method which is used in image processing. In an image, an edge characterizes a boundary. It represents a sharp change in colour in an image i.e., an edge in an image is basically an area with a strong intensity contrast. The process of edge detection significantly reduces data to be analyzed in an image. It filters out useless information and preserves the crucial structural properties of an image. If the edge detection process is properly executed, the subsequent task of interpretation of the in-focus regions of the image becomes a lot easier. In case of multifocus image fusion, the purpose of extracting edge information is to provide strong visual clues that can help the recognition Fig. 7. Depicts the points at which the images are cut after process and can make a clear distinction between the in-focus obtaining the minima values. regions of the source images. There are many types of edge detectors namely Prewitt, Sobel, Laplacian number of A new adaptive threshold is set and again the process of selecting Gaussian, Canny, Roberts and zero cross. In this paper, we blocks having higher edge information are used. This process is have used the canny edge detector carried out up to 100 iterations. Even after all the iterations are performed, there are some blocks in the final fused image which The algorithm proposed by us goes as: remain blank. Before the first iteration is carried out, a copy of Select a source image clicked in two ways that is, having both the source images is made. These remaining blocks are then foreground in focus in the first image and having the background selected from the corresponding copies of source images by in focus in the second image. See Fig. 8a and 8b. comparing the magnitude of the edges in that particular region. The edges with the higher magnitude are then selected for incorporation in the fused image. Once the final output is 0146 This full-text paper was peer-reviewed and accepted to be presented at the IEEE ICCSP 2015 conference. obtained, cross filtering is done. The final image is then Figures. Fig. lla and llb show the input source images. Fig. lla compared with the two source images. By this we get to know has the foreground in focus and Fig. llb has the background in that significant data from which of the two source images is focus. Fig. llc and lld respectively represent the canny edge inculcated in the final image. Some regions of the final image are detection outputs of Fig. lla and lIb. then replaced by the same regions from the source images. This improves the smoothness of the image. VI. EVALUATION AND COMPARISON WITH OTHER METHODS A fusion artefact introduced into the fused image by the fusion process could lead to a benign object being classified as a threat or a valid target; so an efficient fusion method is one that introduces minimum artefacts. There are various parameters using which we can be used to evaluate the final fused image. The most recent parameters which are used for the purpose of evaluation are the petrovic metrics. The parameters which we have considered for the evaluation of our outputs are as follows: Fig. Sa and Sb. Input to our program. Images having foreground 1. Average Pixel Intensity (11 or F): an index of contrast of the and background in focus respectively images 2. Average Gradient (G): Clarity and sharpness measure of the V.RESULTS image 3. Standard Deviation (SD or cr): reflects spread in data in the The following images show the results obtained after running image our algorithm. As seen in the following diagrams, that is, Fig. 9a 4. Entropy (H): evaluates the quantity of information present in and 9b, the canny edge detection of the image having background the image in focus shows more edges on the background clock and the canny edge detection of the image having foreground in focus 5. Mutual Information (MI) or Fusion Factor: measure of the shows more edges in the small clock present in the foreground. correlative information content in fused image with respect to source images 6. Fusion Symmetry (FS): an indication of how symmetric the output image is with the input images. 7. Normalized Correlation (CORR): a measure of relevance of fused image to source images. 8. Petrovic Metric Parameter QABF: an index of edge information preservation. 9. Petrovic Metric Parameter LABF: a measure of loss of edge information. 10. Petrovic Metric Parameter NABF: a measure of noise As seen in Fig. 9a, more edges are present in the back clock which was the 'in-focus' object of Fig. 8a and in Fig. 9b, more edges are present in the back clock which was the 'in-focus' object of image 8b. Fig. lOa represents the output obtained after performing the cross filtering operation. Fig. lOb represents the final fused image output. We used another set of images taken from a 'smart phone' Fig. 11: Input images and canny edge detection outputs of the camera and the output obtained is shown in the following input. 0147 This full-text paper was peer-reviewed and accepted to be presented at the IEEE ICCSP 2015 conference. QABF 0.69 0.705 LABF 0.161 0.141 NABF 0.0003 0.001 VII CONCLUSIONS The results show that the proposed technique is suitable for image fusion of multifocus images in spatial domain. The addition of cross filtering produces smoother images. In addition, the proposed method also yield excellent sharpness, clarity and edge preservation along with increase in mutual information, fusion symmetry and correlation; hence giving better visual quality. ACKNOWLEDGMENT Fig. 12 depicts the final fused image output of the input images Authors would like to thank Dr. Debjani Paul for the The results which are obtained with the proposed algorithm are in valuable guidance. We would also like to thank our friends as follows: Dr. Madhura Mehendale, Tanveer Dapherdar, Meenakshi TABLE I Singh, Prasad, Samrat, Sampath, Amar, Akshi for RESULTS OBTAINED USING THE PROPOSED encouragement and all the members of IITB for their support ALGORITHM and infrastructure. REFERENCES Average Pixel Intensity (f.1) 90.0833 [I] Shutao Li, Bin Yang, "Multifocus Image Fusion by Combining Average Gradient (G) 3.5150 Curvelet and Wavelet Transform, " Pattern Recognition Letters, vo1.29, Standard Deviation (cr) 47.6362 pp.1295-1301, 2008. Entropy (H) 7.1957 [2] S. Arivazhagan, L. Ganesan, T. G. Subash Kumar, "A modified statistical approach for image fusion using wavelet transform, " Mutual Informat9ion (MI) 3.9075 Springer Journal SIViP, Vol.3, pp. 137-144, 2009. Fusion Symmetry (FS) 2.2064 [3] Li, H., Manjunath, B.S., Mitra, S.K.: Multisensor image fusion using CORR 0.9776 the wavelet transform. Graph. Models Image Process 57(3), 235-245 (1995) QABF 0.8934 [4 ] Hamza, AB., He, Y., Krim, H.,Willsky, A: A multiscale approach to LABF 0.1059 pixel-level image fusion. Integr. Comput. Aid. Eng. 12(2), 135-146 NABF 7.1454e-04 (2005) [ 5] Shah, P., Merchant, S.N., Desai, U. 8.: Multifocus and multispectral image fusion based on pixel significance using multiresolution decomposition. 1. SIViP. (2011). doi: 10.1007/s11760-011-0219-7 The proposed fusion technique is well suited for fusion of [6 ] Petrovic, V.: Multisensor pixel-level image fusion. PhD Thesis, Department of Imaging Science and Biomedical Engineering, multifocus images in spatial domain. The major achievements Manchester School of Engineering, United Kingdom (2001) of the proposed method are minimum artifacts (lowest NABF) [7] Carper, 1.W., Lilies, T.M., Kiefer, R.W.: The use of intensity-hue and maximum edge preservation (highest QABF). This is a saturation transformations for merging SPOT panchromatic and multi significant achievement, as artifacts may lead to wrong spectra image data. Photogr. Eng. Remote Sens. 56, 459- 467 (1990) [8] Toet, A: Hierarchical image fusion. Mach. Vis. Appl. 3(1), I-II interpretations which can be catastrophic, especially in (1990) applications like surveillance where it can result into false [9] Haeberli, P.: A multi-focus method for controlling depth of field. alarms. Grafica Obscura (1994) TABLE II RESULTS OBTAINED USING EXISTING METHODS (FBS-AT) Fixed (ABS-AT) Adaptive block size- adaptive block size- adaptive threshold method threshold method J.l 96.646 96.617 cr 47.941 50.739 H 7.263 7.278 G 5.126 5.34 MI 7.3 5.425 FS 1.872 1.582 CORR 0.978 0.978 0148
Enter the password to open this PDF file:
-
-
-
-
-
-
-
-
-
-
-
-