i Preface Welcome to the Volume 2 Number 2 of the International Journal of Design, Analysis and Tools for Integrated Circuits and Systems (IJDATICS). This issue is comprised of extended versions of student papers that were submitted to the Xi’an Jiaotong-Liverpool University (XJTLU) student workshop, which was co-located with the 1st IEEE International Conference on Networked Embedded Systems for Enterprise Applications (IEEE NESEA’10) that was held in Suzhou, Jiangsu Province, China in November 2010. We view the promotion of early-stage Ph.D. student research as a key part of the mission of the NESEA conference series. The inaugural NESEA student workshop provided a high quality forum in which Ph.D. students could present their research to leading international academics who were attending the main NESEA 2010 conference. The papers in this special issue represent a good overview of research being conducted by the Ph.D. students at XJTLU and while the results may be formative in nature, we are sure that the work presented in these papers will go on to produce significant scientific results. We would also like to thank the IJDATICS editorial team, which is led by: Ka Lok Man, Xi’an Jiaotong Liverpool University, China and Myongji University, South Korea Chi-Un Lei, University of Hong Kong, Hong Kong Kaiyu Wan, Xi’an Jiaotong Liverpool University, China Guest Editors: Christophe Huygens, Katholieke Universiteit Leuven, Belgium Kevin Lee, Murdoch University, Australia Danny Hughes, Xi’an Jiaotong Liverpool University, China External Reviewers: Gangmin Li, Xi’an Jiaotong Liverpool University, China David Murray, Murdoch University, Australia Jo Ueyama, University of Sao Paulo, Brazil ii Table of Contents Vol. 2, No. 2, August 2011 Preface ………………………………………………………………………………....... i Table of Contents ……………………………………………………………………….. ii 1. Ordered Incremental Attribute Learning based on mRMR and Neural Networks ………………………..…………....… Ting Wang, Sheng-Uei Guan, Fei Liu 86 2. Low-Complexity Independent Component Analysis Based Semi-Blind Receiver for Wireless Multiple-Input Multiple-Output Systems …………………………………… ………………………...….. Yufei Jiang, Xu Zhu, Enggee Lim, Linhao Dong, Yi Huang 91 3. Integrating Color Vector Quantization and Curvelet Transform for Image Retrieval ….. …………………………………………..........……. Yungang Zhang, Wei Gao, Jun Liu 99 4. Multi-Beam Radar Search Improvement Via Digital Signal Re-Steering ……………… …………………………………………......….……. Dinghong Lu,Yang Li, Jimin Xiao 106 INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR INTEGRATED CIRCUITS AND SYSTEMS The International Journal of Design, Analysis and Tools for Integrated Circuits and Systems (IJDATICS) was created by a network of researchers and engineers both from academia and industry. IJDATICS is an international journal intended for professionals and researchers in all fields of design, analysis and tools for integrated circuits and systems. The objective of the IJDATICS is to serve a better understanding between the community of researchers and practitioners both from academia and industry. Editor-In-Chief Ka Lok Man Xi'an Jiaotong-Liverpool University, China, and Myongji University, South Korea Co-Editor-In-Chief Chi-Un Lei Abhilash Goyal University of Hong Kong, Hong Kong Oracle (SunMicrosystems), USA Editorial Board Vladimir Hahanov Salah Merniz Kharkov National University of Radio Electronics, Ukraine Mentouri University, Algeria Felipe Klein Paolo Prinetto Oscar Valero State University of Campinas, Brazil Politecnico di Torino, Italy University of Balearic Islands, Spain Enggee Lim Massimo Poncino Yang Yi Xi'an Jiaotong-Liverpool University, China Politecnico di Torino, Italy Sun Yat-Sen University, China Kevin Lee Alberto Macii Damien Woods Murdoch University, Australia Politecnico di Torino, Italy University of Seville, Spain Prabhat Mahanti Joongho Choi Franck Vedrine University of New Brunswick, Saint John, Canada University of Seoul, South Korea CEA LIST, France Tammam Tillo Wei Li Bruno Monsuez Xi'an Jiaotong-Liverpool University, China Fudan University, China ENSTA, France Yanyan Wu Michel Schellekens Kang Yen Xi'an Jiaotong-Liverpool University, China University College Cork, Ireland Florida International University, USA Wen Chang Huang Emanuel Popovici Takenobu Matsuura Kun Shan University, Taiwan University College Cork, Ireland Tokai University, Japan Masahiro Sasaki Jong-Kug Seon R. Timothy Edwards The University of Tokyo, Japan LS Industrial Systems R&D Center, South Korea MultiGiG, Inc., USA Vineet Sahula Umberto Rossi Olga Tveretina Malaviya National Institute of Technology, India STMicroelectronics, Italy Karlsruhe University, Germany D. Boolchandani Franco Fummi Maria Helena Fino Malaviya National Institute of Technology, India University of Verona, Italy Universidade Nova De Lisboa, Portugal Zhao Wang Graziano Pravadelli Adrian Patrick ORiordan Xi'an Jiaotong-Liverpool University, China University of Verona, Italy University College Cork, Ireland Shishir K. Shandilya Vladimir PavLov Grzegorz Labiak NRI Institute of Information Science & Technology, India Intl. Software and Productivity Engineering Institute, USA University of Zielona Gora, Poland J.P.M. Voeten Ajay Patel Jian Chang Eindhoven University of Technology, The Netherlands Intelligent Support Ltd, United Kingdom Texas Instruments Inc, USA Wichian Sittiprapaporn Thierry Vallee Yeh-Ching Chung Mahasarakham University, Thailand Georgia Southern University, USA National Tsing-Hua University, Taiwan Aseem Gupta Menouer Boubekeur Anna Derezinska Freescale Semiconductor Inc., USA University College Cork, Ireland Warsaw University of Technology, Poland Kevin Marquet Monica Donno Kyoung-Rok Cho Verimag Laboratory, France Minteos, Italy Chungbuk National University, South Korea Matthieu Moy Jun-Dong Cho Yong Zhang Verimag Laboratory, France Sung Kyun Kwan University, South Korea Shenzhen University, China Ramy Iskander AHM Zahirul Alam R. Liutkevicius LIP6 Laboratory, France International Islamic University Malaysia, Malaysia Vytautas Magnus University, Lithuania Suryaprasad Jayadevappa Gregory Provan Yuanyuan Zeng PES School of Engineering, India University College Cork, Ireland University College Cork, Ireland S. Hariharan Miroslav N. Velev D.P. Vasudevan B. S. Abdur Rahman University, India Aries Design Automation, USA University College Cork, Ireland Chung-Ho Chen M. Nasir Uddin Arkadiusz Bukowiec National Cheng-Kung University, Taiwan Lakehead University, Canada University of Zielona Gora, Poland Kyung Ki Kim Dragan Bosnacki Maziar Goudarzi Daegu University, South Korea Eindhoven University of Technology, The Netherlands University College Cork, Ireland Shiho Kim Dave Hickey Jin Song Dong Chungbuk National University, South Korea University College Cork, Ireland National University of Singapore, Singapore Hi Seok Kim Maria OKeeffe Dhamin Al-Khalili Cheongju University, South Korea University College Cork, Ireland Royal Military College of Canada, Canada Siamak Mohammadi Milan Pastrnak Zainalabedin Navabi University of Tehran, Iran Siemens IT Solutions and Services, Slovakia University of Tehran, Iran Brian Logan John Herbert Lyudmila Zinchenko University of Nottingham, UK University College Cork, Ireland Bauman Moscow State Technical University, Russia Ben Kwang-Mong Sim Zhe-Ming Lu Muhammad Almas Anjum Gwangju Institute of Science & Technology, South Korea Sun Yat-Sen University, China National University of Sciences and Technology, Pakistan Asoke Nath Jeng-Shyang Pan Deepak Laxmi Narasimha St. Xavier's College, India National Kaohsiung University of Applied Sciences, Taiwan University of Malaya, Malaysia Tharwon Arunuphaptrairong Chin-Chen Chang Danny Hughes Chulalongkorn University, Thailand Feng Chia University, Taiwan Xi'an Jiaotong-Liverpool University, China Shin-Ya Takahasi Mong-Fong Horng Jun Wang Fukuoka University, Japan Shu-Te University, Taiwan Fujitsu Laboratories of America, Inc., USA Cheng C. Liu Liang Chen A.P. Sathish Kumar University of Wisconsin at Stout, USA University of Northern British Columbia, Canada PSG Institute of Advanced Studies, India Farhan Siddiqui Chee-Peng Lim N. Jaisankar Walden University, Minneapolis, USA University of Science Malaysia, Malaysia VIT University. India Yui Fai Lam Ngo Quoc Tao Atif Mansoor Hong Kong University of Science & Technology, Hong Kong Vietnamese Academy of Science and Technology, Vietnam National University of Sciences and Technology, Pakistan Jinfeng Huang Steven Hollands Philips & LiteOn Digital Solutions, The Netherlands Synopsys, Ireland Managing Editor Michele Mercaldi Kaiyu Wan Tomas Krilavicius EnvEve, Switzerland Xi'an Jiaotong-Liverpool University, China Vytautas Magnus University, Lithuania Journal Secretary Treasurer Jieming Ma Woonkian Chong Xi'an Jiaotong-Liverpool University, China Xi'an Jiaotong-Liverpool University, China Assistant Editor-In-Chief Nan Zhang Lai Khin Wee Xi'an Jiaotong-Liverpool University, China Technische Universitat Ilmenau, Germany, and Universiti Teknologi Malaysia, Malaysia Publisher Cooperation Name : Solari Co., Hong Kong Address : Unit 1-5, 20/F, Midas Plaza, 1 Tai Yau Street, San Po Kong, Kowloon, Hong Kong Phone : (852) 3966-2536 ISSN: 2071-2987 (online version), 2223-523X (print version) INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR INTEGRATED CIRCUITS AND SYSTEMS http://ijdatics.distributedthought.com/ INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 86 Ordered Incremental Attribute Learning based on mRMR and Neural Networks Ting Wang, Sheng-Uei Guan, and Fei Liu Abstract—Current feature reduction approaches such as feature reduction is not the ultimate technique for coping with feature selection and feature extraction are insufficient for high dimensional problems. dealing with high-dimensional pattern recognition problems when One useful strategy for solving high-dimensional problems all features carry similar significance. An applicable method for coping with these problems is incremental attribute learning is “divide-and-conquer”, where a complex problem is firstly (IAL) which gradually imports pattern features in one or more separated into some smaller modules by features. These size. Hence a new preprocessing called feature ordering should be modules will be integrated after they have been tackled introduced in pattern classification and regression, and the independently. A representative of such methods is Incremental ordering of imported features should be calculated before Attribute Learning (IAL), which incrementally trains pattern recognition. In previous studies, the calculation of feature features in one or more size. It has been shown as an applicable ordering is similar to wrapper methods. However, such a process is time-consuming in feature selection. In this paper, a substitute approach for solving machine learning problems in regression approach for feature ordering is presented, where feature and classification [3-6]. Moreover, some previous studies have ordering is ranked by some metrics such as redundancy and shown that IAL based on neural networks usually obtains better relevance using mRMR criteria. Based on ITID, a neural IAL results than conventional methods which prefer to train all model is derived. Experimental results verified that feature pattern features in one batch [3, 7]. For example, based on ordering derived by mRMR can not only save time, but also machine learning datasets from University of California, Irvine obtain the best classification rate compared with those in previous (UCI), Guan and his colleagues employed IAL to solve some studies. In addition, it is also feasible to apply mRMR to calculate feature ordering for regression problems. classification and regression problems by neural networks. Almost all their results were better than those derived from Index Terms—feature ordering, incremental attribute learning, traditional methods [5, 6]. More specifically, classification mRMR, neural networks errors of IAL using neural network in the datasets of Diabetes, Thyroid and Glass were reduced by 8.2%, 14.6% and 12.6%, respectively [8]. I. INTRODUCTION However, because IAL incrementally imports features into P ROBLEMS like gene analysis and text classification often have a high-dimensional feature space, which consists of a large number of features, also called as attributes. The number systems, it is necessary to know which feature should be introduced in an earlier step. Thus feature ordering should be implemented as a new preprocess apart from conventional of dimensions often stands for the complexity of a problem. preprocessing tasks like feature selection and feature extraction. The more features in a problem, the more complex this problem Usually, feature ordering relies on feature’s discriminative is. Complex high-dimensional problems will cause dimensional ability. In previous studies on neural IAL, feature ordering was disasters that will make systems halt in computing. To solve derived by an approach which is similar to wrappers in feature these problems, some dimensional reduction strategies like selection, where discriminative ability of a single feature is feature selection and feature extraction [1] have been presented calculated by some predictive algorithms like neural networks. [2]. However, these methods are invalid when the problem has The only input consists of the feature to be evaluated, and the a large number of features and all the features are crucial and output refers to the discrimination ability of this feature. have similar importance in the problem simultaneously. Thus However, by comparison with filter, another approach in feature selection, wrapper is more time-consuming. Therefore, it is necessary to do some studies on feature ordering based on This work was supported in part by the National Natural Science Foundation filter methods. of China under Grant 61070085. In this paper, a new feature ordering method of IAL is T. Wang is with the University of Liverpool, Liverpool, L69 3BX UK, and is presented based on a filter feature selection approach called offsite studying at the Xi’an Jiaotong-Liverpool University, Suzhou, 215123 China. Phone: 86-13812296645. E-mail: ting.wang@ liverpool.ac.uk minimal-redundancy-maximal-relevance (mRMR) criterion. S. Guan, is with Xi’an Jiaotong-Liverpool University, Suzhou, 215123 Furthermore, as a neural network algorithm of IAL, ITID will China. E-mail: Steven.Guan@xjtlu.edu.cn be used to test the applicability and accuracy of this new F. Liu, is with La Trobe University, Bundoora, Victoria 3086, Australia. E-mail: f.liu@latrobe.edu.au method. In this paper, some background knowledge of ITID INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 87 and mRMR will be introduced respectively in Section 2 and 3; III. MRMR CRITERION In Section 4, an IAL feature ordering model based on mRMR Minimal-redundancy-maximal-relevance criterion (mRMR) will be presented; Benchmarks with the datasets from UCI will is a method for first-order incremental feature selection [10], be tested out in Section 5 followed by some experimental result which is a hot topic in research. In mRMR criteria, features analysis; and conclusions will be drawn in the last section with which have both minimum redundancy for input features and an outline of future works. maximum relevancy for output classes should be selected. Thus this method is based on two important metrics. One is mutual II. IAL BASED ON NEURAL NETWORKS information between an output and each input, which is used to Incremental attribute learning is a novel approach which measure relevancy, and the other is mutual information imports features gradually in one or more groups. According to between every two inputs, which is used to calculate previous studies, IAL is more feasible to cope with multivariate redundancy between these inputs. dimensional pattern recognition problems. At present, based on More specifically, let S denote the subset of selected some intelligent predictive methods like neural networks, new features, and Ω is the pool of all input features, the minimum approaches and algorithms have been presented for IAL. For redundancy can be computed by example, incremental neural network training with an 1 increasing input dimension (ITID) [9] is an incremental neural min S ⊂Ω S 2 ∑I( f , f i , j ∈S i j ). (1) training method derived from ILIA1 and ILIA2 [7], which have been shown applicable for classification and regression. It where I(fi, fj) is mutual information between fi and fj, and |S| is divides the whole input dimensions into several sub the number of input feature of S. On the other hand, mutual dimensions, each of which corresponds to an input feature. information I(c, fi) is usually employed to calculate Instead of learning input features altogether as an input vector discrimination ability from feature fi to class c = {c1,…,ck}. in a training instance, ITID learns input features one after Therefore, the maximum relevancy can be calculated by another through their corresponding sub-networks, and the 1 structure of neural networks gradually grows with an max S ⊂Ω S ∑ I ( c, f ) . i∈S i (2) increasing input dimension as shown in Fig. 1. During training, information obtained by a new sub-network is merged together Combining (1) with (2), mRMR feature selection criterion can with the information obtained by the old network. Such be obtained as below, either in quotient form: architecture is based on ILIA1. After training, if the outputs of ⎧⎪ ⎡1 ⎤ ⎫⎪ neural networks are collapsed with an additional network max ⎨∑ I (c, f i ) / ⎢ ∑I( f , f i j )⎥ ⎬ (3) sitting on the top where links to the collapsed output units and S ⊂Ω ⎪⎩ i∈S ⎣S i , j ∈S ⎦ ⎪⎭ all the input units are built to collect more information from the or in a different form: inputs, this results in ILIA2 as shown in Fig. 2. Finally, a ⎧⎪ ⎡1 ⎤ ⎫⎪ pruning technique is adopted to find out the appropriate max ⎨∑ I (c, f i ) − ⎢ ∑I( f , f i j )⎥ ⎬ . (4) network architecture. With less internal interference among S ⊂Ω ⎪⎩ i∈S ⎣S i , j∈S ⎦ ⎪⎭ input features, ITID achieves higher generalization accuracy In the solutions of mRMR, features are incrementally added than conventional methods [9]. into the selected feature subset. According to such a process, the sequence of incremental addition can be regarded as an order of discrimination ability of features. Thus feature ordering also can be calculated by (3) or (4), if all features have been put into the selected subset by mRMR. IV. FEATURE ORDERING BASED ON MRMR Feature ordering is unique to data preparation work of IAL. Fig. 1. The basic network structure of ITID Compared with conventional approaches where input features are trained in one batch, features will be gradually imported into pattern recognition one after another in IAL. In this process, how to derive an order for training is very important. Hence feature ordering, seldom used in conventional pattern recognition techniques, is indispensable in IAL. Moreover, the computing procedure of feature ordering is different from that of feature selection methods, where feature selection discards some features from the original feature set, while feature ordering merely puts down all features in a given order which Fig. 2. The network structure of ITID with a sub-network on the top may be different from the original sequence. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 88 Due to the fact that the calculation of feature ordering in feature ordering in the following step. Furthermore, to compare previous studies is based on wrappers, which is time- with previous studies which merely focus on ILIA1, all the consuming compared with filters, using filter methods is able to results about ITID were based on ILIA1 as well. bring benefits for feature ordering in IAL. Moreover, apart To evaluate the performance, two types of metrics were from saving time in preprocessing, there are some other employed for analysis, preprocessing time and error rate of the advantages in the calculation of feature ordering using filters. pattern recognition process. For example, feature reduction and feature ordering can be In terms of time, previous feature ordering computation applied simultaneously, because the former severely relies on employed one feature as the only input of neural networks to discriminability while the calculation of discriminability is the classify or predict all patterns. Such processing can estimate key in the latter. one feature’s discriminability according to the error rate of Although these two mRMR methods are different, both are classification or regression, and such discriminability can rank applicable in the calculation of input feature ordering, and each feature ordering. This processing is similar to wrappers which ordering can be employed in training. Fig. 3 is a model of were widely used in feature selection. ordered IAL, where feature ordering is calculated by mRMR. Compared with wrapper-like feature ordering computing, In this model, there are two phases of ordered IAL: obtaining the calculation using mRMR is quite different where several feature ordering and applying pattern recognition. The step of metrics will be employed to measure the discriminability. Such obtaining feature ordering refers to the computing of Feature processing is similar to filters in feature selection. Obviously, a Ordering Vector, which is derived from Feature Ordering filter-like approach usually takes much less time than a wrapper Calculator. In this step, the discriminability of each feature is method does. Thus computing using mRMR for feature calculated on the basis of training dataset, and the results of ordering is definitely more timesaving. discriminability are placed in a descending order, which will In terms of error rate, both mRMR methods are applicable obtain better results than the other orders [11]. In the second for feature ordering, thus two streams of experiments based on phase, the formal machine learning process starts. Patterns are mRMR were implemented as well. The following subsections randomly divided into three different datasets: training, present the details of different experiments using different validation and testing [12]. The process in this step is based on datasets. these datasets, and feature ordering for importing features is a A. Diabetes foundation for pattern recognition in each round. Diabetes is a two-category classification problem which has 8 continuous input features that are used to diagnose whether a Pima Indian has diabetes or not. There are 768 patterns in this dataset, 65% of which belong to class 1 (no diabetes), 35% class 2 (diabetes). Table I shows the results in comparison with classification using ITID based on different feature orderings which were derived from mRMR (Difference), mRMR (Quotient), wrappers and the conventional method which has no feature ordering. According to the results, mRMR (Difference) obtained the lowest error rate (22.86459%) and the result of mRMR (Quotient) is as good as that of wrappers (22.96876%). All of them are better than those derived from conventional method (23.93229%) which trains patterns in one batch. Thus using mRMR can obtain a better feature ordering for IAL in Diabetes. Fig. 3. The model of Ordered IAL based on mRMR B. Cancer V. EXPERIMENTS AND ANALYSIS Cancer is a classification problem including 9 continuous inputs, 2 outputs, and 699 patterns, which is used to diagnose The proposed ordered IAL method using mRMR and ITID breast cancer. 66% of the patterns belong to class 1 (benign) were tested on four benchmarks from UCI machine learning and 34% of them belong to class 2 (malign). Table II shows datasets. They are Diabetes, Cancer, Thyroid and Flare. The Cancer’s experimental results. By comparison, both mRMR first three are classification problems while the last one is a methods obtained the same best classification results regression problem. In these experiments, all the patterns were (2.29885%) in this test, while those of wrappers and randomly divided into three groups: training set (50%), conventional methods are 2.4999985% and 2.87356%, validation set (25%) and testing set (25%). Especially, the respectively. Hence using mRMR for feature ordering is better training data were firstly used to rank feature ordering based on than using the other approaches in Cancer. mRMR in the first place as a preprocessing task while ITID was employed for classification or regression according to this INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 89 C. Thyroid TABLE I Thyroid diagnoses whether a patient’s thyroid has RESULTS OF DIABETES over-function, normal function, or under-function based on Feature Ordering Classification Error patient query data and patient examination data. This mRMR-Difference 2-6-1-7-3-8-4-5 22.86459% classification problem has 21 inputs features, 3 outputs, and mRMR-Quotient 2-6-1-7-3-8-5-4 22.96876% 7200 patterns where class 1, 2 and 3 have 2.3%, 5.1% and wrappers 2-6-1-7-3-8-5-4 22.96876% 92.6% of all the patterns, respectively. Table III presents the Conventional method 23.93229% classification results of Thyroid. Compared with the error rate TABLE II of wrappers (1.838888%) and conventional method RESULTS OF CANCER (1.8638875%), both mRMR (Difference) and mRMR Feature Ordering Classification Error (Quotient) exhibited better performance, where the result of the mRMR-Difference 2-6-1-7-3-8-5-4-9 2.29885% former is 1.619443% and that of the latter is 1.625001%. mRMR-Quotient 2-6-1-7-8-3-5-4-9 2.29885% Therefore, mRMR is a better approach for feature ordering in wrappers 2-3-5-8-6-7-4-1-9 2.4999985% Thyroid. Conventional method 2.87356% D. Flare TABLE III The Flare problem is a regression problem that predicts three RESULTS OF THYROID outputs of solar flares. There are 10 inputs, 3 outputs and 1066 Classification patterns in this dataset, where the first three inputs have 7, 6, Feature Ordering Error and 4 features, respectively, and each of latter inputs merely 3-7-17-10-6-8-13-16-4-5-12 has one feature. Thus the total number of input features in Flare mRMR-Difference 1.619443% -21-18-19-2-20-15-9-14-11-1 is 24. Table IV exhibits the performance of regression where 3-10-16-7-6-17-2-8-13-5-1-4 mRMR-Quotient 1.625001% the input ordering is derived either from mRMR or wrappers. -11-12-14-9-21-15-18-19-20 According to the column of testing error, mRMR is feasible for 18-17-19-20-11-21-15-10-3 wrappers 1.838888% feature ordering in regression, but the results are not better than -8-13-7-1-2-12-16-6-5-4-14-9 those derived in previous studies using conventional method Conventional method 1.8638875% and wrappers. The reason of such a phenomenon is that, in previous studies, the first three inputs which consist of multiple TABLE IV features were not trained dispersedly and individually, but RESULTS OF FLARE trained group by group. Thus feature grouping may impact on Testing Feature Ordering the final results, which coincides with the idea in [13]. Error According to the analysis presented above, in classification mRMR-Difference 16-23-5-18-10-4-13-6-17-19-11-21 0.568722% problems, feature ordering derived from mRMR exhibits better -7-20-12-1-24-2-3-9-22-15-8-14 16-23-5-18-10-4-13-6-19-17-11-21 performance than that obtained by the conventional method mRMR-Quotient 0.573042% -7-20-14-2-9-15-12-22-8-3-1-24 and wrappers; while in regression problems, the ordering (1-2-3-4-5-6-7)-(8-9-10-11-12-13)- calculated by mRMR is not the best because of the feature wrappers 0.5255421% (14-15-16-17)-18-21-23-22-20-19-24 grouping in Flare. Therefore, as a well-known method of Conventional method 0.55% feature selection, mRMR is also available for feature ordering. It takes less preprocessing time to compute input feature ordering, and brings acceptable results with stable performance traditional methods which train features in one batch in the for simulation and application. However, because of the process of pattern recognition. Nonetheless, there are a number difference among feature ordering, the final results of different of further studies needed to be done in the future. For example, approaches are different. How to get the best feature ordering is although mRMR can attain better performance than the other still unknown. Nevertheless, this is a study which will be taken approaches, how to obtain feature ordering with an optimal in the future. pattern recognition result is still unknown. Moreover, in spite of the predictive results which are not better than others, VI. CONCLUSION whether there exists any improvement of feature ordering for IAL is a novel approach which gradually trains input regression is worthy of being researched. In addition, whether attributes in one or more sizes. Feature ordering in training is a feature grouping of input attributes is a factor which may unique preprocessing step in IAL pattern recognition. In influence pattern recognition also needs to be researched in the previous studies, feature ordering of IAL was derived by future. wrapper methods which are more time-consuming than filter Generally, using mRMR to calculate feature ordering is approaches like mRMR. Moreover, experimental results also applicable for saving time and enhancing the classification rate demonstrated that the feature ordering obtained by mRMR can in pattern classification problems based on neural IAL exhibit better performance than those derived by wrappers or approaches. Although the performance exhibited in regression INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 90 is not the best compared with results derived from some Ting WANG was born in Wuxi, China, on May 28th, 1981. He obtained his BSc degree in computer science at the previous studies, mRMR is still applicable in feature ordering China University of Mining and Technology, in 2003 and calculation for prediction with neural IAL. his MSc degree in computer science at the Guilin University of Technology, China, in 2008, and is now a PhD candidate REFERENCES in computer science, at the University of Liverpool. From July 2003 to July 2004, he was a system analyst at [1] H. Liu, “Evolving feature selection,” IEEE Intelligent Systems, vol. 20, no. the Microstar International Inc. Before he started his PhD 6, pp. 64-76, 2005. program in March 2009, he was a research and development engineer at the [2] S.H. Weiss and N. Indurkhya, Predictive data mining: a practical guide, Jiangnan Institution of Computing Technology. He is now interested in the field Morgan Kaufmann Publishers, CA: San Francisco, 1998. of artificial intelligence and information management. [3] S. Chao and F. Wong, “An incremental decision tree learning methodology regarding attributes in medical data mining,” Proc. of the Sheng-Uei Guan received his M.Sc. & Ph.D. from the 8th Int’l Conf. on Machine Learning and Cybernetics, Baoding, pp. University of North Carolina at Chapel Hill. He is currently 1694-1699, 2009. a professor and head of the computer science and software [4] R. K. Agrawal and R. Bala, “Incremental Bayesian classification for engineering department at Xian Jiaotong-Liverpool multivariate normal distribution data,” Pattern Recognition Letters, vol. University. Before joining XJTLU, he was a professor and 29, no. 13, pp. 1873-1876, Oct. 2008. chair in intelligent systems at Brunel University, UK. [5] S.U. Guan and J. Liu, “Feature selection for modular networks based on Prof. Guan has worked in a prestigious R&D incremental training,” Journal of Intelligent Systems, vol. 14, no. 4, pp. organization for several years, serving as a design engineer, project leader, and 353-383, 2005. manager. After leaving the industry, he joined Yuan-Ze University in Taiwan for three and half years. He served as deputy director for the Computing Center [6] F. Zhu and S.U. Guan, “Ordered incremental training for GA-based and the chairman for the Department of Information & Communication classifiers,” vol. 26, no. 14, Pattern Recognition Letters, pp. 2135-2151, Technology. Later he joined the Electrical & Computer Engineering Oct. 2005. Department at National University of Singapore as an associate professor. [7] S.U. Guan and S. Li, “Incremental learning with respect to new incoming input attributes,” Neural Processing Letters, vol. 14, no. 3, pp. 241-260, Dec. 2001. Fei Liu completed her PhD from The Department of [8] S.U. Guan and S. Li, “Parallel growing and training of neural networks Computer Science & Computer Engineering, La Trobe using output parallelism,” IEEE Trans. on Neural Networks, vol. 13, no. University in 1998. Before joining the department as an 3, pp. 542 -550, May 2002. academic staff member in 2002, she worked as a lecturer in [9] S.U. Guan and J. Liu, “Incremental neural network training with an The School of Computer & Information Science, The increasing input dimension,” Journal of Intelligent Systems, vol. 13, no. 1, University of South Australia, and The School of Computer pp. 43-69, 2004. Science & Information Technology, Royal Melbourne [10] H. Peng, F. Long, C. Ding, “Feature selection based on mutual Institute of Technology. She also worked as a software engineer in Ericsson information: criteria of max-dependency, max-relevance, and Australia. Her research interests include Logic Programming, Semantic Web min-redundancy,” IEEE Transactions on Pattern Analysis and Machine and Security in Electronic Commerce. She has been teaching in Artificial Intelligence, vol. 27, no. 8, pp. 1226-1238, 2005. Intelligence, Programming, and Security in Electronic Commerce. [11] S.U. Guan and J. Liu, “Incremental Ordered Neural Network Training,” Journal of Intelligent Systems, vol. 12, no. 3, pp. 137-172, 2002. [12] Ripley B. D., Pattern Recognition and Neural Networks, Cambridge University Press, UK:Cambridge, 1996. [13] J.H. Ang, S.U. Guan, K.C. Tan, A.A. Mamun, “Interference-Less Neural Network Training”, Neurocomputing, vol. 71, no. 16-18, pp 3509-3524, 2008. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 91 Low-Complexity Independent Component Analysis Based Semi-Blind Receiver for Wireless Multiple-Input Multiple-Output Systems Yufei Jiang, Xu Zhu, Enggee Lim, Linhao Dong, and Yi Huang Abstract—In this paper, a novel semi-blind receiver based on the equalizer coefficients to combat the situation with Doppler independent component analysis (ICA) is proposed for multiple- shift. Although the training overhead is as low as 0.05%, the input multiple-output (MIMO) wireless communication systems actual training is up to 160 symbols for one transmit antenna where a small number of pilots are exploited to resolve phase and permutation ambiguity problems in ICA model, by utilizing each time of initialization. Blind source separation method the correlation between received and original pilot symbols. is able to obtain the channel state information without intro- Unlike the precoding aided receiver proposed previously, the ducing extra training symbols, based on either second order pilot aided semi-blind receiver is more robust to the change of statistics (SOS) or higher order statistics (HOS). The equalizer frame length. Only a small number of pilots are used to solve the coefficient can be directly obtained from the statistics of the indeterminacies, compared to the precoding based method where all of the received and ICA separated symbols are collected to received data without using extra bandwidth and spectral. In eliminate the ambiguities, and thus our proposed method has [11], [13] and [14] , the precoding aided approach is proposed low computational complexity and saves the time of resolving and the blind channel estimation is obtained by exploring the the ambiguity problems, particular for a large amount of data. signal covariance matrix based on second order statistics at Simulation results show that with a very low training overhead of receiver, and is extended to the case with MIMO system in only 0.05%, the proposed semi-blind receiver has a performance extremely close to the case with perfect channel state information [12]. The Second Order Statistics based method is sensitive to (CSI), and outperforms the precoding aided one in outdoor case. the Gaussian noise, but the Higher Order Statistic based one is able to combat the Gaussian noise, when the higher cumulants Index Terms—independent component analysis (ICA), of received signal is exploited in which the higher cumulants multiple-input multiple output (MIMO), semi-blind, pilots of Gaussian noise is assumed to be zero. ICA [5], as one of Higher Order Statistic (HOS) blind source separation approaches, based on the assumption of I. I NTRODUCTION mutual statistical independence between source data [6], can be employed to estimate the data directly from the statistics M ULTIPLE Input Multiple Output (MIMO) [1] wireless systems are shown to provide significantly higher data rate than the Single Input Single Output (SISO) systems and of received signal, by maximizing the non-Gaussianity of re- ceived signal, without knowing the channel state information, are expected to use at wireless electronic applications. Since as long as the received signal corresponds to a linear mixture. multiple transmit and receive antennas are employed, the co- In [15], ICA is used on each subcarrier to separate the received channel interference exists in the single carrier flat fading signal. The statistical correlation between orthogonal subcar- channel, which can be eliminated through the equalization and riers is used to resolve the frequency dependent permutation channel estimation at receiver. The method of equalization and problem. But this approach brings some bit errors which can channel estimation can be divided into training based and blind not be resolved. In [7], the precoding is employed on source source separation approaches. symbols by reference symbols at transmitter side, to eliminate The training based methods make use of training sequences the permutation and phase ambiguities by using the correlation to recognize the channel state information. Thus, the chan- property between received and reference symbols, without nel estimation plays an important role for training based consuming extra spectral. However, it is very sensitive to the equalization. Most of previous works related to the training precoding constant and the frame length. sequence brings considerably extra bandwidth. Ref. [2] and [3] In this paper, a novel semi-blind receiver is proposed which propose an adaptive MIMO single carrier frequency domain benefits from statistic of received data and pilots, by using equalization but the training overhead is up to 13% and 10% ICA for detection. The multiple antennas transmit frames respectively. In [4], the training sequence is used to initialize of data with which a small number of pilot symbols are attached, over the MIMO flat fading channels. However, as Y. Jiang is with Xi’an JiaoTong Liverpool University, , Suzhou, China the instinct drawback of ICA model, the permutation and 215123. He is also with the University of Liverpool, Liverpool, L69 3GJ, U.K. E-mail: yufei10@liverpool.ac.uk phase ambiguities exist in the equalized signal, compared with E. Lim is with Department of Electrical and Electronic Engineering, Xi’an the original one. In order to eliminate these ambiguities, the JiaoTong Liverpool University, Suzhou, China 215123. proposed scheme can be described as following steps: first, X. Zhu, L. Dong and Y. Huang are with Department of Electrical Engineer- ing and Electronics, the University of Liverpool Brownlow Hill, Liverpool, the received signal is separated through the ICA processor. L69 3GJ, U.K. Second, some possible phase deviations exist in the separated INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 92 signal which is corrected by the de-rotation process, enabling antenna is proportional to 1/Nt . Thus,qthe received vector the separated symbols to have the same phase shift in a frame. x(k, i) pre-multiplied by the factor of Pt Nt (Pt is the total Third, the pilot symbols, known at receiver, are extracted to transmit power, Nt is the number of transmit antennas). resolve the problem of permutation and phase ambiguities, by The multiple antennas transmit frames of data s(k) with using the cross-correlation between the equalized and original which a small number of pilot symbols are attached. These pilot symbols. The simulation results show that the proposed pilots are used to resolve indeterminacies, as the instinct of semi-blind receiver has a performance extremely close to the ICA model system. The structure of transmitted signal s(k) is case with perfect channel state information, and outperforms shown as the linear precoding aided one in the outdoor case. pilot symbols: stra,m (k,i) The paper is organized as follows. The MIMO system and z }| { flat fading channel model are provided in Section II. Section sm (k) = [sm (k + 1), sm (k + 2), . . . , sm (k + Ntra ), III describes ICA model and the ambiguity problems resolved sm (k + Ntra + 1), sm (k + Ntra + 2), . . . , sm (k + Nsym )] by pilots aided method. The simulation results are presented | {z } source symbols: ssym,m (k,i) in Section IV. Conclusions can be found in Section V. (3) Throughout the paper, scalar, vectors and matrices and represented by regular small letters, bold small letters and where bold capital letters, respectively. (.)∗ , (.)T and (.)H denotes 1 < m < Nt (4) the complex conjugate, transpose, and complex conjugate Ns = Ntra + Nsym (5) transpose. ℜe(.) denotes the real part of complex-valued signal. E{.} is the expectation operator. diag{v} is a square Ntra denotes the number of pilots, Nsym denotes the number diagonal matrix whose diagonal elements are entries of vector. of source symbols. The pilot symbols are generated randomly, arg max{l} denotes the maximum element in the scale of l. attached in the beginning of data source and known at re- ceivers. The correlation between received and original pilot II. S YSTEM M ODEL symbols is used to resolve the permutation and quadrature ambiguities. We consider a wireless MIMO systems with Nt transmit and Nr receive antennas in the time-invariant flat fading channel as III. P ROPOSED ICA BASED S EMI -B LIND D ETECTION illustrated in Fig 1. The serial of streams are divided into Nt parallel substreams, mapped into QPSK modulation and are From system model (1), it is shown that the received signal organized into the form of frames to propagate. The estimated x(k, i) is linear mixture of transmitted signal s(k, i), ICA [6] channel might be the same as the reality. The channel impulse can be employed to estimate the transmitted data. However, response remains constant for the duration of a frame of Ns as the instinct drawback of ICA models, the permutation and might vary independently from frame to frame. Assume and phase ambiguities exist in the ICA separated signal, the symbol-synchronous receiver sampling and timing are compared with the original one. In order to eliminate these perfect. The received symbols ambiguities, the received signal is passed through the ICA processor to obtain separated signal. Second, some possible x(k, i) = [x1 (k, i), x2 (k, i), . . . , xNr (k, i)]T (1) phase deviations in the equalized signal is corrected by the de- rotation process, enabling the signal to have the same phase at time instant k on the symbol i(i = 1, 2, . . . , Ns ) over the shift. Third, the pilots, known at the receiver, are extracted flat fading channel can be written as: to resolve the problem of permutation and phase ambiguities, r Pt by using the cross-correlation between equalized and original x(k, i) = H(k)s(k, i) + n(k, i) pilot symbols. N t r h1,1 (k) · · · h1,Nt (k) s1 (k, i) Pt .. .. .. .. A. Independent Component Analysis = . . . . Nt ICA is aimed at extracting the original source data from hNr ,1 (k) · · · hNr ,Nt (k) sNt (k, i) the linear mixture based on statistics of received symbols. + n(k, i) In order to use ICA to separate the received signal without (2) the knowledge of channel state information, the following where s(k, i) with size of (Nt × Ns ) and x(k, i) with size of assumption has to be met (Nr × Ns ) are the complex valued baseband signal, H(k) with 1) Transmitted symbols have to be statistic independence. size of (Nr × Nt ) is the flat fading MIMO channel impulse 2) Transmitted symbols have nongaussian distributions. response. The elements in H(k) are independent identically 3) Transmitted symbols have zero mean. distributed entries of complex value, with Rayleigh distributed 4) The number of receive antennas have to be equal/more amplitude and uniformly distributed phase. n(k, i) with size than the number of transmit antennas. of (Nr × Ns ) is the additive white Gaussian noise (AWGN). JADE [9] is one of the well-established methods in ICA, The elements in n(k, i) are complex valued with zero mean resulting in parameter free and shorter data sequences. Thus, and a variance of 12 σn2 . The total radiated power is fixed and JADE is employed in this paper to extract the source data. independent of Nt but the power lunched by each transmit In order to obtain the spatially uncorrelated signals, Principal INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 93 7UDQVPLWWHU 5HFHLYHU k x 1 i s k 3LORWV ([WUDFWLRQ 1 i 3LORW 'DWD s t Ú r a k i M M ,&$ 3KDVH 'HYLDWLRQ $PELJXLW\ 'HFLVLRQ 'HWHFWLRQ 5HVROXWLRQ - k Ú &RUUHFWLRQ i ~ i s t ^ i k k i N k k s s s s i 3LORW 'DWD k i x r N Fig. 1. Wireless MIMO system model with ICA based semi-blind pilot aided receiver structure of frequency flat channel and ambiguity elemination Component Analysis (PCA) is used firstly to whiten the B. Phase Deviation Correction received signal. The whitening matrix W(k) is obtained from Firstly, some phase deviation is in the ICA separated signal the Eigenvalue Decomposition of the autocorrelation matrix of resulting in different phase shift in s̃(k, i), due to noise and received signal as the drawback of the ICA based equalization. The de-rotation process can correct possible phase deviation in the ICA Rxx (k) = E{x(k, i)xH (k, i)} (6) equalized signal, to make sure that all of the data, including = U(k)Λ(k)UH (k) equalized pilots and symbols in a frame, has the same phase shift on each substream. The equalized signal š(k, i) with size The whitening matrix is given as of (Nt × Ns ) after phase deviation correction is denoted as W(k) = Λ−1/2 (k)UH (k) (7) š(k, i) = L−1 (k)s̃(k, i) −1 L1 (k) 0 ··· 0 s̃1 (k, i) such that −1 0 L 2 (k) ··· 0 s̃2 (k, i) H H = .. .. .. .. .. W(k)E{x(k, i)x (k, i)}W (k) = IN t (8) . . . . . 0 0 ··· L−1 Nt (k) s̃Nt (k, i) JADE is applied on the received signal to get the unitary matrix (12) V(k). The extracted signal s̃(k, i) with size of (Nt × Ns ) at th the k th instant time and ith symbol is given by: where L−1 m (k) on the m substream is deoneted as αm (k) s̃(k, i) = V(k)W(k)x(k, i) (9) L−1 m (k) = (13) | αm (k) | However, the ICA equalized signal is not the same as the where αm (k) is the de-rotation factor obtained from the transmitted symbols, there exists ambiguity matrix G(k) in statistical characteristics of s̃(k, i) with QPSK modulation as s̃(k, i), compared with s(k, i) as ( )− 41 Ns 1 X 4 π s̃(k, i) = G(k)s(k, i) (10) αm (k) = s̃m (k, i) ej 4 (14) Ns i=1 where the ambiguity matrix G(k) is composed by three where šm (k, i) has phase rotation of θ(θ ∈ {0, π2 , 3π 2 , π}), indeterminacies as which might vary from frame to frame, and from transmitted substream to transmitted substream. This remained ambiguity G(k) = L(k)P(k)D(k) (11) problem can be resolved by a small number of pilots as following. where L(k) is the phase deviation matrix, P(k) is the permu- tation ambiguity matrix, and D(k) is the quadrant ambiguity matrix. C. Permutation and Quadrant Ambiguity Resolution The ambiguity which remains in the ICA separated signal Since the equalized signal after the de-rotation process has can be resolved by proposed pilot aided method described as the same quadrant and permutation ambiguity in a frame of the th further steps. πm separated receive substream, the cross correlation property INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 94 Indoor Case, Nt=4 and Nr=4 −4 10 −5 10 BER −6 10 Semi−blind pilot Precoding a=0.1 [7] Precoding a=0.3 [7] ZF perfect CSI MMSE perfect CSI −7 10 0 5 10 15 20 25 30 SNR dB Fig. 2. Performance of precoding and pilot aided for Nt = 4 and Nr = 4 MIMO system in indoor case Outdoor Case, Nt=4 and Nr=4 −3 10 −4 10 BER −5 10 −6 Semi−blind pilot 10 Precoding a=0.3 [7] Precoding a=0.4 [7] ZF perfect CSI MMSE Perfect CSI −7 10 0 5 10 15 20 25 30 SNR dB Fig. 3. Performance of precoding and pilot aided for Nt = 4 and Nr = 4 MIMO system in outdoor case between the original and equalized pilot symbols is employed under the noiseless assumption is th to find the correct order and phase shift for the πm equalized Ntra signal. The pilot symbols stra,πm (k, i) generated randomly 1 X |ρπm ,m | = štra,πm (k, i)s∗tra,m (k, i) and known at receiver, are attached on the transmitted source Ntra i=1 (16) symbols sm (k) on the mth transmit antenna traveling over the 1 πm = m flat fading channel. At receiver, the pilot symbols štra,πm (k, i) = th 0 πm 6= m are extracted from the equalized signal šπm (k, i) on the πm received substream. The cross correlation ρπm ,m between the In the noiseless case, the absolute value of cross correlation original and equalized pilot symbols is defined as achieves the maximum of one only when πm = m, and the th phase shift θπm on the πm substream can be found via Ntra stra,m (k, i) 1 X jθπm ∗ ejθπm = (17) ρπm ,m = [štra,πm (k, i)e ]stra,m (k, i) štra,πm (k, i) Ntra i=1 (15) However, in reality, the noise always exists in the wireless where θπm = π2 l , l ∈ {0, 1, 2, 3} is the phase shift on the channel, the value of cross correlation might be less than one. th πm detected substeam. Since the pilots in different substreams Alternatively, we search for the largest absolute value of the are statistically independence, the absolute value of the ρπm ,m cross corelation ρπm ,m to find the highest possibility of being INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 95 0 a=0.1, Nt=4 and Nr=4 10 SNR = 20 dB SNR = 25 dB Precoding −1 10 BER −2 10 Pilot −3 10 100 200 300 400 500 600 700 800 900 1000 Frame length Fig. 4. Impact of frame length Ns for Precoding (a = 0.1) and pilot aided for Nt = 4 and Nr = 4 MIMO system correct order is given as cor T πm = arg max |ρπm ,m | (18) cor ] ŝ(k, i) = D−1 (k)[šπ1cor , šπ2cor , . . . , šπN t πm jθcor e π1 0 ··· 0 šπ1cor cor The correct order is arranged into a matrix so that the 0 ejθπ2 · · · 0 šπ2cor (23) permutation ambiguity is resolved = .. .. .. .. .. . . . . . P−1 (k) = [π1cor , π2cor , · · · , πm cor cor ] (19) jθπ šπN cor 0 0 ··· e Nt t With the equation above, the quadrant ambiguity still remains Then the ŝ(k, i) is passed to the decision device to obtain the cor (k, i). The cross correlation ρπ cor ,m can be still in the šπm hard estimate s̄(k, i). m used to search the phase rotation θπm cor , by finding the largest s̄(k, i) = D[ŝ(k, i)] (24) cor real part of the ρπm cor ,m on the correct π m substream The phase and permutation ambiguity can still be resolved θπcor m = arg max ℜe(ρ cor ,m πm ) by a non-redundant linear precoding aided method, by adding l the reference signal on the source symbols [7] as n 1 N Xtra 1 jθπm cor = arg max ℜe [štra,πm cor (k, i)e ] sm (k, i) = √ [d(k, i) + adref (k, i)] (25) l Ntra i=1 1 + a2 o where a (0 < a < 1) is the precoding constant as a tradeoff s∗tra,m (k, i) of the power allocation between the source data and the (20) reference data. The correlation between the received data and the reference data can be used to resolve the permutation and where π phase ambiguity in the ICA separated signal at receiver as θ πm cor = l l ∈ {0, 1, 2, 3} (21) 2 1 X Ns ρπm ,m = šπm (k, i)d∗ref,m (k, i) The reason why we choose the real part of the ρπm cor ,m is Ns i=1 because the real part of ρπmcor is less than one while the value Ns 1 X 1 of imaginary part is not zero any more due to the noise. The = √ [dˇπm (k, i) + adˇref,πm (k, i)] larger the real part is, the less the imaginary part will be. Ns i=1 1 + a2 Thus, The θπcorm has the highest possibility of being the correct d∗ref,m (k, i) phase rotation when the real part is the largest one. The phase Ns rotation θπcor are arranged into a diagonal matrix D−1 (k) 1 1 X m =√ [dˇπm (k, i)d∗ref,m (k, i)] 1 + a2 Ns i=1 cor cor jθ cor D−1 (k) = diag [ejθπ1 , ejθπ2 , . . . , e πNt ]T (22) Ns 1 X + [adˇref,πm (k, i)d∗ref,m (k, i)] Thus, the estimated signal ŝ(k, i) with size of [N t × (Ns − Ns i=1 Ntra )] after the permutation and quadrant ambiguity resolution (26) INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 96 TABLE I PARAMETERS S ET FOR T WO E NVIRONMENTS Indoor case Outdoor case Maximum speed 3 km/h 120 km/h Doppler shift 5.55 Hz 222 Hz Coherence time 32.28 ms 806.94 µs RMS Delay Spread (τrms ) 70 ns 4,000 ns Symbol period (Ts ) 700 ns 40 µs Signal bandwidth 1.5 MHz 25.0 KHz Frame length (Ns ) 30,000 bits 500 bits Overhead 0.05% 3.2% Data rate 1.5M bit/s × 4 = 6.0M bit/s 25.0Kbit/s × 4 = 100.0Kbit/s where An RMS delay spread of τrms = 70 ns is set in an indoor E[dˇπm (k, i)d∗ref,m (k, i)] = 0 (27) environment with a maximum mobile velocity of 3 km/h. The symbol duration Ts is set to 700 ns (Ts >> τrms ) as frequency flat channel, and thus the number of symbols are 30,000 bits 1 πm = m in a frame. The Doppler frequency is 5.55 Hz operating in the E[dˇref,πm (k, i)d∗ref,m (k, i)] = (28) 0 πm 6= m 2GHz band, much smaller than the signal bandwidth of 1.5 Therefore the absolute value of the cross correlation ρπm ,m MHz, as slow fading channel. has The scenario 2 is a outdoor channel with mobile velocity a of 120 km/h and an RMS delay spread of τrms = 4000 ns. |ρπm ,m | = √ , when πm = m (29) 1 + a2 The symbol duration Ts is 40 µs to meet the requirement of The permutation ambiguity is resolved and the phase rotation frequency flat channel, and thus the number of symbols are can be found on the correct order of substream of the equalized 500 bits in a frame. A maximum Doppler frequency of 222 Hz signal šπm cor (k) via is smaller than signal bandwidth of 25 KHz, as slow fading " #−1 channel. Both of two extreme cases are considered as doubly −j π ρπm cor ,m j π flat fading channel: frequency flat fading and slow fading. bπ m cor (k) = e 4 sign e 4 (30) |ρπm cor ,m | Monte Carlo method is used to obtain the accurate evaluation of the performance for two cases. The decoding is used to obtain the estimate of source data from signal ŝ(k, i) of being correct order and p B. Results d(k, i) = 1 + a2 ŝ(k, i) − adref (k, i) (31) Fig. 2 shows the comparison of BER performance of All of received and separated symbols are collected to wipe between the precoding aided and pilot aided semi-blind ICA out the ambiguity in the ICA equalized signal while only based structure for MIMO systems in indoor environment. The a small number of pilot symbols are used in the proposed ZF (Zero-Forcing) and MMSE (Minimum Mean Square Error) pilot aided method. Therefore, the pilot aided one has low based equalizations with perfect channel state information are computational complexity and saves the time for ambiguity used as benchmarks. The proposed pilots aided ICA based elimination, particular for a large amount of data. equalization outperforms the ZF based equalization with per- fect CSI, showing the ability to resolve ambiguity effectively, due to the deviation correction process which can reduce the IV. S IMULATION R ESULTS impact of noise on the separated symbols, enabling all of the A. Simulation setup symbols having a same phase shift in a frame. This proposed We use the simulation results to demonstrate the perfor- one is also close to the MMSE based equalization with perfect mances of the proposed ICA based pilot aided semi-blind CSI but the BER performance improvement is owning to the detection for MIMO systems of Nt = 4 transmit antennas noise reduction in the MMSE based one. For precoding aided and Nr = 4 receive antennas. The minimum pilots, 8 symbols ICA based detection, the difference of BER performance is for 4 transmit antennas, are chosen to provide a good BER shown in Fig. 2 between precoding constant a chose as 0.1 performance over a wide ranges of SNRs, as shown in Fig. 5. and 0.3, and the optimal value is 0.1 in indoor case providing The flat fading channel of single propagation path is modelled a BER performance almost the same to pilot aided one. In using Rayleigh block fading [1] which remains constant during precoding aided method, all of received and separated symbols a frame of symbols with QPSK modulation. Assume the are collected to resolve the ambiguity problems in the ICA symbol-synchronous receiver sampling and timing are perfect. equalized signal while only a small number of pilot symbols The frame duration is set to 20 ms. Two extreme scenarios of are used in the proposed pilot aided method. For instant, in indoor and outdoor environments [10], operating on the 2 GHz precoding aided approach, 15,000 symbols in receiver are used band, are considered for simulation as shown in Table 1. to find the correct order and phase shift on each ICA separated INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 97 TABLE II T HE MINIMUM PILOT FOR DIFFERENT TRANSMIT ANTENNAS The minimum pilot Overhead in indoor case Overhead in outdoor case 1 by 1 MIMO system 5 0.033% 2.0% 2 by 2 MIMO system 6 0.040% 2.4% 3 by 3 MIMO system 7 0.047% 2.8% 4 by 4 MIMO system 8 0.053% 3.2% Nt=4 and Nr=4 0 10 SNR=10dB SNR=15dB SNR=20dB −1 SNR=25dB 10 SNR=30dB BER −2 10 −3 10 −4 10 0 5 10 15 20 25 30 Number of pilots Fig. 5. Impact of number of pilots Ntra on BER performance with Nt = 4 transmit antennas and Nr = 4 receive antennas substream, compared to the 8 symbols in pilot aided one. Fig. 5 illustrates the influence of the number of pilots on Therefore, the proposed pilot aided one has low computational BER performances at from SNR = 10 dB to SNR = 30 complexity and saves the time for ambiguity elimination, dB, for Nt = 4 transmit antennas and Nr = 4 receive particular for this large amount of data. antennas MIMO systems. The BER performance improves as Fig. 3 demonstrates the BER performance of pilot aided the number of pilots increase until it achieves a minimum level ICA based detection in outdoor case with high speed, com- of the number of pilot symbols when there is no significant pared with precoding aid, ZF based and MMSE based equal- BER improvement. At higher SNR, some more pilots are ization. The pilot aided method, is close to the ZF based needed to obtain better performance. For instant, the minimum and MMSE based equalization, and significantly outperforms number of pilots of 5 is a minimum at low SNR = 10 dB, while the precoding aided one with optimum value of precoding 8 at high SNR = 30 dB, and then these BER performances keep constant a = 0.3, even about 6 dB gains more, showing the constant at the larger number of pilots. Thus, the minimum effectiveness of our proposed method in ambiguity elimination number of pilots is 8 for Nt = 4 transmit antennas, resulting in outdoor case. The BER performance gap between the two in the training overhead of 0.05% in indoor case and 3.2% in precoding constant a = 0.3 and a = 0.4 is still as big as that outdoor case. The minimum number of pilots varies depending in Fig. 2 over a wide ranges of SNRs. It means that the BER on the number of transmit antenna as shown in Table II. performance of precoding aided equalization is very sensitive to the choice of a and frame size. V. C ONCLUSION Fig. 4 demonstrates the impact of frame length of Ns A new pilot aided ICA based semi-blind detection has on precoding aided and pilot aided ICA based detection, been proposed for MIMO wireless communication systems. A respectively. The precoding constant a is set to 0.1. The small number of pilots are attached in the source symbols at precoding aided method in Fig. 4 has significant change in trasmitter, to resolve the permutation and phase ambiguities in BER performance at both SNR = 20 dB and SNR = 25 dB, ICA model, by utilizing the cross-correlation between received and is shown to be very sensitive to the change of frame and original pilots. The minimum number of pilots is selected length from small to medium size, while pilot aided method properly to obtain a good BER performance, eliminating the almost keeps flat over the change of frame size, being able to ambiguity completely. All of received and separated symbols eliminate ambiguity effectively. In precoding aided detection, are collected to wipe out the ambiguity in the ICA equalized a large number of data is very required to reduce the ambiguity signal while only a small number of pilot symbols are used in of ICA model. the proposed pilot aided method. Therefore, the pilot aided INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 98 one has low computational complexity and saves the time Yufei Jiang received the M.Sc degree in Telecommunications and Computer for ambiguity elimination, particular for a large amount of Networks from the London South Bank University, London, U.K., in 2008. He is currently working towards a Ph.D. in the joint programme between the data. In addition, the proposed ICA based semiblind receiver University of Liverpool, Liverpool, U.K., and the Xi’an JiaoTong Liverpool has a good BER performance close to the ZF/MMSE based University, Suzhou, China. His research interests include MIMO, space-time equalization with perfect CSI, and much better than that of process, OFDM, cooperative communication, and the blind source separation. percoding aided receiver in outdoor case. Particularly, the proposed pilots aided receiver is insensitive to the change of frame length from small to medium size but consumes little Xu Zhu received the B.Eng. degree (with first class honors) from the Depart- bandwidth. ment of Electronics and Information Engineering, the Huazhong University of Science and Technology, Wuhan, China, in 1999, and the Ph.D. degree from the Department of Electrical and Electronic Engineering, the Hong Kong R EFERENCES University of Science and Technology, Hong Kong, in 2003. Since May 2003, she has been with the Department of Electrical Engineering and Electronics, [1] A. Goldsmith, Wireless Communications. London, U.K.: Cambridge the University of Liverpool, Liverpool, U.K., where she is currently a senior University Press, 2005. lecturer. Dr. Zhu was the vice chair of the 2006 and 2008 ICA Research [2] J. Coon, S. Armour, M. Beach, and J. McGeehan, ”Adaptive frequency Network International Workshops. She has acted as session chair and technical domain equalization for single-carrier MIMO systems,” in Proc. IEEE program committee member for more than 20 international conferences, such International Conference on Communications (ICC 2004), pp. 2487-2491, as IEEE GLOBECOM 2009 and IEEE ICC 2010. Her research interests Paris, France, Jun. 2004. include cooperative communications, cognitive networks, cross-layer design, [3] Y. Wu, X. Zhu, and A. K. Nandi, ”Adaptive layered space-frequency MIMO and adaptive equalization, etc. equalization for single-carrier MIMO systems,” in Proc. 13th European Signal Processing Conference (EUSIPCO 2005), Antalya, Turkey, Sep. 2005. [4] L. Sarperi, X. Zhu and A. K. Nandi, ”Semi-blind layered space-frequency equalization for single-carrier MIMO system with block transmission,” Enggee Lim received the BEng(Hons) and PhD degrees in Electrical and IEEE Transactions on Wireless Communications, vol. 7, no. 4, pp. 1203- Electronic Engineering from the University of Northumbria, UK in 1998 and 1207, Apr. 2008. 2002 respectively. Dr Lim worked for Andrew Ltd, a leading communications [5] J.F. Cardoso, ”High-Order Contrasts for Independent Component Analy- systems company in the United Kingdom from 2002 to 2007. His respon- sis,” Neural Computation, vol. 11, pp. 157-192, 1999. sibilities were to conduct the research and development for a cost-effective [6] A. Hyvarinen, J. Karhunen, and E. Oja, Independent Component Analysis. solution for all terrestrial microwave systems operating at frequencies between New York, U.S.A.: John Wiley & Sons, 2001. 400 MHz and 90 GHz. Dr Lim joined Xian Jiaotong-Liverpool University in [7] J. Gao, X. Zhu, and A. K. Nandi, ”Non-redundant precoding and PAPR 2007. His research interests are Antennas, RF/microwave engineering, EM reduction in MIMO OFDM systems with ICA based blind equalization,” measurements/simulations, and Wireless Communication Network. IEEE Transactions on Wireless Communications, vol. 8, no. 6, pp. 3038- 3049, Jun. 2009. [8] O. Weikert and U. Zolzer, ”New approach for resolving ambiguities for semi-blind equalization of MIMO frequency selective channels,” in Proc. 17th International IEEE Symposium on Personal, Indoor and Mobile Linhao Dong received two B.Eng. degrees of both Electrical Engineering Radio Communications (PIMRC 2006), pp. 1-5, Helsinki, Finland, Sep. and Avionics from the Dalian Nationalities University, Dalian, China, and 2006. the Glyndwr University, Wrexham, U.K., in 2009. He is currently working [9] J. F. Cardoso and A. Souloumiac, ”Blind beamforming for non-Gaussian towards the Ph.D. degree at the Department of Electrical Engineering and signals,” IEE Proc. F on Radar and Signal Processing, vol. 140, no. 6, Electronics, the University of Liverpool, Liverpool, U.K. His research interests pp. 362-370, Dec. 1993. include cooperative communications, cross-layer design for wireless network systems, and adaptive equalisation for MIMO systems. [10] ETSI/SMG/SMG2, ”The ETSI UMTS Terrestrial Radio Access (UTRA)ITU-R RTT Candidate Submission,” ETSI Proposal for IMT- 2000. [11] F. Gao and A. Nallanathan, ”Blind channel estimation for OFDM systems via a generalized precoding,” IEEE Transactions on Vehicle Yi Huang received BSc in Physics (Wuhan, China), MSc (Eng) in Microwave Technology, vol. 56, no. 3, pp. 1155-1164, May. 2007. Engineering (Nanjing, China), and DPhil in Communications from the Univer- [12] F. Gao and A. Nallanathan, ”Blind channel estimation for MIMO OFDM sity of Oxford, UK in 1994. He has been conducting research in the areas of systems via nonredundant linear precoding,” IEEE Transactions on Signal radio communications, applied electromagnetics, radar and antennas over the Processing , vol. 55, no. 2, pp. 784-789, Jan. 2007. past 23 years or so. His experience includes 3 years spent with NRIET (China) [13] R. Lin and A. Petropulu, ”Linear precoding assisted blind channel esti- as a Radar Engineer and various periods with the Universities of Birmingham, mation for OFDM systems,” IEEE Transactions on Vehicle Technology, Oxford, and Essex as a member of research staff. He worked as a Research vol. 56, no. 3, pp. 1155-1164, May. 2007. Fellow at British Telecom Labs. Dr Huang joined the Department of Electrical [14] A. Petropulu, R. Zhang, and R. Lin, ”Blind OFDM channel estimation Engineering & Electronics, the University of Liverpool as a faculty in 1995, through simple linea precoding,” IEEE Transactions on Wireless Com- where he is now the Head of High Frequency Engineering Research Group munications, vol. 3, no. 2, pp. 647-655, Mar. 2004. and MSc Programme Director. [15] D. Obradovic, N. Madhu, A. Szabo, and C. S. Wong, ”Independent Prof Huang has published over 200 refereed international journals component analysis for semiblind signal separation in MIMO mobile and conference papers, and is the principal author of the popular book frequency selective communication channels,” in IEEE international Antennas: from Theory to Practice (John Wiley, 2008). He has received Conference on Neural Networks, pp. 53-58. Budapest, Hungary, Jul. 2004. many research grants from research councils, government agencies, charity, EU and industry, acted as a consultant to various companies, and served on a number of national and international technical committees (such as the UK Location & Timing KTN, IET, EPSRC, and European ACE) and been an Editor-in-Chief or Guest Editor of four of international journals. He has been an invited/keynote speaker at many conferences/workshops (e.g. IEEE iWAT, WiCom and Oxford Engineering Programmes) and is the Leader of Focus Area D of European COST-IC0603 (Antennas and Sensors), a Senior Member of IEEE and a Fellow of IEE (now IET). INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 99 Integrating Color Vector Quantization and Curvelet Transform for Image Retrieval Yungang Zhang, Wei Gao, and Jun Liu Abstract—Since most of image databases today are pooly gap, the main reason of semantic gap is the extracted visual indexed or annotated, there is a great need for developing features mismatch human’s judgements on similarity. automated, content-based image retrieval (CBIR) systems to In human visual perception system, humans use a combina- help users to get images they want. The focus of our research is mining image features which can represent image in the tion of features (color, shape and texture) to recognize objects way of human visual perception. Our image retrieval approach and do not rely on any one individual feature [6]. The focus depends on the extracted color and shape features. Vector of our research is mining image features which can represent Quantization (VQ) can provide a way of better exploiting the image in the way of human visual perception. Therefore, spatial information to generate different color histograms than our image retrieval approach depends on the extracted color scalar color quantization, thus VQ is employed in this work to extract color pattern of images. The shape feature of images is and shape features, which are then weighted for similarity extracted by curvelet transform, as it has been proved that the measurement. curvelet transform is an almost optimal sparse representation of Although there are various color scalar quantization (SQ) objects with edges. The extracted color and shape features are methods [7], they have apparent drawbacks when using them combined and weighted by using Genetic Algorithm (GA), then in image retrieval work, since they do not consider the spatial used for image similarity measurement. Experimental results show that the GA combined features can bring about good relationship between pixels. VQ (Vector Quantization) [8] can retrieval precision and speed simultaneously. provide a way of better exploiting the spatial information to generate different histograms in such case. For color his- Index Terms—color vector quantization, curvelet transform, genetic algorithm, image retrieval. togram generation, we use a specific neural network-traing algorithm, the frequency sensitive competitive learning (FSCL) algorithm [9] to design the color codebooks for chromatic I. I NTRODUCTION and achromatic colors respectively. According to [10], FSCL is insensitive to the initial choice of codewords, and the S one of the most important applications of image A analysis and understanding, CBIR (Content-Based Im- age Retrieval) has received more and more attention. The codewords designed by FSCL are more efficiently utilized than those designed by methods such as the LGB algorithm. The shape feature of images is extracted by curvelet tremendous growth of the quantities and sizes of digital image transform, as it has been proved that the curvelet transform and video require powerful tools for searching in image and is an almost optimal sparse representation of objects with video databases. Since a lot of image databases are pooly edges [11]. The discrete curvelet transform (DCT) has been indexed or annotated, there is a great need for developing widely use in image denoising, image segmentation, texture automated, content-based systems to help users to get images analysis, and image retrieval [12]. The DCT decomposites an they want. There have been a large number of CBIR systems image into several levels of curvelet coefficients, each level developed in the recent years such as IBM’s QBIC project [1], has a number of sub-bands locate at different directions, in VisualSeek [2], PicSOM [3], PicToSeek [4] and lot more. consideration of searching speed, we use two levels of curvelet Most of these systems decompose images into bags of local transform coefficients within all directions to represent spatial features that measure peoperties such as texture, edginess and feature rather than using all of the coefficients. color. When facing with a query, the system extracts features We use Genetic Algorithm to optimize weights for all from the query, compares them to that of the images stored in the curvelet and color features. The combined and weighted the database, the distance between the query image and each features are used for similarity measurement. Experimental image in database is evaluated according to the similarity of results show that the combined feature can bring about good features. This retrieval procedure is often called QBVE (Query retrieval precision and speed simultaneously. By Visual Example) [5]. Sometimes the searching result can be quite different from user’s expectation because of the semantic II. C URVELET T RANSFORM AND S PATIAL F EATURE E XTRACTION Yungang Zhang is with the Department of Computer Science, Xi’an JiaoTong-Liverpool University, Suzhou, Jiangsu, 215123 China. He is also A. Curvelet Transform with the Department of Computer Science, Yunnan Normal University, Kunming, Yunnan, 650092 China. E-mail: yungang.zhang@liv.ac.uk. Although wavelets have been widely used in image analysis, Wei Gao is with Department of Computer Science, Yunnan Normal Uni- traditional wavelets perform well only at representing point versity, Kunming, Yunnan, 650092 China. E-mail: gaowei@ynnu.edu.cn Jun Liu is with Computing Center, Kunming Metallurgy College, Kunming, singularities, since they ignore the geometric properties of Yunnan, 650033 China. E-mail: junliu@kmyz.edu.cn structures and do not exploit the reularity of edges. Curvelet INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 100 transform was proposed in order to overcome the drawbacks the positions bj,l j,l −1 k1 k2 T k = bk1 ,k2 := Rθj,l ( 2j , 2j/2 ) , with k1 , k2 ∈ of conventional wavelet transform, the curvelet transform has Z and Rθ denotes the rotation matrix with angle θ. an almost optimal sparse representation of objects with C 2 – The choice of positions yields a parabolic scaling of the singularities [11], combined with other methods, excellent grids with the relationship length≈ 2−j/2 and width≈ 2−j . performance of the curvelet transform has been shown in The DCT thus provides a decomposition of the image f image processing [12]. into J detail sub-bands(levels), with Lj directions on each 1) The Continous Curvelet Transform (CCT): In this sec- level, and Kjl,1 × Kjl,2 spatial translations for each of these tion we briefly introduce CCT in [13], [14]. Curvelets func- directions [15]. tions can be constructed from two window functions V (t) The discrete curvelet transform can be defined through its and W (r) with three parameters, the scale a ∈ (0, 1], the discrete Fourier transform as location b ∈ R2 and the orientation θ ∈ [0, 2π), using Φ̂j0k (n1 , n2 ) = Uj (n1 , n2 )e−2πi(k1 n1 /Kj0,1 +k2 n2 /Kj0,2 ) (6) the polar coordinates(r, ω) in frequency domain, the a-scaled window can be defined as: and 3 ω Φ̂jlk = SθTl Φ̂j0k . (7) Ua (r, ω) := a 4 W (ar)V ( √ ) (1) a Sθl is a shearing matrix, which shears the grid on which 2 Let the Fourier transform for a function f ∈ L (R ) be defined2 the curvelet is evaluated by an angle θl . The slopes defined by by the angles θl are equispaced. Uj is a frequency window Z 1 function with compact support. Therefore, DCT decomposites fˆ(ξ) := f (x)e−ihx,ξi dx. (2) the frequency space into dyadic rectangular coronae, each of 2π R2 which is divided into wedges, the number of wedges doubles Designate the window Ua as the Fourier transform of the with every second level. This is how the frequency coronae in curvelet function Φa,0,0 , we can get: Figure 2 be constructed. Φ̂a,0,0 (ξ) := Ua (ξ). (3) The curvelet family can be constructed by translation and rotation of Φa,0,0 , Φa,b,θ := Φa,0,0 Rθ (x − b) , (4) 2 cos θ − sin θ where the translation b ∈ R and Rθ = sin θ cos θ is the rotation matrix with angle θ. Fig.1 is an example graph of a curvelet function. The continuous curvelet transform Γf of the function f ∈ L2 (R2 ) is given as Z Γf (a, b, θ) := hΦa,b,θ , f i = Φa,b,θ (x)f (x)dx, (5) R2 Γf is the product of a given function f with every curvelet element Φa,b,θ . Fig. 2. Discrete curvelet tiling coronae Figure 3(b) shows the curvelet coefficients of a 6-level decomposition of a 512×512 Lena in Fig.3(a). On the coarsest level, j = 1, the curvelets are isotropic, the low-pass image is located at the center of the coronae, the sub-bands curvelet coefficients located around the low-pass image according to their scales and orientations, and on the finest level, j = J (j = 6 in Fig.3(b)), one can choose to use curvelet or wavelet in the implementation, we have used wavelets on the finest level since the shorter execution time and smaller Fig. 1. Graph of Φa,b,θ , a = 210 , b = 0, θ = 120 ◦ memory requirements. Actually, there is a rule to determine the decomposition levels according to the size of the image, 2) The Discrete Curvelet Transform(DCT): The mind of the number of decomposition levels DL can be calculated as discrete curvelet transform is simple—choose a suitable sam- DL = log 2(n) − 3, where n = min(M, N ) for a M × N pling at the range of scales, locations and directions: size image. the scales aj := 2−j , j ≥ 0; the equidistant sequence of rotation angles θj,l , B. Spatial Feature Extraction Through Curvelet Transform πl2 −⌈j/2⌉ Once the curvelet coefficients have been obtained from θj,l := , l = 0, 1, . . . , 4 · 2⌈j/2⌉ − 1; DCT, the standard deviation of the curvelet sub-bands is 2 INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 101 vectors from the input data space onto a finite set of typical reproduction vectors. VQ has been widely used in different applications, such as pattern recognition, image compression, speech recognition face detection and so on [7]. A. Definition of VQ A vector quantizer–or for short quantizer–Q is defined as a mapping of a k dimensional vector space Rk onto a finite subset Y ∈ Rk : (a) Lena Q : Rk 7→ Y (9) The set Y = y1 , y2 , . . . , yN of reproduction or prototype vetors yi is also refered to as the codebook. The size N of the codebook is the essential parameter for characterizing a set of vector quantizers. With every vector quantizer Q of size N there always is associated a partition of the considered vector space Rk into the regions or cells R1 , R2 , . . . , RN . In the cell Ri lie all those vectors x ∈ Rk , which were mapped to the prototype or code word yi by the quantizer. One obtains the respective cell as the inverse image of the prototype vector under the mapping Q of the vector quantizer: (b) Decomposition Coefficients Ri = Q−1 (yi ) = x ∈ Rk |Q(x) = yi (10) Fig. 3. 6-level DCT decomposition As a quantizer Q maps every vector x from the input space to exactly one prototype yi , it defines implicitly a complete and computed as the shape features for the curvelet, since this disjoint partition of Rk into cells Ri , i.e.: S N k feature has shown good capability in desicription of wavelet i=1 Ri = R and Rj ∩Rj = ∅, ∀i, j with i 6= j. The behavoir and curvelet sub-bands [16] [17]. of a quantizer Q is then uniquely defined by the specification In our feature extraction step, in consideration of com- of the codebook Y used and the associated partition {Ri } of putational complexity, not all levels of curvelet coefficients the vector space considered. are used. Only level 2 and level 5 sub-bands coefficients are selected for feature extraction in a 6 levels decomposition of B. Quantization of Color Space with VQ images with size 512 × 512. The reason for this selection is Although there are various color scalar quantization (SQ) level 2 coefficients are in ‘finer’ details of an image, they can methods, they have apparent drawbacks when using them in be used as the texture description of the image, and coefficients image retrieval work. Since they do not consider the spatial in level 5 on the contrary, they are in a ‘coarser’ level, they are relationship between pixels, as we can see from Figure III-B, fittable for description of the edges in an image. The selection VQ can provide a way of better exploiting the spatial infor- is approriate since level 2 describe the image in a coarser mation to generate different histograms in such case. view and the level 5 coefficients are detail descriptions. The There are many methods developed for designing the VQ standard deviation σ of each sub-band in level 2 and level 5 codebook. The K-means types algorithm, such as the LGB is calculated as following: algorithm, which is also called the Generalized Lloyd Algo- v rithm (GLA) [8]., the neural network based algorithms, such u M X N u 1 X 2 σk = t Ck (i, j) − µk (8) as the Kohonen feature map [18] and Gauss mixture model M × N i=1 j=1 based algorithms [19] are popular tools. In this work, we used a specific neural network-traing algorithm, the frequency where Ck (i, j) represents the coefficient at location (i, j) in sensitive competitive learning(FSCL) algorithm [9] to design the k th DCT decomposed sub-band, µk is the mean value of our codebook. According to [10], FSCL is insensitive to the the k th sub-band. M × N is the size of the k th sub-band. initial choice of codewords, and the codewords designed by FSCL are more efficiently utilized than those designed by III. V ECTOR Q UANTIZATION methods such as the LGB algorithm. When processing color data in images one is always faced The FSCL VQ design algorithm is: with the problem that color information on the one hand 1) Initialize the codewords, Ci (0), i = 1, 2, . . . , I, to ran- needs to be quantized as compactly as possible and on the dom numbers and set counters associated with each other hand must be represented with sufficient accuracy. As codeword to 1, i.e., ni (0) = 1. digital representations are necessarily always also finite, it is, 2) Present the training sample, X(t), where t is the se- therefore, the goal of a so-called vector quantizer to map quence index, and calculate the distance between X(t) INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 102 image block. The stimulus strength of the block is calculated as m n 1 XX S= y(i, j) (11) m × n i=0 j=0 Then the ASP pattern vector, ASP = {asp(i, j), i = 0, 1, 2, m; j = 0, 1, 2, n} of the block, is formed as y(i, j) asp(i, j) = (12) (a) Two different images have same numbers of two colors S An N-dimensional vector quantizer (N = m × n), QP , with a codebook Cp = {CP (i); i ∈ I} of size I, where CP (i) is the ith N-dimensional codeword, is then designed for ASP using many training samples. In our work, three hundred 384 × 256 color images, which form more than 10 million samples have been used to design CP . 2) VQ Codeing of CSP (Chromatic Spatial Pattern): The CSP vector is formed by sub-sampling the two chro- matic channels, Cb and Cr . The CSP vector is also nor- (b) Histogram of the left image. Left: SQ. Right: VQ malized by S. In the case where a m × n block size is used, the CSP vector is a 2M -dimensional vector (M = m × n/4). Let Cb = {cb (i, j), i = 0, 1, 2, m; j = 0, 1, 2, n} and Cr = {cr (i, j), i = 0, 1, 2, m; j = 0, 1, 2, n} be the two corresponding chromatic blocks of the Cb and Cr channels. The the sub-sampled Cb signal, SCb = {scb (k, l), k = 0, 1, 2, m/2; l = 0, 1, 2, n/2} is obtained as follows: 1 1 1 XX scb (k, l) = cb (2k + i, 2l + j) (13) 4S i=0 j=0 (c) Histogram of the right image. Left: SQ. Right: VQ Fig. 4. Comparison of histograms generated by SQ and VQ Similarly, the sub-sampled Cr signal, SCr = {scr (k, l), k = 0, 1, 2, m/2; l = 0, 1, 2, n/2} is ob- tained as follows: and the codewords, Di (t) = D(X(t), Ci (t), and modify 1 1 the distance according to D̂i (t) = ni (t)Di (t). 1 XX scr (k, l) = cr (2k + i, 2l + j) (14) 3) Find j, such that D̂j (t) ≤ D̂i (t) for all i, update the 4S i=0 j=0 codeword and counter Cj (t + 1) = Cj (t) + a X(t) − Cj (t) The CSP vector, CSP = {csp(k), k = 0, 1, . . . , 2M } is nj (t + 1) = nj (t) + 1 formed by concatenating SCb and SCr . A 2M -dimensional where 0 < a < 1 is the training rate. vector quantizer, Qc , with a codebook Cc = {Cc (j); j ∈ J} 4) Repeat by going to 2. of size J is then designed. Again, over 10 million samples have been used to design Cc . 3) ASP and CSP Codebook Generation: In our work, we C. VQ Coding for Color using FSCL have used 300 images from Li et al [21], which contains 1000 G.Qiu has developed a colored pattern appearance model color JPEG images whose sizes are 384 × 256 and 256 × 384. (CPAM) [20], which has two channels capturing the char- It is a subset of the well-known Corel DB and has 10 classes acteristics of the chromatic and achromatic spatial patterns. with 100 images each. We selected 30 images of each class This model is working in Y Cb Cr color space. In CPAM, the for codebook training and generation. visual appearance of a small image block is modelled by three Although it is possible to choose various block sizes, we components: the stimulus strength(S), the spatial pattern(P) have chosen the size of the pattern to be 4 × 4 pixels for and the color pattern(C). computational convenience. Therefore, the ASP vectors are The P channel of CPAM captures the achromatic spatial 16-dimensional and the CSP vectors are 8-dimensional. We pattern of the input colored image pattern. The C channel used the FSCL algorithm to train the codebooks and a 256- captures the chromatic spatial pattern. The P and C channels codeword quantizer for both the ASP and CSP vectors. are called achromatic spatial pattern(ASP) and chromatic For each image, an ASP histogram ASPH and a CSP spatial pattern (CSP), respectively. histogram CSPH are generated as: 1) VQ Coding of ASP (Achromatic Spatial Pattern): Let Y = {y(i, j), i = 0, 1, 2, m, j = 0, 1, 2, n} be the m × n Y ASP H(i) = P r(QP (ASP ) = i), f ori = 1, 2, . . . , I, (15) INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 103 where I is size of QP . which are belong to the same image category K. the EuDis of Ki and Kj is calculated as ACSP H(j) = P r(QP (CSP ) = j), f orj = 1, 2, . . . , J, v u 5 (16) uX where J is size of QP . Therefore, these histograms’ dimen- EuDis(Ki , Kj) = t (wl × Fl (Ki ) − wl × Fl (Kj ))2 sionality are the size of the VQ codebooks, which is 512. l=1 (19) where Fl (·) is the lth feature of an image, and wl is the weight IV. I MAGE R ETRIEVAL C OMBINING C URVELET AND VQ assigned to Fl (·). Thus, the in-class distance of an image category K is a A. Features Used in Image Retrieval set of distances between any two images Ki and Kj in the After generating histograms according to Section 4.1.3, they same category. For example, in our training scheme, we use were combined with features extracted by curvelet transform 60 images to train the weights for each image category, then to be used in similarity measurement of images. For curvelet the number of elements in a ICD is 60 × 59 = 3540, but features, the level-2 and level-5 coefficients were chosen, but actually Di,j = Dj,i , so the number is halfed to 1770. different from Chapter 3, we calculated the mean and standard For an image category K in the image database T , the deviation of the first half of the total subbands at each level. between-class distance BCD is defined as: The reason why only first half of the total subbands at a BCDK = {D(Ki , Ric )|D(Ki , Ric ) = EuDis(Ki , Ric )} resolution level are considered for feature calculation is that (20) the curvelet at angle θ produces the same coefficients as the where Ric represents an image Ri which belongs to any image curvelet at angle (θ + π) in the frequency domain. These category c, c 6= K in the image database T . If we totally have subbands are symmetric in nature. Therefore, considering half C image categories and each category has M images, then, of the total number of subbands at each level reduces the total the number of elements in an BCD set is (C − 1) × M . For computation time for the feature vector formation. example, if we have 10 image categories and each category The total features for each image I then can be described has 60 images, then the number of elements in an BCD set is as: 540. F (I) = µI1 , σ1I , µI2 , σ2I , C I , (17) 2) GA-based Weights Assignment: Genetic Algorithm (GA) is an optimization tool based on the ideology of evolution. where µ1 and µ2 are mean values of level-2 and level-5 It has been widely used for many scientific and engineering curvelet coefficients; σ1 and σ2 are standard deviation of level- optimization and search problem [22]. The GAs typically 2 and level-5 curvelet coefficients; C I is the color histogram. main a population of individuals which represents the set of According to curvelet transform, there are 16 sub-bands and solution candidates for the optimization problem to be solved. 32 sub-bands in level-2 and level-5 respectively, thus, the The goodness of each candidate solution is evaluated based dimensionalitie of µ1 and σ1 are 8, the dimensionalitie for on its fitness value. The population of the GAs evolves by µ2 and σ2 are 16, and the dimensonality of C I is 512 as we a set of genetic operators. The basic genetic operators are have explained before. selection, crossover and mutation. In the selection process, some individuals are selected to be copied into a tentative next population. Individual with higher fitness value is more B. Weights Training Scheme likely to be selected. the selected individuals are altered by Our image database has 1000 images, the images in the the mutation and crossover and form a new population of database are in 10 categories, such like Africans, beach, solutions. GAs can be considered as a good solution for dinosaus, buses, moutains, achitecture etc [21]. Every cate- the feature selection and weighting problems, because the gory has 100 images. For each image category, 60 images feature selection and weighting are complicated combinational were chosen for weights training, the training process was problem in a large search space [23]. based on two distances which we called in-class distance and • Chromosome Representation between-class distance. The goal of weights selection is to let A chromosome representation is used to describe an the selected weights produce the biggest difference in-class individual in the population of interest. A chromosome distance and between-class distance. Genetic Algorithm was in our GA is defined as: selected as our optimization tool for weights selection in our c = (w1 , w2 , . . . , wi , . . . , wn ), (21) work. 1) In-class Distance and Between-class Distance: For an where wi is the weight assigned to feature vector set i image category K in the image database T , the in-class and n is the number of feature vector sets, which is 5 in distance ICD is defined as: our work. A population P is defined as: P = {c1 , c2 , . . . , ci, . . . , cP opSize } (22) ICDK = {Di,j |Di,j = EuDis(Ki , Kj), i 6= j} (18) where P opSize is the number of individuals in the where EuDis(·, ·) is the weighted Euclidean distance between population and ci is a chromosome. In our work, the the five features(see equation17) of two images Ki and Kj , P opSize = 100. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 104 • Selection Function where i=1,2,3,4,5 and f i is the mean value of the ith feature A selection function plays a vital role in a GA since it of all the training images belong to category K. When a query selects individuals to reproduce successive generations. image Q comes in, the features of Q were extracted first, form A probabilistic selection is performed based on the indi- a feature vector F (Q) = {f qi } , i = 1, . . . , 5, then, use the vidual’s fitness such that the better individuals have an following strategy to decide which weight should Q use. increased chance of being selected. 1) For each weight vector, WK (K = 1, 2, . . . , 10)(Table. The selection method, Roulette wheel, proposed by Hol- I), calculate FQK = f qi × wiK , i = 1, . . . , 5 . land [24] has been used in our implementation. The 2) For the mean feature vector of category K, K = probability, Pi , for each individual is defined by 1, 2, . . . , 10, F K (Equation 24), calculate its Euclidean Fi Distance with FQK , find out the minimum distance and P [Individual i is chosen] = PP opSize (23) Fj the corresponding category R. j=1 3) Use a uniform weight vector like {1, 1, 1, 1, 1} to do where Fi is equal to the fitness of individual i. A things in step 1 and 2 again, finding another minimum series of N random numbers is generated Pand compared distance and its corresponding category R1. i against the cumulative probability, Ci = j=1 Pj , of the 4) If R = R1, the weights assigned to the query image population. If Ci−1 < U (0, 1) ≤ Ci , the individual is Q is WR . Otherwise, a uniform weight vector will be selected. assign to Q. • Genetic Operators Simple crossover and uniform mutation have been used With above weight selection strategy, we now have two in our implementation [25]. The crossover and mutation groups of weight vectors to decide which one to use. This probabilities are 0.75 and 0.05 respectively. can guarantee the robust of our retrieval sheme, if the two • Fitness Function weight vectors can point to the same category, the trained Given an image category K with a set of weights w, if weights will be used, if they can’t, a uniform weight vector the in-class distance set ICDK has M elements and the like {1, 1, 1, 1, 1} will be employed. between-class distance set BCDK has N elements, then the fitness function is sorting all the distance elements D. Results from both ICDK and BCDK , and counting the first smallest N1 elements which come from the in-class set We use two metrics to measure to retrieval performance of ICDK , we wish to let N1 → N on the selection of an image retrieval system: recall and precision [26]. In general, proper w. precision and recall are used together in a graph so that they After GA weights training, each image category K, K = can show the change of precision values with respect to the 1, 2, . . ., 10 in the image database can obtain a proper weight recall values. Every image in the image database has been vector wiK , i = 1, . . . , 5 as listed in Table I. used for testing the precision and recall. The methods we have compared are our proposed VQ and TABLE I curvelet combined method, only VQ retrieval method in [10], W EIGHTS OBTAINED BY GA uniform weight retrieval and Gauss Mixture VQ retrieval Image Category w1 w2 w3 w4 w5 method in [19]. The retrieval result can be seen in Fig. 5. 1 0.0777 0.06967 0.0891 0.0551 0.7514 Traditional quantization methods such as SQ not included in 2 0.8895 0.2420 0.4849 0.8722 0.4156 the comparison because the research works in [10] and [19] 3 0.1323 0.0277 0.9768 0.8184 0.4156 4 0.2578 0.3026 0.8575 0.4278 0.6938 have already done the comparison, their results have proved 5 0.4518 0.7859 0.9949 0.1289 0.9108 the VQ methods have much better performances in image 6 0.9172 0.2591 0.9363 0.0369 0.5882 retrieval than SQ-based methods. 7 0.1174 0.7958 0.9783 0.8305 0.1062 8 0.4067 0.1181 0.9989 0.3282 0.5810 9 0.5104 0.5978 0.6298 0.1858 0.4478 10 0.5508 0.3120 0.1137 0.2070 0.5160 C. Automatic Weight Selection When a query image submitted into the image retrieval system, although we have obtained weight vector for each image category, we must to know which weight vector should be assigned to the query image, because we don’t know the query image belongs to which class of images in our image database. An automatic weight assignment must be done before starting the retrieval stage. The mean values of features for training images in each image category were defined first as: Fig. 5. Comparison of different image retrieval methods(1000 images’ FK = fi (24) average) INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 105 As we can see from the result curves, our proposed method [13] E. J. Candès and D. L. Donoho, “Continuous curvelet transform: I. performs the best in these methods. The combination of resolution of the wavefront set,” Appl. Comput. Harmon. Anal., vol. 19, pp. 162–197, 2003. curvelet features and VQ has greatly improved the precision [14] ——, “Continuous curvelet transform: II. discretization and frames,” than using them seperately. The GMVQ has nearly the same Appl. Comput. Harmon. Anal., vol. 19, pp. 198–222, 2003. results with our method before recall ≤ 40, but the precison of [15] T. Gebäck and P. Koumoutsakos, “Edge detection in microscopy images using curvelets,” BMC Bioinformatics, vol. 10(75), pp. 1–14, 2009. GMVQ retrieval method drops quickly after the recall > 40, [16] S. Arivazhagan and L. Ganesan, “Texture classification using wavelet this is the reason why we did not chose GMVQ as our VQ transform,” Pattern Recognition Letters, vol. 24, pp. 1513–1521, 2003. method. We want to provide more correlated images to users. [17] I. J. Sumana, M. M. Islam, D. Zhang, and G. Lu, “Content based image retrieval using curvelet transform,” in IEEE 10th workshop on Multimedia signal processing, 2008, pp. 11–16. V. C ONCLUSION [18] T.Kononen, Self-organization and associative memory. Berlin: Springer, 1989. In this paper, we have introduced a GA-based method [19] S. Jeong, C. S. Won, and R. M. Gray, “Image retrieval using color to combine curvelet transform and color vector quantization histogram generated by gauss mixture vector quantization,” Computer for image retrieval. The Genetic Algorithm was employed Vision and Image Understanding, vol. 94, pp. 44–66, 2004. [20] G.Qiu, “Image coding using a coloured pattern appearance model,” in for training the weights of features, an automatic weights Visual Communication and Image Processing 2001, 2001. selection strategy is also explained. Through the experiments, [21] J.Li, J.Z.Wang, and G.Wiederhold, “Irm: integrated region matching for we demonstrated that the scheme could be used for index- image retrieval,” in Proc. 2000 ACM Multimedia Conf. [22] D. Coley, An Introduction to Genetic Algorithms for Scientists and ing images for content-based image retrieval from image Engineers. World Scientific, 1999. databases. We further demonstrated that the proposed method [23] I.Oh, J.Lee, and B.Moon, “Hybrid genetic algorithms for feature selec- is better than or at least as good as some state-of-art image tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 1424–1437, 2004. retrieval techniques. Our future work will include studying the [24] H. J.H., Adaption in Natural and Artifical Systems. The University of weight selection scheme’s performance and developing it into Michigan Press, Ann Arbor, MI, 1975. an big amount image database indexing scheme. The other [25] G. D. E., Genetic algorithms: In search, optimization and machine learning. Addison-Wesley, 1987. weights optimization tools such as particle swarm optimization [26] J. R. Smith and S. F. Chang, “Tools and techniques for color image (PSO) will also be considered to use for our work. retrieval,” in Proc. SPIE storage and retrieval for image and video database IV, vol. 2670, 1996, pp. 426–437. ACKNOWLEDGMENT The project is funded by China Jiangsu Provincial Natural Science Foundation “Intelligent Bioimages Analysis, Retrieval and Management”(BK2009146) and Xi’an JiaoTong-Liverpool Research Development Fund “Learning Multiscale Kernels for Yungang Zhang is a Ph.D. student at the Depart- ment of Computer Science and Software Engineer- Image Analysis” (RDF 01-10-30). ing, Xi’an JiaoTong-Liverpool University, China. His research fields include pattern recognition, ma- R EFERENCES chine learning and image analysis. He got his bach- elor degree and master degree on computer science [1] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, from Yunnan Normal University in 2002 and 2005, M. Gorkani, J. Hafner, D. Lee, D. Pettovic, D. Steele, and P. Anker, respectively. Now he is pursuing his doctoral degree “Querying by image and video content: the QBIC system,” IEEE in computer science at Xi’an JiaoTong-Liverpool Transactions on Computers, vol. 25, pp. 23–32, 1995. University. [2] J. R. Smith and S. F. Chang, “VisualSeek: a fully automated content- based image query system,” Proceedings of the ACM Multimedia, pp. 87–98, 1996. [3] J. Laaksonen, M. Koskela, and E. Oja, “PicSOM: self-organizing image retrieval with mpeg-7 content descriptors,” IEEE Transactions on Neural Networks, vol. 13(4), pp. 841–852, 2002. [4] T. Gevers and A. Smeulders, “PicToSeek: Combining color and shape invariant features for image retrieval,” IEEE Transactions on Image Wei Gao is a Ph.D. student at the Department Processing, vol. 9, pp. 102–119, 2000. of Mathematics, Soochow University, China. His [5] N. Vasconcelos, “From pixels to semantic spaces: Advances in content- research fields are graph theory, statistical learning based image retrieval,” Computer, vol. 40(7), pp. 20–26, 2007. theory, information retrieval, and artificial intelli- [6] A. K. Jain and A. Vailaya, “Image retrieval using color and shape,” gence. He got two bachelor degrees on computer Pattern Recognition, vol. 29(8), pp. 1233–1244, Aug. 1996. science from Zhejiang industrial university in 2004 [7] C. W. Tsai, C. Y. Lee, M. C. Chiang, and C. S. Yang, “A fast vq code- and mathematics education form College of Zhejiang book generation algorithm via pattern reduction,” Pattern Recognition education in 2006. Then, he enrolled in department Letters, vol. 30, pp. 653–660, 2009. of computer science and information technology, [8] A. Gersho and R. M. Gray, Vector Quantization and signal compression. Yunnan normal university, and got Master degree Boston: Kluwer Academic Publishers, 1992. there in 2009. Now, he is Ph.D. student in depart- [9] S. C. A. et al., “Competitive learning algorithms for vector quantization,” ment of Mathematics, Soochow University, China. Neural Networks, vol. 3, pp. 277–290, 1990. [10] G.Qiu, “Indexing chromatic and achromatic patterns for content-based colour image retrieval,” Pattern Recognition, vol. 35, pp. 1675–1686, 2002. [11] E. J. Candès and D. L. Donoho, “New tight frames of curvelets and optimal representations of objects with piecewise-C 2 singularities,” Comm. on Pure and Appl. Math, vol. 57, pp. 219–266, 2004. Jun Liu is with Computing Center, Kunming Metallurgy College, China. [12] J. Ma and G. Plonka, “The curvelet transform,” IEEE Signal Processing Magazine, vol. 27(2), pp. 118–133, Mar. 2010. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 106 Multi-Beam Radar Search Improvement Via Digital Signal Re-Steering Dinghong Lu,Yang Li, and Jimin Xiao Abstract—This paper studies a method of producing multi- ple receiving beams by applying re-steering of digital signals exported from subarrays inside the 3dB beam width of the transmitting beam to remedy the loss caused by mismatching between target location and the center of the transmitting beam. This improves the average detection performance inside the 3dB beam width of the transmitting beam, and finally enhances the search performance of Radar. The simulation experiments have verified the validity of the proposed method. Index Terms—re-steer, receiving multi-beam, average detection performance, subarray I. I NTRODUCTION T HE search performance of Radar in a certain area can be denoted by false alarm probability, average detection probability and scanning time. As the Radar equation indi- Fig. 1. Typical subarray structure cating, the detection capability of Radar is determined by the ratio of the signal energy to the power of noise (SNR). At a performance by adopting multiple receiving beams formed in fixed transmitting power, the energy of target’s echo signal is systems with element digitalization was studied in [3]. determined by the Radar Cross Section (RCS) of the target, This paper proposes a method aiming at Radar systems with the gain of transmitting antenna and that of receiving antenna. subarray digitalization. Re-steering of digital signals exported With the development of stealthy technique, the RCSs of from subarrays to form multiple receiving beams (called multi- stealth aircrafts such as F-117 and B-2 are very small, which beam receiving system) covering the 3dB beam width of means that it becomes more and more difficult for Radar to the transmitting beam was studied. The average detection find the target correctly. As the Radar side can do nothing performance inside the 3dB beam width, and further the search with the RCS of a target, the problem that how to improve performance of Radar in the whole scanning area can be detection performance by utilizing the wave energy sufficiently improved by adopting the proposed method. In section II becomes important. Based on the principle of Radar target the system realization of forming multiple receiving beams detection, it is clearly that the energy of Radar wave can be through re-steering of subarray digital output signals will explored sufficiently when the target locates at the center of the be introduced first, then in section III we will derive the transmitting beam so that the scattered energy from the target equivalent weighted value and the test statistic for every is maximum, and the center of the receiving beam is also receiving beam taking a standard line array for example, later directed to the target’s location so that the received energy on simulation validation will be shown in section IV, and is maximum. However, in the practical Radar systems, the finally a conclusion will be given in section V. beam step for scanning is decided by the 3dB beam width, so the true location of target can be different from the center II. F ORMING M ULTI - BEAM R ECEIVING S YSTEM THROUGH of transmitting or receiving beam of Radar, therefore, some R E - STEERING OF S UBARRAY D IGITAL O UTPUT S IGNALS energy of the Radar wave was wasted ( called beam shape Practical phased array Radar systems often consist of many loss). elements. It is unfeasible to deploy a digital receiver for every The literature [1] and [2] focused on the performance element because of the limitation of space and the cost of improvement by applying different kinds of electronic syn- system. So practical phased array Radar systems often adopt thesized wide transmitting beams to reduce the transmitting subarray digitalization structure, namely deploy an analog beam shape loss. A method to improve search and tracking phase shifter for every element and a digital receiver for every Dinghong Lu and Yang Li are with the Department of Electrical En- subarray formed by several elements. The typical array system gineering, Beijing Institute of Technology, Beijing, China. E-mail: lud- is shown in Fig. 1. inghong@bit.edu.cn. For Radars with subarray digitalization structure, because Jimin Xiao with Department of Electrical and Electronic Engineering, University of Liverpool, UK, and Department of Electrical and Electronic there are two stage steerings including the analog phase shift Engineering, Xian Jiaotong Liverpool University, Suzhou, China. of elements and the digital phase shift of subarray output INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 107 signals, so the final receiving pattern of an array of such kind is decided by both of them. Shifting phases of signals received by elements by controlling phase shifters of elements enables the pattern of subarray to focus on the pointing direction of the transmitting beam, while re-steering of the output digital signals exported from AD samplers attached to subarrays forms the final pattern of the whole array. Because the re- steering of subarray output signals is carried on in digital domain, we can re-steer digital output signals differently Fig. 2. The diagram of matching and mismatching of array factor and subarry to form multiple receiving beams with different directions patterns simultaneously. Here we take a standard line array for example to tell the story of re-steering for the sake of convenience. All discussions So we get the receiving pattern F (µ, µa , µ0 ) finally, which is in the following part of this paper will turn solely on the a gain function of arriving angles. It can be denoted as standard line array. First of all, we will show the synthesizing process of a so (t) F (µ, µa , µ0 ) = receiving pattern which is a gain function of the arriving angle. s(t) (5) We assume that a standard line array has N elements which = (WsH (µ0 )Vs (µ))(WaH (µa )Va (µ)) can be divided into M subarrays equally. Each subarray has K = N/M elements. The distance between elements and subarray △ Actually the quantity AF (µa , µ) = WaH (µa )Va (µ) is the centers are dx and ds respectively. If the echo signal is denoted receiving pattern of the super array(array factor), while the as s(t) at the reference point (the first left element), then △ the N × 1 signal vector received by N elements is described quantity Fs (µ0 , µ) = WsH (µ0 )Vs (µ) is the receiving pattern as(under narrow band assumption) of the subarrays. If the steering of the signals exported from subarrays is 2π 2π V (µ, t) = s(t)[1 e−j λ dxsin(θ) . . . e−j λ (N −1)dxsin(θ) ]T consistent to that of subarray, namely µ0 = µa , then it can (1) be called as matching of both sides, a normal receiving beam △ where µ = sin(θ), θ ∈ [− π2 , π2 ] is sine value of the arriving with the same pointing direction as that of the transmitting angle variable, θ is the arriving angle variable, T denotes beam is formed in this way. If µ0 6= µa , and µa = µ0 + △u, transpose of matrices or vectors. Here we assume that elements it can be called mismatching of both sides, then as indicated in are isotropic. We define a K × 1 vector as literature[4], some problems will appear in the final receiving pattern, such as rising sidelobe and loss of main lobe gain. 2π 2π △ Vs (µ) = [1 e−j λ dxsin(θ) . . . e−j λ (K−1)dxsin(θ) ]T However, when △u is small enough, the negative effects mentioned above can be tolerated. The diagram of matching and, a M × 1 vector as and mismatching of subarray patterns and array factor is shown △ 2π 2π in the following Fig. 2. As mentioned before, because the Va (µ) = [1 e−j λ dssin(θ) . . . e−j λ (M −1)dssin(θ) ]T re-steering imposed on subarrays output signals is done in digital domain, multiple receiving beams covering the 3dB then (1) can be rewritten as width of the transmitting beam can be formed simultaneously. Their receiving patterns and output signals of the multi-beam V (µ, t) = s(t)Va (µ) ⊗ Vs (µ) (2) receiving system are denoted as(assume L receiving beams were formed) Actually, Vs (µ) is the response vector of a subarray, while Va (µ) can be regarded as the response of a array with its F (µ, µa,l , µ0 ) = Fs (µ, µ0 )AF (µ, µa,l ), l = 1, 2 . . . L (6) elements set at the subarray centers(called super array). The effect of the analog phase shifters can be denoted as Ws (u0 ), a K × 1 weighted vector. Here, µ0 = sin(θ0 ), θ0 is the steering so,l (t) = s(t)(WsH (µ0 )Vs (µ))(WaH (µa,l )Va (µ)), l = 1, 2 . . . L angle of subarray. The signal vector out of subarrays which is (7) a M × 1 vector can be described as where µa,l = sin(θa,l ), l = 1, 2 . . . L are the directions super array steered to. Sos (t) = s(t)(Ws (µ0 )H Vs (µ, t))Va (µ) (3) The multiple receiving beams covering the 3dB width of the transmitting beam, which are processed simultaneously, can The output signal vector of subarrays is then re-steered in remedy the receiving shape loss caused by the disagreement digital domain. The re-steering operation can be denoted as between real location of target and the pointing direction of a weighted vector Wa (µa ) which is a M × 1 vector. Here the transmitting beam. Using the multi-beam receiving system µa = sin(θa ), θa is the steering angle of the super array. to detect, namely using signals {so,l , l = 1, 2 . . . L}, can Then the output signal of the whole process can be described improve the detection performance of Radar. We will analyze as the detection performance of the simultaneous multi-beam so (t) = s(t)(WsH (µ0 )Vs (µ))(WaH (µa )Va (µ)) (4) receiving system in the following part. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 108 III. T HE A NALYSIS ON D ETECTION P ERFORMANCE OF where N0 is a N × 1 dimension complex Gauss noise vector M ULTI - BEAM R ECEIVING S YSTEM and obeys N (0, C), C is the noise covariance matrix. We If we want to discuss detection performance of the multi- assume that the noises of elements are independent and obey beam receiving system proposed, we should start from the the same distribution, then C = Pn I, where Pn is the power expression of the test statistic of every receiving beam with of noise and I is N × N dimension unit matrix. s = bejϕ two stage steering structure, then we will discuss the detection is the complex amplitude of received signal, where b is the performance of each beam under the condition of Gauss White amplitude of echo signal representing the intension of the echo Noise(GWN) assumption, finally, we will study the average signal, and ϕ is the instantaneous phase of the signal. Vg (θ) detection performance inside the 3dB beam width of the is the response vector of the whole array mentioned above. transmitting beam using the multi-beam receiving system. According to the equivalent weighted value of array processing deduced above, for the lth receiving beam, the relevant test A. The Equivalent Weighted Value and Test Statistics of Re- statistic Tl (X) obtained can be given by ceiving Beams H Tl (X) = |Wg,l X| (13) Here, we assume that the 3dB width of the transmitting beam is θb , and L(for the sake of convenience, L is an odd where | • | denotes the operation of getting the modulus. number) receiving beams covering θb are formed, the center Actually the test statistic of the lth receiving beam is the direction of each beam is decided as modulus of the output signal of the receiving beam. l µa,l = sin(sin−1 (µ0 ) + θb ) (8) L−1 B. The Detection Performance of Each Receiving Beam where l = −(L − 1)/2, . . . , (L − 1)/2. When l = 0, Because N0 is a N × 1 dimension Gauss noise vector with µa,0 = µ0 , the direction of the receiving beam pointing is components irrelevant mutually and the operation of Wg,lH X consistent to that of the transmitting beam. Observing (1) H imposed on X is linear, so the output signal Wg,l X of the and (2), we will find that the kronecker product of Va (µ) lth beam obtained by two stage steering including element and Vs (µ) forms the response of the whole array to echo level phase shift and digital re-steering processing imposed signal s(t) from the direction of angle θ, it can be defined △ on receiving signal X also obeys complex Gauss distribution. as Va (µ) ⊗ Vs (µ) = Vg (µ). Actually, V (µ, t) = s(t)Vg (µ). Under hypothesis H0, the signal received only contains noise, For the lth receiving beam, its receiving pattern denoted by so the mean and variance of the output signal of the lth (5) or (6) can be developed as receiving beam are given by M −1 K−1 X X H H F (µ, µ0 , µa,l ) = Wa∗ [m]Ws∗ [k]e−j 2π λ (mds+kdx)µ E(Wg,l X) = Wg,l E(X) (14) m=0 k=0 =0 WsH (µ0 ) 0 ... H H 0 WsH (µ0 ) ... V ar(Wg,l X) = E(Wg,l XX H Wg,l ) = WaH (µa,l ) .. .. .. Vg H = Wg,l E(XX H )Wg,l (15) . . . H 0 0 H Ws (µ0 ) = Wg,l CWg,l = (WaH (µa,l ) ⊗ WsH (µ0 ))Vg (µ) H So, the distribution of the output complex signal Wg,l X of lth (9) receiving beam under H0 can be written as N (0, WgH CWg ). So, as (9) indicating, the equivalent weighted value of the lth Under hypothesis H1, the signal received by the array contains receiving beam obtained by element phase shift and re-steering target echo and noise, the mean and variance of the output of digital signals exported from subarrays can be defined as signal of the lth receiving beam are given by △ Wg,l (µ0 , µa,l ) = Wa (µa,l ) ⊗ Ws (µ0 ) (10) H E(Wg,l H X) = Wg,l E(X) Then, the pattern of lth receiving beam can be written as H = Wg,l (sVg ) (16) F (µ, µ0 , µa,l ) = (WsH (µ0 )Vs (µ))(WaH (µa,l )Va (µ)) = H sWg,l Vg = (Wa (µa,l ) ⊗ Ws (µ0 ))H (Va (µ) ⊗ Va (µ)) H H H H H = H Wg,l (µ0 , µa,l )Vg (µ) V ar(Wg,l X) = E((Wg,l X − sWg,l Vg )(Wg,l X − sWg,l Vg )H ) (11) H = Wg,l E(N0H N0 )Wg,l After We have obtained the equivalent weighted value of the H array with two stage steering structure, we will show the test = Wg,l CWg,l (17) statistics used to detect targets. The target detection problem is H So the distribution of the output complex signal Wg,l X an issue of binary hypothesis test. As stated in [5], the model of lth receiving beam under H1 can be written as of receiving data vector X under the two hypothesizes for H N (sWg,l Vg , WgH CWg ). Then as stated in [6], its relevant phased array Radar is given by H test statistic Tl (X) which is the modulus of Wg,l X obeys H1 : X = sVg (θ) + N0 Rayleigh distribution under hypothesis H0, and obeys Rice (12) H0 : X = N0 distribution under H1 hypothesis. The detection threshold νt,l INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 109 of the lth receiving beam is decided by (18) when the false and 0(•) denotes infinitesimal of higher order. So (23) can be alarm probability level is set as Pf a,l . deduced to q H W )log(1/P P (Z) = 1 − (1 − Lα + 0(α2 )) νt,l = Pn (Wg,l g,l f a,l ) (18) (25) ≈ Lα The relevant detection probability of the lth receiving beam According to the analysis above, we can conclude that the can be described by overall false alarm probability of the multi-beam receiving s s H V |2 |sWg,l g 1 system is L times of that set for a single receiving beam. Pd,l = Q( HW , 2log( )) (19) In other words, for fairly comparing, the false alarm level 0.5Pn Wg,l g,l P f a,l of every receiving beam of the multi-beam receiving system approximately, where Q is Marcum Q function. Actually, the should be set as two quantity describe the detection performance of the lth Pf a,l = Pf a,g /L (26) receiving beam entirely. where l = −(L − 1)/2, · · · , 0, · · · , (L − 1)/2, and Pf a,g is the false alarm level controlled for the whole multi-beam C. The Detection Performance Analysis of Multi-beam Receiv- receiving system, so that the overall false alarm level will be ing System consistent when we contrast the detection performance of the If we regard the multiple receiving beams formed by re- multi-beam receiving system with that of the traditional single steering subarray output signals as a system to detect targets beam receiving system. lying in the 3dB width of the transmitting beam, then how Under hypothesis H1, the situation is little different since is the detection performance of this multi-beam receiving the correlation of detection events of receiving beams become system? stronger significantly. But we can use (27) similar to (23) to We assume Zl , l = −(L − 1)/2, · · · , 0, · · · , (L − 1)/2 is predict the detection performance if we neglect the correlation the detection event of the lth beam, then here we define the of detection events. detection event of the multi-beam receiving system Z as (L−1)/2 Y Pd,g = 1 − (1 − Pd,l ) (27) (L−1)/2 [ l=−(L−1)/2 Z= Zl (20) l=−(L−1)/2 where Pd,g is the overall detection probability of the multi- S beam receiving system, Pd,l is the detection probability of where represents the union of events. the lth receiving beam. However, because of some extent Under hypothesis H0, the detection events(false alarm) Zl , correlation between detection events of beams, the real overall l = −(L − 1)/2, · · · , 0, · · · , (L − 1)/2 can be regarded as in- detection probability will be different from that predicted by dependent mutually, so the probability of detection event(false (27). The following simulation results will show the difference. alarm) of the multi-beam receiving system can be decided by (L−1)/2 IV. S IMULATIONS Y P (Z) = 1 − P (Zl ) After the realization of the multi-beam receiving system and l=−(L−1)/2 analysis on detection performance above, simulations will be (21) taken to verify. (L−1)/2 Y =1− (1 − P (Zl )) l=−(L−1)/2 A. Simulation Conditions here, Q represents intersection of sets, Zl represents comple- We adopt a standard line array with N = 32 elements ment of Zl , and P (•) represents the probability of the event. which is divided into M = 4 subarrays equally. Each We assume the false alarm probability of each receiving beam subarray contains K = 8 elements. Elements are assumed is controlled as to be isotropic. The pointing direction of the transmitting Pf a,l = α (22) beam is θ0 = 70◦ , and the operation wavelength of Radar is 0.02m. The 3dB beam width of transmitting beam is where l = −(L − 1)/2, · · · , 0, · · · , (L − 1)/2, then (21) can θb = 2arcsin(0.891/32) = 3.2◦ . In this simulation, L = 3 be rewritten as receiving beams are generated. (L−1)/2 Y P (Z) = 1 − (1 − α) B. Detection Performance Comparison inside the 3dB Beam l=−(L−1)/2 (23) Width = 1 − (1 − α)L We check each position inside the 3dB beam width of the transmitting beam respectively, calculate the detection Because the false alarm level controlled is usually very probability of the traditional single beam receiving system and small, so we can apply Taylor’s expansion to (1 − α)L in the proposed multi-beam receiving system whose detection (23) at zero, and we obtain event is defined in (20). The simulation results are shown in (1 − α)L = 1 − Lα + 0(α2 ) (24) Fig. 3. INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 110 Fig. 3. Detection performance comparison inside the 3dB beam width of Fig. 4. Pfa = 0.001 comparison of average detection probability transmitting beam In Fig. 3, abscissa is the diverging angle between the real position of a target and the center of the transmitting beam, ordinate is detection probability. The solid curve represents the detection probability of a traditional single beam receiving system, which is predicted by (19) with l = 0. The dotted line represents detection probability of the multi-beam receiving system predicted by (27). the dash-and-dot line represents the detection probability of the multi-beam receiving system obtained by applying Monte Carlo simulation. It can be seen from Fig. 3 that with the real position of target diverging from the pointing direction of the transmitting beam, the detection performance of the single beam receiving system declines quickly because of the double mismatching Fig. 5. Pfa = 0.0001, comparison of average detection probability loss, while the multi-beam receiving system adopting multiple receiving beams covering the 3dB beam width of the trans- mitting beam remedies the receiving shape loss, so its de- tection performance declines with the diverging much slowly dotted lines represent the average detection probability curves comparing with that of the single beam receiving system. changing with SNR in the multi-beam receiving system given Consequently, the average detection performance inside the by Monte Carlo simulation. 3dB beam width of the transmitting beam can be improved by We can conclude from Fig. 4, Fig. 5 and Fig. 6 that for the proposed method. the traditional single beam receiving system and the multi- Comparing the detection probability curve predicted by (27) beam receiving system described in this paper, if you want to with the result curve of Monte Carlo simulation, we can obtain equivalent detection performance, the SNR requirement conclude that because there are correlations to some extent of the multi-beam receiving system is lower than that of the between detection events of beams in the multi-beam receiving system under hypothesis H1, so the overall detection proba- bility predicted by (27) diverges from real overall detection probability. The accurate measurement of this deviation needs to be studied further. C. The Average Detection Performance Comparison inside the 3dB Beam Width Here we will show the average detection performance changing with SNR inside the 3dB beam width of the trans- mitting beam and make comparison under different false alarm probabilities. The simulation results are shown in Fig. 4, Fig. 5 and Fig. 6. In Fig. 4, Fig. 5 and Fig. 6, the results are similar. The solid lines represent the average detection probability curves changing with SNR in the single beam receiving system, the Fig. 6. Pfa = 0.00001, comparison of average detection probability INTERNATIONAL JOURNAL OF DESIGN, ANALYSIS AND TOOLS FOR CIRCUITS AND SYSTEMS, VOL. 2, NO. 2, AUGUST 2011 111 single beam receiving system by 1-1.5dB, in other words, if R EFERENCES you apply the multi-beam receiving system proposed in this [1] J. Clayton Kercel,George C. Brown and Mark A. Mitchell, “Phase-Only paper you can obtain extra SNR gain. Accordingly, the dwell Transmit Beam Broadening for Improved Radar Search Performance”, time of transmitting beams can be reduced, and then the search Radar Conference IEEE, 2007. [2] J. Clayton Kercel,George C. Brown,and Mark A. Mitchell, “Extreme efficiency of Radar can be improved eventually. Beam Broading Using Phase only Pattern Synthesis”, Fourth IEEE Workshop on Sensor Array and multi-channel Processing(SAM 2006), Waltham, MA,12-14 july 2006. V. C ONCLUSIONS [3] Kai-Bor Yu, “Digital beamforming of multiple simultaneous beams for This paper has proposed a method aiming at phase array improved target search”, Radar Conference IEEE 2009, pp.1-5. [4] David D. Aalfs and Robert L. Howard, “Wideband Subarray Sizing Radar systems with subarray digitalization to remedy the shape Requirements For Digital Beamforming Arrays Employing Multiple loss. The detection strategy (the detecion event definition of Digitally Re-steered Beams”, Proc. SPIE Vol. 4374. multi-beam receiving system) of the multi-beam receiving [5] Steven M. Kay, Fundamentals of Statistical Signal Processing,Vol.II: Detection Theory, NJ : Prentice Hall, 1998. system has been discussed, and the detection performance [6] Van Trees H. , Detection, Estimation, and Modulation Theory: Part I, of the multi-beam receiving system under detection event New York : Wiley,1968. definition in (20) has been compared to that of the traditional [7] Houg S.M. , Kwag Y. K., “Approximations for detection probability and measurement accuracy taking into account antenna beam pointing loss”, single beam receiving system. The simulation results show that IECE Trans. Comm. Vol. E89-B no. 7, July2006, pp. 2106-2110. the receiving beam shape loss was remedied by applying the [8] Zatman M. , “Digitization requirements for digital radar arrays”, Radar proposed multi-beam receiving system. In further study, the Conference, 2001, Proceedings of the 2001 IEEE, May 2001, pp:163- 168. correlation between the detection events of receiving beams [9] Bassem R. Mahafza, Matlab Simulations for Radar Systems Design, should be studied deeply, and other detection strategies based London : CRC Press, 2004 on the described multi-beam receiving system may proposed [10] Robert J. Mailoux, Phase Array Antenna Handbook, London : CRC Press, 2004 to improve the performance further. [11] Levanon N. , Radar principles, New York : Wiley-Interscience,1988. [12] Van Trees H. , Optimum array processing, New York : Wiley,2002. ACKNOWLEDGMENT The writers appreciate the illuminating suggestions pro- vided by colleagues in Radar Technology Institute of Beijing Institute of Technology and Department of Electrical and Dinghong Lu received the B.S degree in information and electronic engineer- ing from Nanjing University of Science and Technology, China, in 2004. He is Electronic Engineering, Xi’an Jiaotong Liverpool University. currently working towards the Ph.D. degree in communication and information engineering at the Beijing Institute of Technology. His research interests include Radar system, detection and estimation and their applications to phased array Radar. Yang Li received the B.S degree in electronic engineering from Beijing In- stitute of Technology, China, in 2002 and the Ph.D. degree in communication and information engineering from Beijing Institute of Technology in 2007. Now he is an Associate Professor of Beijing Institute of Technology. His research interests include Radar system, target tacking and digital signal processing. Jimin Xiao received the B.S degree and the M.S degree both in information and electronic engineering from Najing University of Posts and Telecommu- nications, China, in 2004 and 2006. He is working towards the Ph.D. degree at Xian Jiaotong Liverpool University. His research interests include digital signal processing ,image processing and statistic signal processing.
Enter the password to open this PDF file:
-
-
-
-
-
-
-
-
-
-
-
-