AN EFFICIENT PSO OPTIMIZED INTEGRATION WEIGHT ESTIMATION USING D-PRIME STATISTICS FOR A MULTIBIOMETRIC SYSTEM

International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 DOI : 10.5121/ijbb.2012.2304 31 AN EFFICIENT PSO OPTIMIZED INTEGRATION WEIGHT ESTIMATION USING D-PRIME STATISTICS FOR A MULTIBIOMETRIC SYSTEM S. M. Anzar1 and P. S. Sathidevi2 1Department of Electronics & Communication Engineering, National Institute of Technology, Calicut, India, 673601 p090004ec@nitc.ac.in 2Department of Electronics & Communication Engineering, National Institute of Technology, Calicut, India, 673601 sathi@nitc.ac.in ABSTRACT The efficiency of a multibiometric system can be improved by weighting the scores obtained from the degraded modalities in an appropriate manner. In this paper, we propose an efficient PSO (Particle Swarm Optimization) based integration weight optimization scheme using d-prime statistics to determine the optimal weight factors for the complementary modalities, under different noise conditions. Instead of treating the weight estimation process from an algebraic point of view, an attempt is made to consider the same from the principles of linear programming techniques. The performance of the proposed method is analysed in the context of fingerprint and voice biometrics using sum rule of fusion. The d-prime statistics of fingerprint and voice modalities are measured and the integration weights are computed as the ratio of these two statistics. The computed d-prime ratio is then optimized against the recognition accuracy. The optimizing parameter is estimated in the training/validation phase using Leave-One-Out Cross Validation (LOOCV) technique. Experimental studies show that the proposed method improves recognition rate under normal operating conditions and reduces the FAR (False Acceptance Rate) considerably even at extreme low SNR (Signal to Noise Ratio) conditions. The proposed biometric solution can be easily integrated into any multibiometric system with score level fusion. Moreover, it finds extremely useful in applications where there are less number of available training samples. KEYWORDS Multibiometric System, Integration Weight, d-prime Statistics, Particle Swarm Optimization, Robustness 1. INTRODUCTION In this paper, the primary focus is on the determination of the optimal integration weights using d-prime statistics, for the score level fusion of fingerprints and voice biometrics. Although the recognition accuracy of the voice biometrics is high in clean conditions, its performance tends to be significantly degraded under the presence of background noise. This is not desirable in real world applications. At high SNR, the voice matcher outperforms the fingerprint matcher and the final decision heavily relies on the score values of the voice matcher. When the voice biometric is contaminated by noise, the fingerprint matcher outperforms the voice matcher. In this case, the score values of the fingerprint matcher contribute more to the final decision. Therefore, it is crucial to estimate the optimal weight factor dynamically, to combine both the modalities for the maximum recognition accuracy [1]. Environmental noise and the quality in acquisition do affect the matching score distributions. Two examples of this are scanner quality in fingerprints and microphone quality for voice recognition [2]. Integration weights depend on the matching score International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 32 distributions of the voice biometrics since the matching score vectors are heavily affected by the amount of acoustic noise. Figure 1: Block diagram representing Score Level Fusion The smaller the overlap between the genuine and the impostor scores the better the recognition rate. As the amount of noise increases, the overlap between the genuine and the impostor score distributions will also increase. Thus, the score separability measures from the matching score space at different noise conditions give an indication of the quality of the matcher. It is well known that the d-prime statistics gives the ratio of separation to spread (separation/spread) of the genuine and the impostor score distributions. Here, we use this auxiliary information to find the best integration weights for the optimal fusion of the two modalities. The estimation of optimal integration weight is important because it determines the amount of contribution of each modality towards the final decision. Otherwise, the system will show attenuating fusion. We are motivated by [3, 4] to develop a bimodal biometric system that is more robust to environmental and sensor noise. Figure 1 shows the overall block diagram of the score level fusion strategy. 2. RELATED WORKS As the inputs tend to be noisy, the sum rule of fusion is proved more effective [5]. However, this fusion strategy results in attenuating fusion, when the integration weights are not optimal. One can manually determine the integration weight, but this is a non-optimal approach, which often needs much iteration and lot of expertise [6]. Previously, the integration weight estimation was treated from an algebraic point of view. Optimal integration weights estimation using least squares technique was reported in [7]. Reliability based optimal integration weights estimation for audio-visual decision fusion was reported in [4, 6]. Optimal integration weights for a multibiometric system with fingerprints and voice were reported in [1] by the author’s of this paper. We compared the proposed PSO based integration weight optimization scheme with the integration weights optimization scheme reported in [1]. In order to automate the estimation of the “best available weight factor”, we formulated the integration weight estimation problem using mathematical programming techniques. The normalized match scores from the two modalities were combined by the weighted sum rule to produce the final decision. Given the speaker scores (sc) S and the finger scores ( fc) S the fused scores were obtained by linearly combining the two scores as follows, ( ) ( ) ( ) 1( ) fus sc fc S = βS + − β S (1) International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 33 The weight factor β ( 0 ≤ β ≤1) determined the amount of contribution of each modality to the final decision. The objective was to find the optimal β that maximise the recognition accuracy. LOOCV strategy was employed for the estimation of the optimal integration weight. The performances of the proposed method were evaluated with a direct search optimization method (Grid Search (GS) [8]) and a random search optimization method (Genetic Algorithm (GA) [9]). Even though the recognition accuracy of the proposed methods were pretty better than the unimodal systems under normal operating conditions, the demerits of the method was that, at the extreme noise conditions the fusion module contributed zero weighting to the voice modality. In these conditions, the overall performances of the multimodal system solely depend on the unimodal fingerprint matcher. Moreover the FAR and FRR hardly show any substantial improvements at low SNR conditions. In order to improve the global recognition accuracy and reduce the FAR under various noise conditions, we proposed PSO optimized integration weight estimation scheme using the d-prime statistics. To the best of our knowledge, the proposed optimal integration weight estimation using d-prime statistics has not been attempted until now. 3. PROPOSED METHOD In order to improve the performance of score level fusion at various noise conditions, we present a new weighting strategy using the d-prime statistics [10]. For judging the quality of a matcher, it is sufficient to measure the separation between non-match score probability density and the match score probability density. The sensitivity index or the d-prime statistic could effectively characterise these distance measures. The d-prime statistics provides the separation between the means of the genuine and the impostor score distributions, in units of the standard deviation of the impostor score distribution [10]. The separation/spread ' d is defined as, ( ) 2 2 ' m n m n d σ σ µ µ + − = (2) where, µ m = mean of genuine scores; σ m = variance of genuine scores; µ n = mean of impostor scores; σ n = variance of impostor scores. Numerator (separation) gives the indication that how much the mean of the distributions are separated and the denominator (spread) gives an indication of their overlap. A high d-prime value indicates that the genuine scores can be readily detected. Thus, the discriminability of a class depends on both the separation and the spread of the genuine and impostor score distribution curves. The sensitivity index captures both the separation and the spread of the genuine and impostor score distributions effectively. The global recognition rate of a multibiometric system can be improved by incorporating these separability measures as integration weights in the fusion module. The d-prime statistics of each modality were captured from the genuine and impostor scores during the training/validation stage. When the voice samples do not contain any noise, the d-prime statistic exhibits high values. As the voice samples become noisy, these values tend to become small. The d-prime ratio can be calculated as follows, ' ' ' V F V d d d + α = (3) where ' V d and ' d F are the d-prime indices of the voice and fingerprint modality respectively. Even though the integration weight obtained using equation (3) can improve the noise robustness under certain noise conditions, it is not always the optimal. Hence, a modified integration weight β as given by equation (4) is employed to obtain better performance under low SNR conditions [6]. International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 34 + = ' ' ' V F V opt d d d β k (4) Here opt k is the scaling factor that needs to be optimized. We can automate the estimation of the “best available weight factor”, by formulating the problem using stochastic optimization techniques. The proposed PSO based optimization method systematically chooses the best ‘scale factor’ opt k from a defined domain 0 ≤ kopt ≤ 1to maximize the objective function (Recognition Accuracy). The performances of the proposed method were compared with that of the baseline systems [1]. Robust feature vectors were extracted from the two biometric traits. We considered minutiae based system that used elastic matching algorithm for fingerprint matching [11]. MFCC features were extracted from the voice samples and Gaussian mixture model (GMM) was considered for representing the acoustic feature vectors [12]. Model with 16 MFCC feature vectors and 12 Gaussian mixtures were used. Score normalization was done to transform the match scores of the two matchers into a comparable (common) domain. score score unnormalised score score normalised score max_ min_ _ min_ _ − − = (5) The optimal integration weights were obtained in the training/validation stage. The normalized match scores from the two modalities were combined by the equation (6) to produce the final decision. ( ) ' ' ' ( ) ' ' ' ( ) 1 fc V F V opt sc V F V opt fus S d d d S k d d d S k + + − + = (6) The objective function is given by: ( ) Re _ = − ×100 ∑∑ ∑ Mat Mat C diag C cognition Accuracy (7) CMat is the confusion matrix. LOOCV strategy was employed for the estimation of the optimal integration weights. 3.1. Particle Swarm Optimization (PSO) Algorithm The PSO algorithm is a stochastic optimization strategy, inspired by the social behaviour of the flock of swarms. Here the underlying concept is that, for every time instant, the velocity of each particle (potential solution), changes between its pbest and lbest locations [13]. The particle associated with the best solution (fitness value) seems to be the leader and each particle keeps track of its coordinates in the solution space. This fitness value is stored which is referred to as pbest. Another “best” value that is tracked by the particle swarm optimizer is the best value, obtained so far by any particle in the neighbours of the particle. This location is called lbest. When a particle takes all the population as its topological neighbours, the best value is a global best and is called gbest. The algorithm is as follows: 1. Randomly generate initial candidate solutions. 2. Assign the position and velocity of the associated particles randomly. 3. Evaluate the fitness (objective function) of each particle. International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 35 4. Compare each particles objective function value with this particles personal best value. If better, update pbest and take record current position as the particles personal best position. 5. Find the lowest objective function value of the whole particles. If the value is better than gbest, replace gbest with this objective function value, and take record the global best position. 6. Change velocities and positions. Velocity and the position update is given by the following equations, X = c1* rand1*(pbest[old] − present[old]) v[new] = w* v[old] + X + c2 * rand 2* (gbest[old] − present[old]) (8) present[new] = present[old] + v[new] (9) where ‘w’ is the inertia weight, v[] is the particle velocity, present[] is the current particle (solution). rand1 and rand2 are the random number between [0,1]. c1, c2 are learning factors. Usually c1 = c2 = 2. 7. Repeat 3 to 7 until stop criteria are satisfied. Figure 2: Performance of the system with Equal Weighting (Baseline) International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 36 Table 2: Fusion with Manually determined Integration Weight No SNR Accuracy of Classifiers wf ws Combined in dB Fingerprints Voice Accuracy 1 20 95.6522 98.6956 0.30 0.70 100.0000 2 15 95.6522 91.4493 0.35 0.65 97.8261 3 10 95.6522 69.4203 0.35 0.65 97.2464 4 5 95.6522 33.1884 0.65 0.35 95.6522 5 0 95.6522 23.1884 1.00 0.00 95.6522 6 -5 95.6522 9.1304 1.00 0.00 95.6522 7 -10 95.6522 5.6521 1.00 0.00 95.6522 Table 3: Optimal Integration Weights Estimated using LOOCV No SNR Baseline Proposed Method in dB GS β GA β ' d F ' V d α PSO opt k PSO β 1 20 0.7071 0.8087 0.8823 2.1176 0.7059 0.9795 0.6914 2 15 0.6928 0.7387 0.8823 1.8633 0.6786 0.9699 0.6582 3 10 0.6500 0.6605 0.8823 1.5593 0.6386 0.8449 0.5396 4 5 0.1255 0.1255 0.8823 1.2692 0.5899 0.7790 0.4595 5 0 0.0000 0.0000 0.8823 0.9403 0.5159 0.3556 0.1835 6 -5 0.0000 0.0000 0.8823 0.6222 0.4136 0.3452 0.1428 7 -10 0.0000 0.0000 0.8823 0.2503 0.2210 0.1751 0.0387 Table 4: Recognition Accuracy with the Baseline and the Proposed methods No SNR Accuracy of Classifiers Baseline Systems Proposed in dB Fingerprints Voice Equal weighting GS GA PSO 1 20 95.6522 98.6956 97.8261 100.0000 100.0000 100.0000 2 15 95.6522 91.4493 97.6812 97.8261 97.8261 100.0000 3 10 95.6522 69.4203 95.6522 97.2464 97.1015 97.8261 4 5 95.6522 33.1884 95.6522 95.6522 95.6522 97.7174 5 0 95.6522 23.1884 85.6522 95.6522 95.6522 93.4783 6 -5 95.6522 9.1304 68.6956 95.6522 95.6522 93.4783 7 -10 95.6522 5.6521 57.3913 95.6522 95.6522 93.4783 3.2. Leave-One-Out Cross Validation (LOOCV) As the number of available biometric samples was limited, LOOCV strategy was employed to fine-tune the training/validation phase and estimate the best optimal weights. LOOCV involves partitioning the data samples into complementary subsets and using a single observation from the data sample as validation data, and the remaining observations as training data. To reduce variability, multiple rounds of cross-validation were performed using different partitions, and the validation results were averaged over the rounds. Here, seven training samples from each modality were used for cross validation. So, seven cross validation sets were considered from International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 37 both modalities (Val.1 to Val.7). We choose each sample from the training set, for cross validation testing and the remaining six samples for cross validation training one at a time. The average training/validation accuracy of the individual classifiers is depicted in Table 1. The d-prime statistics and the scale factor opt k for various noise conditions were obtained with LOOCV technique. After performing the validation (testing), match score outputs from each validation set were stored separately. The d-prime values for each noise conditions were obtained from the genuine and impostor scores of the match score outputs of the two modalities Figure 3: GS based optimization (Baseline) Figure 4: GA based optimization (Baseline) International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 38 (validation sets) using equation (2). The d-prime values for each noise conditions were used to find the best scaling factor opt k , using equation (6) to maximize the objective function. A more detailed description of the LOOCV procedure we adopted is given in [1]. 4. RESULTS AND DISCUSSIONS Finger images from the FVC2002 fingerprint database [14] and voice samples from ELSDSR database [15] have been employed for the experimentation. ELSDSR database contains nine text independent speech samples of twenty-three persons. So, finger images of twenty-three different persons with nine impressions per finger were considered. Out of these nine samples per biometric, seven samples were used for training the individual classifiers and two samples Figure 5: Score density plot for GS based optimization (Baseline) Figure 6: Score density plot for GA based optimization (Baseline). were used for testing. As the fingerprint biometric is more robust, the performance of the system under varying noise conditions was not considered. The performance of the weak, voice biometric system under varying noise conditions were investigated by artificially degrading the test samples with additive white Gaussian noise. The performance of the system with varying SNR conditions were considered systematically from the feature extraction and model building International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 39 stage to the testing stage. MFCC feature vectors of the order 12, 16 and 20 and the GMM with 12 and 16 mixtures were considered for the simulation studies, as they are widely used. Different model combinations were considered to select the best model that gives better recognition accuracy. The voice model with 16 MFCC feature vectors with 12 Gaussian mixtures gave improved recognition accuracy under normal operating conditions (10 db SNR to 20 db SNR). So, this model combination was considered for the subsequent analysis. 4.1. Fusion with Baseline Systems The outputs of the different classifiers were consolidated into a single vector of scores using the sum rule of fusion. We explored the sum rule of fusion with equal and different weights to the two modalities. 4.1.1. Fusion with Equal Weightings In this case a constant value of β = 0.5 was assigned as an integration weight at all SNR levels. The score transformation could be achieved by the following equation. ∑= = n j j fus S n S 1 ( ) 1 ;n = 2 (10) [ ] ( ) ( ) ( ) 2 fus 1 sc fc S = S + S (11) where (sc) S - Speaker Scores; ( fc) S - Finger Scores. This technique will not favour one modality over another. More over the combined recognition accuracy may not be the maximum always. The testing accuracy of the bimodal system is shown in Table 4. Figure 2 depicts the DET performance curve. 4.1.2. Manually Determined Integration Weights This is a heuristic approach and often needs a lot of expertise. Weighted averages of the individual scores were considered here. The integration weights were randomly chosen on a trial and error basis for different SNR conditions and the weights that give higher accuracy were considered as the best choice. Sum rule of fusion with integration weights can be represented as follows. j n j j fus w s n S ∑= = 1 ( ) 1 ; n = 2 (12) where 1 1 ∑ = = n j wj . This process-required lot of time and effort and the results are not guaranteed to be optimal. The testing accuracy of the manually determined integration weight is shown in Table 2. The parameters wf and ws are the individual weights given to the fingerprint and voice modality respectively. The results show that for different noise conditions we need different weighting factors for both the modalities. 4.1.3. Fusion with Optimal Integration weights without Separability Information Blind optimization of the parameter β using equation (1) was performed in [1] to maximise the recognition accuracy. Grid Search (GS) and Genetic Algorithm (GA) based optimization were performed to obtain the best integration weights. The testing accuracy of this method is shown in the Table 4. The method does show improved performance than the sum rule of fusion method with equal weighting and it also helps to automate the integration weights estimation International Journal on Bioinformatics & Biosciences (IJBB) Vol.2, No.3, September 2012 40 process. Moreover, it showed improved recognition accuracy than any of the unimodal systems in the normal operating conditions and maintained the accuracy of the better unimodal ones for all adverse conditions. Further insight could be obtained from the DET plots [16] (Figure 2- Figure 4) and the score density plots (Figure 5- Figure 6). The disadvantage of the method is that at the extreme noise conditions the fusion module contributed zero weighting to the voice modality and there is no substantial reduction in FAR and FRR. 4.2. Performance of the Proposed Method From the validation stage, we obtained optimal integration weights (β α) opt = k for different noise conditions from -10dB to 20 dB.