Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 Subjective and Objective Quality Assessment of Image: A Survey Pedram Mohammadi1, Abbas Ebrahimi-Moghadam2, Shahram Shirani3 1, 2- Department of Electrical Engineering, Ferdowsi University of Mashhad, Mashhad, Iran Email: pedram.mohammadi@stu.um.ac.ir (Corresponding author) Email: a.ebrahimi@um.ac.ir 3- Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada Email: shirani@mcmaster.ca Received: August 2014 Revised: November 2014 Accepted: January 2015 ABSTRACT With the increasing demand for image-based applications, the efficient and reliable evaluation of image quality has increased in importance. Measuring the image quality is of fundamental importance for numerous image processing applications, where the goal of image quality assessment (IQA) methods is to automatically evaluate the quality of images in agreement with human quality judgments. Numerous IQA methods have been proposed over the past years to fulfill this goal. In this paper, a survey of the quality assessment methods for conventional image signals, as well as the newly emerged ones, which includes the high dynamic range (HDR) images, is presented. A thorough explanation of the subjective and objective IQA, and their classification is provided. Six widely used subjective quality datasets, and performance measures are overviewed. Emphasis is given to the full-reference image quality assessment (FRIQA) methods, and 9 often-used quality measures (including mean squared error (MSE), structural similarity index (SSIM), multi-scale structural similarity index (MS-SSIM), visual information fidelity (VIF), most apparent distortion (MAD), feature similarity measure (FSIM), feature similarity measure for color images (FSIMC), dynamic range independent measure (DRIM), and tone-mapped images quality index (TMQI)) are thoroughly described. Moreover, the performance and computation time of these metrics on four subjective quality datasets are evaluated. KEYWORDS: Image Quality Assessment (IQA), High Dynamic Range (HDR) Images, Full-Reference (FR), Reduced-Reference (RR), No-Reference (NR). 1. INTRODUCTION Digital images are rapidly finding their way into our daily lives due to the explosion of information in the form of visual signals. Digital images often pass through several processing stages before they reach to their end users. In most cases, this end user is a human observer. Through different processing stages, e.g., acquisition, compression, and transmission, images will be subjected to different types of distortions which degrade the quality of them. For example, in image compression, lossy compression schemes would introduce blurring and ringing effects in the final result, which leads to quality degradation. Moreover, in the transmission stage, due to limited bandwidth of the channels, some data might be dropped or skipped, which results in quality degradation of the received image. In order to maintain, control, and enhance the quality of images, it is essential for image communication, management, acquisition, and processing systems to assess the quality of images at each stage. IQA plays an important role in visual signal communication and processing. The application scope of IQA includes, but is not confined to, image acquisition [1], segmentation [2], printing and display systems [3,4], image fusion [5], and biomedical imaging [6,7]. IQA methods can be categorized into subjective and objective methods. Since human observers are the ultimate users in most of the multimedia applications, the most accurate and also reliable way of assessing the quality of images is through subjective evaluation. However, subjective evaluations are expensive and time consuming, which makes them impractical in real-world applications. Moreover, subjective experiments are further complicated by many factors including viewing distance, display device, lighting condition, subjects’ vision ability, and subjects’ mood. Therefore, it is necessary to design mathematical models that are able to predict the quality evaluation of an average human observer. The goal of objective IQA is to design mathematical models that are able to predict the quality of an image 55 Majlesi Journal of Electrical Engineering accurately and automatically. An ideal objective IQA method should be able to mimic the quality predictions of an average human observer. Based on the availability of a reference image which is considered to be distortion-free and have perfect quality, the objective quality assessment methods can be classified into three categories. The first category is full-reference image quality assessment (FR-IQA) where the undistorted, perfect quality reference image is fully available. The second category is reduced-reference image quality assessment (RR-IQA) where the reference image is not fully available. Instead, some features of the reference image are extracted and employed as side information in order to evaluate the quality of the test image. The third category is noreference image quality assessment (NR-IQA) where we don’t have access to the reference image. Since in many real-world applications the reference image is not accessible, NR-IQA methods are very practical in practice. This paper aims at an overview of the subjective and objective IQA. Classification of both subjective and objective IQA is presented. Six widely used subjective quality datasets and performance measures are reviewed. Emphasis is given to FR-IQA measures and 9 often-used quality measures (including MSE, SSIM [8], MS-SSIM [9], VIF [10], MAD [11], FSIM [12], FSIMC [12], DRIM [13], and TMQI [14]) are thoroughly described. Moreover, the computation time and performance of these methods are evaluated on four subjective datasets. This paper is organized as follows: In Section 2, subjective IQA is reviewed, international standards on designing subjective experiments are presented, and four common standardized subjective IQA measures are overviewed. In Section 3, objective IQA and its three main categories are reviewed. Moreover, a thorough description of six FR-IQA methods for gray-scale images (including MSE, SSIM [8], MS-SSIM [9], VIF [10], MAD [11], and FSIM [12]) is provided. In Section 4, a brief introduction to color images quality assessment is presented, and one FR-IQA method for color images (namely FSIMC [12]) is thoroughly described. In Section 5, a brief introduction to HDR images quality assessment is provided. Moreover, two FR-IQA methods for images with different dynamic ranges (including DRIM [13] and TMQI [14]) are comprehensively described. In Section 6, six widely used subjective quality datasets and performance measures are overviewed. In Section 7, performance and computation time of FR-IQA measures described in previous sections are evaluated on four subjective quality datasets (including LIVE dataset [15], CSIQ dataset [16], TID2008 dataset [17], and the dataset presented in [18]). In Section 8, a concise introduction to 3-D IQA is provided, some of 56 Vol. 9, No. 1, March 2015 the objective 3-D IQA methods are briefly reviewed, and some 3-D image datasets are presented. Finally, Section 9 concludes the paper. 2. SUBJECTIVE IMAGE QUALITY ASSESSMENT The most reliable method for assessing the quality of images is through subjective testing, since human observers are the ultimate users in most of the multimedia applications. In subjective testing a group of people are asked to give their opinion about the quality of each image. In order to perform a subjective image quality testing, several international standards are proposed [19-25] which provide reliable results. Here, we briefly describe some of these international standards: ITU-R BT.500-11 [19] proposes different methods for subjective quality assessment of television pictures. This is a widely used standard, which contains information about viewing condition, instructions on how to perform subjective experiments, test materials, and presentation of subjective results. ITU-T P.910 [21] proposes the standard method for digital video quality assessment with transmission rate below 1.5 Mbits/sec. ITU-R BT.814-1 [22] is proposed in order to set the brightness and contrast of the display devices. ITU-R BT.1129-2 [23] is proposed for assessing the quality of the standard definition (SD) video sequences. In the following subsections, we will briefly describe some of the standardized subjective IQA methods: 2.1. Single stimulus categorical rating In this method, test images are displayed on a screen for a fixed amount of time, after that, they will disappear from the screen and observers will be asked to rate the quality of them on an abstract scale containing one of the five categories: excellent, good, fair, poor, or bad. All of the test images are displayed randomly. In order to avoid quantization artifacts, some methods use continuous rather than categorical scales [19]. 2.2. Double stimulus categorical rating This method is similar to single stimulus method. However, in this method both the test and reference images are being displayed for a fixed amount of time. After that, images will disappear from the screen and observers will be asked to rate the quality of the test image according to the abstract scale described earlier. 2.3. Ordering by force-choice pair-wise comparison In this type of subjective assessment, two images of the same scene are being displayed for observers. Afterward, they are asked to choose the image with higher quality. Observers are always required to choose Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 one image even if both images possess no difference. There is no time limit for observers to make the decision. The drawback of this approach is that it requires more trials to compare each pair of conditions [26]. In [27,28], two methods for reducing the number of trials are proposed. 2.4. Pairwise similarity judgments As we mentioned before, in force-choice comparison, observers are required to choose one image even if they see no difference between the pair of images. However, in pair-wise similarity judgment observers are asked not only to choose the image with higher quality, but also to indicate the level of difference between them on a continuous scale. One might be tempted to use the raw rating results such as: excellent, good, fair, and etc. for quality scores. However, these rating results are unreliable. One reason for this is that observers are likely to assign different quality scales to each scene and even distortion types [29]. Here, we briefly introduce two scoring methods used in the subjective IQA. 2.5. Difference mean opinion score (DMOS) Instead of directly applying rating results, modern IQA metrics use differences in quality between images. DMOS is defined as the difference between the raw quality score of the reference and test images. DMOS is calculated using the following equation: d r j r i,j i ,ref   i , j (1) where r is the raw score for the i  th subject and i,j r j i ,ref   denotes the raw score given by the i  th subject to the reference the j  th image. Also, image corresponding to the j  th test image. 2.6. Z-score In order to easily compare each observer's opinion about the quality of images, a linear transform that makes the mean and variance equal for all observers is employed. The outcome of such transform is called Zscore and it can be computed using the following equation: d d i,j i (2) z  i,j i The mean DMOS, i d , and the standard deviation, i , are computed across all images that are rated by the i  th subject. Subjective quality assessment methods provide accurate and reliable measurements of the quality of visual signals. However, these methods suffer from different drawbacks that limits their applications:  They are time consuming and expensive. This is due to the fact that subjective results are obtained through experiments with many observers.  They cannot be incorporated into real-time applications such as image compression, and transmission systems.  Their results depend heavily on the subjects’ physical conditions and emotional state. Moreover, other factors such as display device and lighting condition affect the results of such experiments. Therefore, it is necessary to design mathematical models that are able to predict the perceptual quality of visual signals in a consistent manner with subjective evaluations. 3. OBJECTIVE IMAGE QUALITY ASSESSMENT The goal of objective IQA is to design mathematical models that are able to predict the quality of an image accurately and also automatically. An ideal objective IQA method should be able to mimic the quality predictions of an average human observer. Objective IQA methods have a wide variety of applications [30]:  They can be used to monitor image quality in quality control systems. For example, image acquisition systems can employ an objective IQA metric to monitor and automatically adjust themselves in order to obtain the best quality image data.  They can be used to benchmark image processing algorithms. For example, if a number of image enhancement algorithms are available, an objective IQA metric can be employed to choose the algorithm that provides the higher quality images.  They can be used to optimize image processing and transmission systems. For example, in a visual communication network, an objective IQA metric can be employed to optimize pre-filtering and bit assignment algorithms at the encoder and post-filtering and reconstruction algorithms at the decoder. Based on the availability of a distortion-free, perfect quality reference image, objective IQA methods can be classified into three categories. The first category is full-reference image quality assessment (FR-IQA) where the reference image is fully available. The second category is reduced-reference image quality assessment (RR-IQA) where only partial information about the reference image is available. And the third category is no-reference image quality assessment 57 Majlesi Journal of Electrical Engineering (NRIQA) where neither the reference image nor its features are available for quality evaluation. Objective IQA methods can also be categorized based on their application scope [30]. General purpose methods are the ones that do not assume a specific distortion type. Therefore, these methods are useful in a wide range of applications. On the other hand, application specific methods are the ones that are designed for specific distortion types. An example of these methods are the algorithms designed for image compression applications. Many quality metrics in image compression are designed for block-DCT or wavelet-based image compression. In the following subsections, the characteristics of the three main categories of the objective IQA are described. 3.1. No-reference image quality assessment (NRIQA) In many real-world applications, such as image communication systems, the reference image is not available and the quality evaluation is solely based on the test image. NR-IQA is a more difficult task in comparison to RR-IQA and FRIQA methods. However, human beings usually can efficiently assess the quality of a test image without using any reference image. This is probably due to the fact that our brain holds a lot of information about how an image should or should not look like in real world [30]. Some NR-IQA methods can be found in [31-36]. 3.2. Reduced-reference image quality assessment (RR-IQA) In RR-IQA, the reference image is not fully accessible. Instead, a number of features are extracted from the reference image. These features are employed by the quality assessment method as the side information for evaluating the quality of the test image. RR-IQA methods can be employed in a number of applications. For instance, they can be used to track the level of visual quality degradation of image and video data transmitted via real-time visual communication networks. Fig.1 shows the framework of an RR-IQA system. At the transmitter, a feature extraction process extracts certain features from the reference image and transmits them through an auxiliary channel. Feature extraction process is also applied to the test image at the receiver. The feature extraction process at the receiver can also be adopted to the side information at the receiver, which is shown as a dashed arrow in the figure, or it can be the same as in the transmitter. In order to obtain a single score for the overall quality of the test image, the features extracted from both, the reference and test images, are employed. An important parameter in the design of an RR-IQA system is the data rate used to 58 Vol. 9, No. 1, March 2015 encode the side information. If a high data rate is available, it is possible to include more information about the reference image, which allows more accurate quality predictions. If the data rate is high enough that all the information about the reference image can be transmitted, then the RR-IQA metric can be considered as a FR-IQA metric. On the other hand, if a low data rate is used, then only a small amount of information about the reference image can be transmitted. This results in less accurate quality predictions. In the case of zero data rate, the RR-IQA metric is considered as an NR-IQA metric. In real-world RR-IQA systems, the maximally allowed data rate is usually low [30]. Limited values for the data rate limits the feature selection process in RR-IQA systems. Therefore, selected features should satisfy following criteria: Transmitter Reference Image Receiver Distortion channel Feature extraction Auxiliary channel Test image Feature extraction RR-IQA metric Quality score RR features Fig. 1. The framework of a RR-IQA system    They should be able to provide an efficient summary of the reference image. They should be sensitive to a variety of distortion types. They should possess good perceptual relevance. 3.2.1. Methods based on the models of the image source The methods of this type are often statistical models that capture a priori of low level statistical features of natural images. These methods often have a low data rate. This is due to the fact that the parameters of these methods are able to summarize the image information in an efficient manner. Some of the methods in this category can be found in [37-40]. 3.2.2. Methods based on capturing image distortions The methods in this category are most useful when sufficient information about the image distortions is available. The application scope of these methods is limited, since they are unable to capture the distortions that they are not designed for. Some of the methods in this category can be found in [41-44]. 3.2.3. Methods based on the models of human visual system Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 In designing the methods in this category, physiological and/or psychophysical studies may be employed. These methods have shown good performance for JPEG and JPEG2000 compression schemes. Some of the methods in this category can be found in [45,46]. 3.3. Full-reference image quality assessment (FRIQA) In FR-IQA metrics, the perfect quality reference image is fully available for quality prediction process. The application scope of these metrics includes image compression [47], watermarking [48,49], and so on. In the following subsections, we will comprehensively describe six FR-IQA methods. The selected methods are widely cited in the literature, and have been reported to have good performance by researchers. Moreover, the authors of the selected metrics have released the source codes of their respective metrics. Therefore, results of the selected metrics are easy to reproduce. The six FR-IQA metrics described in the following subsections include mean squared error (MSE), structural similarity index (SSIM) [8], multi scale structural similarity index (MS-SSIM) [9], visual information fidelity (VIF) [10], most apparent distortion (MAD) [11], and feature similarity index (FSIM) [12]. It is important to note that all of these six quality evaluation metrics are designed for gray-scale images. In all of the following subsections, I ref and I tst denote the reference and test images respectively, and subscript ref denotes reference and tst test. Moreover, W and H represent the width and height of images respectively. 3.3.1. Mean squared error (MSE) MSE denotes the power of the distortion, i.e., the difference between the reference and test images. MSE value can be calculated using the following equation: 1 MSE  WH H W   I  i , j   I  i , j   j 1 i 1 ref 2 (3) tst MSE is often converted to peak-signal-to-noise ratio (PSNR). PSNR is the ratio of maximum possible power of a signal and power of distortion, and it is calculated by:  D2  (4) PSNR  10log    MSE  where D denotes the dynamic range of pixel intensities, e.g., for an 8 bits/pixel image we have D  255 . MSE possesses some characteristics that make it a widely used performance measure in the field of signal processing. Following are some of these characteristics [50]:  It is a parameter free and computationally inexpensive method.  It has a physically clear meaning, i.e., it is a natural way of defining the energy of an error signal.  Since MSE satisfies properties like convexity, symmetry, and differentiability, it is considered as an excellent measure in optimization applications.  It is considered as a convention, i.e., it is extensively used for optimization and assessment in a wide range of signal processing applications. Despite the above interesting features of MSE, when it comes to predicting human perception of image quality, MSE shows poor performance. This is due to the fact that some of the important physiological and psychophysical characteristics of the human visual system (HVS) are not accounted for by this measure. An instructive example is shown in Fig. 2, where the reference image (a) is altered by two types of distortions: white Gaussian noise (b), and quantization of the LH subbands of a 5-level discrete wavelet transform (DWT) of the image with equal distortion contrast at each scale (c). (a) (b) MSE = 181.770 (c) MSE = 180.922 Fig. 2. Harbor image altered with two types of distortions: (a) reference image; (b) white Gaussian noise; (c) quantization of the LH subbands of a 5-level DWT of the image with equal distortion contrast at each scale. All images are extracted from [51] It is important to note that images (b) and (c) have nearly similar MSE values. However, they have different visual qualities. There exist some implicit assumptions when using the MSE measure which makes it a poor measure of image quality. These assumptions are listed as follows [50]:  If the reference and test images are randomly re-ordered in a similar manner, the MSE between them will remain unchanged. This demonstrates that MSE is independent of temporal or spatial relationship between samples of the reference image.  For a specific distortion signal, MSE remains unchanged despite of which reference signal it is added to.  MSE is independent of the error signal sample’s sign. 59 Majlesi Journal of Electrical Engineering  Image signals are considered important when MSE is computed. Vol. 9, No. 1, March 2015 equally c  I ref , I tst  , 3.3.2. Structural similarity index (SSIM) The SSIM algorithm [8] assumes that HVS is highly adapted for extracting structural information from a scene. Therefore, this algorithm attempts to model the structural information of an image. The SSIM algorithm is based on the fact that pixels of a natural image demonstrate strong dependencies and these dependencies carry useful information about the structure of a scene. Therefore, a method that is capable of measuring structural information change can provide a good approximation of perceived image distortion. The SSIM algorithm defines image degradation as perceived change in structural information. In [8], it is stated that the structure of the objects in a scene is independent of local luminance and contrast. Therefore, to extract the structural information, we should separate the effect of illumination. In this algorithm, structural information in an image is defined as those traits that represent the structure of objects in that image, independent of the local luminance and contrast. The SSIM algorithm performs similarity measurement in three steps: luminance comparison, contrast comparison, and structure comparison: First, the luminance of each image signal is compared. The estimated mean intensity is computed as follows: ref 1  WH H W  I  i , j  j 1 i 1 (5) ref The luminance comparison function, function of  ref and tst . l (I ref , I tst ) , is a  H W 1  I ref  i , j   ref  W H  1 j 1 i 1  2    1 2 (6) The contrast comparison function, c  I ref , I tst  , is a function of  ref and  tst . Third, the structure of each image signal is compared. Structure comparison function, s (I ref , I tst ) , is a function of I ref   ref  ref and S  I ref , I tst  l (I ref , I tst ) , s (I ref , I tst ) . The function satisfies following conditions: S (I ref , I tst )  S (I tst , I ref ) .  Symmetry:  Boundedness:  Unique maximum: only if 1  S (I ref , I tst )  1 . S (I ref , I tst )  1 , if and I ref  I tst . Definitions of l  I ref , I tst  , c  I ref , I tst  , For luminance comparison function we have: l (I ref , I tst )  Where 2ref tst T1 ref2  tst2 T1 (7) T 1 is a positive stabilizing constant chosen to prevent the denominator from becoming too small. We have: T1  t 1D  2 (8) Where D is the dynamic range of pixel values and t 1  1 is a small constant. For contrast comparison function we have: c (I ref , I tst )  Where And (9) T 2  t 2 D  is a positive stabilizing constant. 2 t 2  1 . For structure comparison function we have: s (I ref , I tst )  Where  ref 2 ref  tst T 2 2   tst2 T 2  ref  ref , tst T 3  ref  tst T 3 (10) T 3 is a positive stabilizing constant. In (10), , tst is the correlation coefficient between the reference and test images. In the discrete form,  ref , tst can be estimated via following equation:  ref , tst  H W 1  I ref i , j   ref  WH  1 j 1 i 1   I i , j     tst (11) tst I tst  tst . Finally, three comparison functions are  tst Finally, structural similarity index is defined as: combined and an overall similarity measure is produced. The overall similarity measure, SSIM (I ref , I tst )  l (I ref , I tst ) c (I ref , I tst ) s (I ref , I tst ) and 60 and s  I ref , I tst  , are as follows: Second, the contrast of each image signal is compared. For estimating the contrast, standard deviation is being used. An unbiased estimate of standard deviation in discrete form is as follows:  ref   S (I ref , I tst ) , is a function of    (12) Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 Where  ,  , and  are positive constants chosen to indicate the relative importance of each component. The universal quality index (UQI) [52,53] is a special case of the SSIM index when: T1  T 2  T 3  0 and       1 . In [8], it is mentioned that it is useful to apply SSIM index locally instead of globally. In order to achieve this, authors used an 11 11 circular symmetric Gaussian weighting function w  w i , j i  1, 2, ..., W and j  1, 2, ..., H  1.5 samples, normalized to with standard deviation of H unit sum W w j 1 i 1 i,j  1 . Using this function, the estimates of local statistics  ref ,  ref , and  ref , tst are calculated as follows: H (13) j 1 i 1  ref H W   w i , j  I ref  i , j   ref  j 1 i 1 H W  w i , j  I ref  i , j   ref , tst j 1 i 1 1  2 2   (14)   I i , j     tst (15) tst In order to have a single overall quality measure for the entire image, authors of [8] use a mean SSIM (MSSIM) index to evaluate the overall quality: MSSIM  I ref , I tst   where I i ref 1 Mw Mw  SSIM (I i 1 i ref , I tsti ) (16) M w is the total number of local windows, and i and I tst are image contents at the i  th local window. Reference image 3.3.2.1. Parameter specification in the SSIM algorithm There are several parameters in the SSIM algorithm that need to be specified. First, for computing (8) the values of t1 and D are set to be 0.01 and 255 respectively. Second, for computing (9) the value of t 2 is set to be 0.03. Third, in (10) we have: T3  T2 2 . It is stated in [8] that the performance of the SSIM algorithm is fairly insensitive to the values of T1 , T2 , T3 . Finally, in order to simplify (12), SSIM algorithm sets       1 . and Authors of [8] have provided a MATLAB implementation of the SSIM algorithm that is available at [57]. W ref  w i , j I ref  i , j   ref sensing [55], and visual surveillance [56]. Luminance measurement computed only at Contrast measurement + + 3.3.3. Multi-scale structural similarity index (MSSSIM) The SSIM algorithm described earlier is considered a single-scale approach that achieves its best performance when applied at an appropriate scale. Moreover, choosing the right scale depends on the viewing conditions, e.g., viewing distance and the resolution of the display. Therefore, this algorithm lacks the ability to adapt to these conditions. This drawback of the SSIM algorithm motivated researchers to design a multi-scale structural similarity index (MSSSIM) [9]. The advantage of the multi-scale methods, like MS-SSIM, over single-scale methods, like SSIM, is that in multi-scale methods image details at different resolutions and viewing conditions are incorporated into the quality assessment algorithm. The block diagram of the MS-SSIM algorithm is presented in Fig. 4. After taking the reference and test images as input, this algorithm performs low-pass filtering and downsampling (by factor of 2) in an iterative manner. At each scale, (9) and (10) are calculated. At each scale, (9) and (10) are calculated. However, (7) is Luminance comparison M s  th scale. The final MS-SSIM index is calculated via following equation: ÷ Contrast comparison Test image Luminance measurement + SSIM measure Structure comparison + Combination Reference image Contrast measurement L c1(Iref , Itst) s1(Iref , Itst) ÷ Fig. 3. The block diagram of the SSIM algorithm The block diagram of the SSIM algorithm is presented in Fig. 3. Some applications of the SSIM algorithm are image fusion [5], image watermarking [54], remote Test image L ↓2 L ↓2 L c2(Iref , Itst) s2(Iref , Itst) ↓2 L ↓2 cM (Iref , Itst) s sM (Iref , Itst) s ↓2 L lMs(Iref , Itst) MS-SSIM measure ↓2 Fig. 4. The block diagram of the MS-SSIM algorithm. L: low-pass filter; ↓2: downsampling by factor of 2 61 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 MS  SSIM (I ref , I tst )  l M s  I ref , I tst  where M s (17) Ms i . c i  I ref , I tst  s i  I ref , I tst  i i 1 ci  I ref , I tst  and s i  I ref , I tst  are the contrast and the structure comparison function at the i  th scale respectively, and l M s  I ref , I tst  is the luminance comparison function at the Moreover, M s  th scale.  M , i , and  i are positive constants s chosen to indicate the relative importance of each component. In [9],  i   i   i Ms  i 1 i for all j , and of 3.3.4.1. Source model As stated earlier, VIF algorithm models natural images in wavelet domain using GSM model. A GSM is defined as a random field that can be determined as a product of two independent random fields [60]. In other words, a GSM like c can be expressed as: c  zu  1. (18) Where z is a random field containing positive scalars, and u is a Gaussian vector random field with zero 3.3.3.1. Parameter specification in the MS-SSIM algorithm An image synthesis-based approach is used in order to calculate the exponents of (17). In [9], for a given original 8 bits/pixel gray-scale test image, a matrix of test images is constructed. Each element in the matrix is an image that is related to a specific MSE value and a specific scale. Each test image in the matrix is created by randomly adding white noise to the original test image. 5 scales and 12 distortion levels are used that yields a matrix of total of 60 images. Moreover, 10 original test images of size 64  64 with different contents are used in order to create 10 sets of test images, resulting in the total number of test images to be 600. As mentioned in [9], 8 subjects (including one of the authors) have participated in the subjective experiment for calculating the exponents of (17). Subjects had general understanding of the human vision, but were unaware of the goal of the experiment. After seeing all 10 sets of test images in a fixed viewing distance, they were asked to choose one image in each of 5 scales that they think have the same quality. After that, the positions of chosen images in each scale is saved and averaged across all test images and subjects. Test results are then normalized so that their sum be equal to 1. The resulting exponents for each wavelet domain using Gaussian scale mixtures (GSMs). Images and videos that are taken from natural environment by using high quality capturing devices operating in visual spectrum are classified as natural scenes. For a review of natural scene models see [59].VIF algorithm consists of three components: source model, distortion model, and HVS model. 5 scales  2   2  0.2856 , are: 1   1  0.0448 , 3   3  0.3001 ,  4   4  0.2363 , 5   5   5  0.1333 . Authors of [9] have provided a MATLAB implementation of the MS-SSIM algorithm that is available at [58]. mean and covariance Cu . In [10], it is assumed that u constitutes of independent components. VIF algorithm models each subband of image’s wavelet decomposition as a GSM random field. Each subband coefficients are grouped into non-overlapping blocks of size Mn. 3.3.4.2. Distortion model Distortion is modeled in the wavelet domain as signal attenuation and additive noise. This model is defined as follows: d  gc  v (19) where c is a random field from a subband in the reference image, d is a random field from a subband in the test image, g is a deterministic scalar field, and v is a random field from a stationary white additive Gaussian noise with zero mean and covariance Cv   v2 I . In [10], random fields v , z , and u are assumed to be independent from one another. Moreover, random field g is considered to be slow varying. 3.3.4.3. HVS model HVS is modeled as a distortion channel that adds noise to the input signal, limiting the amount of information that flows through the channel. This visual noise is characterized as a zero mean stationary additive white Gaussian noise modeled in the wavelet domain. HVS noise is modeled as stationary random fields n  and n which are zero mean, uncorrelated multivariate Gaussians with the same covariance Cn  Cn    n I , 2 3.3.4. Visual information fidelity (VIF) VIF algorithm [10] models natural images in the 62 where  n2 is considered the variance of visual noise). Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 calculated as follows: The outputs of the HVS channels are as follows: e c n f  d  n (20) (21) where e is the output of the HVS channel when the input is the reference image and f is the output of the same channel when the input is the test image. Random fields n and n  are assumed to be independent of u , z , and v . With the source, distortion, and HVS models described earlier, the VIF quality measure can be calculated.    to be a collection of C  c1 , c 2 , ..., c M r Consider  z  z 1 , z 2 , ..., z M r Mr 3.3.4.4. Calculating reference image’s information The amount of information that can be extracted from a particular subband in the reference image, I C ; E z  , is calculated as follows: Mr i 1  z i2Cu   n2 I 1 =  log 2   2 i 1  n2 I  Mr where h . and . (22)     denote the differential entropy of a continuous random vector and determinant operator respectively. Since Cu is symmetric, by using matrix factorization we can write it as Cu  Q Q , where T Q is an orthonormal matrix, and  is a diagonal matrix containing eigenvalues k . Using this factorization, (22) can be written as: I C ; E z    z i2k  1 M r Me log  2 1   2  2 i 1 k 1 n   (23) Where M e is the total number of eigenvalues in  . 3.3.4.5. Calculating test image’s information The amount of information that can be extracted from a particular subband in the test image, I C ; F z  , is (24) i 1    g i2 z i2Cu   v2,i   n2 I  1 . =  log 2    2 i 1  v2,i   n2 I   Mr   Using the same factorization as before, (24) can be written as: I C ; F z   and realization from the random field c and z respectively. Moreover, let D, E, and F be defined in a similar manner in terms of d, e, and f. In [10], all GSM vectors are constructed from a non-overlapping 3  3 neighborhood. In order to calculate the VIF measure, information content of the reference and the test images needs to be calculated. I C ; E z  =   h c i  n i z i   h  n i z i   Mr I C ; F z     h  g i c i  v i  n i z i   h v i + n i z i    g 2 z 2  1 M r Me log 2 1  2i i k 2       2 i 1 k 1 v ,i n   (25) It has been discovered in [10] that the ratio of equations (23) and (25) relates well with visual quality. Therefore, by using the assumption that each subband is completely independent of others in terms of their respective random fields as well as the distortion model parameters, the VIF quality measure is calculated as follows:  I C j ;F j z j (26) j subbands VIF  j j j  I C ;E z   j subbands j Where     j j is the subband index, and I C ; F z j j and I C ; E z j  j  are the corresponding mutual information of the j  th subband. In [10], summation is performed over subbands at the finest scale. The VIF measure can be calculated by using an entire subband of image or by using a spatially localized region of subband coefficients. In the first case, VIF measure is a single number that quantifies the overall quality of the image, and in the second case, a sliding window could be used to obtain a quality map of the image. The block diagram of the VIF algorithm is presented in Fig. 5. For all practical distortion types, the VIF measure takes its values in the interval [0,1] . V IF  0 means that all the information about the reference image has been lost due to presence of distortions. For images with higher perceptual quality, the value of the VIF measure is close to 1. A linear contrast enhancement of the reference image, that doesn’t add distortion to it, results in the VIF measure greater than 1. Therefore, VIF  1 means that the test image has a superior visual quality than the reference image. 3.3.4.6. Parameter specification in the VIF algorithm In order to compute (26), values of Cu , z i , g i ,  v , n must be estimated. The estimation of Cu is done using the wavelet coefficients of the reference image in 63 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 each subband: ˆ  1 C u Mr Mr c c i (27) T i i 1 in appearance-based strategy, variations in local statistics of spatial frequency components are being employed. Here, we summarize each strategy in more details. 2 By using maximum-likelihood estimation, z i can be estimated using the following equation [61]: zˆi2  1 T ˆ 1 c i Cu c i Mm Where Mm parameters (28) is the dimensionality of c. The g i and  v ,i can be computed using simple regression, since both the reference and test image coefficients are available. Reference image Natural image source Test image Distortion channel HVS channel 3.3.5.1. Detection-based strategy It is argued in [11] that when HVS views high quality images, it tries to look beyond image’s subject matter, looking for distortions. Detection-based strategy consists of two stages: determining the locations of visible distortions, and computing perceived distortion due to visual detection. First, the locations of visible distortions should be determined. In order to describe the non-linear relationship between pixel values and physical luminance of display device, MAD algorithm primarily transforms pixels of the reference and test images to luminance values using the following equation: L     I   HVS channel (29) Where L is the luminance image, I is the reference (or test) image, and  ,  , and  are device specific constants. Applying (29) to I ref and Mutual information ÷ I tst yields L ref and L tst respectively. Since HVS has a non-linear VIF measure response to luminance, it should be converted to perceived luminance via: 1 Information content Fig. 5. The block diagram of the VIF algorithm Lˆ  L 3 where to Finally, n is estimated by running the VIF algorithm for different values of this parameter and then choosing the value that yields the best performance in terms of overall image quality prediction accuracy. Authors of [10] have provided a MATLAB implementation of the VIF algorithm that is available at [62]. 3.3.5. Most apparent distortion (MAD) MAD algorithm [11] assumes that HVS employs different strategies when judging the quality of images. It is mentioned in [1 1] that when HVS attempts to view images containing near-threshold distortions, it tries to move past the image, looking for distortions. This approach is called detection-based strategy. Moreover, it is also stated in [11] that when HVS attempts to view images containing clearly visible distortions, it tries to move past the distortions, looking for image’s subject matter. This approach is called appearance-based strategy. For estimating distortions in detection-based strategy, local luminance and contrast masking are used. Moreover, for estimating distortions 64 (30) L̂ L ref denotes perceived luminance. Applying (30) and Ltst results in Lˆref and Lˆtst respectively. After computing perceived luminance, an error image is computed: Lˆerr  Lˆref  Lˆtst (31) To describe variations in sensitivity due to spatial frequency, authors of [11] employ contrast sensitivity function (CSF) as introduced in [63] with adjustments as in [64]. CSF is applied to both, the reference and  and I err  respectively. error images which yields I ref Since presence of an image’s content can affect the detection of distortions, a spatial domain measure of contrast masking is employed. To model this, first  is divided into blocks of size 16  16 with 75 I ref percent overlap between neighboring blocks. Afterward, rms contrast (in the lightness domain) of each block is calculated. The rms contrast for block b of  is calculated via: I ref C ref b    ref b  ref b  (32) Majlesi Journal of Electrical Engineering where ref b  b in the reference d detect takes its values in the interval 0,  . If is the minimum of the standard d detect  0 , there are no visible distortions in the test is the mean of block  ref b  image, and b . The same deviation of the four subblocks in procedure is done for Vol. 9, No. 1, March 2015  with the exception that the I err rms contrast for this image is calculated via following equation:   err b  , ref  0.5  (33) C err b    ref b  0 , otherwise  Where  err b  is the standard deviation of block b in  . In (33), the threshold of 0.5 denotes the fact that I err HVS is relatively insensitive to changes in extremely dark regions. After computing C ref b  , a local distortion visibility map, and C err b   b  , is computed as follows: ln C err b    ln C ref b   , ln C err b    ln C ref b       , ln C err b      ln C ref b    b   ln C err b      0 , otherwise   Reference Luminance image image generation Perceived luminance image generation CSF Calculating rms contrast Local distortion visibility map generation + Test image Luminance image generation Perceived luminance + image generation - CSF (34) Combination ddetect image. As the value of d detect increases, perceived distortion increases and consequently, visual quality decreases. The block diagram of detection-based strategy is presented in Fig. 6. 3.3.5.2. Appearance-based strategy It is argued in [11] that when viewing low quality images, HVS tries to move past the distortions, looking for image’s content. To model this strategy, MAD algorithm uses log-Gabor filter responses. Similar to detection-based strategy, this strategy is also consists of two stages; log-Gabor decomposition of the reference and test images, and computing the local statistical difference map. First, the reference and test images are decomposed into number of subbands via a 2-D log-Gabor filter bank with frequency responses of the form:    log   s G s ,o  ,    exp    2 r   2  2     exp     0    2 02      where indices s and o correspond to spatial scale and orientation respectively, parameters  and  are normalized radial frequency and orientation respectively,  s is normalized center frequency,  r controls the filter’s bandwidth, and Calculating rms contrast (37) 0 and 0 center orientation and angular spread of the filter respectively. In [11], five scales ( s  1,2,...,5 ) and Local MSE calculation Fig. 6. The block diagram of the detection-based strategy in the MAD algorithm four orientations ( o  1, 2,..., 4 ) are used for logGabor decomposition, which result in twenty subbands per image.  b  , Where  is a threshold value (   5 , as in [11]). Second, the perceived distortion due to visual detection Second, a local statistical difference map, ( d detect ) is calculated. d detect is calculated via: the test image. For each block of size 16  16 , d detect 1  B  b  b    b   2 (35) 1 2  b     I err (i , j )  16 16 i , j M p p generated. This map is defined by comparing local subband statistics of the reference image with those of  b   (b ) 5 the local MSE of block b of size 16  16 , that can be calculated via following equation: is set of pixels in block is is calculated via following equation: 1 2 Where B is the total number of blocks, and  b  is where M are (36) 4   s 1 o 1 s   sref,o b    stst,o b   2  sref,o b    stst,o b    sref,o b    stst,o (b )    (38) Where  s ,o standard deviation, skewness, and kurtosis of 16  16 subband coefficients associated with scale s , orientation b. b  ,  s ,o b  , and s ,o b  correspond to o, and block b . In (38), s is a scale specific weight which takes into account the preference 65 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 of HVS for coarser scales over fine ones. (in [11], s  0.5, 0.75, 1, 5, and 6 for finest to coarsest  b  , scales, respectively). After computing a final scalar value of perceived distortion, d appear , is calculated as follows: 1  B d appear 1 2  b    b  (39) 2 d appear takes its values in the interval 0,  . If d appear  0 , there is no perceived distortion in the test 3.3.5.3. Parameter specification in the MAD algorithm There are several parameters in the MAD algorithm that need to be specified. First, for computing (29) the values of  ,  , and  are set to be 0, 0.02874, and 2.2 respectively. These parameters are calculated using 8 bit pixel values and a sRGB display. Second, the logGabor filter parameters are assigned as follows: s  0.6666, 1.3333, 2, 2.6666, 3.3333 for finer to coarser 0   6 scales rad ,  r  0.0413 , respectively, and   0  0,  , 4  , 2 1 3 4  rad . 2 image. As the value of d appear increases, perceived Finally, for computing (41) the values of distortion increases and consequently, visual quality decreases. The block diagram of appearance-based strategy is presented in Fig. 7. After computing d detect and d appear , these two values are set to be 0.467 and 0.130 respectively. It is important to note that the parameters  1 and  2 are are combined to yield an overall measure of perceived distortion. In [11], it is hypothesized that HVS uses a combination of detection-based strategy and appearance-based strategy for assessing the quality of images. To model the relation between these two strategies, a weighted geometric mean of d detect and d appear is employed that has the form: MAD  d detect  d appear  1  (40) where  is a weighting constant chosen to indicate the relative importance of each term. MAD measure takes its values in the interval 0,  . In [11], it is argued that Reference image log-Gabor decomposition Statistical difference map generation Test image (.)2 Combination dappear log-Gabor decomposition Fig. 7. The block diagram of the appearance-based strategy in the MAD algorithm selecting the value for  based on d detect can yields good overall performance. Therefore, via:  1 1  1 d detect  Where 66  2 1 and  2 are free parameters. is calculated (41) and calculated for the Cornell-A57 dataset [51]. Authors of [11] have provided a MATLAB implementation of the MAD algorithm that is available at [65]. 3.3.6. Feature similarity index (FSIM) The FSIM algorithm [12] is based on the fact that HVS understands an image mainly due to its low-level characteristics, e.g., edges and zero crossings [66-68]. In order to assess the quality of an image, FSIM algorithm uses two kinds of features. Physiological and psychophysical experiments have demonstrated that at points with high phase congruency (PC), HVS can extract highly informative features [68-72]. Therefore, PC is used as the primary feature in the FSIM algorithm. However, PC is contrast invariant and our perception of an image’s quality is also affected by local contrast of that image. As a result of this dependency, the image gradient magnitude (GM) is used as the secondary feature in the FSIM algorithm. Calculating FSIM measure consists of two stages: computing image’s PC and GM, and computing the similarity measure between the reference and test images. 3.3.6.1. PC and GM computation The PC model states that Fourier components with maximum phase contain the points where features are perceived by HVS. This model provides a simple structure on how mammalian visual system handles detection and identification of features in an image [6872]. First, by applying (37) to the reference and test images, a set of set of response vectors are created at location x , scale s , and orientation o . Second, the local amplitude of these vectors at scale s and orientation o is calculated. Moreover, the local energy at orientation o is also computed. Finally, the PC Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 value at location x is calculated via following equation:  E x    A  x  o PC  x   Reference image (42) o PC computation GM computation Combination SL Combination s ,o s Where Eo  x is the local energy at orientation orientation o , and  s o, is a positive stabilizing constant. interval  0,1 . In order to compute the gradient magnitude of the reference and test images, three different gradient operators are employed. These operators are: Sobel operator [73], Prewitt operator [73], and Scharr operator [74]. 3.3.6.2. Similarity measure computation Consider PC ref and PC tst are PC maps computed for I ref and I tst respectively, and G ref and G tst are GM maps for these images. The final similarity measure between the reference and test images consists of two components: similarity measure between PC ref and PC tst or S PC  x  , and similarity measure between G ref and G tst or S G  x  . S  x  is calculated via PC following equation: 2PC ref  x  PC tst  x  T 4 PC ref2  x   PC tst2  x  T 4 Test image and PC  x  is a real number that takes its values in the T 4 is a positive stabilizing constant chosen to prevent the denominator from becoming too small. S PC  x  takes its values in the interval  0,1 . S G  x  SG  x  Where G ref2  x   Gtst2  x  T 5 (44) T 5 is a positive stabilizing constant. S G  x  takes its values in the interval  0,1 . SG PCm GM computation T 4 , and T 5 depend on the dynamic range of PC and GM values respectively. The final similarity measure between I ref and I tst , S L  x  , The values of is computed via following equation:  S L  x   S PC  x   S G  x   (45) Where  and  are two constants chosen to indicate the relative importance of each component (in [12],     1 ). Since our perception of an image is affected differently by different location in an image, and also PC value at a location reflects whether or not that location is perceptibly significant [72], therefore, if anyone of PC ref  x  and PC tst  x  be greater than the other, it implies that position x has a high impact on HVS when evaluating As a result, PC m  x   max  PC ref between S L  x S L  x FSIM between algorithm  x  , PC  x   tst I ref and uses as a weighting in the overall similarity measure I ref and I tst . Finally, the FSIM index between the reference and test images is defined as follows: is calculated via following equation: 2G ref  x Gtst  x  T 5 Combination FSIM measure Fig. 8. The block diagram of the FSIM algorithm I tst . (43) PC computation function for Where Combination o A s ,o  x  is the local amplitude at scale S PC  x   SPC FSIM   S  x  .PC  x   PC  x  x m L x (46) m Where  is the whole image spatial domain. The block diagram of the FSIM index is presented in Fig. 8. 3.3.6.3. Parameter specification in the FSIM algorithm In order to specify parameters in the FSIM algorithm, authors of [12] used a subset of the Tampere image dataset 2008 (TID2008) which contained the first 8 reference images and their corresponding 544 test images. Parameters that achieve the highest spearman’s rank order correlation coefficient (SRCC) are chosen, and are fixed for all conducted experiments. In [12], 67 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 four scales ( s  1, 2,..., 4 ) and four orientations ( o  1, 2,..., 4 ) are used for log-Gabor decomposition. The parameters’ value in FSIM index are:  r  0.5978  0  0.6545 rad , T  0.85 , and T 5  160 . 4 1 1 1 1 , , for finer to coarser Moreover, s  , 6 12 24 48 ,   scales respectively and  0  0,   4 ,  2 , 3 4  rad . It is mentioned in [12] that Scharr gradient operator [74] yields the highest SRCC among Sobel and Prewitt operators. Therefore, this operator is used to compute GM of the reference and test images. Authors of [12] have provided a MATLAB implementation of the FSIM algorithm that is available at [75]. 4. OBJECTIVE QUALITY ASSESSMENT OF COLOR IMAGES Objective FR-IQA methods described thus far are designed specifically for grayscale images and they don’t make use of images’ color information. Color information simplifies the identification and extraction of objects in a scene. Therefore, it affects human observers’ judgment when assessing the quality of an image. In many areas that deal with digital images, there is always a demand for objective quality metrics that can predict the quality of a test color image with respect to its reference version. Applications of such a metric can be found in computer graphics when comparing the level of photorealism of two different rendering methods, image coding when comparing the performance of two different compression schemes, image processing when evaluating the performance of color image enhancement methods, and false-color multispectral image fusion [76]. In general, objective IQA metrics for gray-scale images can, in principle, be extended to incorporate color images. This is accomplished by applying these metrics to each of three RGB color channels individually, and then combining the quality score for each channel together. However, this approach does not relate with human perception, and this is because RGB color space does not represent color as it is perceived by HVS [76]. The first color image quality measure is proposed in [77]. In this work, a simple model of human color vision is presented which quantitatively describes different perceptual parameters, e.g., brightness and saturation. The perceptual space is considered as a vector space with spatial filtering characteristics. Moreover, a norm on the vector space is introduced that enables measuring the distances and defines a distortion measure that correlates well with perceptual 68 evaluations. Some of the researches that address color image quality assessment can be found in [78-82]. Here, we only describe feature similarity index for color images (FSIMC): 4.1. Feature similarity index for color images (FSIMC) FSIM index described earlier is designed for gray-scale images or the luminance component of color images. In order to extend FSIM index to incorporate color images, first the reference RGB color image is transformed into another color space in which the luminance component can be separated from chrominance. In [12], RGB color image is transformed to YIQ color space, where Y denotes luminance component and I and Q denote chrominance components. RGB color space is transformed to YIQ color space via [83]:  Y  0.299 0.587 0.114   R   I   0.596 0.274 0.322 G       Q   0.211 0.523 0.312   B  (47) Iref and Q ref are chromatic components of the reference image, and Itst and Qtst are chromatic Suppose components of the test image. The similarity measures between chromatic components are computed as follows: S I  x  S Q  x  2Iref  x  Itst  x  T 6 (48) 2Qref  x  Qtst  x  T 7 (49) 2 I2ref  x   Itst  x  T 6 2 Qref  x   Qtst2  x  T 7 Where T 6 and T 7 are two positive stabilizing constant chosen to prevent the denominators from becoming too small. In [12], the values of T 6 and T 7 are set to be equal with each other. The final similarity measure between chromatic components, S C  x  , is the product of S I  x  and S Q  x  : SC  x  S I  x S Q  x (50) The FSIM index for color images is calculated via:  FSIM C  S  x  S  x  PC  x    PC  x  x L m C x m (51) Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 where  is a positive weighting constant chosen to indicate the relative importance of chromatic components. Note that for color images PC and GM are computed via their luminance component Y. Moreover, the calculation process of PC and GM for color images is the same as gray-scale images described in Sec. 3.3.6. The block diagram of the FSIMC algorithm is presented in Fig. 9. 4.1.1. Parameter specification in the FSIMC algorithm The values of s , o ,  s ,  r ,  0 , and  0 in the FSIMC algorithm are the same as their values in the FSIM algorithm. Reference image Test image Color space conversion Color space conversion Iref Combination Qref SI SL × Itst Combination Qtst SC Combination FSIMC measure SQ PCm Fig. 9. The block diagram of the FSIMC algorithm Moreover, in the FSIMC T 6  T 7  200 , and   0.03 . algorithm we have: Authors of [12] have provided a MATLAB implementation of the FSIMC algorithm which is available at [75]. 5. QUALITY ASSESSMENT OF HIGH DYNAMIC RANGE IMAGES There has been a growing interest in recent years in HDR images that have greater dynamic range of intensity values than low dynamic range (LDR) images. In order to visualize HDR images on standard display devices, tone mapping operators (TMOs) [84-87] are employed. Since TMOs reduce the dynamic range of HDR images, they result in information loss and quality degradation. Therefore, it is important to assess the quality of each tone-mapped image to see which TMO provides better quality LDR images. On the other hand, due to the advent of various display technologies, e.g., HDR display, digital cinema projections, and mobile devices’ displays, it is important to measure the quality of images with different dynamic ranges to evaluate the capability of each displaying device in producing higher quality images. Subjective evaluation is the most reliable method for assessing the quality of HDR and LDR images [88-93]. However, as we mentioned before, these methods are expensive, time consuming, and cannot be embedded into optimization algorithms. Therefore, it is important to develop objective IQA methods for evaluating the quality of HDR images and their corresponding tonemapped versions. The FR-IQA methods described thus far cannot be employed for this purpose. This is due to the fact that the described methods assume that the dynamic range of the reference and test images is similar. In the following subsections, we will describe two FRIQA methods for evaluating the quality of images with different dynamic ranges. These methods are: dynamic range independent quality measure (DRIM) [13], designed for evaluating the quality of images with arbitrary dynamic ranges, and tone-mapped images quality index (TMQI) [14], designed for evaluating the quality of tone-mapped images with respect to their reference HDR images. 5.1. Dynamic range independent quality measure (DRIM) In [13], an image quality metric capable of assessing the quality of images with arbitrary dynamic ranges is proposed. Reference Contrast detection image predictor Cortex transform Test Contrast detection image predictor Cortex transform Loss of visible contrast predictor Band-pass filter Amplification of invisible contrast predictor Band-pass filter Reversal of visible contrast predictor Band-pass filter Visualization of Distortion map structural changes Fig. 10. The block diagram of the DRIM algorithm The output of this metric is a distortion map that indicates the loss of visible features, the amplification of invisible features, and the reversal of contrast polarity. The DRIM algorithm is sensitive to three types of structural changes:  Loss of visible contrast: this case describes the situation when a contrast that was visible in the reference image becomes invisible in the test image. This usually happens when a TMO compresses the details in the HDR image to a level that they become invisible in the resulting LDR image.  Amplification of invisible contrast: this case describes the situation when a contrast that was invisible in the reference image becomes visible in the test image. This usually happens when an inverse TMO, i.e., an operator that converts LDR images to HDR images, introduces contouring artifacts in the resulting HDR image.  Reversal of visible contrast: this case happens when a contrast is visible in both the reference and test images, but with different polarity. This usually occurs in image locations possessing strong distortions. The block diagram of the DRIM algorithm is presented in Fig. 10. The inputs for this metric are luminance 69 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 maps corresponding to the reference and test images. First, the detection thresholds are predicted and a perceptually normalized response map is generated. In order to predict detection thresholds, authors of [13] employ the detection model in [94], which is designed specifically for HDR images. This model takes into account spatial sensitivity changes due to local adaption, non-linear response of the photo receptors, and light scattering in the eye’s optics. To ensure the accuracy of predictions, the DRIM algorithm calibrates its detection model with measurements in [95]. For optical transfer function (OTF) and CSF, models in [96] and [97] are employed respectively. Second, the perceptually normalized response is decomposed into several bands of different orientations and scales. In order to do this, cortex transform, i.e., the collection of the band-pass and orientation selective filters, as proposed in [97] is employed. Third, for prediction of three distortion types separately for each band, the conditional probability of each distortion type is calculated as follows: s ,o Ploss  Prefs ,o/vis Ptsts ,/oinv s ,o ampl P s ,o rev P s ,o ref / inv P P s ,o ref /vis s ,o (53) P s ,o tst /vis P where Ploss , (52) s ,o tst /vis R s ,o s ,o s ,o , and Prev Pampl (54) denote the conditional probability of loss of visible contrast, amplification of invisible contrast, and reversal of visible contrast in the scale s and orientation o respectively. Subscripts ref / . and tst / . denote the reference and test images respectively, also . /vis and . / inv correspond to visible and invisible contrast. The parameter R is equal to 1 if the contrast polarity in the reference and test images differs from one another, and is zero otherwise. Fourth, because (52) through (54) possess s ,o may non-linear operators, the probability map P contain spurious distortions. In order to prevent this problem, each probability map is filtered one more time s ,o using its corresponding cortex filter B probability map is computed as follows:    s ,o s ,o Pˆloss  F 1 F Ploss B s ,o . The filtered (55) 1 where F and F denote the 2-D Fourier and inverse Fourier transforms respectively. Although (55) is s ,o written for Ploss , the filtered probability maps for s ,o Pampl s ,o and Prev are computed in a similar manner. Finally, the probability of detecting a distortion in any subband is calculated as follows: 70 Ms Mo  s ,o Ploss  1   1  Pˆloss s 1 o 1 Where  (56) M o and M s are total numbers of orientations and scales respectively. Equ. 56 is based on the assumption that detecting each distortion in each subband is an independent procedure. The probability maps Prev and Pampl are calculated in a similar manner. In order to visualize each of the three distortion types, an in-context distortion map approach similar to [97] is employed, and a custom viewer application for detailed inspections is introduced. In order to generate the incontext map, luminance of the test image is copied to all three RGB channels, and each channel is scaled using the detection probability of their corresponding distortion types. In [13], only the distortion types with highest probability of detection at each pixel location is used for visualization purposes. Green is chosen for loss of visible contrast, blue corresponds to amplification of invisible contrast, and red denotes reversal of visible contrast. By using custom viewer application employed in [13], one can dynamically set the level of distortion types and the background image to an appropriate level in order to investigate each distortion types separately. Applications of the DRIM algorithm, as stated in [13], are: comparison of TMOs, evaluation of inverse TMOs, and comparison of different types of display devices. To the best of our knowledge, authors of [13] have not published a publicly accessible source code for the DRIM algorithm. However, in [98], authors have provided an online implementation of the DRIM algorithm where the reference and the test images can be uploaded and after assigning the parameters by the user, the probability maps and the final in-context distortion map is generated. 5.2. Tone-mapped images quality index (TMQI) In [14], an objective IQA method for tone-mapped images is proposed. This metric is a combination of multi-scale structural fidelity measure and statistical naturalness measure. The TMQI algorithm consists of two stages: structural fidelity measurement, and statistical naturalness measurement. Since TMOs compress the dynamic range of HDR images, they result in the loss of information. Moreover, this loss of information may not be visible in the LDR images for the human observers to see. Therefore, structural fidelity is an important part of tone-mapped images quality assessment. Consider x and y to be two local image patches obtained from the HDR and tone-mapped LDR images respectively. TMQI algorithm defines its structural fidelity measure Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 as follows: S local  x , y   Where  x , 2 x  y T 8  x , y T 9  x 2   y 2 T 8  x  y T 9  y , and  x (57) ,y a   z  a  exp   2 a2 2   a a is the standard deviation of normal k  are the local standard positive stabilizing constants designed to prevent the denominators from becoming too small. Compared with (12), the luminance comparison function is missing, and the structure comparison function, denoted by the second part of (57), is exactly the same. The reason for the absence of luminance comparison function is that since TMOs change the local luminance and contrast, the direct comparison of these two characteristics is inappropriate. The first component of (57) is a modified version of (9) that compares the strength of two image signals. This modification is based on two intuitive considerations:  When the signal strength of the HDR and LDR image patches are either above the visibility threshold or below it, the difference between them should not be penalized.  The difference in signal strength between HDR and LDR image patches should be penalized when signal strength in one patch is above visibility threshold and is below it in the other patch. In order to compensate above considerations, the local standard deviation  is passed through a non-linear mapping that yields   in (57). This mapping has the following characteristics:  Signal strengths above visibility threshold are mapped to 1.  Signal strengths below visibility threshold are mapped to 0.  Smooth transition between 0 and 1. The non-linear mapping described above is related to the visual sensitivity of contrast. HVS follows a gradual increasing probability in observing contrast changes. Some psychometric functions that describe the detection probability of signal strength have been used to model the data taken from psychophysical experiments [99,100]. TMQI algorithm employs a commonly used psychometric function known as Galton’s ogive [101]. This function has the form of cumulative normal distribution function denoted by: 1 threshold, and is the modulation distribution. It has been shown that the ration deviations and cross correlation between image patches x and y respectively, and T 8 and T 9 are two P a   a amplitude of sinusoidal stimuli, 2   dz  Where P is the probability of detection, (58) a is the a a is approximately a constant, known as Crozier’s law [101,102]. Usually, k takes its values between 2.3 and 4, and k  3 results in the probability of detection to be considerably low [101]. In order to quantify visual contrast sensitivity, CSF is used. TMQI algorithm uses the following equation for CSF [63]: A f   2.6 0.0192  0.114f  exp[  0.114f  1.1 (59) ] Where f is the spatial frequency. In order for CSF to be compatible with psychological data, it needs to be scaled by a constant  . In TMQI algorithm, CSF measurement, as presented in [103], is used. The modulation threshold,  a a f   f  , is calculated via: 1 A f (60)  Equ. 60 is the threshold value based on contrast sensitivity measurement with assumption of pure sinusoidal stimuli.  a  f  needs to be converted into a signal strength threshold. In order to achieve this, it is important to note that signal amplitude scales with contrast and mean signal intensity. Therefore, the threshold value defined on signal standard deviation, HDR image ↓2 L S1 LDR image ↓2 L L ↓2 S2 SMs ↓2 L ↓2 L L Structural fidelity measure ↓2 Fig. 11. The block diagram of the structural fidelity measure in the TMQI algorithm. L: low-pass filter; ↓2: downsampling by factor of 2 s f  , is computed as follows: s f   Where  a  f 2    A f  (61) 2 is the mean intensity of the signal. According to Crozier’s law [101,102]:   f    s  f  . s Finally, the non-linear mapping between defined as follows:  k and  is 71 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 s   z   s 2  exp   2 s2  dz 2     x and  y in (57) are the mapped versions of  x and  y respectively. Equ. 57 is computed using a   1  (62) sliding window approach which yields a map containing the variations of structural fidelity across the entire image. TMQI algorithm adapts a multi-scale approach same as MS-SSIM algorithm, in which HDR image and its corresponding LDR version are iteratively low-pass filtered and downsampled (by factor of 2). The block diagram for computing structural fidelity measure is presented in Fig. 11. At each scale, the local structural fidelity map is computed and averaged in order to obtain a single score: 1 Ss  Mp where Mp S x i 1 local i ,yi  (63) x i and y i are the i  th image patch in the HDR and LDR images respectively, and M p is the total number of image patches in the scale s . The overall structural fidelity score is calculated as follows: Ms S   S ss (64) s 1 where M s is the total number of scales, and  s is a constant chosen to indicate the relative importance of the scale s . Structural fidelity alone is not a sufficient measure for evaluating the overall quality of images. Another important characteristic of a high quality LDR image is that it should look natural. According to the results of a subjective experiment conducted in [104], TMQI algorithm uses brightness and contrast for its statistical naturalness model. Calculating fm LDR image 1/K × Statistical naturalness measure Calculating fd Fig. 12. The block diagram of the statistical naturalness measure in the TMQI algorithm This model is based on statistics of about 3000 8 bits/pixel gray-scale images available at [105,106]. In order to compute statistical naturalness measure, TMQI algorithm computes the histograms of mean and the 72 standard deviation of these images. It is mentioned in [14] that these histograms can be well-fitted by Gaussian and Beta probability density function respectively: fm  m   fd  d   Where 1 m   m  m 2  exp    2 m2  2   1  d  d  1 B  d ,  d  B .,. d 1 (65) d (66) denotes Beta function. According to [107], brightness and contrast are mostly independent characteristics in terms of natural image statistics and biological computation. Therefore, the joint probability density function of contrast and brightness is the product of their respective probability density functions. As a result, TMQI algorithm defines its statistical naturalness measure via following equation: N  1 f m fd K Where (67) K  max fm ,fd  is a normalization factor designed to bound N in the interval  0,1 . The block diagram of the statistical naturalness measure is presented in Fig. 12. After computing structural fidelity and statistical naturalness measure, the overall quality index is calculated via: Q   S   1    N  Where 0   1 (68) is a constant chosen to indicate the relative importance of each component, and  and  are two constants chosen to indicate each component’s sensitivity. The overall quality measure, Q , takes its values in the interval [0,1] . Two application of the TMQI algorithm, as mentioned in [14], are the parameters tuning in TMOs and adaptive fusion of tone-mapped images. 5.2.1. Parameter specification in the TMQI algorithm There are several parameters in the TMQI algorithm that need to be specified. First, for computing (57) the values of T 8 and T 9 are set to be 0.01 and 10 respectively. It is mentioned in [14] that the overall performance of the TMQI algorithm is insensitive to the values of parameters T 8 and T 9 up to an order of magnitude. Second, TMQI algorithm employs the same procedure as SSIM algorithm for creating the fidelity Majlesi Journal of Electrical Engineering map of each scale, i.e., using a Gaussian sliding window of size 11 11 and standard deviation of 1.5 samples. Third, the viewing distance is set to be 32 cycles/degree. Therefore, the spatial frequency parameter in (59) is set to be 16 cycles/degree for the finest scale measurement. The spatial frequency parameter employed for the remaining finer scales are 8, 4, 2, and 1 cycles/degree. Fourth, the value of mean intensity in (61) is set to be equal to the dynamic range of LDR images. In other words,   128 . Fifth, according to psychophysical experiment in [9], the parameters in (64) are defined as: M s  5 and s  0.048, 0.2856, 0.3001, 0.2363, 0.1333 for scales 1 to 5 respectively. Finally, since TMQI algorithm is designed specifically for gray-scale images, color images are first converted from RGB color space to Yxy color space and then the structural fidelity measure is applied to the Y component only. The parameters of (65) and (66) are estimated by first, fitting the histograms of means and standard deviations of images in [105,106] using Gaussian and Beta probability density functions, and then using regression. These parameters, are found to be m  115.94 and  m  27.99 in (65), and  d  4.4 and  d  10.1 in (66). Parameters of (68) are determined in a way that best fit the subjective evaluation data presented in [108]. In this subjective experiment, subjects were trained to look simultaneously at two LDR images generated via two different TMOs, and then pick the LDR image that they think has higher overall quality. In order to find the best parameters, an iterative learning method is employed. In this method, at each iteration a pair of images are chosen randomly from a random dataset. If the output of the overall quality measure is of the same order as the subjective rank, then the model parameters are left unchanged. Otherwise, each parameter is updated to lower the difference between subjective and objective scores. The iteration process continues until convergence occurs. It has been reported in [14] that this process has good convergence property. In order to evaluate the robustness of the proposed iterative learning process, a leave-one-out cross validation procedure is employed. It is mentioned in [14] that although this procedure ended up with a different value for parameters at each time, the results were fairly close to one another and they were all of the same rank orders for all datasets. Finally, the parameters of (68), are found to be:   0.8012 ,   0.3046 , and   0.7088 . Authors of [14] have provided a MATLAB implementation of the TMQI algorithm that is available at [18]. Vol. 9, No. 1, March 2015 6. SUBJECTIVE DATASETS AND PERFORMANCE MEASURES IN IMAGE QUALITY ASSESSMENT 6.1. Subjective datasets In order to evaluate the performance of a newly proposed IQA method, many subjective quality datasets have been introduced. Here, we briefly introduce six most widely used subjective quality datasets. These datasets include: Cornell-A57 dataset [51], IVC dataset [109], Tampere image dataset 2008 (TID2008) [17], LIVE dataset [15], Toyoma-MICT dataset [110], and categorical image quality (CSIQ) dataset [16]. The Cornell-A57 [51 ] dataset constitutes of 54 distorted images with six types of distortions. The distortions in this dataset are: quantization of the LH subbands of a 5-level DWT of the images using 9/7 filters, additive white Gaussian noise, baseline JPEG compression, JPEG2000 compression without visual frequency weighting, blurring via a Gaussian filter, and JPEG2000 compression with the dynamic contrastbased quantization algorithm. The IVC dataset [109] consists of 10 reference images and 185 distorted versions of them. Distortion types in this dataset are: JPEG2000 compression, JPEG compression, blurring, and local adaptive resolution coding. The TID2008 dataset [17] consists of 1700 test images generated from 25 reference images with 17 distortion types at four different distortion levels. 654 observers from three different countries participated in subjective ratings. Lightening condition, screen size, monitor type, and color gamma are varied between experiments in collecting TID2008 dataset. Distortion types in this dataset are: additive Gaussian noise, additive noise in color components is more intensive than its counterpart in the luminance component, masked noise, spatially correlated noise, high frequency noise, impulse noise, Gaussian blur, image denoising, JPEG compression, JPEG2000 compression, transmission errors in JPEG compression, transmission errors in JPEG2000 compression, contrast change, intensity shift, local block-wise distortions of different intensity, and noneccentricity pattern noise. Quality ratings for each image in TID2008 dataset are reported as mean opinion score (MOS). The LIVE dataset [15] consists of 29 reference images. Distortions in this dataset are: JPEG compression, JPEG2000 compression, white Gaussian noise, blurring, and fast fading channel distortion of JPEG2000 compressed stream. Total number of distorted images in this dataset is 779. Quality ratings for each image in this dataset are reported as DMOS. The Toyoma-MICT dataset [110] consists of 14 original images. Totally, it consists of 196 images (168 test images and 28 reference images). Distortions in 73 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 this dataset are: JPEG and JPEG2000 compression. Method used for subjective rating in this dataset is single stimulus categorical rating. Quality ratings for each image in this dataset are reported as MOS. The CSIQ dataset [16] consists of 30 reference images each distorted using six types of distortions at four to five distortion levels. Distortions in this dataset are: JPEG and JPEG2000 compression, global contrast decrements, additive white and pink Gaussian noise, and Gaussian blurring. Total number of distorted images in this dataset is 866. 6.2. Performance measures By taking into account the non-linearity of subjective ratings introduced during the subjective experiments, it is necessary to perform a non-linear mapping on the objective scores before measuring the correlation between the subjective and objective scores. According to the video quality experts group (VQEG) research [111], in order to obtain a linear relationship between an objective IQA method’s score for an image and its corresponding subjective score, each metric score x is mapped to q x  q x  . The non-linear mapping function is given by the following equation: 1  1 q  x   1      x  5  2 1  exp   2  x   3    4   The parameters 1, 2 , 3 , 4 , 5  (69) are calculated through minimizing the sum of squared differences among the subjective and the mapped scores. In order to compare the performance of a newly proposed IQA method with the existing ones, performance evaluation metrics are used. Here, we describe six commonly used performance measures in IQA: The Pearson’s linear correlation coefficient (PLCC) is the linear correlation coefficient between the predicted MOS (DMOS) and subjective MOS (DMOS). PLCC is a measure of prediction accuracy of an IQA metric, i.e., the capability of the metric to predict the subjective scores with low error. The PLCC can be calculated via following equation: Md PLCC   q i 1 i  q  s i  s  1 2 Md  Md 2  2   q i  q      s i  s    i 1   i 1  Where (70) 1 2 s i and qi are the subjective score and the mapped score for the i  th image in an image dataset of size M d respectively, and q and s are the means of the mapped scores and subjective scores respectively. The Spearman’s rank correlation coefficient (SRCC) is 74 the correlation coefficient between the predicted MOS (DMOS) and the subjective MOS (DMOS). SRCC measures the prediction monotonicity of an IQA metric, i.e., the limit to which the quality scores of a metric agrees with the relative magnitude of the subjective scores. The SRCC can be calculated via following equation: Md SRCC  1  Where 6 d i2 (71) i 1 M d  M d2  1 d i is the difference among the i  th image’s rank in the objective and subjective experiments. SRCC is independent of any monotonic non-linear mapping between objective and subjective scores. The Kendall’s rank correlation coefficient (KRCC) is a non-parametric rank correlation measure that can be calculated via following equation: KRCC  Where M c  M dc 1 M d  M d  1 2 (72) M c and M dc are the numbers of concordant and disconcordant pairs in the dataset respectively. Like SRCC, KRCC is a measure of the prediction monotonicity. The outlier ratio (OR) is defined as the percentage of the number of the predictions outside the interval of 2 times the standard deviation of the subjective scores. OR can be calculated via following equation: M (73) OR  d M Where M  is the number of outliers. OR measures the prediction consistency of an IQA metric, i.e., the limit to which the metric maintains the accuracy of its predictions. The root mean square error (RMSE) can be calculated via following equation: 1 2  1 Md 2 (74) RMSE   q i  s i     M d i 1  Like PLCC, RMSE is a measure of prediction accuracy. The mean absolute error (MAE) can be calculated via following equation: MAE  1 Md Md q i 1 i  si (75) Like PLCC and RMSE, MAE is a measure of prediction accuracy. A good IQA metric should have higher PLCC, KRCC, and SRCC while lower RMSE, MAE, and OR. Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 7. EVALUATION RESULTS 7.1. Evaluation of prediction performance In this subsection, we will evaluate the prediction performance of the FR-IQA methods described in previous sections: PSNR, SSIM [8], MS-SSIM [9], VIF [10], MAD [11], FSIM [12], FSIMC [12], and TMQI [14]. For all these methods, we have used their original MATLAB implementation provided by their respective authors. Since the DRIM algorithm [13] does not generate a single quality score for the entire image, it is impossible to compare its results with subjective evaluations. Therefore, we have not included this metric in all evaluations. Moreover, since the described FR-IQA methods are for different category of images (some for grayscale images, some for color images, and some for HDR images), we evaluate the performance of each category separately. The performance evaluation process for TMQI algorithm [14] is done on the dataset presented in [18]. For the remaining algorithms, we choose three datasets, these datasets include: TID2008 dataset [17], LIVE dataset [15], and CSIQ dataset [16]. It is important to note that in all our evaluations, the reference images are excluded and only test images are employed. Table 1 shows our test results of the 8 FR-IQA methods on four subjective quality datasets. To provide an evaluation of the overall performance of the image quality metrics under consideration, Table 2 gives the average SRCC, KRCC, PLCC, RMSE, and MAE results over three datasets, where the average values are calculated in two cases. In the first case, the performance measures’ scores are directly averaged, while in the second case, different weights are assigned to different datasets depending on their sizes (measured as the number of images, i.e., 1700 for TID2008, 866 for CSIQ, and 779 for LIVE datasets respectively). Since TMQI algorithm’s performance is measured in only one dataset, it is not included in Table 2. As it can be seen from Table 1, for TMQI algorithm only SRCC and KRCC measures are calculated. This is due to the fact that PLCC, RMSE, or MAE are used when subjects rank the quality of images in a specific range, e.g., from 1 to 10. However, in the subjective experiment in [18] subjects were asked to rank the images from best to worst quality and thus the scores given by subjects do not represent the quality of images. Hence, only the SRCC and KRCC measures are calculated for evaluation of TMQI algorithm. 7.2. Evaluation of computation time We have also evaluated the computation time of each selected FR-IQA methods. Since authors of [13] have not published a publicly available source code of their algorithm, we have not included DRIM algorithm in our evaluation. As we mentioned before, since the selected methods are for different category of images, we evaluate their computation time separately. SSIM PSNR MAD FSIM VIF MS-SSIM Table 1. Performance evaluation of 8 FR-IQA algorithms described in this paper CSIQ dataset KRCC SRCC PLCC MAE 0.6907 0.8756 0.8613 0.0991 0.6084 0.8058 0.8000 0.1195 0.7970 0.9466 0.9502 0.0636 0.7567 0.9242 0.9120 0.0797 0.7537 0.9195 0.9277 0.0743 0.7393 0.9133 0.8991 0.0870 RMSE 0.1334 0.1575 0.0818 0.1077 0.0980 0.1149 SSIM PSNR MAD FSIM VIF MS-SSIM LIVE dataset SRCC PLCC 0.9479 0.9449 0.8756 0.8723 0.9669 0.9675 0.9634 0.9597 0.9636 0.9604 0.9513 0.9489 RMSE 8.9455 13.3597 6.9037 7.6780 7.6137 8.6188 KRCC 0.7963 0.6865 0.8421 0.8337 0.8282 0.8045 MAE 6.9325 10.5093 5.2202 5.9468 6.1070 6.6978 75 Majlesi Journal of Electrical Engineering SSIM PSNR MAD FSIM VIF MS-SSIM Vol. 9, No. 1, March 2015 TID2008 dataset SRCC PLCC 0.7749 0.7732 0.5531 0.5734 0.8340 0.8308 0.8805 0.8738 0.7491 0.8084 0.8542 0.8451 KRCC 0.5768 0.4027 0.6445 0.6946 0.5860 0.6568 MAE 0.6547 0.8327 0.5562 0.4926 0.6000 0.5578 RMSE 0.8511 1.0994 0.7468 0.6525 0.7899 0.7173 KRCC 0.7690 CSIQ dataset SRCC 0.9310 PLCC 0.9192 MAE 0.0762 RMSE 0.1034 FSIMC KRCC 0.8363 LIVE dataset SRCC 0.9645 PLCC 0.9613 MAE 5.8403 RMSE 7.5296 FSIMC KRCC 0.6991 TID2008 dataset SRCC 0.8840 PLCC 0.8762 MAE 0.4875 RMSE 0.6468 FSIMC Dataset in [18] KRCC 0.5579 TMQI SRCC 0.7385 Table 2.Average performance over three datasets SSIM PSNR MAD FSIM VIF MS-SSIM SSIM PSNR MAD FSIM VIF MS-SSIM FSIMC FSIMC 76 KRCC 0.6879 0.5659 0.7612 0.7617 0.7226 0.7335 KRCC 0.6574 0.5220 0.7300 0.7431 0.6858 0.7126 KRCC 0.7681 KRCC 0.7491 SRCC 0.8661 0.7448 0.9158 0.9227 0.8774 0.9063 Direct Average PLCC 0.8598 0.7486 0.9162 0.9152 0.8988 0.8977 Dataset Size-Weighted Average SRCC PLCC 0.8413 0.8360 0.6943 0.7017 0.8941 0.8935 0.9111 0.9037 0.8432 0.8747 0.8921 0.8833 Direct Average SRCC 0.9265 MAE 2.5621 3.8205 1.9467 2.1730 2.2604 2.4475 RMSE 3.3100 4.8722 2.5774 2.8127 2.8339 3.1503 MAE 1.9729 2.9016 1.5148 1.6559 1.7464 1.8658 RMSE 2.5504 3.7018 2.0085 2.1476 2.1999 2.4015 PLCC 0.9198 MAE 2.1347 RMSE 2.7599 Dataset Size-Weighted Average SRCC PLCC 0.9149 0.9072 MAE 1.6276 RMSE 2.1090 Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015 Table 3. Evaluation of computation time Time Computation Time for an image of size 512*512 (in seconds/image) SSIM PSNR MAD FSIM VIF 0.0293 0.0035 2.0630 0.3508 1.3647 Time Computation Time for an image of size 512*512 (in seconds/image) FSIMC 0.3776 Time Computation Time for an image of size 512*512 (in seconds/image) TMQI 0.4087 We measured the average computation time required to evaluate the quality of images of size 512  512 , Experiments were performed on a laptop with Intel Core i7 processor at 1.6 GHz. The software platform was MATLAB R2013a. The results are listed in Table 3. 8. QUALITY ASSESSMENT OF 3-D IMAGES The number of digital 3-D images available for human consumption has increased at a fast pace in recent years. According to the statistics collected by the motion picture association of America (MPAA), half of all moviegoers saw at least one 3- D movie in 2011, and those under 25 years old saw more than twice that number [112]. In order to meet this increasing demand, the number of 3-D movies has been increasing at least 50 percent annually over the recent years [112,113]. Aside from movies, other forms of 3-D contents are finding their way into our daily lives via 3-D television broadcasts [114], and 3-D on mobile devices [115]. These contents bring with themselves a variety of complex technological and perceptual problems. For a consistent, comfortable, and plausible perception of depth, a large number of parameters in the imaging and processing stages need to be determined in a perceptually meaningful way. However, due to some inevitable trade-offs in real-world applications, the visual quality of these 3-D contents will degrade. Therefore, in order to maintain and improve the quality of experience (QoE) of 3- D visual contents, subjective and objective quality assessment methods are needed. These methods are of high importance for display manufacturers, content providers, and service providers. Compared to its 2-D counterpart, 3-D IQA faces more new challenges. These include depth perception, virtual view synthesis, and asymmetric stereo compression. One natural question is the applicability of 2-D IQA methods to the 3-D images. The works in [116,117] try to answer this question. The results demonstrated that 2-D objective IQA methods can well evaluate the quality of 3-D images only in the case of symmetric images, i.e., the PSNR’s of the two- MS-SSIM 0.0834 eye images are approximately the same. Some of the proposed quality descriptors of 3-D contents that quantify the overall viewing experience of a 3-D representation are as follows [118]:  Depth quality: the depth characteristics of 3D data need to be examined in order to validate the suitability of the content for viewing [119].  Naturalness: the limit that enables viewers to easily fuse left and right views into a naturallooking 3-D image with smooth depth representation [120].  Presence: a natural-looking 3-D scene enhances the viewers’ sense of presence [121].  Value-add: the perceived benefit of displaying a content in 3-D over displaying the same content in 2-D [122].  Discomfort: the overall subjective perception resulting from physiological and/or psychological effects of 3-D viewing content [123].  Overall 3-D QoE as typically measured in terms of DMOS. It is important to note that there are no commonly accepted methods for quantifying the above descriptors yet. However, Standards have recently been introduced to address this issue. Here, we summarize some of these standards:  ITU-R [124] has released a new recommendation on subjective quality assessment of 3-D TV systems. The focus of this recommendation is on picture quality, depth quality, and visual comfort.  The VQEG is addressing three main areas, including finding ground truth data for subjective evaluation methodology validation, validating objective 3-D video quality evaluation, and determining the effects of viewing environment on 3-D quality assessment. 77 Majlesi Journal of Electrical Engineering  IEEE initiated work on a standard for quality assessment of 3-D contents, 3-D displays, and 3-D devices based on human factors. This work looks into characteristics of display, device, environment, content, and viewers. The classification of 2-D IQA methods (namely FRIQA, RR-IQA, and NR-IQA) can be used in the case of 3-D images. However, the definitions do not apply in quite the same way [125]. This is due to the fact that it is impossible to gain access to the reference and test 3D images as they are perceived. This results since we only can access the left and right views of a scene, and we cannot access the Cyclopean image, i.e., a single mental image of a scene generated by the brain through combining the images received from the two eyes. This applies to both, the reference and test Cyclopean images. Therefore, the problem of 3-D IQA is doubleblind [125]. The first objective IQA for 3-D images is presented in [126]. In this work, a quality metric which uses the reliable 2-D IQA methods (including SSIM [8], UQI [52,53], method in [45], and the metric in [38]) is proposed. It is noteworthy that this method doesn’t take into account the depth information of 3-D images. Based on utilized information, 3-D IQA methods can be classified in two categories [127]: methods based on color information only, and methods based on color and disparity information. 8.1. Methods based on color information The methods in this category are based solely on color information [128-132]. In [128], quality scores on the SIFT-matched feature points are computed. In [129], a multiple channel model is employed to estimate the 3D image quality. In [130], an RR-IQA method for 3-D images is proposed. This method makes use of extracted edge information. In [131], the Gabor response of binocular vision is modeled for measuring the quality of 3-D images. In [132], a state of the art 3D IQA method for 3-D video compression is proposed. 8.2. Methods based on color and disparity information The methods in this category make use of both, color and disparity information in order to evaluate the overall quality of 3-D data [133-135]. In [133], an RRIQA method for 3-D images is proposed which is based on eigenvalues/eigenvectors analysis. In [134,135], two NR-IQA methods for 3-D images are proposed. 8.3. Subjective 3-D image quality datasets In this subsection, some of the subjective 3-D image quality datasets are introduced: LIVE 3-D IQA dataset [136] consists of 20 reference image, 5 distortion categories, and total number of 365 test images. The quality scores in this dataset are in the 78 Vol. 9, No. 1, March 2015 form of DMOS. This dataset is the first publicly available 3-D IQA dataset that makes use of true depth information along with stereoscopic pairs and human opinion scores. Distortion types in this dataset are: JPEG compression, JPEG2000 compression, additive white Gaussian noise, Gaussian blur, and fast fading model based on the Rayleigh fading channel. IVC 3-D images dataset [137] consists of 6 reference images and 15 distorted version of each image plus their respective subjective scores. The distortion types in this dataset are: JPEG compression, JPEG2000 compression, and blurring. Total number of images in this dataset is 96. To the best of our knowledge, the only 3-D dataset for HDR images and their corresponding tone-mapped versions is available in [138]. In this dataset, 9 reference 3-D HDR images are tone-mapped using 8 TMOs. The total number of images in this dataset is 81. Moreover, the statistics of these images (including min, max, and mean luminance) and their histograms are also available in this dataset. 9. CONCLUSION The growing demand for digital image technologies in applications like medical imaging, biomedical systems, monitoring, and communications has highlighted the need for accurate quality assessment methods. Many processes can affect the quality of images, including compression, transmission, display, and acquisition. Therefore, accurate measurement of the image quality is an important step in many image-based applications. IQA aims at quantifying the quality of image signals including 3-D images, retargeted images, and HDR images by means of objective quality metrics. The goal of objective IQA is to design algorithms that can automatically evaluate the quality of images in a perceptually consistent manner. These methods are crucial to multimedia systems since they remove or reduce the need for extensive subjective evaluation. In this paper, an overview of subjective and objective IQA was presented. Four most commonly used subjective IQA methods were briefly introduced. Moreover, the three main categories of objective IQA were described. 3-D, Color and HDR images quality assessment were also reviewed. The central theme of this study was on FR-IQA methods and we thoroughly described 9 methods of this category. The prediction performance and computation time of these methods were also evaluated. IQA has been a rapidly developing field of research in recent years. The number of algorithms that are being proposed are growing at a fast pace. Of course, only a small number of methods have been discussed in detail in this paper. The selected methods are widely cited in the literature and have been reported to have good performance by researchers. Another criterion for our Majlesi Journal of Electrical Engineering selection is that the source code for most of these methods has been made available online, so the interested readers can implement them and regenerate the reported results. There are number of factors that need to be taken into account when selecting an IQA method for a specific application. Some of these factors include the availability of the reference image, computation time, implementation complexity, application goal, and quality prediction accuracy. By considering all these factors, one can make the right choice for each specific application. We have also provided a brief introduction to 3-D IQA, and summarized the issues associated with this field of research. It is important to note that with the advances in 3-D coding, transmission, and displays, the quality assessment of 3-D images has been studied independently for each of these areas. A number of elements must be taken into account for achieving a 3D IQA method. Among these are: dependencies between display, content, and the viewer, also individual user constraints, preferences, and perception of depth must be considered. Once we are able to further develop our knowledge of the perception of stereoscopic distortions, we can achieve better 3-D IQA algorithms. Vol. 9, No. 1, March 2015 [9] [10] [11] [12] [13] [14] [15] [16] REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] F. Xiao, J.E. Farrel, and B.A. Wandell, "Psychophysical thresholds and digital camera sensitivity: the thousand-photon limit," Proc. SPIE, Vol. 5678, pp. 75-84, Feb. 2005. J. Chen, T.N. Pappas, A. Mojsilovic, and B.E. Rogowitz, "Perceptually-tuned multiscale colortexture segmentation," in IEEE Int. Conf. on Image Processing, Oct. 2004. X.Zhang, D.A. Silverstein, J.E. Farrell, and B.A. Wandell, "Color image quality metric S-CIELAB and its application on halftone texture visibility," presented at IEEE Computer Conf., Feb. 1997. N., Damera-Venkata, T.D., Kite, W.S., Geisler, B. L., Evans, and A.C., Bovik, "Image quality assessment based on a degradation model," IEEE Trans. Image Processing, Vol. 9, pp. 636-650, April 2000. G. Piella, H. Heijmans, "A new quality metric for image fusion," presented at IEEE Int. Conf. on Image Processing, Sept. 2003. H.H. Barret, "Objective assessment of image quality: effects of quantum noise and object variability," J. Opt. Soc. Am. A, Vol. 7, pp. 12661278, 1990. H.H. Barret, J.L. Denny, R.F. Wanger, and K.J. Myers, "Objective assessment of image quality. II. Fisher information, Fourier crosstalk, and figures of merit for task performance," J. Opt. Soc. Am. A, Vol.12, pp. 834-852, 1995. Z. Wang, A.C. Bovik, and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Trans. Image [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] Processing, Vol. 13, pp. 600 612, April 2004. Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale structural similarity for image quality assessment," presented at IEEE Asilomar Conf. on Signals, Systems, and Computers, Nov. 2003. H.R. Sheikh, A.C. Bovik, "Image information and visual quality," IEEE Trans. Image Processing, Vol. 15, pp. 430-444, Feb. 2006. E.C. Larson, D.M. Chandler, "Most apparent distortion: full-reference image quality assessment and the role of strategy," J. Electron. Imag, Vol. 19, pp. 011006:1 011006:21, Jan. 2010. L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A feature similarity index for image quality assessment," IEEE Trans. Image Processing, Vol. 20, pp. 2378-2386, Aug. 2011. T.O. Aydin, R. Mantiuk, K. Myszkowski, and H.P. Seidel, "Dynamic range independent image quality assessment," ACM Trans. Graph. , Vol.27, pp.1 -10, Aug. 2008. H. Yeganeh, Z. Wang, "Objective quality assessment of tone-mapped images," IEEE Trans. Image Processing, vol. 22, pp. 657-667, Feb. 2013. H.R. Sheikh, K. Seshadrinathan, A.K. Moorthy, Z. Wang, and A.C. Bovik, "LIVE image quality assessment database," [Online]. Available: http://live.ece.utexas.edu/research/quality/subjective .htm. E.C. Larson, D.M. Chandler, "Categorical image quality dataset," [Online]. Available: http://vision.okstate.edu/csiq. N. Ponomarenko, K. Egiazarian, "Tampere image database 2008 TID2008," [Online]. Available: http://www.ponomarenko.info/tid2008.htm. H. Yeganeh, Z. Wang, "TMQI: Tone-mapped image quality index," [Online]. Available: http://ece.uwaterloo.ca/~z70wang/research/tmqi/. ITU-R Recommendation BT.500-11 , "Methodology for the subjective assessment of the quality of television pictures," ITU, Geneva, Switzerland, 2002. ITU-R Recommendation BT.710-4, "Subjective assessment methods for image quality in highdefinition television," ITU, Geneva, Switzerland, 1998. ITU-T Recommendation P.910, "Subjective video quality assessment methods for multimedia applications," ITU, Geneva, Switzerland, 2008. ITU-R Recommendation BT.814-1, "Specification and alignment procedures for setting of brightness and contrast of displays," ITU, 1994. ITU-R Recommendation BT.1129-2, "Subjective assessment of standard definition digital television (SDTV) systems," ITU, 1998. ITU-R Recommendation BT.1361, "Worldwide unified colorimetry and related characteristics of future television and imaging systems," ITU, 1998. ITU-R Recommendation BT.815-1, "Specification of a signal for measurement of the contrast ratio of displays," ITU, 1994. R. Mantiuk, A. Tomaszewska, and R. Mantiuk, 79 Majlesi Journal of Electrical Engineering [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] 80 "Comparison of four subjective methods for image quality assessment," Computer Graphics Forum, Vol.31, pp. 2478-2491, 2012. H. Gulliksen, L.R. Tucker, "A general procedure for obtaining paired comparisons from multiple rank orders," Psychometrika, Vol. 26, pp. 1 73183, June 1961. D.A. Silverstein, J.E. Farrell, "Efficient method for paired comparison," J. Electron. Imag., Vol. 10, pp. 394-398, April 2001. A.M. van Dijk, J.B. Martens, and A.B. Watson, "Quality assessment of coded images using numerical category scaling," Proc. SPIE, Vol. 2451, pp. 90-101, Feb. 1995. Z. Wang, A.C. Bovik, “Modern image quality assessment. Synthesis Lectures on Image”, Video & Multimedia Processing, Morgan & Claypool Publishers, 2006. H. R. Sheikh, A.C. Bovik, and L. Cormack, "Noreference quality assessment using nature scene statistics: JPEG2000," IEEE Trans. Image Processing, Vol.14, pp.1918-1927 , Nov. 2005. L. Liang, S. Wang, J. Chen, S. Ma, D. Zhao, and W. Gao, "No-reference perceptual image quality metric using gradient profiles for JPEG2000," Signal Processing: Image Communication , Vol. 25, pp. 502-516, Aug. 2010. T. Brando, M.P. Queluz, "No-reference image quality assessment based on DCT domain statistics," Signal Processing, Vol. 88, pp. 822-833, April 2008. Z. Wang, H.R. Sheikh, and A.C. Bovik, "Noreference perceptual quality assessment of JPEG compressed images," presented at IEEE Int. Conf. on Image Processing, Sept. 2002. R. Ferzli , L.J. Karam, "A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB)," IEEE Trans. Image Processing, Vol. 18, pp. 717-728, April 2009. Z. Wang, A.C. Bovik, "Reduced-and no-reference image quality assessment: the natural scene statistic model approach," in IEEE Signal Processing Magazine, Vol. 28, Nov. 2011, pp. 2940. A. Rehman, Z. Wang, "Reduced-reference image quality assessment by structural similarity estimation," IEEE Trans. Image Processing, Vol. 21, pp. 3378-3389, Aug. 2012. Z. Wang, E.P. Simoncelli, "Reduced-reference image quality assessment using a wavelet-domain natural image statistic model," Proc. SPIE, Vol. 5666, pp. 149-159, 2005. Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E.H. Yang, and A.C. Bovik, "Qualityaware images," IEEE Trans. Image Processing, Vol. 15, pp. 1680-1689, June 2006. Q. Li, Z. Wang, "Reduced-reference image quality assessment using divisive normalizationbased image representation," IEEE Journal of Selected Topics in Signal Processing, Vol.3, pp. 202-211, April 2009. S. Wolf, M.H. Pinson, "Spatial-temporal Vol. 9, No. 1, March 2015 [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] distortion metric for in-service quality monitoring of any digital video system," Proc. SPIE, Vol. 3845, pp. 266-277, Nov. 1999. T.M. Kusuma, H.J. Zepernick, "A reducedreference perceptual quality metric for in-service image quality assessment," Joint First Workshop on Mobile Future and Symposium on Trends in Commun., pp. 71 -74, Oct. 2003. I.P. Gunawan, M. Ghanbari, "Reduced reference picture quality estimation by using local harmonic amplitude information," presented at London Commun. Symposium, Sept. 2003. K. Chono, Y.C. Lin, D. Varodayan, Y. Miyamoto, and B. Girod, "Reduced-reference image quality assessment using distributed source coding," in IEEE Int. Conf. on Multimedia and Expo, April 2008. M. Carnec, P. Le Callet, and D. Barba, "An image quality assessment method based on perception of structural information," in IEEE Int. Conf. on Image Processing, Sept 2003. M. Carnec, P. Le Callet, and D. Barba, "Visual features for image quality assessment with reduced reference," in IEEE Int. Conf. on Image Processing, Sept. 2005. L. Ma, S. Li, and K.N. Ngan, "Visual horizontal effect for image quality assessment," IEEE Signal Processing Letters, Vol. 17, pp.627-630, July 2010. F. Zhang, W. Liu, W. Lin, and K.N. Ngan, "Spread spectrum image watermarking based on perceptual quality metric," IEEE Trans. Image Processing, Vol. 20, pp. 3207-3218, Nov. 2011. Y. Niu, M. Kyan, L. Ma, A. Beghdadi, and S. Krishnan, "A visual saliency modulated just noticeable distortion profile for image watermarking," in European Signal Processing Conf., 2011. Z. Wang, A.C. Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures," in IEEE Signal Processing Magazine, Vol. 26, Jan. 2009, pp. 98-117. D.M. Chandler, S.S. Hemami, "A57 dataset," [Online].Available:http://foulard.ece.cornell.edu/dm c27/vsnr/vsnr.html. Z. Wang, “Rate scalable foveated image and video communications”. PhD thesis, Dept. of ECE, the University of Texas at Austin, Dec. 2001. Z. Wang , A.C. Bovik, "A universal image quality index," IEEE Signal Processing Letters, Vol. 9, pp. 81 -84, March 2002. A.M. Alattar, E.T. Lin, and M.U. Celik, "Digital watermarking of low bit-rate advanced simple profile MPEG-4 compressed video," IEEE Trans. Circ. Syst. Video Tech. , Vol. 13, pp. 787-800, Aug. 2003. E. Christophe, D. Leger, and C. Mailhes, "Quality criteria benchmark for hyperspectral imagery," IEEE Trans. Geosci. Remote Sensing, Vol. 43, pp. 2103-2114, Sept. 2005. L. Snidaro, G.L. Foresti, "A multi-camera approach to sensor evaluation in video Majlesi Journal of Electrical Engineering [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] surveillance," presented at IEEE Int. Conf. on Image Processing, Sept. 2005. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "The SSIM index for image quality assessment,",[Online].Available: http://ece.uwaterloo.ca/~z70wang/research/ssim/ Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale structural similarity for image quality assessment," [Online]. Available: http://ece.uwaterloo.ca/~z70wang/research/ssim/mss sim. A. Srivastava, A.B. Lee, E.P. Simoncelli, and S.-C. Zhu, "On advances in statistical modeling of natural images," J. Math. Imag. Vis. , Vol. 18, pp. 17-33, Jan. 2003. M.J. Wainwright, E.P. Simoncelli, and A.S. Wilsky, "Random cascades on wavelet trees and their use in analyzing and modeling natural images," Appl. Comput. Harmon. Anal., Vol. 11, pp. 89-123, July 2001. V. Strela, J. Portilla, and E. P. Simoncelli, "Image denoising using a local Gaussian scale mixture model in the wavelet domain," Proc. SPIE, Vol. 4119, pp. 363-371, Dec. 2000. H.R. Sheikh, A.C. Bovik, "Visual information fidelity (VIF) measure for image quality assessment," [Online]. Available: http://live.ece.utexas.edu/research/quality/vifvec_rel ease. J. Mannos, D.J. Sakrison, "The effects of a visual fidelity criterion of the encoding of images," IEEE Trans. Inf. Theory, vol. 20, pp. 525-536, July 1974. S. Daly, "Subroutine for the generation of a human visual contrast sensitivity function," Eastman Kodak Tech. Report 233203y, 1987. E C. Larson , D.M. Chandler, "Full-reference image quality assessment and the role of strategy: The most apparent distortion,"[Online]. Available: http://vision.okstate.edu/mad/. D. Marr, Vision. MIT Press, July 2010. D. Marr, E. Hildreth, "Theory of edge detection," Proc. R. Soc. Lond. B, Vol. 207, pp. 187-217, Feb. 1980. M.C. Morrone, D.C. Burr, "Feature detection in human vision: A phase-dependent energy model," Proc. R. Soc. Lond. B, Vol. 235, pp. 221 245, Dec. 1988. M.C. Morrone, J. Ross, D.C. Burr, and R.A. Owens, "Mach bands are phase dependent," Nature, Vol. 324, pp. 250-253, Nov. 1986. M.C. Morrone, R.A. Owens, "Feature detection from local energy," Pattern Recognit. Letters, Vol. 6, pp. 303-313, Dec. 1987. P. Kovesi, "Image features from phase congruency," Videre: J. Comp. Vis. Res., Vol.1, pp. 1 -26, 1999. L. Henriksson, A. Hyvärinen, and S. Vanni, "Representation of cross-frequency spatial phase relationships in human visual cortex," J. Neurosci., Vol. 29, pp. 14342-14351, Nov. 2009. R. Jain, R. Kasturi, and B.G. Schunck, “Machine Vol. 9, No. 1, March 2015 [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] vision.” ,McGraw-Hill, 1995. B. Jähne, H. Haussecker, and P. Geissler, “Handbook of computer vision and applications” , Academic Press, 1999. L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A feature similarity index for image quality assessment," [Online]. Available: http://www.comp.polyu.edu.hk/~cslzhang/IQA/FSI M/FSIM. tm. A. Toet , M. P. Lucassen, "A new universal colour image fidelity metric," Displays, Vol. 24, pp. 197207, Dec. 2003. O.D. Faugeras, "Digital color image processing within the framework of a human visual model," IEEE Trans. Acoust. Speech Signal Processing, Vol. 27, pp. 380-393, Aug. 1979. P. Le Callet , D. Barba, "Perceptual color image quality metric using adequate error pooling for coding scheme evaluation," Proc. SPIE, vol. 4662, May 2002. Y.K Lai, J.Guo, and C.C.J. Kuo, "Perceptual fidelity measure of digital color images," Proc. SPIE, Vol. 3299, pp. 221-231, 2002. M.S. Lian, "Image evaluation using a color visual difference predictor (CVDP)," Proc. SPIE, Vol. 4299, June 2001. J. Preiss, F. Fernandes, and P. Urban, "Color-image quality assessment: From prediction to optimization," IEEE Trans. Image Processing, Vol. 23, pp. 1366-1378, March 2014. A. Kolaman, O. Yadid-Pecht, "Quaternion structural similarity: A new quality index for color images," IEEE Trans. Image Processing, Vol. 21, pp. 1526-1536, April 2012. C.C. Yang , S.H. Kwok, "Efficient gamut clipping for color image processing using LHS and YIQ," Opt. Eng., Vol. 42, pp. 701 -711, March 2003. E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, "Photographic tone reproduction for digital images," ACM Trans. Graph. , Vol. 21, pp. 267276, 2002. G.W. Larson, H. Rushmeier, and C. Piatko, "A visibility matching tone reproduction operator for high dynamic range scenes," IEEE Trans. Visual. Comp. Graph. , Vol. 3, pp. 291 -306, 1997. R. Fattal, D. Lischinski, and M. Werman, "Gradient domain high dynamic range compression," ACM Trans. Graph. , Vol. 21, pp. 249-256, July 2002. F. Drago, K. Myszkowski, T. Annen, and N. Chiba, "Adaptive logarithmic mapping for displaying high contrast scenes," Computer Graphics Forum, Vol. 22, pp. 419-426, Sept. 2003. F. Drago, W. L. Martens, K. Myszkowski, and H.P. Siedel, "Perceptual evaluation of tone mapping operators," presented at ACM SIGGRAPH 2003 Sketches & Appl., 2003. J. Kuang, H. Yamaguchi, C. Liu, G.M. Johnson, and M.D. Fairchild, "Evaluating HDR rendering algorithms," ACM Trans. Appl. Perception, Vol. 4, July 2007. P. Ledda, A. Chalmers, T. Troscianko, and H. 81 Majlesi Journal of Electrical Engineering [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] 82 Seetzen, "Evaluation of tone mapping operators using a high dynamic range display," ACM Trans. Graph. , Vol. 24, pp. 640-648, July 2005. A. Yoshida, V. Blanz, K. Myszkowski, and H.P. Siedel, "Perceptual evaluation of tone mapping operators with real-world scenes," Proc. SPIE, Vol. 5666, pp. 192-203, 2005. M. Čadík, M. Wimmer, L. Neumann, and A. Artusi, "Image attributes and quality for evaluation of tone mapping operators," in Proc. 14th Pacific Conf. on Comput. Graph. Appl., pp. 35-44, 2006.. M. Barkowsky, P. Le Callet, "On the perceptual similarity of realistic looking tone mapped high dynamic range images," in IEEE Int. Conf. on Image Processing, Sept. 2010. R. Mantiuk, S.J. Daly, K. Myszkowski, and H.P. Siedel, "Predicting visible differences in high dynamic range images: model and its calibration," Proc. SPIE, Vol. 5666, pp. 204- 214, March 2005. A.B. Watson, "Visual detection of spatial contrast patterns: Evaluation of five simple models," Opt. Express, Vol. 6, pp. 12-33, 2000. [96] R. J. Deeley, N. Drasdo, and W. N. Charman, "A simple parametric model of the human ocular modulation transfer function," Ophthalmic and Physiol. Opt. , Vol. 11, pp. 91 -93, 1991. S.J. Daly, "Visible differences predictor: an algorithm for the assessment of image fidelity," Proc. SPIE, Vol. 1666, Aug 1992. T.O. Aydin, R. Mantiuk, K. Myszkowski, and H.P. Seidel, "Dynamic range independent metrics online," [Online]. Available: http://driiqm.mpiinf.mpg.de/. J.P. Guilford, “Psychometric methods”, 2nd ed. McGraw-Hill, Dec. 1954. Y. Le Grand, “Light, color and vision”. Dover, 1957. P.G.J. Barten, “Contrast sensitivity of the human eye and its effects on image quality”, Vol. PM72. SPIE Press, Dec. 1999. W.J. Crozier, "On the variability of critical illumination for flicker fusion and intensity discrimination," J. General Physiol., Vol. 19, pp. 503-522, Jan. 1936. D.H. Kelly, "Effects of sharp edges on the visibility of sinusoidal gratings," J. Opt. Soc. Am., Vol. 60, pp. 98-102, 1970. M. Čadík, P. Slavik, "The naturalness of reproduced high dynamic range images," presented at 9th Int. Conf. on Inf. Visual., July 2005. "Computer vision test images," [Online]. Available: http://www.cs.cmu.edu/afs/cs/project/cil/www/v images.html. "UCID - uncompressed colour image database," [Online]. Available: http://homepages.lboro.ac.uk/~cogs/datasets/ucid/uc id.html. V. Mante, R.A. Frazor, V. Bonin, W.S. Geisler, and M. Carandini, "Independence of luminance and contrast in natural scenes and in the early visual Vol. 9, No. 1, March 2015 [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] system," Nature Neurosci., Vol. 8, pp. 1690-1697, Nov. 2005. M. Song, D. Tao, C. Chen, J. Bu, J. Luo, and C. Zhang, "Probabilistic exposure fusion," IEEE Trans. Image Processing, Vol. 21, pp. 341 -357, Jan. 2012. P. Le Callet, F. Autrusseau, "Subjective quality assessment IRCCyN/IVC database," [Online]. Available: http://www.irccyn.ec-nantes.fr/ivcdb/. Y. Horita, K. Shibata, and Y. Kawayoke, "MICT image quality evaluation database," [Online]. Available: http://mict.eng.utoyama.ac.jp/mictdb.html. Video quality expert group (VQEG), "Final report from the video quality experts group on the validation of objective models of video quality assessment II," [Online]. Available: http://www.vqeg.org. "Theatrical market statistics," MPAA. Washington, DC, USA [Online]. Available: http://www.mpaa.org/wp-content/uploads/2014 /03 /MPAA-Theatrical-Market-Statistics2013_032514-v2.pdf., 2011. "List of 3-D movies," [Online]. Available: http://en.wikipedia.org/wiki/List_of_3D_films, 2005. "ESPN 3-D broadcasting schedule," ESPN. Bristol, CT, USA [Online]. Available: http://espn.go.com/espntv/3d/. H. Lee, S. Cho, K. Yun, N. Hur, and J. Kim, "A backward-compatible, mobile, personalized 3DTV broadcasting system based on T-DMB," in Three-dimensional television, H. M. Ozaktas and L. Onural, Eds.: Springer Berlin Heidelberg, pp. 11 -28, 2008. C.T.E.R. Hewage, S.T. Worral, S. Dogan, and A.M. Kondoz, "Prediction of stereoscopic video quality using objective quality models of 2-D video," Electron. Lett., Vol. 44, pp. 963-965, Jul. 2008. C.T.E.R. Hewage, S.T. Worral, S. Dogan, S. Villette, and A.M. Kondoz, "Quality evaluation of color plus depth map-based stereoscopic video," IEEE Journal of Selected Topics in Signal Processing, Vol. 3, pp. 304-318, April 2009. S. Winkler , D. Min, "Stereo/multiview picture quality: Overview and recent advances," Signal Processing: Image Communication, Vol. 28, pp. 1358-1373, Nov. 2013. P. Lebreton, A. Raake, M. Barkowsky, and P. Le Callet, "Evaluating depth perception of 3D stereoscopic videos," IEEE Journal of Selected Topics in Signal Processing, Vol. 6, Oct. 2012. P.J. Seuntiëns, I.E. Heynderickx, W.A. IJsselsteijn, P.M.J. van den Avoort, J. Berentsen, I.J. Dalm, M.T. Lambooij, and W. Oosting, "Viewing experience and naturalness of 3D images," Proc. SPIE, Vol. 6016, Nov. 2005. W.A. IJsselsteijn, "Presence in depth," PhD dissertation: Eindhoven University of Technology, 2004. J. Hakala, "The added value of stereoscopy in Majlesi Journal of Electrical Engineering [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] still images," Master's Thesis: Alato University, 2010. M.T.M. Lambooij, W.A. IJsselsteijn, and M.F. Fortuin, "Visual discomfort and visual fatigue of stereoscopic displays: A review," Journal of Imaging Technology and Science, Vol. 53, pp. 1 14, 2009. ITU-R Recommendation BT.2021, "Subjective methods for the assessment of stereoscopic 3DTV systems," International Telecommunication Union, Geneva, Switzerland 2012. A.K. Moorthy, C.C. Su, A. Mittal, and A.C. Bovik, "Subjective evaluation of stereoscopic image quality," Signal Processing: Image Communication, Vol. 28, pp. 870- 883, Sept. 2013. A. Benoit, P. Le Callet, P. Campisi, and R. Cousseau, "Quality assessment of stereoscopic images," EURASIP Journal on Image and Video Processing,2008. Y.H. Lin, J.L. Wu, "Quality assessment of stereoscopic 3D image compression by binocular integration behaviors," IEEE Trans. Image Processing, Vol. 23, pp. 1527-1542, April 2014. P. Gorley, N. Holliman, "Stereoscopic image quality metrics and compression," Proc. SPIE, Vol. 6803, Feb. 2008. L. Shen, J. Yang, and Z. Zhang, "Stereo picture quality estimation based on a multiple channel HVS model," in Int. Congr. Image and Signal Processing. Tianjin, pp. 1 -4, 2009. C.T.E.R. Hewage and M.G. Martini, "Reducedreference quality metric for 3D depth map transmission," in 3DTV-Conf. True Vis., Capture, Transmiss. Display 3D Video. Tampere, pp. 1 -4, 2010. R. Bensalma, M.-C. Larabi, "A perceptual metric for stereoscopic image quality assessment based on the binocular energy," Multidimen. Syst. Signal Processing, Vol. 24, pp. 281 -316, June 2013. V. De Silva, H.K. Arachchi, E. Ekmekcioglu, and A. Kondoz, "Toward an impairment metric for stereoscopic video: A full-reference video quality metric to assess compressed stereoscopic video," IEEE Trans. Image Processing, Vol. 22, pp. 33923404, Sept. 2013. A. Maalouf, M.C. Larabi, "CYCLOP: A stereo color image quality assessment metric," in IEEE Int. Conf. Acoust., Speech and Signal Processing. Prague, pp. 1161 -1164, 2011. R. Akhter, Z.M. Parvez Sazzad, Y. Horita, and J. Baltes, "No-reference stereoscopic image quality assessment," Proc. SPIE, Vol. 7524, Feb. 2010. M.J. Chen, L.K. Cormack, and A.C. Bovik, "Noreference quality assessment of natural stereopairs," IEEE Trans. Image Processing, Vol. 22, pp. 3379-3391, Sept. 2013. A.K. Moorthy, C.C. Su, and A.C. Bovik, "LIVE 3D image quality dataset," [Online]. Available: http://live.ece.utexas.edu/research/quality/live_3di mage.html, 2013. A. Benoit, P. Le Callet, P. Campisi, and R. Vol. 9, No. 1, March 2015 [138] Cousseau, "IVC 3D images dataset," [Online]. Available:http://130.66.64.103/spip.php?article876 &lang=. Z. Mai, C. Doutre, P. Nasiopoulos, and R.K. Ward, "Rendering 3 -D high dynamic range images: Subjective evaluation of tone-mapping methods and preferred 3-D image attributes," [Online]. Available:http://www.ece.ubc.ca/~zicongm/subjecti ve_test_3dtmo/3D MO_HTMLReport .html, 2012. 83