ESIM: Edge Similarity for Screen Content Image Quality Assessment

Zhangkai Ni¹, Lin Ma², Huanqiang Zeng¹, Jing Chen¹, Canhui Cai¹, and Kai-Kuang Ma³

¹School of Information Science and Engineering, Huaqiao University, Fujian, China

²Tencent AI Lab, Shenzhen

³School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

Fig. 1. Samples of the reference SCIs utilized in the SCID.

SCID: Screen Content Image Database

Fig. 2. The profiles of the obtained MOS in our established SCID: (a) Scatter plot of the entire 1,800 distorted SCIs and their MOS values; (b) Histogram of the MOS values of the entire 1,800 distorted SCIs.

In our proposed SCI database (denoted as SCID), both the reference SCIs and their distorted versions are included. For the former, extensive effort on searching representative SCIs over the Internet have been conducted through large manpower. With thoughtful evaluations, 40 images were selected from several hundreds of collected SCIs as the reference6 SCIs, as they cover a wide variety of combinations on various image contents, including texts, graphics, symbols, patterns, and natural images. The sources of these SCIs could come from web pages, PDF files, power-point slides, comics, digital magazines through screen snapshot, and so on. All of them are cropped to a fixed size, with the resolution of 1280 x 720. To conduct the performance evaluation of our proposed ESIM model, a new and the largest SCI database (denoted as SCID) is established in our work and made to the public for download. Our database contains 1,800 distorted SCIs that are generated from 40 reference SCIs. For each SCI, 9 distortion types are investigated, and 5 degradation levels are produced for each distortion type.

ESIM: Edge-SIMilarity based IQA model for SCI

Fig. 3. The framework of our proposed ESIM for SCI.

The proposed IQA model, edge similarity (ESIM), is used for conducting an objective image quality evaluation of a given distorted SCI, with respect to its reference SCI. The flowchart of the proposed ESIM algorithm is shown in Fig. 3, which consists of three processing stages. In the first stage, a parametric edge model is used to extract two salient edge attributes, edge contrast and edge width. As a result, the extracted edge contrast and edge width information are expressed in terms of maps---i.e., the edge contrast map (ECM) and the edge width map (EWM), respectively. In addition, the edge direction is considered and incorporated into our proposed IQA model. The edge direction map (EDM) will be generated directly from each SCI via our proposed edge direction computation method. In the second stage, the computed edge feature maps, one from the reference SCI and the other from the distorted SCI, will be compared to yield their edge similarity measurement. For example, the two ECMs, respectively obtained from the distorted SCI and the reference SCI, will be compared to arrive at the edge contrast similarity (ECS) map. Likewise, the edge width similarity (EWS) map and edge direction similarity (EDS) map will be generated based on the corresponding pair of EWMs and EDMs, respectively. The three generated similarity measurement maps will be combined to yield one measurement map, which is used as the input of the third, and the last, stage to compute the final ESIM score using our proposed edge-width-based pooling process.

Experimental Results

Table. 1. Performance Comparison of Different IQA models on our Established SCID Database.

From the experimental results documented in Fig 4, one can see that the proposed ESIM model outperforms all the state-of-the-art models under comparison, as it is able to achieve the highest correlation or consistency with the subjective quality ratings judged by the HVS. Note that the proposed ESIM model also significantly outperforms other state-of-the-art edge-based IQA models; i.e., ES, GSIM, and GMSD. This reveals that the proposed ESIM model that exploits three edge attributes (i.e., edge width, edge contrast, and edge direction) is much more effective to explore the edge information for conducting SCI perceptual quality assessment. Another interesting observation is that all the models specifically designed for the SCI quality assessment (i.e., SIQM, SPQA, SQI, EMSQA, and our proposed ESIM) can achieve better performance than those IQA models developed for the naturalimages in SIQAD database. This is mainly due to the fact that they consider the special characteristics of the SCI on the design of SCI quality assessment models.

Fig. 4. Scatter plots of the MOS versus the IQA metrics using our established SCID database: (a) PSNR; (b) SSIM; (c) MSSIM; (d) IWSSIM; (e) VIF; (f) IFC; (g) MAD; (h) FSIM; (i) GSIM; (j) GMSD; (k) VSI; and (l) Proposed ESIM, respectively. The blue-color line as shown in each sub-plot is obtained by using a curve fitting procedure according to (20). One can see that the data points on the scatter plot of the proposed ESIM are much closer to this curve (thus, a better fitting to the curve).

To further visualize the performance yielded by the IQA models under comparison, Fig. 4 shows the scatter plots of the subjective scores against the objective scores as predicted by some representative IQA models (i.e., PSNR, SSIM, MSSIM, IWSSIM, VIF, IFC, MAD, FSIM, GSIM, GMSD, VSI, and the proposed ESIM) based on our SCID database for a demonstration. The curve as shown in each sub-plot of Fig. 4 is obtained by a nonlinear curve fitting procedure according to equation (20) in the paper. One can see that our proposed ESIM has a ‘tighter’ (thus, better) curve fitting, compared with that of other IQA models.

Download

@article{ni2017esim,
	title={ESIM: Edge similarity for screen content image quality assessment},
	author={Ni, Zhangkai and Ma, Lin and Zeng, Huanqiang and Chen, Jing and Cai, Canhui and Ma, Kai-Kuang},
	journal={IEEE Transactions on Image Processing},
	volume={26},
	number={10},
	pages={4818--4831},
	year={2017},
	publisher={IEEE}
}

You can download the SCID as well as the supporting file via the OneDrive: Download SCID

References

Contact

If you have any questions, please feel free to contact Dr. Zhangkai Ni .

Last update: Oct. 7, 2018