Gradient Direction for Screen Content Image Quality Assessment

Zhangkai Ni1, Lin Ma2, Huanqiang Zeng1, Canhui Cai1, and Kai-Kuang Ma3

1School of Information Science and Engineering, Huaqiao University, Fujian, China

2Huawei Noah's Ark Lab, Hong Kong

3School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

Paper | Code

Abstract

In this letter, we make the first attempt to explore the usage of the gradient direction on the perceptual quality assessment of the screen content image (SCI). Specifically, the proposed approach firstly extracts the gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI. A deviation based pooling strategy is subsequently utilized to generate the corresponding image quality index. Moreover, we investigate and demonstrate the complementary behaviors of the gradient direction and magnitude for SCI quality assessment. By jointly considering them together, our proposed SCI quality metric outperforms the state-of-the-art quality metrics in terms of correlation with human visual system perception.

The contributions of this work:

  • The gradient direction presents an important attribute of the edge. The proposed approach to compute the image gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI.
  • Incorporating gradient direction with magnitude. The gradient magnitude be jointly utilized to describe the perceptual quality degradation of SCI. Experimental results on public SCI database show that our proposed IQA metric outperforms the state-of-the-art quality metrics.
Gradient Similarity Score for SCI Quality Assessment

Dummy Image

Fig. 1. The SCIs and the corresponding direction distributions.

The framework of our proposed IQA for SCI is illustrated in Fig. 1. Our proposed IQA firstly computes the gradient magnitude and direction of the reference and distorted SCI, respectively. The deviation pooling is then used to pool the magnitude and direction similarities together to a final quality score.

Dummy Image

Fig. 2. (a) Original SCIs; (b)–(d) are the distorted versions with Gaussian blurring, motion blurring, contrast change, respectively; (e)–(h) are the corresponding gradient direction distributions of (a)–(d) (blue bars) and the distribution differences between original and distorted versions (red bars).

Observations: The distribution of gradient magnitude from an SCI is normally quite different from that of a natural image.

Dummy Image

Fig. 3. The gradient magnitude of SCI (red curve) versus gradient magnitude of natural image (blue curve) in the log domain.

Gradient Direction Computation: The gradient direction map is calculated in terms of twelve directions from 0 to π by individually convolving the SCI with different convolution kernel at different directions. After obtaining the gradient direction maps for all twelve directions, the final gradient direction map is generated by selecting the maximum value among the responses overall directions. Fig. 2 demonstrates an example of SCI with Gaussian blurring, motion blurring, contrast change. The blue bars denote the gradient direction distributions of the corresponding SCIs, while the red bars indicate the difference between the original and distorted SCIs.

Experimental Results

Table. 1. Performance Comparison of Different IQA models on the SIQAD Database.

Dummy Image

Table I lists the PLCC, SROCC, and RMSE values of the proposed and other IQA metrics using the test SCIs drawn from the database SIQAD. Overall, GSS performs the best as the objective measurement from this metric is mostly consistent with the subjective judgment made by the HVS. The performance of the VIF metric is the second best. Compared with the other metrics based on the gradient magnitude (e.g., GSIM and GMSD), our proposed GSS witnesses an evident advantage that the gradient direction plays an important role for SCI quality assessment. And the success of GSS actually stems from the complementary behavior of the gradient direction to the gradient magnitude.

Download

If you use the code in your research, we kindly ask that you reference this website and our paper listed below.

Zhangkai Ni, Lin Ma, Huanqiang Zeng, Canhui Cai, Kai-Kuang Ma, "Gradient Direction for Screen Content Image Quality Assessment," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1394-1398, August 2016. Paper | Code | BibTex

@article{ni2016gradient,
	title={Gradient direction for screen content image quality assessment},
	author={Ni, Zhangkai and Ma, Lin and Zeng, Huanqiang and Cai, Canhui and Ma, Kai-Kuang},
	journal={IEEE Signal Processing Letters},
	volume={23},
	number={10},
	pages={1394--1398},
	year={2016},
	publisher={IEEE}
}

References

GSS
Z. Ni, L. Ma, H. Zeng, C. Cai, and K.-K. Ma, "Gradient Direction for Screen Content Image Quality Assessment," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1394-1398, Aug. 2016. Paper
SSIM
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. Paper
MSSIM
Z. Wang, E. P. Simoncelli, and A. C. Bovil, "Multi-scale Structural Similarity for Image Quality Assessment," IEEE Conference on Signals, Systems and Computers, vol. 2, pp. 1398–1402, November 2003. Paper
IWSSIM
Z. Wang, Q. Li, "Information Content Weighting for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 5, pp. 1185-1198, Apri. 2011. Paper
VIF
H. R. Sheikh, A. C. Bovik, "Image Information and Visual Quality," IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, Feb. 2006. Paper
IFC
H. R. Sheikh, A. C. Bovik, and G. de Veciana, "An Information Fidelity Creation for Image Quality Assessment Using Natural Scene Statistics," IEEE Transactions on Image Processing, vol. 14, no. 12, pp. 2117– 2128, December 2005. Paper
MAD
E. C. Larson, D. M. Chandler, "Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy," Journal of Electronic Imaging, vol. 19, no. 1, Jan. 2010. Paper
FSIM
L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 8, pp. 2378-2386, Aug. 2011. Paper
GSIM
A. Liu, W. Lin, and M. Narwaria, "Image Quality Assessment Based on Gradient Similarity," IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1500-1512, Apr. 2012. Paper
GMSD
W. Xue, L. Zhang, X. Mou, and A. C. Bovik, "Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index," IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 684-695, Feb. 2014. Paper
VSI
L. Zhang, Y. Shen, and H. Li, "VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 23, no. 10, pp. 4270-4281, Oct. 2014. Paper
SIQAD
H. Yang, Y. Fang, and W. Lin, "Perceptual Quality Assessment of Screen Content Images," IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4408-4421, Aug. 2015. Paper | Download
Contact

If you have any questions, please feel free to contact Dr. Zhangkai Ni .

Last update: Oct. 7, 2018

Top