Gradient Direction for Screen Content Image Quality Assessment

Abstract

In this letter, we make the first attempt to explore the usage of the gradient direction on the perceptual quality assessment of the screen content image (SCI). Specifically, the proposed approach firstly extracts the gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI. A deviation based pooling strategy is subsequently utilized to generate the corresponding image quality index. Moreover, we investigate and demonstrate the complementary behaviors of the gradient direction and magnitude for SCI quality assessment. By jointly considering them together, our proposed SCI quality metric outperforms the state-of-the-art quality metrics in terms of correlation with human visual system perception.

The contributions of this work:

• The gradient direction presents an important attribute of the edge. The proposed approach to compute the image gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI.
• Incorporating gradient direction with magnitude. The gradient magnitude be jointly utilized to describe the perceptual quality degradation of SCI. Experimental results on public SCI database show that our proposed IQA metric outperforms the state-of-the-art quality metrics.

Gradient Similarity Score for SCI Quality Assessment

Fig. 1. The SCIs and the corresponding direction distributions.

The framework of our proposed IQA for SCI is illustrated in Fig. 1. Our proposed IQA firstly computes the gradient magnitude and direction of the reference and distorted SCI, respectively. The deviation pooling is then used to pool the magnitude and direction similarities together to a final quality score.

Fig. 2. (a) Original SCIs; (b)–(d) are the distorted versions with Gaussian blurring, motion blurring, contrast change, respectively; (e)–(h) are the corresponding gradient direction distributions of (a)–(d) (blue bars) and the distribution differences between original and distorted versions (red bars).

Observations: The distribution of gradient magnitude from an SCI is normally quite different from that of a natural image. Fig. 3. The gradient magnitude of SCI (red curve) versus gradient magnitude of natural image (blue curve) in the log domain. Gradient Direction Computation: The gradient direction map is calculated in terms of twelve directions from 0 to π by individually convolving the SCI with different convolution kernel at different directions. After obtaining the gradient direction maps for all twelve directions, the final gradient direction map is generated by selecting the maximum value among the responses overall directions. Fig. 2 demonstrates an example of SCI with Gaussian blurring, motion blurring, contrast change. The blue bars denote the gradient direction distributions of the corresponding SCIs, while the red bars indicate the difference between the original and distorted SCIs.

Experimental Results

Table. 1. Performance Comparison of Different IQA models on the SIQAD Database.

Table I lists the PLCC, SROCC, and RMSE values of the proposed and other IQA metrics using the test SCIs drawn from the database SIQAD. Overall, GSS performs the best as the objective measurement from this metric is mostly consistent with the subjective judgment made by the HVS. The performance of the VIF metric is the second best. Compared with the other metrics based on the gradient magnitude (e.g., GSIM and GMSD), our proposed GSS witnesses an evident advantage that the gradient direction plays an important role for SCI quality assessment. And the success of GSS actually stems from the complementary behavior of the gradient direction to the gradient magnitude.

Download

If you use the code in your research, we kindly ask that you reference this website and our paper listed below.

@article{ni2016gradient,
	title={Gradient direction for screen content image quality assessment},
	author={Ni, Zhangkai and Ma, Lin and Zeng, Huanqiang and Cai, Canhui and Ma, Kai-Kuang},
	journal={IEEE Signal Processing Letters},
	volume={23},
	number={10},
	pages={1394--1398},
	year={2016},
	publisher={IEEE}
}

References

Contact

If you have any questions, please feel free to contact Dr. Zhangkai Ni .

Last update: Oct. 7, 2018