Volume 11 Issue 4 - November 6, 2009 PDF
Optimized Scale-and-Stretch for Image Resizing
Yu-Shuen Wang1, Chiew-Lan Tai2, Olga Sorkine3 and Tong-Yee Lee1,*

1Department of Computer Science and Information Engineering, College of Electrical Engineering and Computer Science, National Cheng Kung University
2Department of Computer Science and Engineering, Hong Kong University of Science and Technology
3Department of Computer Science, New York University

Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee,"Optimized Scale-and-Stretch for Image Resizing," ACM Transactions on Graphics (SIGGRAPH Asia 2008 issue), Vol. 27, No. 5, Dec. 2008, Article 118.

Font Enlarge
Project Web Site: http://graphics.csie.ncku.edu.tw/Image_Resizing/

Recently, content aware image resizing is becoming important, since current display devices, such as television, notebooks, PDAs and cell phones, all come in different aspect ratios and resolutions. Traditional methods perform homogeneous resizing without considering the image content, equally propagating the distortion over the entire image and noticeably squeezing prominent objects. To achieve resizing without distortion, many approaches attempt to remove the unimportant information from the image periphery. For example, based on a face detection technique and a saliency measure, the image is cropped to fit the target aspect ratio and then uniformly resized by traditional interpolation. However, cropping methods may potentially remove prominent objects close to the image boundary. Another clear drawback of cropping methods is that important regions may lie at opposite edges of the image—a common problem in real images.

Recently proposed retargeting methods try to retain prominent objects while reducing or removing other image content. Seam carving methods [1,2] reduce the image size in a certain direction by removing monotonic 1D seams of pixels that run roughly in the orthogonal direction (image expansion is achieved by duplicating such seams instead). To reduce artifacts, they search for minimal-cost seams that pass through homogeneous regions by computing their forward [2] or backward energy [1]. These methods produce very impressive results, but their discrete nature may cause noticeable jags in structural objects. Moreover, these two methods only propagate distortion along the resizing direction. So, if the homogeneous information in the required spatial direction runs out, removing seams in that direction to change the aspect ratio would inevitably generate significant distortion.

In this paper [4](see Figure 1), we present a “scale-and-stretch” warping method that allows resizing images into arbitrary aspect ratios while preserving visually prominent features. The method operates by iteratively computing optimal local scaling factors for each local region and updating a warped image that matches these scaling factors as closely as possible. The amount of deformation of the image content is guided by a significance map (see Figure 2) that characterizes the visual attractiveness of each pixel; this significance map is computed automatically using a novel combination of gradient and salience-based measures. Our technique allows diverting the distortion due to resizing to image regions with homogeneous content, such that the impact on perceptually important features is minimized. Unlike previous approaches, our method distributes the distortion in all spatial directions, even when the resizing operation is only applied horizontally or vertically, thus fully utilizing the available homogeneous regions to absorb the distortion. We develop an efficient formulation for the nonlinear optimization involved in the warping function computation, allowing interactive image resizing. Experimental results show our method outperforms previous methods for a variety of images (see Figure 3). More results can be found in http://graphics.csie.ncku.edu.tw/Image_Resizing/
Figure 1: We partition the original image (left) into a grid mesh and deform it to fit the new desired dimensions (right), such that the quad faces covering important image regions are optimized to scale uniformly while regions with homogeneous content are allowed to be distorted. The scaling and stretching of the image content is guided by a significance map which combines the gradient and the saliency maps.

Figure 2: We define the significance map as the product of the gradient magnitude and the saliency measure. Compared with the gradient map, the significance map is less sensitive to the disturbance of trees and leaves, focusing on the old man and the little girl. We compare the results of narrowing the original image using the gradient map and our significance map. The shapes of the old man and little girl are preserved better with our significance map.

Figure 3: Comparison of our results with those of improved seam carving [2], i.e., by Rubinstein et al. 2008, and the warping method of [3], i.e., by Wolf et al. 2007. The results of [3] and our method tend to be smoother than those of seam carving. Notice the discontinuities in the people, San Francisco Heart and the house roof, which are due to the pixels being directly removed. Compared with [3], our method can preserve the aspect ratios of prominent features better.

[1] AVIDAN, S., AND SHAMIR, A. 2007. Seam carving for content aware image resizing. ACM Trans. Graph. 26, 3, 10.
[2] RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3.
[3] WOLF, L., GUTTMANN, M., AND COHEN-OR, D. 2007. Non-homogeneous content-driven video-retargeting. In Proceedings of IEEE ICCV, 1–6.
[4] Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee,"Optimized Scale-and-Stretch for Image Resizing,"  ACM Transactions on Graphics (SIGGRAPH Asia 2008 issue), Vol. 27, No. 5, Dec. 2008, Article 118.
< Previous
Next >
Copyright National Cheng Kung University