Video Screening

Year: 2005 Authors: Jinnah Yu; Ergun Akleman

Core claim

The paper claims video screening extends artistic screening to video images, enabling multiple animations to be visually combined with frame-to-frame coherence and no spatial or temporal aliasing.

Topics

aesthetic images, color halftoning, animated screening, frame coherence, anti-aliasing

Domains

digital halftoning, dithering, error diffusion, spatial sampling, generative art, visual illusion, graphic design, artistic screening

Methods

pattern-based halftoning, control image modulation, grid-based dot placement, hybrid rectangular-hexagonal layout

Media

low-resolution color images, video animations, black-and-white screen images, repeating dot patterns

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

JINNAH YU and ERGUN AKLEMAN

Visualization Sciences Program, Department of Architecture

Texas A&M University, College Station, TX 77843-3137, USA

E-mail: ergun@viz.tamu.edu

Abstract

This paper presents video screening, a new concept for the creation of aesthetic images. We have developed a variety of techniques to create video screening images and animations. Using these techniques we have created a variety of video screening animations. These techniques allows us to create artistic looking images easily. The techniques provide frame to frame coherence, spatial and temporal anti-aliasing in animations.

1. Introduction

This paper a new artistic concept that extends the application area of artistic screening [16, 10] from high-resolution Black & White images to low-resolution color images. Since, most common examples of such low-resolution color images are video images, we call the new artistic concept video screening. The images created by using video screening are called video screening images. Figure 1 shows examples of video screening images.

Figure 1: Example frames from video screening animations.

Similar to artistic screening, in video screening user specified patterns are used to generate halftones. Video screening allows artists to combine two images and one repeating pattern to create one image. These are the following: Goal image. This image defines what the final image will look like, Dot pattern. This is a black and white image that defines the shape of the dot, Control image. This image helps to control the size

and the colors of the dots. The final video screening images look like a composite of repeating dot patterns with different sizes. If the goal image, dot pattern, and control images are replaced by a series of images (animations), we create a series of video screening images, which are called video screening animations.

Differences between artistic and video screening can be classified as follows: (1) Artistic screening is developed for representing halftones with limited number of colors. Therefore, in artistic screening images, the number of colors is severely limited (i.e. the number of inks in printing). On the other hand, in video screenings, there is no practical limitation in the number of colors. This freedom over the number of the colors allows us to control dot color and size independently. (2) artistic screening images can have very high resolution and they are particularly well-suited for large-scale poster production. [16, 10]. On the other hand, the resolution of video images is severely limited.

Here are five requirements we have identified to create successful video screening images. (1) The results should be aesthetic. (2) If we use images to create animations, the results should show clearly the correlation between two animations on the screen. (3) At least two animations should be clearly visible. (4) There should be frame by frame coherence. (5) There should be no spatial or temporal aliasing. These conditions present a variety of interesting and unique technical challenges for generating the video screen images. Based on these requirements, we have developed several mathematical and artistic techniques for the creation of video screening images. Our techniques assure to create aesthetic looking images.

2 Background

Our motivation for this work comes from puzzle-like artworks that creates hidden patterns and unexpected illusions with the unique combination and arrangement of various types of objects. An early pioneer of such artworks is a $16^{th}$ century Italian painter, Giuseppe Archimboldo. He painted human portraits by combining a variety of objects such as animals, flowers, fruits [19]. Until $20^{th}$ century, we do not see many examples such artworks.

During the $20^{th}$ century many artists, such as Ken Knowlton, [14], Zeke Berman [9] and Salvador Dali [5], created a variety of puzzle-like artworks. Most notable of these artists is M.C. Escher who created impossible structures, spatial illusions, and complicated repeating geometric patterns (tessellations) [6, 7]. Artistic screening [16, 10] is motivated by his well-known illustration, “Day And Night”, [6], Escher represented changing illumination from day to night with transition from tesselated black birds to white birds.

The traditional screening techniques, which are also called half-toning and dithering, has been known before Escher. They were developed for printing photographs with various gray tones using limited number of inks in the 1890’s [4]. The term ”screening” came from the etched glass screens used to create traditional halftone images. Traditional halftone images were created by using the etched glass screen with finely ruled grids to divide the image into many small dots. The etched glass screen is placed close to high contrast film and then as exposed to the light, the light source through the screen remains dots of varying sizes on the high contrast film. The sizes of dots under the glass screen are proportional to the amount of the light source passing through the screen grids. With this processing, the original photograph converts to dots of varying size on the high contrast film, and these small dots represent different intensity areas of a photograph.

Digital half-toning was introduced to play a similar role to traditional halftone for digital media. During the last two decades a variety of digital half-toning methods have been developed to create variable intensity level images using limited palettes and resolutions [11]. These algorithms can roughly be classified as

dithering [15, 20, 3, 12, 21] and error diffusion [8, 13].

Artistic screening, which was developed by Ostromoukhov and Hersch in 1995, allow unlimited resolution and provide a new concept of half-toning [16]. Since there is no limit on resolution, artistic screening can allow to adapt a wide variety of shapes as screen dots, such as letter shapes, artistic patterns and ornamental shapes, to generate halftones. This freedom of adapting any pattern allows images of high aesthetic quality to be created. These techniques can be useful for printing large posters, credit cards and banknotes [16].

In this paper, we present we present video screening, a new concept that is inspired by artistic screening. In video screening we combine more than two animations within one screen in such a way that two animations will be clearly visible.

3. Foundations

This section introduces foundations for the development of video screening. Similar to artistic screening, we first create a Black & White Screen Image, $I_S$ , that consists of repeating dot patterns. Repeating patterns are usually created by placing dot patterns inside of rectangular or hexagonal grids [15, 20, 3, 12] (see Figure 2). Hexagonal grids are visually more desirable since dot patterns placed in rectangular grids creates an illusion of vertical or horizontal lines. On the other hand, using rectangular grids greatly simplifies algorithms. In video screening, we use a hybrid solution: We create an illusion of hexagonal grid on a rectangular grid by organizing dot patterns as shown on the Figure 3.

Figure 2: Repeating dot patterns.

In creating Screen Image we change the sizes of dot patterns. Let $a$ denote the ratio of area of dot pattern to the area of the rectangle. This ratio is a number between 0 and 1. The value of $a$ can be controlled by scaling the dot patterns. If there is no overlapping of dot images, the value of $a$ can easily be computed based on the scaling and ratio of black and white regions. Even if there is an overlapping, it is still possible to compute the value of $a$ . However, overlapping dot patterns do not look aesthetic. To solve this problem, when overlapping occurs we switch to white dot pattern over black background. As a result, using a single dot pattern we create a Black & White Screen Image that consists of dots with changing sizes (see Figure 3).

Our goal is to create a final color image $I_F$ that satisfy two conditions: (1) $I_F$ must have the same pattern in $I_s$ ; (2) $I_F$ must look like any given goal image $I_G$ when we squint our eyes. The first condition can be achieved by replacing black and white colors with two locally different colors. The second condition can be

Figure 3: Screen image formation from a given dot pattern with fake hexagonal grid and changing dot sizes.

expressed by the following equation:

where $C_G(K,L)$ is the color of the pixel $(K,L)$ the goal image $I_G$ and $C_F(i,j)$ is the color of the pixel $(i,j)$ of the final image $I_F$ , and $2N+1$ is the size of the square regions of rectangular grid. Without loss of generality, we assume that color $C$ is a real number between 0 and 1, and it represents either red, green or blue.

We now assume that there exists only two distinct colors inside of any $(2N+1)\times (2N+1)$ square region in $I_F$ . Let these two colors are called dot color, $C_D$ , and background color, $C_B$ . These colors come from replacing black and white colors of screen image $C_F(i,j)=C_B$ if $C_S(i,j)=1$ and $C_F(i,j)=C_D$ if $C_S(i,j)=0$ . Under this assumption, equation 1 can simplify as follows:

In this equation, only goal color $C_G$ is given. The rest of the parameters, i.e. $C_B$ , $C_D$ and $a$ , can be freely chosen as far as they satisfy the equation 2. This freedom gives us two methods to provide users an intuitive control using a control image $I_C$ .

3.1. Direct control of the dot color

In this case, $C_D(i,j)$ is chosen to be $C_C(i,j)$ where $C_C(i,j)$ is the color of the pixel $(i,j)$ of the control image $I_C$ . Then, the background color is computed as a function of the goal color, background color and $a$ based on equation 2. Since $a$ can also be chosen by the users, this provides us an additional control. However, the value of $a$ cannot directly be controlled. Instead we use an indirect control. First note that the relationship between $a$ and $C_B$ would be $a=\frac{C_G-C_B}{C_D-C_B}$ . This equation must be between 0 and 1 as well. According to this condition, if C_D > C_G then $C_G$ must be larger than $C_B$ . If C_D < C_G then $C_G$ must be smaller than $C_B$ . Based on this observation, we can provide an additional control as

where $t$ is a user specified number again between $0$ and $1$. In this equation, $t=0$ gives a result like traditional screening and $a$ gets maximum value. On the other hand, $t=1$ makes $a=0$ and does not create any screening. In other words, the value of $t$ controls both the size of dots and amount of blending between goal image and dot image. As the dot size gets smaller it creates more blending. We generally use a $t$ value around $0.5$.

We also try to keep $a$ value smaller than $0.5$. This is also easy to get. If $a\ge 0.5$ then we simply use $C_D$ as background color and $C_B$ as dot color. Thus, the area of the dot never exceeds $50\%$ of its region.

3.2 Direct control of the dot size

In this case, In this case, $a(i,j)$ is chosen to be $C_C(i,j)$ and $C_B$ and $C_D$ are computed to choose maximum possible color. We have identified two possible cases. Without loss of generality, we assume that $1-a\ge a$, also let $b=1-a$.

1. In this case, we want to choose maximum possible color for $C_D$. Then the equation becomes

$C_D=1$ and $C_B=\frac{C_G-a}{b}$ if $C_G\ge a$ $C_B=0$ and $C_D=\frac{C_G}{a}$ otherwise 2. In this case, we want to choose minimum possible color for $C_D$. Then the equation becomes

$C_B=1$ and $C_D=\frac{C_G-b}{a}$ if $C_G\ge b$ $C_D=0$ and $C_B=\frac{C_G}{b}$ otherwise

The first case allows dots to look brighter, the second one makes the dots look darker. Since we can apply a different method for each channel, we can make dots looks red, green, blue, green+blue, red+blue and red+green in addition to white and black. This method enable us to create distinctive variation of dot size and coherent color dots through the whole image.

4 Artistic Methods

We have developed a prototype system to create artistic video screenings. Our system is written in C++. It reads three image files: (1) goal image, (2) dot pattern, (3) control image. To compute colors, we use RGB space, i.e., we compute $R$, $G$ and $B$ separately by using the equations presented earlier. Although the other color spaces can effectively be used, we did not observe any problem with using RGB. We have developed three artistic methods to give intuitive control with acceptable video screenings over the results.

4.1 Using Complementary Colors for Dot and Background

This method is based on the direct control of dot color. The control images are created by reducing a complementary color from a given goal image. Therefore, in this case, $C_D\le C_G$ and we compute $C_B=t+(1-t)C_G$. If we choose $t\simeq C_D/C_G$ then $C_B$ becomes the complement of $C_D$, i.e. the color of the background automatically becomes $C_D$’s complementary. The Figure 4 shows a gray-scale goal image converted to a video screening image using the complementary color method. Blue image as control image was created by eliminating its complementary color,yellow.

Figure 4: (A) is the goal image, (B) is the control image that is a blue image that is created by eliminating its complementary color (in this case yellow), (C) and (E) are images that define dot patterns, (D) and (F) are video screening images that is created by using dot shapes in (C) and (E) respectively.

Figure 5 show another example in which a color photograph is used as goal image. The resulting image shows the effect caused by using a blue tone image as a control image which was created by eliminating yellow from a goal image for each control image. From result images, you can see that the eliminated colors from control images appear in the dots. Using complementary colors is a common method in painting [22] to create aesthetic results. As clearly seen in the examples in this section, this method creates a very good local palette and images created by this method look much more aesthetically interesting than original goal images.

Figure 5: An example of using complementary colors. In this example, blue control image is created by eliminating its complementary color (in this case yellow).

4.2. Indirect Control of the Dot Size

Using direct control of dot color, it is still possible to indirectly change the sizes of dots. For instance, if the color of the control image is the same as the color of the goal image ina region, dots shrink and become invisible, i.e., dots are completely eliminated in that region.

4.3. Direct Control of the Dot Size

The sizes of dots can also be directly defined by the control animation, and then the color of background on each square region is calculated by the ratio of dot sizes. Figure 6 shows how this method works for generating images. In Figure 6 we choose maximum possible color for the dot color. As seen in Figure 6, we chose the boundaries of objects on the control image are darker. Based on our equations, the darker the area is, the smaller the sizes of dots becomes. Thus, our system creates very clear boundaries in the result image as seen in Figure 6.

Goal image

Dot pattern

Control image Figure 6: An example that shows the effect of control images for direct dot size control in a one channel (Black & White) image.

Final video screening image

5. Conclusion and Future Work

Video screening allows to create unique and interesting animations and images. Unlike the conventional screening techniques this method allows unlimited number of colors and animated shapes for elements in video resolution. This method enables the user to create complex moving screenings by three different animations which are freely chosen by the user. Video screening could convey an artistic charm by allowing to view several animations simultaneously. In the future, we plan to use freely moving animated dots that goes beyond static regular tessellations since freely floating animated dots can provide more variety of illusions.

References

[1] A. Atsalakis, N. Kroupis, D. Soudris, and N. Papamarkos, A Window-based Color Quantization Technique and Its Architecture Implementation, http://ipml.ee.duth.gr/ papamark/wsofm.htm (1999) [2] Adobe Systems Incorporated, Print Publishing Technical Guides Scanning and Halftones: webpage www.adobe.com/support/techguides/printpublishing/scanning/psscanning. (2000) [3] B.E. Bayer, An optimum method for two level rendition of continuous-tone pictures. Proc. IEEE Int. Conf. Commun, Conference Record, 1973, (26-11)-(26-15). [4] A. Donnelly, Halftoning basics, webpage: www.cl.cam.ac.uk/ and1000/newsprint/halftone. (1998) [5] S. Dali and D. Ades, Dali’s Optical Illusions Yale Univ Press (January 2000). [6] B. Ernst and M. C. Escher, Magic Mirror of M. C. Escher Taschen America Llc; (February 1995). [7] M. C. Escher and J.L. Locher, Infinite World of M.C. Escher Abradale Press; (May 1984). [8] R. W. Floyd and L. Steinberg, An Adaptive Algorithm for Spatial grayscale. SID Symposium, 1975, 36-37.

[9] D. Heimerdinger, Optics: Zeke Berman, Friends of Photography Bookstore; (June 1992), San Francisco, CA, U.S.A. Also see: www.laurencemillergallery.com/bermanlist.htm and www.exploratorium.edu/exhibits/postcard_illusions/.

• [10] R. D. Hersch, Peripheral Systems Laboratory(EPFL/LC-LSP) Microstructure Imaging: ArtScreen : Artistic Screening http://diwww.epfl.ch/w3lsp/research /microstructureimaging/ (1995).
• [11] P.S. Heckbert , 1982. Color image Quantization for frame buffer display, in Proceedings of SIGGRAPH’82, in Computer Graphics Proceedings, vol. 16, n. 3, 297-307
• [12] J. Jarvis, C. Judice, and W. Ninke, 1976. A Survey of Techniques for The Display of Continuous Tone Pictures on Bilevel Displays. Computer Graphics and Image Processing, n. 5, 13-40.
• [13] D. Knuth, 1987. Digital Halftones by Dot Diffusion. ACM Transactions on Graphics, V. 6 N. 4, 245-273.
• [14] K. Knowlton, www.knowltonmosaics.com.
• [15] J. O. Limb, 1996. Design of Dither Waveforms for Quantized Visual Signals, Bell Systems Technical Journal, v.48, n. 7, 2555-2582.
• [16] V. Ostromoukhov, R. D. Hersch, 1995. Artistic Screening, in Proceedings of SIGGRAPH’95, in Computer Graphics Proceedings, Annual Conference Series, 219, 221-222
• [17] V. Ostromoukhov, and R. D. Hersch, 1999. Multi-Color and Artistic Dithering, In Proceedings of SIGGRAPH 99, in Computer Graphics Proceedings, Annual Conference Series, 425-432
• [18] N. Rudaz, R. D. Hersch, V. Ostromoukhov, 1998. An Interface for The Interactive Design of Artistic Screens, In: Electronic Publishing, Artistic Imaging and Digital Typography, Lecture Notes in Computer Science 1375, Springer Verlag, 1-10
• [19] C. Strand, Hello, Fruit Face!: The Paintings of Guiseppe Arcimboldo (Adventures in Art), Prestel USA; (March 1999).
• [20] R. Ulichney, Digital Halftoning. MIT Press, Cambridge, Mass., 1987
• [21] L. Velho , J. Gomes, 1991. Digital halftoning with space filling curves, In Proceedings of SIGGRAPH 91, in Computer Graphics Proceedings, Annual Conference Series, 81 - 90
• [22] C. Willard, Watercolor Mixing, the 12-Hue Method Rockport Publishers; (April, 2000).