Evaluating Multiple Viewpoint Images

Allen Klinger

Background

There are numerous uses for picture information derived from different places. They range from three-dimensional reconstruction of solid models to interpretation of object geometry. Computer programs to process images and build realistic models are more important than ever due to declining cost of imaging and digital processing devices. There are many very likely application benefits from work on those programs.

The use of many digital cameras simultaneously creates an image base. The visual data there forms a valuable resource. One needed item to increase the value of such an image base is a set of reliable image quality measures. In order to address the diverse purposes of such an image base we need to put into place practical evaluations based on practice. That involves processing many images derived from video cameras engaged in purposeful activities. One such activity involves gaining a panoramic overview. Another creates a set of varied standpoint scene images.

Algorithms

One main issue is combining multiple views. Procedures suitable for teaching and entertainment applications are needed (still images; moving pictures), but even more important is the ability to interface with a stored base of many images. One aspect involves combining slightly different viewpoints. The typical use of image-acquisition cameras places the cameras producing the views at places from about five to ten degrees apart. In cases where two images taken from slightly different viewpoints are used to estimate a third image in between them, the source images are ten to twenty degrees apart.

We propose to develop new image handling algorithms to address the key concerns. They fit into three main areas: 1. Reducing the burden on a human when there is a fairly large number of cameras in a system; 2. Characterizing the communication bandwidth needed to convey images over networks to remote sites; and, 3. Rating the quality and complexity of available pictures.

Applications

Technical change makes availability of software the most important bottleneck in terms of bringing into the market products in the following domains: 1. Online Entertainment — Means for presentation of interactive multi-viewpoint three-dimensional video images of live performances; examples involve concerts, sporting/theatrical events. 2. Commercial Internet — Devise new visual platforms for web designers, service providers and large retailers to offer goods to individuals at a distant location. 3. Training and Distance Learning — Develop interactive three-dimensional problem support techniques. Tailor education systems to customer needs. 4. Image Support - Facilities for the disabled; tools for a film director; enabling the synthesis of diverse sensor sources for ground modeling.

Individuals in education, industry (including entertainment firms), and government have expressed interest in past achievements. This proposal is designed to create new software to support expanded interaction with those individuals and organizations. Expansion of published items could yield new proposals.

Research

Past research reported in papers [1, 2] and a book [3] concerned two issues. The first has to do with determining how to make local measurements within image domains and from them obtain global information that characterizes the value of the picture itself to a human observer. In [2] that work resulted in numerical measures associated with information, and described by ordinary generally understood words such as temperature, harmony and life (liveliness). Those recent results (in [2]) show how to produce or extract useful images from a set: usefulness involves the question of acceptability by humans.

That humans are part of the system since they are the observers of the pictorial image is itself the key issue discussed in [3]. The book [3] collects papers with a common theme, namely, user-impact. It puts forward a system characteristic: the human is a central actor. One way people interact with images is by saying or writing a description of things seen. Twelve textual files [4], over five thousand words, gives this project, a linguistic as well as a numerical character for image description. Nevertheless, the research is for the development of software tools to help people characterize by quantitative means, compare, and generally work with multiple viewpoint images. Description may be an added aspect with respect to structure, either arbitrary or detected in an image by means presented in [1, 2]. Mathematics education [5, 6], and assessing accomplishment [7] could be accomplished by using images and inter-net software tools.

References

[1] Klinger, A., "Searching Images for Structure," Structured Computer Vision, Tanimoto, S., Klinger, A. eds., NY: Academic Press, 1980, 151-167.

[2] Klinger, A., Salingaros, N., "A Pattern Measure," Environment and Planning B: Planning and Design 2000, 27-4, pp. 537-547, July 2000.

[3] Klinger, A. ed., Human Machine Interactive Systems, NY: Plenum Press, 1991.

[4] Klinger, A., http://www.cs.ucla.edu/~klinger/tenpp/index2.html; also replace last eleven characters by:

1_emerson.html, 2_do_it.html, 3_work.html, 4_mistakes.html, 5_human.html, 6_faith.html, 7_road.html

8_learn.html, 9_genius.html, 10_truth.html, 11_einstein.html, or 12_life.html

[5] Klinger, A., http://www.cs.ucla.edu/~klinger/math.html, Overview of Accessible Mathematical Books.

[6] Klinger, A., http://www.cs.ucla.edu/~klinger/tet1.jpg, Solid Model Image Constructed Using Mathematica.

[7] Klinger, A., "Experimental Validation of Learning Accomplishment," UCLA CSD Technical Report Number 970019, Proc. Conf.  Frontiers in Education, 1997. See http://www.p-mmm.com/ (Explorers).

Participants

Dr. Allen Klinger, Professor Emeritus, Computer Science Department, UCLA; Postgraduate Research Engineers and Industrial Associates [e.g., Mr. Ping Liu, M.S. Univ. Texas (Austin), Director, Hardware Development, Reality Commerce Corp (Vancouver, British Columbia, Canada); see [4], 4_mistakes.html, "Coming together is a beginning; keeping together is progress; working together is success. - Henry Ford"].

Schedule

Month 1: Initiate Staffing. Month 2: Install Video and Computer Hardware

Month 3,4: Create Image Processing Efficiency and Quality Measures

Month 5~7: Algorithm Design Month 8~9: Prepare Research Report

Months 10-12: Preparation of Proposals

Budget Total: $ 24,000.00

Desk-Top Computers: $ 5,000.00 (2 @ $ 2,500 each) Video Capturing Cards: $ 1,000.00

Video Cameras: $ 1,000.00 (2 @ $ 500 each)

Travel: $ 3,000.00 Investigator's Salary: $ 5,000.00 Software: $ 1,000.00 Student Researchers' Salary: $ 8,000.00

Appendix A

A labeling/categorizing function akin to the names of the files listed in [4] above can help incorporation of images or portions of them into video camera source image databases. Four examples, two from ethnographic art, the others from Microsoft clipart (see two PowerPoint presentation items below) follow.