Center for Digital Innovations (CDI)

Associate Director Steven Peterson

3871 Slichter Hall

156704

Allen Klinger

Panoramic and multi-viewpoint images enable three-dimensional reconstruction of solid models of scenes. Digital computer programs process the constituent images. In some situations these programs are made efficient by special purpose hardware. That kind of hardware implements video capture and processing functions. The value of resultant three-dimensional information and the low cost of imaging devices makes research in this area likely to have application benefits.

Most panoramic techniques are either fisheye or use adjoined images stitching based. There are significant technical challenges in capturing and processing the huge amount of video data in the existing systems for acquiring multi-viewpoint data. The result is that current technology can only produce still images: it cannot handle time-varying or video data.

This proposal proceeds in cooperation with a commercial company that owns proprietary technology. That technology includes techniques to capture both panoramic and multi-viewpoint imagery. The company, Reality Commerce Corporation (RCC), will supply technical knowledge, information and practical experience. That information will be based on their systems - see description below (System Architecture).

We will be conducting joint research regarding enabling new methods for handling video imagery. That activity will be based on specific RCC technology, including proprietary real-time data compression.

Two RCC patents pending are the foundation of the research:

1. Parallel Multi-Viewpoint Video Capturing and Compression (PMVCC): Method and Apparatus, U.S. Ser. No. 60/191,721.

2. Subject Video Streaming: Methods and Systems, U.S. Ser. No. 60/191,754.

Several media processors are capable of capturing and compressing one or two channels of real-time video images. We propose in depth study of technical and economic aspects of designing and implementing a multiple input multiple data (MIMD) video parallel processing system, in order to extend that capability to the massive information handling needed in practical immersion or three-dimensional imaging situations. We further will investigate related compression algorithms for such hardware.

Online entertainment — interactive multi-viewpoint 3D video images of live performances, sports, concerts and theatre presentations.

E-commerce Industry — e-commerce platforms for businesses and consumers through web host designers, service providers and large retailers.

Online and Training and e-learning Industry — interactive 3D training techniques tailored to distance learning, training and customer support facilities.

Surveillance and Remote Sensing — agricultural crops and natural disaster monitoring from air and space.

We have identified several competing technologies with products tailored to the 3D image and video marketplace. Currently, there are no competing 3D multi-viewpoint video imaging solutions available. Competing technologies include the following methods:

Digital Image Capture — several companies have developed hardware capable of overlapping a series of still images and stitching them together to form a flat panoramic image. The process is time consuming as the process is not automated and requires tedious human intervention. In contrast, RCC’s 3D multi-viewpoint image capturing device is a fully automated high performance capture device that can be used for both still image and video production without stitching artifacts.

Model Based Rendering Techniques — software modeling technology has advanced and several companies are aggressively developing graphic renderings of animated objects for e-commerce marketing. The process is extremely time consuming and expensive to complete as a model must be developed for each image. Perception is also a problem as the final product is an animated view of the source object.

Panoramic 3D Images — several groups have concentrated on the development of an optical system capable of capturing a 360 degree image around the camera in a single frame. This avoids the need of stitching, however, the image suffers from nonlinear distortion or "fish eye" viewing. Multi-viewpoint capture and content production is more challenging than panoramic content. Panoramic content captures only one viewing position. Multi-viewpoint content demands the capture and realization of picture taking at multiple spatial positions.

Mr. Ping Liu, M.Sc, Post Engineer in UCLA, Director of Hardware Development, Reality Commerce Corp,.

(if resume is needed please let me know)

1. National Tele-immersion Initiative Web site:

http://http://www. advanced.org/teleimmersion.html

2. Tele-immersion at Brown University:

http://www. cs.brown.edu/~lsh/telei.html

Andries van Dam, Loring Holden, Robert C. Zeleznik

http://www. cs.brown.edu/~lsh/telei.html

3. Tele-immersion at the University of North Carolina at Chapel Hill:

http://www.cs.unc.edu/Research/stc/teleimmersion/

Team Members: Henry Fuchs, Herman Towles, Greg Welch, Wei-Chao Chen, Ruigang Yang, Sang-Uok Kum, Andrew Nashel, Srihari Sukumaran

http://www. cs.unc.edu/Research/stc/ teleimmersion/

4. Tele-immersion at the University of Pennsylvania:

http://www.cis.upenn.edu/~sequence/teleim1.html

Ruzena Bajcsy, Kostas Daniilidis, Jane Mulligan, Ibrahim Volkan Isler

http://www. cis.upenn.edu/~sequence/teleim2.html

5. Tele-immersion site at Internet2:

http://www.internet2.edu/html/tele-immersion.html

6. Advanced Networks and Services:

http://www.advanced.org/teleimmersion.html

Jaron Lanier, Amela Sadagic