Center for Digital Innovations (CDI)

Associate Director Steven Peterson

3871 Slichter Hall


(Ping, Please eliminate all bullets/dots at left in the item below.)

Capture and Compression of Multi-Viewpoint Video

Allen Klinger


Panoramic and multi-viewpoint images enable three-dimensional reconstruction of solid models of scenes. Digital computer programs process the constituent images. In some situations these programs are made efficient by special purpose hardware. That kind of hardware implements video capture and processing functions. The value of resultant three-dimensional information and the low cost of imaging devices makes research in this area likely to have application benefits.

Most panoramic techniques are either fisheye or use adjoined images stitching based. There are significant technical challenges in capturing and processing the huge amount of video data in the existing systems for acquiring multi-viewpoint data. The result is that current technology can only produce still images: it cannot handle time-varying or video data.

This proposal proceeds in cooperation with a commercial company that owns proprietary technology. That technology includes techniques to capture both panoramic and multi-viewpoint imagery. The company, Reality Commerce Corporation (RCC), will supply technical knowledge, information and practical experience. That information will be based on their systems - see description below (System Architecture).

We will be conducting joint research regarding enabling new methods for handling video imagery. That activity will be based on specific RCC technology, including proprietary real-time data compression.

Improved System Architecture

Two RCC patents pending are the foundation of the research:

1. Parallel Multi-Viewpoint Video Capturing and Compression (PMVCC): Method and Apparatus, U.S. Ser. No. 60/191,721.

2. Subject Video Streaming: Methods and Systems, U.S. Ser. No. 60/191,754.

Accelerated Algorithmic Processing

Graphics-based approaches to creating multi-viewpoint images involve first establishing a geometric model. This is followed by a texture rendering process. In contrast, the proposed system is based on individual images. This approach uses the actual images. They are taken simultaneously at multiple cameras. Each camera captures a different viewpoint.

The processing of individual images improves the overall system performance. Nevertheless the key is the careful registration and that is based on data structures and algorithms [1]. Where graphics-based methods are usually not real-time, our methods have high processing speed. The image-based system we propose can process both still image and moving pictures.

Several media processors are capable of capturing and compressing one or two channels of real-time video images. We propose in depth study of technical and economic aspects of designing and implementing a multiple input multiple data (MIMD) video parallel processing system, in order to extend that capability to the massive information handling needed in practical immersion or three-dimensional imaging situations. We further will investigate related compression algorithms for such hardware.

Applications and Markets

Online entertainment — interactive multi-viewpoint 3D video images of live performances, sports, concerts and theatre presentations.

E-commerce Industry — e-commerce platforms for businesses and consumers through web host designers, service providers and large retailers.

Online and Training and e-learning Industry — interactive 3D training techniques tailored to distance learning, training and customer support facilities.

Surveillance and Remote Sensing — agricultural crops and natural disaster monitoring from air and space.

As soon as I have an agreement with RCC I will supply information about this added application, possibly leading to a future source of business.



We have identified several competing technologies with products tailored to the 3D image and video marketplace. Currently, there are no competing 3D multi-viewpoint video imaging solutions available. Competing technologies include the following methods:

Digital Image Capture — several companies have developed hardware capable of overlapping a series of still images and stitching them together to form a flat panoramic image. The process is time consuming as the process is not automated and requires tedious human intervention. In contrast, RCC’s 3D multi-viewpoint image capturing device is a fully automated high performance capture device that can be used for both still image and video production without stitching artifacts.

Model Based Rendering Techniques — software modeling technology has advanced and several companies are aggressively developing graphic renderings of animated objects for e-commerce marketing. The process is extremely time consuming and expensive to complete as a model must be developed for each image. Perception is also a problem as the final product is an animated view of the source object.

Panoramic 3D Images — several groups have concentrated on the development of an optical system capable of capturing a 360 degree image around the camera in a single frame. This avoids the need of stitching, however, the image suffers from nonlinear distortion or "fish eye" viewing. Multi-viewpoint capture and content production is more challenging than panoramic content. Panoramic content captures only one viewing position. Multi-viewpoint content demands the capture and realization of picture taking at multiple spatial positions.


Research Objectives:

Design and analyze MIMD video parallel processing system architecture.

Design and analyze compression algorithm.

Compare the solution we proposed with other completing technologies

Research Team

Dr. Allen Klinger, Professor of computer science department, UCLA

Mr. Ping Liu, M.Sc, Post Engineer in UCLA, Director of Hardware Development, Reality Commerce Corp,.

(if resume is needed please let me know)

Research Schedule

Month 1: more investigation on related research

Month 2~4: Media Parallel Processing System Hardware Architecture Design and analysis

Month 5~7: Parallel video compression algorithm design and analysis

Month 8~9: Research project reporting

Research Project Cost

Desk-top Computer: $2,500.00

Travel : $8,000.00

Literature and industrial standard purchasing: $1,000.00

Video capturing card: $1,000.00

Two Video Camera: $1,000.00($500 each)

Video utility software: $3,000.00

Total: $16,500.00

Immersion Systems

1. National Tele-immersion Initiative Web site:


2. Tele-immersion at Brown University:


Andries van Dam, Loring Holden, Robert C. Zeleznik


3. Tele-immersion at the University of North Carolina at Chapel Hill:

Team Members: Henry Fuchs, Herman Towles, Greg Welch, Wei-Chao Chen, Ruigang Yang, Sang-Uok Kum, Andrew Nashel, Srihari Sukumaran

http://www. teleimmersion/

4. Tele-immersion at the University of Pennsylvania:

Ruzena Bajcsy, Kostas Daniilidis, Jane Mulligan, Ibrahim Volkan Isler


5. Tele-immersion site at Internet2:

6. Advanced Networks and Services:

Jaron Lanier, Amela Sadagic