This collage show a few screen grab of an upcoming video tool based on Dx9.
The core concept is based on a chroma keyer, pre-rendered 3D animated scenes
and real-time 3d compositing & tracking.
The main UI shot show how the raw video footage is processed.
On the top left we see that the talent is shoot over a green backdrop
The center row of icon show the component that build the final set
On the top right we see the final produced image with video processing, and direct3d scene compositing.
The bottom of the UI show the control over the alpha channel generation and color control for the selected source clip. Those control let user adjust real scene shadow extraction, highlight levels, and other keying settings.
The backdrop feature the highlight features and spill suppression (Removing any trace of green from the video)
Here is a link to see the video output featuring 3d environment with multiple camera angle:
Thanks to the new DirectX NVidia & ATI drivers we are able to render those animation at over 15 frame second to disk in 720x480 uncompressed on a 2.4ghz P4 equipped with a Ti4200.
Just 6 month ago we didn't even reach 3 frame second with the same gear.
Their is still head room in the performance area by shifting more CPU based processing to pixel shaders,
being optimistic it seem possible to render the sample video above in real-time at full resolution.