Not logged in, Join Here! or Log In Below:  
 
News Articles Search    
 

 Home / 3D Theory & Graphics / color keying and Opengl Account Manager
 
Archive Notice: This thread is old and no longer active. It is here for reference purposes. This thread was created on an older version of the flipcode forums, before the site closed in 2005. Please keep that in mind as you view this thread, as many of the topics and opinions may be outdated.
 
kiddygames

April 27, 2005, 06:06 AM

Hi,

is there a way to do some color keying in openGL ?

The link to Nate Miller Tutorials page is dead, and I can't find any other explanation about it ?

thanks

Stéphane

 
Rui Martins

April 27, 2005, 06:17 AM

OGL has something better for the job.
Stencil buffer!
Check it out.

Basic setting ideia:
- Change The Stencil test to match your "key color", filter by color.
- Draw your "key color" input, into the stencil buffer.
- Use the Stencil buffer, to "Color KEY" whatever you want to write over the image.

 
kiddygames

April 27, 2005, 09:37 AM

Yeah !

Didn't think about it but stencil should do the job...

thanks a lot

Stéphane

 
Victor Widell

April 27, 2005, 07:52 PM

Or you could use alpha testing... (?)

 
Pogacha

April 28, 2005, 05:48 AM

I think that alpha masking could be cheaper than using stencil making ...

 
Rui Martins

April 28, 2005, 08:09 AM

I'm not sure about that one, because you can have a single bit stencil buffer, but with alfa you usually get more, like from 4 to 8.
You would also have to encode/preprocess your input so that a key color is converted into a specific Alpha value. Color keying Input don't usually have a alpha channel.

However rendering could be faster, because you just need to disable color write (excepth alpha) on first pass, then it's similar.

Also, stencil (hardware) support must exist, which used to be a downside, but not anymore, with the new cards.

 
kiddygames

April 28, 2005, 08:25 AM

Here is what I'm trying to do exactly:

- Some stuff are drawn in a not openGL window (GDI or similar), without alpha channel.
- I get the HBITMAP from this window, and get the associated data (the pixel buffer)
- I copy the buffer in a openGl texture (RGBA format)
- I can now use this texture without problems in my openGL window

but I want all the white or black or whatever background color to become invisible in my openGL texture...

I just can't see any way to do this with alpha testing (except a huge loop to test and set an alpha component on each pixel before sending it to the video ram).

The stencil buffer seems to be the easier way

Stéphane

 
DavidR

April 28, 2005, 10:28 AM

Stencil buffer is not the way to go. You have to allocate memory for the buffer and get the right stenciling operations, etc... Definitely not the way to go for simple color keying. What you want to do is load your RGB bitmap into memory first. For an RGB image (assuming 24-bit), there are 8-bits per pixel. You know the width and the height of the image, so the total image size would be width*height*3 (3 channels -- RGB). An RGBA image has 32-bits, with 8-bits per pixel. You have to allocate width*height*4 (4 channels -- RGBA) in a NEW memory location. You then go through the RGB image, if the pixel components are the specified color for your color key, let's say black (which would be R=0, G=0, B=0), you save a 0 to the RGBA image's alpha channel (which is indexed with i+3 based on 0-index counting). If the pixel doesnt match your color key, save the components in the RGBA image and set the alpha channel to 1. You now have an RGBA image at your disposal :)

-DavidR-

 
DavidR

April 28, 2005, 12:07 PM

ok, so I made one mistake... If the color doesnt match your color key, instead of setting it to one, you set it to 255 (full opaque)... I was thinking of 0.0 to 1.0 clamped colors, rather than 0-255 color values :D

Anyway, I have some working code for it. This is just the actual process of converting an RGB image to an RGBA image:

  1.  
  2.         int i = 0;
  3.         int j = 0;
  4.         for(i, j; i < rgb_size; i += 3, j += 4)
  5.         {
  6.                 if(pRGB->data[i] == key[0] &&
  7.                         pRGB->data[i+1] == key[1] &&
  8.                         pRGB->data[i+2] == key[2])
  9.                 {
  10.                         pRGBA[j+3] = 0; // transparent
  11.                 }
  12.                 else
  13.                         pRGBA[j+3] = 255;       // opaque
  14.                
  15.                 pRGBA[j] = pRGB->data[i];
  16.                 pRGBA[j+1] = pRGB->data[i+1];
  17.                 pRGBA[j+2] = pRGB->data[i+2];
  18.         }
  19.  


Hope that helps!

-DavidR-

 
Rui Martins

April 28, 2005, 12:20 PM

But David, this approach requires a copy of the texture, which is doen entirelly in the CPU, instead of delegating any copy to the GPU.

If you want to animate a video stream, this can have a significant impact on performance.
It is even worse, if the origin of the stream is 3D output from your application, color keyed with an outside stream, because you have to get the texture from the 3D board (render to texture!).

Note however, that color key is hard to get right, because you can't trust you will have a single unique color value as key, in a tipical video stream, usually some filtering/processing is required, unless the source of the stream is a 3D render, which you can garante to have a specific color as background.

 
DavidR

April 28, 2005, 06:24 PM

I wasn't aware she was animating a video stream. From her posts, it sounded like she had a bitmap already stored in memory that she was copying over to an RGBA texture to display using OpenGL. So she is basically already copying the texture to an RGBA image, but instead of filling the alpha channel with transparent or opaque components, she is not doing anything with that channel. I merely suggest that rather than performing operations on the stencil buffer for color keying, that she actually fill in that alpha channel with correct values.

Your suggestion of using the stencil buffer is interesting, however. It isn't a bit overkill to use the stencil buffer for color keying on a simple texture? Rather than using the stencil buffer every frame, why not just save the RGB image as an RGBA image once and not worry about the stencil operations for every frame?

I'm curious about the way the bitmaps are being used. Is it an operation that must be done on-the-fly? Or is it an operation where you can save out the RGBA images to a file and load them at startup? I can definitely see the advantage of using the stencil buffer if the images are being streamed. My method would completely suck on streamed images. I'm interested in learning more about the stencil buffer approach.

-DavidR-

 
Rui Martins

April 29, 2005, 04:42 AM

DavidR wrote: I wasn't aware she was animating a video stream. ...


I don't exactly know what Stéphane is doing, but the usual context of color keying is for "live" video streams.

DavidR wrote: Your suggestion of using the stencil buffer is interesting, however. It isn't a bit overkill to use the stencil buffer for color keying on a simple texture? Rather than using the stencil buffer every frame, why not just save the RGB image as an RGBA image once and not worry about the stencil operations for every frame?


If you are using a single static texture then yes, your way is simple and is local to the texture, you don't have to set a full framebuffer with stencil.

DavidR wrote: ... I can definitely see the advantage of using the stencil buffer if the images are being streamed. ... I'm interested in learning more about the stencil buffer approach.


I believe the main issue here is how color keying is being used!

Do we have a single subject lost in full screen of "blue" or similar, where the goal is to provide a background (3D or otherwise)?

Or do we have several color keyed subjects and the goal is to provide a background?

OR we have a background with specific color keyed parts (example TV sets), and we want to provide the contents for those parts ?

Depending on what the case may be the best correct approach will vary.

Stencil is usefull, if we can exploit the frame buffer coherency, i.e. we will color key several stuff into the same frame buffer.

 
kiddygames

April 29, 2005, 06:01 AM

first thanks for all your answers, here are some replies:

- DavidR : first I'm a guy not a girl (in French, Stéphane is for guys...).

- Yes I want to do "on the fly" color keying on a animated (not video but animated) bitmap, so adding the good values in the alpha channel on the fly for a 1024 * 1024 texture is too much CPU consuming.

- I only have 1 color for my color keying input (it's not a video or any compressed stuff, so no noise or dithering on the input color key).

So my conclusion is that I should use a stencil buffer as there's no straight way to do it with "basic" OpenGL calls...

Stéphane

 
JCash

April 29, 2005, 07:17 AM

Maybe I'm running out on a limb here but why not use a pixel shader?
Should be easy enough even if you'd want to specify a range of colors?

 
DavidR

April 29, 2005, 11:50 AM

Excuse my assumption of your gender! I apologize deeply.

-DavidR-

 
This thread contains 15 messages.
 
 
Hosting by Solid Eight Studios, maker of PhotoTangler Collage Maker.