Not logged in, Join Here! or Log In Below:  
 
News Articles Search    
 


Submitted by Parashar Krishnamachari, posted on May 23, 2001




Image Description, by Parashar Krishnamachari



Not that long ago, someone posted an IOTD about an engine that made a scene look hand-drawn... When I saw that, I just slapped myself on the forehead and realized I never posted a screenshot of the 3rd version of aEmber to IOTD... D'oh!!!

Anyway, for those of you who are unaware, this is a product of UIUC SIGGraph which paints a 3d model to the screen in realtime. The screen you are seeing was running at 1.7 FPS on a P-200 with no 3d hardware, and in fact... an ISA video card *gasp*. Bear in mind that at SIGGraph conference 2000, we were given Nvidia Quadro's, so the models are tesselated to an extent that reflects that.

The older screenshots:
  • IOTD March 09, 2000 (Painter v1 screenshot)
  • IOTD August 26, 2000 (Painter v2 screenshot)
  • IOTD September 13,2000 (Comparison of Painter to ordinary rendering)
  • This is an entirely different method from our previous versions. This one is dependent on the projected coordinates of the triangles and an offscreen GL render of the scene. We tesselate the 2d triangles and sample from the image at the resulting pixels (jittered of course). Then paintstrokes are rendered as textured tris to the window with said color and location.

    The artist who creates the model has the control over
  • specific paintbrush for a given model (you can get vertex-specific actually)
  • Direction and curvature of the brushstrokes
  • size, skewing, and other features of brushstrokes
  • background image (if any)
  • depth of tesselation for each object
  • degree of jittering for tesselation and color
  • morph response (brushstrokes changing randomly or to sound input)
  • What you see here is the result of 1 guy locked in his hotel room coding by the seat of his pants at the very last minutes before presenting the App to SIGGRAPH... Oh, well... it was fun and the project really reached the final goal, so here's the pretty part... behold the coolness of fruit.

    - Parashar K.


    [prev]
    Image of the Day Gallery
    www.flipcode.com

    [next]

     
    Message Center / Reader Comments: ( To Participate in the Discussion, Join the Community )
     
    Archive Notice: This thread is old and no longer active. It is here for reference purposes. This thread was created on an older version of the flipcode forums, before the site closed in 2005. Please keep that in mind as you view this thread, as many of the topics and opinions may be outdated.
     
    Buster

    May 23, 2001, 04:14 PM

    Man that's some hairy fruit... time to hit Albertsons!

     
    Clemens Hofreither

    May 23, 2001, 04:17 PM

    Looks pretty nice.

    If the image is recreated from scratch for every frame, does that mean there's no consistency between frames? That would look a bit chaotic.
    I'd like to see it in real-time though.

    clemens

    [second... and actually my first post on this site ever! hi to all!]

     
    Jesse Chounard

    May 23, 2001, 04:22 PM

    So, you actually see the images being painted? (As a person would paint on a canvas?)

    If so, would it be possible to render the finished images fast enough to be used in a game? (I'd really like to see some games that made good use of some trippy cartoon rendering/watercolor effects/etc.)

    The banana on the right kind of looks like it is being thrown, and you can see motion blur.

    I love the way this looks, btw.

    Jesse

     
    Outline

    May 23, 2001, 04:27 PM

    Currently I am working on investigating types of non-photo realistic rendering. Where could I find papers describing the different types of approaches? Are there any examples with source out there?

    That's a very interesting image. However, I think it would look a lot better without fuzzy hair-like lines on the silhouette. Is there a demo available?

     
    stoo

    May 23, 2001, 04:41 PM

    Wow, that looks really nice. Did you spend a lot of time experimenting, or did it all come together pretty quick?

    Outline:
    Check out http://www.red3d.com/cwr/npr/ that has loads of stuff on NPR, including lots of real-time stuff. Other than that, have a look at the ACM digital library, for relevant papers.

    Stoo

     
    DarkReaper

    May 23, 2001, 06:22 PM

    That's really cool. I'm glad to finally see a (semi) original post. That doesn't even look like a 3D app...and it doesn't have terrain! w00t! =) Great job!

    Is there an exe? and/or source?

     
    DragonWorx

    May 23, 2001, 08:11 PM

    It'd be nice to dissolve from a phong shaded version, to this style - in realtime. Nice work!

     
    David Ko

    May 24, 2001, 03:54 AM

    haha, hey parashar,

    Nice to see you finally submitted some pictures of the latest versions of aEmber. I also want to thank you for patiently answering all my annoying email questions for my own painterly npr work (do you know who i am now?). :)

    Anyways, IMHO, real-time painterly rendering has a bit to go (but not THAT far off) before they'll be ready for use in real-time games. The biggest reason for this is the sheer number of brush strokes required to adequately portray a "painted" (or whatever medium) model. And, as parashar had mentioned before, the aEmber system must also render the geometry, and then sample the resulting pixels to determine the color of the overlaying brushstrokes, and then render the brushstrokes themselves. In essence, you're rendering twice, although it's not as bad as it might seem. On my PIII 700 (TNT2), I get about 10-15 fps for that fruit model you've posted. The coherence of the brush strokes ain't bad either. I think painted models can easily be intergrated into a normal fps game, with careful management of off-screen rendering buffers. But, you'd be limited to small models that don't require a lot of brush strokes. Since the "look" of the painted models are almost solely dependent on the characteristics of the brush stroke texture, you can simulated many styles of paintings for any given model using an adequate texture. Of course, it'll be a long-time before computers become fast enough to allow normal users to walk through massive, entirely painted worlds. I understand the champagne team had also investigated a 3d method for real-time painterly rendering, but nixed that because it proved to be too slow.

    Now, I have a question for you, parashar. aEmber allows you to change the shape parameters of the brushstrokes using the "draw functions" and "brush width variances". From that, I'm guessing you're not using quads to render the brush strokes, but possibly a 14-vertex polygon so you can stretch the brushstroke shape around. Am I correct? So, do you think that level of control over the brush stroke shape changes the "look" of the scene significantly? Why not just use a 6-vertex polygon, or even a quad that can be warped to produce different stroke styles?

    One more thing...any screenshots of that recursive monte carlo thingy you were telling me out. :)

    dave

     
    Parashar Krishnamachari

    May 24, 2001, 09:47 AM

    There actually IS consistency between frames because of the facts that the tesselation results in the same points every time. Granted it's not 100% perfect like the first version had, but it can fool your eyes enough unless you've enabled random morphing or audio morphing...

    The 2nd version on the otherhand, didn't have consistency from frame to frame... that's fixed here, but the GL renderer gives us color to sample, which we CAN jitter if jitter is enabled for that object.

     
    Parashar Krishnamachari

    May 24, 2001, 09:53 AM

    The hairy effect is a result of the brushstrokes being really fuzzy, and the options being set such that the strokes are free to overrun the shapes of the objects (which we found to be more convincing, so it's default). When the brushes were confined to the poly-mesh of the objects, it ended up looking like a painterly textured object rather than something that was actually painted...

     
    Parashar Krishnamachari

    May 24, 2001, 09:59 AM

    No you don't see the painting. The painting actually IS in realtime... When running this engine on a machine with... say... a GeForce 2 MX card, it runs at 23 fps easily, and you only get to see the painted results, anyway. (We are after all, rendering the scene twice per frame)

    So it's more than ready for games, but managing your scenes is kind of involved because you have to think about the brushstrokes for each object... Like you mentioned with the banana, which had nearly horizontal strokes on the lit side, which ended up looking like motion blur...

     
    Parashar Krishnamachari

    May 24, 2001, 10:04 AM

    We spent a lot of time experimenting -- we went through 3 versions, after all... One glide, one D3D, and this one which is GL... The first one came together pretty slowly, but we got around to understanding a lot more.

    The second one was worked on little by little over vacation, while someone else was busy porting our code to D3D.

    The final version came from suggestions of people who were at SIGGraph conference, and was put together almost immediately before we had to present.

     
    Parashar Krishnamachari

    May 24, 2001, 10:14 AM

    FYI to all...
    There is an executable, and at last, I managed to grab the source code.
    Currently, though, the UIUC-SIGGraph site is undergoing an overhaul, and you might have to wait a bit before the executables are posted... but the site is at -- http://www.acm.uiuc.edu/siggraph/

    BTW -- the "one guy locked in the hotel room" was NOT me...
    I'm the numbskull who stepped onto the project one week before we had to present to the reviewers at UIUC-EOH, and wrote a simple lighting engine for it while running a fever and spitting up blood every few hours... And that, my friends, is enjoyable.

     
    Machin Shin

    May 24, 2001, 10:26 AM

    yes.. isn't it always? ;)

    hey, parashar how is it going?
    you may not remember me, but we knew each other way back when you
    were in junior high .. (somewhat)
    (govinda's a friend of mine, i'm srivatsan :))

    btw, whatever happened to that 'shark game' you and govinda wanted
    to make? .. i remember some interesting little "programmer art" diagrams you & gov drew..

     
    MC BAXTON

    May 24, 2001, 10:31 AM

    I would NEVER eat this ..

     
    Parashar Krishnamachari

    May 24, 2001, 10:54 AM

    > Anyways, IMHO, real-time painterly rendering has a bit to go (but
    > not THAT far off) before they'll be ready for use in real-time
    > games.

    We weren't really thinking about games when we wrote it... we were more or less thinking about making certain people look like complete idiots, which we certainly did accomplish. Of course, now... the admins at Champaign utterly hate the guts of SIGGraph. And I DO mean HATE -- not just because of this, but because we've made a habit of projects that make fools out of several profs. And we're all undergrads (so far), which only adds insult to injury.

    > On my PIII 700 (TNT2), I get about 10-15 fps for that fruit model
    > you've posted

    That's not bad... We tried it out on a PIII-750 w/ a GeForce2 MX, and got about 23 fps on the grass huts scene (dunno if I gave you that one).

    > I understand the champagne team had also investigated a 3d method
    > for real-time painterly rendering, but nixed that because it proved
    > to be too slow.

    Mmmm... yeah... that's what we thought until we discovered that Viz Studio had a feature called "Release build". But even, so, it wasn't running any faster than what you're getting with your TNT2 in version 3, and it wasn't anywhere near the quality.
    If you want to see that, that's the version 1 IOTD -- March 6.

    > From that, I'm guessing you're not using quads to render the brush
    > strokes, but possibly a 14-vertex polygon so you can stretch the
    > brushstroke shape around. Am I correct?

    I didn't really work on that part, and I only recently started looking through the code, so I don't really know. However, I happen to know Don & Shanon (who wrote the brushstroke functions) well enough to be pretty certain they used quads. You can easily rotate, shear, and stretch/scale quads enough to get all the effects we were trying to get. And I do know that the original version used quads, so I doubt that's changed other than the fact that the brushstroke variations seem to be based on the original polygon orientation (from the mesh itself), rather than the camera orientation (so brushes would always face you by default, albeit an overridable default)

    > One more thing...any screenshots of that recursive monte carlo
    > thingy you were telling me out. :)

    I would if it were actually working... ^_^

     
    Parashar Krishnamachari

    May 24, 2001, 10:59 AM

    > (govinda's a friend of mine, i'm srivatsan :))

    Ah, yes...

    > btw, whatever happened to that 'shark game' you and govinda wanted
    > to make? .. i remember some interesting little "programmer art"
    > diagrams you & gov drew..

    Haven't actually done anything for that other than writing some of the music for it -- which ended up being used in SIGGraph demos anyway, ^_^
    Oh, well. Not like I've kept that much in touch with Govinda and Hari/Ram, etc...

     
    Parashar Krishnamachari

    May 24, 2001, 11:04 AM

    > Check out http://www.red3d.com/cwr/npr/ that has loads of stuff on
    > NPR, including lots of real-time stuff. Other than that, have a

    I just noticed that link you gave also has OUR abstract listed on it... yipee... You look for the name "Josh Michaels" on that page, and you'll find it. Not that my name is on the abstract, but I'm one of 3 people whose names aren't there. Oh, well.

     
    EGreg

    May 24, 2001, 11:06 AM

    Hey, dude, cool job. I wonder, if you render a few frames could you get that hand-drawn-cartoon type thing where the picture changes a bit and "dances"? Cuz it be kool like dat.

    -Greg

     
    Parashar Krishnamachari

    May 24, 2001, 11:27 AM

    The option is there to do that, actually. We can set the strokes to wobble with time ever so slightly (or radically, if you choose). Otherwise, the strokes stay absolutely the same... although there is a slight coherency issue when you move around (but it's hardly noticeable, which we figured is good enough)

    That's what I was referring to with the "morph response" thing.

    You can also make them change with sound. If you speak into your microphone, we can to an FFT of the input and make the brushes wobble and stretch based on a the spectrum... (just involves linearly mapping the polygons to frequencies w/ wraparound if there are more polygons than points in the FFT analysis).
    That has an interesting effect when you put the mic up to speakers that are playing music -- the brushes dance... Quite literally, in fact... ^_^

    As it so happens, that really wasn't an important feature. It's just that the EOH judges tend to be complete morons who don't know a damn thing about technical aspects of ANY science, let alone programming. These people who can't even spell IBM basically judge you based on "wow" factor alone. That's the only reason we even put in the sound-affected brush morphing.

    PLO at this year's EOH wrote a boxing game, which they claimed, "has important applications in e-commerce." o_O whatever... but the fact remains that the judges believed that...

    The SIGMobile group didn't even have their project working when they had to present last year... what they did at the last minute was to slap pictures of Pikachu all over their booth and that got them 1st place in the "Just for Fun" category.

     
    [-WD40-]

    May 24, 2001, 11:37 AM

    With the vertex and pixel shaders on the GeForce 3 I bet you should be able to do that in real-time easily (maybe not TOO easily :) but still..)

     
    Thunderbird

    May 24, 2001, 11:49 AM

    How were you making fools of profs by writing this? Did they say this was impossible to do, or that undergrads couldn't do this stuff? Either way this is beautiful. I can't believe how bad your judges are though.

    -Blackstream

     
    Sebastian Sylvan

    May 24, 2001, 12:22 PM

    Oh this is the coolest thing I've ever seen... Does anyone else think "Alice 2"?
    Think about just stepping inside an oil painting on the wall and fighing monsters in a bizarre world rendered using this technique.

    How optimized is it? Is there anything that would benifit a lot from SSE/3DNOW type assembler?

    I think this could be made to run at 60 frames per second on current hardware. I mean, you can turn off stuff like bump-mapping, and shadows and stuff doesn't really have to be that accurate (so you can save some time there). Basically since it smudges the image so much you can turn of loads of other stuff to get a higher framerate.

    And if you use pixel/vertes shaders (if it's possible) you can probably make it run even faster.

    I wanna see a game using this type of rendering!

     
    Parashar Krishnamachari

    May 24, 2001, 01:44 PM

    > How were you making fools of profs by writing this? Did they say
    > this was impossible to do, or that undergrads couldn't do this
    > stuff?

    Both...

    > I can't believe how bad your judges are though.

    That's how it is... People from all over sign up to be judges for this event -- the idea is to be impartial so no CS professors judge favorably to programming projects... no Physics profs judge favorably to physics society projects and that sort of thing. The end result though, is that we end up with English majors and PE teachers who are long over the hill to be our judges for all events.

     
    Parashar Krishnamachari

    May 24, 2001, 01:51 PM

    Yeah, that's probably true...
    Well, the thing is that we already get 23 fps on a grass huts scene with a regular old GeForce2 MX, and this is without doing any particular optimizations (like efficient tristripping and such)

    The 1.7 fps I mentioned was on my machine which has no 3d hardware accel.

    For the fruits scene you see there, a GF2 MX pulls a good 45+ fps...

    So it would probably seem that efficient use of the bandwidth is kinda our biggest problem... which makes sense considering how many strokes there are for one scene. Texture caching is not that big a deal unless it's a problem in the original, unpainted scene. For the painting itself, you really only end up reusing a handful of really small textures.

     
    Parashar Krishnamachari

    May 24, 2001, 02:03 PM

    > How optimized is it? Is there anything that would benifit a lot
    > from SSE/3DNOW type assembler?

    Not very optimized, actually...
    The biggest issue, by my guess is bandwidth. We're sending across a LOT of quads one by one... making them quads helps a little, but tristrips/fans would help far more.

    A lot of the work is done by the 3d hardware. The only things that the CPU is left to do is sample from a raw rendering and modifiers on the polygon shapes (based on all the parameters for brush direction, size, skewing, etc...)

    > I think this could be made to run at 60 frames per second on current
    > hardware. I mean, you can turn off stuff like bump-mapping, and
    > shadows and stuff doesn't really have to

    On the fruit scene, we get 45+ fps easily with a PIII-750 and GF2 MX. There's a grass huts scene which is really complex, and that gets around 23 fps.

    Currently, we're cheating on shadows -- it's being done with negative lightsources. Which are scene-specific... the only lights you can control in the program is the "sun." Of course, you don't see much in the way of shadows in that scene, because there's no ground or anything... ^_^

     
    SientStrike

    May 24, 2001, 02:51 PM

    I spent like 25 hours learning OpenGL, then another probably 30 hours writing code for a gravity simulation project. I lost to a VB keylogger that emails recorded keystrokes to a given email address.

    Dumb judges. Ugh :(.

     
    Parashar Krishnamachari

    May 24, 2001, 03:43 PM

    What's scary is that among other universities I've attended, UIUC is one that actually had MORE respect for undergrads than other schools...

    Despite hating SIGGraph, and being rather surly rather often, they at least cater to the basic needs like reserving lecture halls and meeting rooms and such. Although, the most annoying thing we had to deal with this year was that we reserved 4 tables, 11 chairs, and 6 monitors for EOH -- what we got was 1 table, 3 chairs, and zero monitors -- SIGGRAPH gets no monitors?

    We also tried to get the school to pay for conference fees/hotel/flight/etc for everyone to go to SIGGraph 2000 to show aEmber... what we got from them was "Here's a check for $50. Now you get the hell out of this office before I have you thrown out"

    Regardless, whenever we wanted to broadcast over the TV channels, we got it for nothing... Whenever we wanted to rent out a lecture hall to do a demoparty, we got it for nothing... We even got them to pay for mixed drinks (no, I'm not kidding).

    There are worse places, though...

    There was also a time when video-conferenced lectures at other universities (splitting up)... and I got Texas -- according to what Sir Paul (Nettle) told me, his lecture over there went way above the heads of the grads there when he mentioned "Z-buffers"... Their words were something like "You're lecturing to them? No you're not, you're just an undergrad."

     
    Bemmu Sepponen

    May 25, 2001, 08:20 AM

    I agree. Hairy bananas just aren't my thing.

     
    This thread contains 29 messages.
     
     
    Hosting by Solid Eight Studios, maker of PhotoTangler Collage Maker.