flipCode - Tech File - Adam Robinson [an error occurred while processing this directive]
Adam Robinson
(aka Iok)
Click the name for some bio info

E-Mail: Iok@bigwig.net
Website: http://www.furryfoxy.freeserve.co.uk

   07/23/1999, In the Beginning

I, for the past 3 weeks have been converting all my old Dos code written in Watcom 10.6 to Windows and Msvc 5. This has not been fun.

Reason 1: DirectX

Firstly, what was I converting. I am in the process of writing a game engine for an idea that a friend and myself have come up with. The game is not going to be a Quake beater but isn't bad. There was a skeleton of a game engine and an almost fully operational 3D engine. I designed the engines structure to enable everything to be dynamic, from the lighting and shadowing to the terrain and levels, weather, atmospheric conditions e.t.c. The whole thing ran in 16bpp. There were also all the tools to acompany this i.e. texture converters, tools for setting the attributes of polygons in models, the skeletal animation tool e.t.c.

I really had no choice but to convert all my code to windows, Dos is no longer a viable platform for selling games in but it does make for a very nice and stable development environment. Windows on the other hand is not a stable and nice development environment.

After a week I had learned how to write windows applications [not really very hard]. Another week later I had learned how to use DirectX, also not really very hard [and I actually wrote my own code instead of just coping the examples!]. Then I started to convert all my old code. Most time consuming was getting Msvc to link to my Asm code [which is still in Wasm] correctly no matter which function parameter convention was being used [I use little inline asm wrapper functions in C++ to put values in registers and call my code, very important to save ebp,ebx and esi and edi across routines]. Anyway I got this done and tested it and all was well. [although there was one function that crashed the compiler when it was trying to build it, I put extra spaces between operators in the function and then it compiled correctly [very weird]].

Then came the time to integrate my old engine with DirectX. I did and ran my program. BANG. Some how, when the dust settled, the display drivers for my Rage Pro had been wiped off the machine. This sort of thing isn't ment to happen in windows. I fixed the bug causing that 'little' problem and continued. [Isn't the Win16 Mutex annoying! I currently lock and then unlock my surfaces straight away so the debugger is useable inside my rendering loops].

The big problem that DirectX causes is the very one that it was designed to solve. Before DirectX you had to code your own drivers for everything. This was bad. Most coders didn't have the time or resources to do so and had to resort to VGA only or the simpler SVGA video modes. Directdraw makes it very easy to set nice new video modes but makes the mistake of having far too many features, most of which do not work on all video cards the same way. This is bad because it means we need to tweak our own code or give the user a load of options to tweak performance. Direct3D has the same problem, if you use any of the nicer features they are bound to fail on at least 50% of machine out there. H.E.L. is rubbish.

Anyway, it turns out that blitting a screen to video ram from system ram is faster than flipping it in fullscreen mode with my video card. Confusing. [Probably because I have to keep the surface in system memory due to my excessive use of transparancies].

Reason 2: Msvc development environment

Too complex, too slow and it goes wrong too much - what more can I say. + I can't get any of my code to profile. I used to use Edit in Dos and batch files to handle making e.t.c. It was simple and effective and never went wrong [well most of the time it didn't].

Reason 3: Code Patching

You need to create a new selector to access the code segment under windows [writing to the code segment under Dos4gw was fine]. Patching in surface memory addresses e.t.c. is useful and causes no significant speed impact. At the moment I've put all the patching code in a data segment [where writes are enabled] but this isn't a briliant situation since some macros, like ALIGN work differently in the data to code segments [ALIGN outputs 'or eax,[eax]' instead of 'nop'], it may also have a speed impact but I'm not sure about this [since I can't profile].

What I am doing right now:

The game [a bit like Descent 3] runs with combined dedicated indoor and outdoor 3D engines, the two are coupled together. Everything is software rasterised. I've played a lot of hardware rasterised games and for the most part the only extra feature they use is bilinear/trilinear filtering which doesn't really impress me, so far only Q3test seems to be using 3D cards in a worth while manner. You should be able to use 3D cards to pump through at least 10 times as many polygons as a software rasteriser but so far not many games I have seen do. Anyway, while I'd like to accelerate my game there is no chance of it, the main reasons being cost and market size. I would need (in order to make a game that could possibly be commercially successful) several 3D cards to test on + another monitor e.t.c. for my computer and I would tweak the code for each one to get good throughput. I know in America (a huge market) most games players have 3D cards in their pc's (I bet tonnes of those are slow Voodoo 1's though) but in europe and escpecially in Britain almost no one installs 3D cards in their machines. It'll be 4 or 5 years before there are enough machines around with 3D cards here for a game to survive purly on 3D acceleration [in just the european marketplace]. Remember, there are still a huge number of people who do not own 3D cards.

Anyway, the lands 3D engine runs a multi level implicit dynamic bsp tree [I made up that name - its just a bsp tree that is always centered around the camera - I'll explain better another time] with nodes clipped to the world space view frustrum. All land polygons are cbuffered. I designed the engine to be able to be able to easily handle dynamic modifications to the land data set [what I really want is too be able to set off a nuclear weapon and see the shock waves travel out across the map]. It also runs a lod system that morphs down 4x4 blocks of land to big 1x1 blocks at distance. The land polygons fill a zbuffer [which is 32bit]. At the moment there are about 7megs of unlit [mipmapped] textures drawn to the screen per frame, this was the start of my problems. To handle lighting I wanted to run a surface cache at first - this is impossible there is too much to calculate (want true rgb lighting). So I wanted to run vertex shading on the perspective textures. In a 16bpp mode using mmx is the fastest way [the other way is 3 table lookups per pixel i.e. slow] but mmx won't work directly on compressed 16bpp colours and require approx 6 cycles per pixel to unpack and repack it. Thats why the other day I converted the 3D engine to run in 32bpp. This of course caused the frame rate to drop by 20% but has the advantage of much much faster lighting and transparancies [additive and subtractive at a few cycles per 4 pixels!]. The dynamic lighting on the land is now free (i.e. adding more lights won't cause a drop in frame rate).

Why do I want to light the land? Dymanic weather. I want cloud shadows [with real 3D clouds], lighting and shadowing of the land depending on the position of the sun. (as you can probably tell I've been looking at screen shots of Black and White far too much).

Anyway thats the end of this very long update - list of stuff I'm currently trying to do:
  • Add a dynamic wind system to the weather system so tree and stuff sway as waves in air move across the map
  • Use dynamic vertex lighting on the portal engine (Probably split and cache polygons using clipped light frustrums)
  • Use dynamic CSG on the portal engine. (had some ideas about this - want to use big cubes with portals on the edge as room boundaries with dynamic csg running inside the cubes - only need to calc cubes we can see).
  • Put the land maps inside sectors so land maps can be positioned anywhere in the portal engines maps.
  • Write more stable editing tools
  • Add a nice gourard shaded sky sphere (need a few more control points for the interpolation that a cube has) so that day to night transitions will work nicely.
  • Add the wave generators to the water that exists on the land map (not sure at the moment wheather I'm gonna texture the waters surface or run transparent gourard shaded polygons)
  • I need a good 3D model editor.
  • Write a bilinear span drawer (got all the tables already set up)
  • Get the locational music thing working
  • Test the speed of the engine on a P166MMX [Base machine] and PII400 [Fast machine owned by friend]. I've got a Celeron 333 with 32Mb ram.
  • Find a way to profile my code [I ran about 10 profiling cycles a day under Watcom 10.6]
  • -As you can see I've got a lot to do. I'll post a smaller update in a week detailing my progress.
    -Email me if you've got any comments e.t.c.

    p.s. - Msvc 5 crashes whenever I run the debugging version of my code [my program then continues to run correctly], there's a fault in dm.dll which appears to be a task spawner, anyone know why this could be happening?

  • 04/01/2002 - Tech File Update
  • 03/08/2001 - Tech File Update
  • 02/11/2000 - Tech File Update
  • 01/06/2000 - Adsy Loves Microsoft...
  • 10/27/1999 - Gone For A Burton - Or Not - Maybe.
  • 09/14/1999 - Update 3
  • 08/23/1999 - Update 2
  • 07/23/1999 - In the Beginning

  • This document may not be reproduced in any way without explicit permission from the author and flipCode. All Rights Reserved. Best viewed at a high resolution. The views expressed in this document are the views of the author and NOT neccesarily of anyone else associated with flipCode.

    [an error occurred while processing this directive]