 |
04/03/2000, Tech File Update
|
hi folks...
I just came home from my gf and as there is nobody online whom I could talk with
I decided it would be a good idea to write down my latest ideas/expriences with
scripts,compilers,c/asm code and shader files..and if I find enough time about
my implementation of a dll plugin system....
some time ago I decided that it would be good to have a script...
so I wrote some simple parser which recognized tokens and encoded the read
source file(c similar) into some binary file..
that was read in by the game and parsed realtime...
the parsing was a goddamn ugly huge if else combination...I just wonder that it
worked at all...but it was possible to call real functions(compiled and linked
ones) from the script..what is quite handy..but it was far too slow...
also I was interested to see how a real compiler works..
as I wanted to have it done my self I stayed away from yacc/lex and extended my
old 'compiler'...I just opened the old file containing the lexed code and
translated into pseudo asm..well..'just' is the wrong word..that's the real
trickey part
I had to rewrite that 3 times before it worked out..I'll give a short
describtion how I made it(it is prolly shure NOT the best solution)
C to ASM
1st pass - initialize global vars
I look through all symbols which are (global) variables and and notetheir size
and weather they are initialized or not...
(the size of the dataseg is written to file firstly)
then I parse the code which initializes the global vars...
(code starting at 'var=' and ending at ';')
the parsing func(used for all the parsing stuff)
1. put the line to be parsed into a cache
2. compress the cache
-if there is a functioncall in the cache then the code to execute the
func is written
to the output file and the part of the cache containing the function
call is replaced by the register which contains the return value of
the func
-constants are loaded into registers and the numbers in the cache are
replaced by
the registers containing em
-all the rest code which is not ambiguous or dependent on brakets is
written to the output file and the respective parts of the cache are
replaced with the registers
3. parse the cache
-recursively look for brakets
-start in the deepest braket and look for operations with descending
priority
(cast,indirection,arrays[],*,/,+,-,...)
this is similar for most operations...so I'll just make one eg for cast
so let's say we found a cast in the cache :'(float*)reg1'
now we check if the cast is solveable..means that there is no operation
with higher order right to the cast(brakets for eg)...if so we write the
code casting reg1(=eax)
to float* and remove the cast from the cache (actually there is no code
needed in this case..but we remember the new type of the
register..so we can do typechecks later)
4. write the result to the adress of our var
-as soon the deepest braket level is done (the complete braket is
replaced by some register containing the result of the operations within
the braket)
we do the same thing one level higher..so if there is no syntax error in
the cache there will be one register in the cache after having
processed it completely..that's the result..which is assigned to the
adress of the variable which is inited with that code
(if there is more left than one register then there is a syntax error(we
were not able to solve this line of code))
2nd pass - write strings
next I write all the strings in the sourcefile to the dataseg
eg:
"TexFuncs.dll" is 3*4 characters long..so it fits into 3 DWORDs
(88 is the offset of the string in the dataseg)
so we can write :
(the number at the beginning of the line indicates the offset of the
command)
48: mov reg1 , const 1182295380
72: mov [ const 88 ] , reg1
104: mov reg1 , const 1935896181
128: mov [ const 92 ] , reg1
160: mov reg1 , const 1819042862
184: mov [ const 96 ] , reg1
216: mov reg1 , const 4293632
240: mov [ const 100 ] , reg1
|
the numbers moved to reg1 represent "TexFuncs.dll"
now we cann access the string with adress 88...
3rd pass - compile the functions
now we compile all the funtions...
we start with the standart 'header' saving esp and incrementing esp by
the amount we need for local vars...
then we parse for vars needing initialization(like we did for the global
vars)
in the second run over the function we look at each line(seperated by ';')
weather it is a var def(then we forget it as vars are already inited)
or some keyword(like for,if else,while,...,asm)
or something else...
if it's a keyword we add the code needed to perform what the keyword implys
(if the keyword is 'asm' then just labels are solved and adresses of
vars are loaded..
the rest of the code goes unchanged)
if it's something else:
-we look weather there is an '=' sign in it..if so then this codeline
assign some value
to some adress..so we load everything till the '=' sign into the cache
and parse it..
(and cry out loud if no adress is returned)..then we put the rest of the
line in the cache and assign the result to the adress gathered
before
-if there is no '=' we just put the complete line into the cache and
parse it..
4th pass - link info
last but not least we write the items we have no adress for (extarnal
vars/funcs)
and their adress within the output file to the outputfile..so the game can
link that at load time...
here comes something interesting...
I added a new flag to my virtual cpu which indicates (if set) that a
function call or access to some adress ([adress]) is
external..external here means that this var/function
is a true one(no script..but pentium compatible compiled)..
that flag is set beofere calls and [adress]..
another thing I added I call realtime linking...
if the VirtualMachine(VM) cannot link(find) some adress it just notes
that...
now you can somewhere in your code load a dll which registers the adress
needed to link your script(this MUST happen before the unlinked adress is
used)
and after you loaded all the dlls you call a func called RunTimeLink which
fills in the
missing adresses (and crys if there is still one missing)
there is hell a lot of more stuff in it..this is just a rough describtion how I
did it..if you got any quests..feel free to mail me..
I just wanted to add that the implementation of arrays in msvc is quite
strange...
generate an array (DWORD m_array[10];)
now m_array==&m_array ..what is quite confusing
in my implementation you have to write &m_array to get the adress of the
array
(like you have to do for any other variabletype in msvc)
in my script m_array==m_array[0]
I added a logfile from my compiler which shows the different outputs for each
step of the compile process...
------- dynamic shaders -----
everybody seams to use em today..shaders...q3a started using em...
and the system nehind em is rather cleverand gives you much freedom..
I read that q3a compiles the shaders ...so they can be executed fastly...
but that adds a problem..if there is a new card coming out with a new
special effect
(opengl extentionmaybe)
which was not implemented in the compiler/game then the level designer
cannot access that feature....
my shaders are written using my c script
I added some new asm instructions (like blend,texbegin,dpeth,...)
which access their opengl counterparts rather directly so you can fastly do
your shader stuff :
example doing a 3 pass blend:
__asm
{
sne //all call rever to external funcs
texbegin [TexIds] //activate marble.tga
blend GL_ONE,GL_ONE //blend mode
tcmod TCM_SCALE,2.0,2.0 //scale texture
tcmod TCM_SCROLL,0.2,0.2 //scoll tex
push 0.2
push 0.2
se
call TexFuncs_Turb
sub data,8
texend //draw or upload tex to tmu
sne
texbegin [TexIds+4] //activate env.tga
blend GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA //blend mode
tcgen TCG_ENVIRONMENT //generate evironment tex coords
texend //draw or upl 2nd tex
texbegin lm_id
blend GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA //blend mode
texend //draw or upl 2nd tex
flush //draw tex not drawn yet
}
//se/sne enable/disable the extern flag
|
the nice thing here (which soves the problem I mentioned at the beginning) is
that
you can call external functions (like 'TexFuncs_Turb' )
which may contain code initing a new opengl extention and using it or just
anything else..
but the dll and the textures used in this shader need to be loaded first..so
when the shader is used for the first time the LoadTexture function (standart
interface) is executed once:
#define NUM_TEXTURES 20
extern void* Importer;
extern void* GLRasterizer;
DWORD TexIds[NUM_TEXTURES];
__cdecl void LoadTextures()
{
float area[4];
area[0]=0.0;
area[1]=0.0;
area[2]=0.0;
area[3]=0.0;
ImportDll(Importer,"TexFuncs.dll",area);
RunTimeLink();
TexIds[0]=TexLoad(GLRasterizer,"marble.tga");
TexIds[1]=TexLoad(GLRasterizer,"env.tga");
}
|
here the dll containing(and registering 'TexFuncs_Turb' via some standart
interface) is loaded and afterwards the still missing adress (in the asm code
snipped) is filled by calling RunTimeLink().
//I'll add the complete shader file as attachment..so you can have a look
(the constants for opengl are not properly yet..)
when the shader is beeing unloaded a func call CleanUp is called which
..well..cleans up the shit :)
__cdecl void CleanUp()
{
UnloadDll(Importer,"TexFuncs.dll");
TexDelete(GLRasterizer,TexIds[0]);
TexDelete(GLRasterizer,TexIds[1]);
}
|
I atlked about registering function from a dll and mention my plugin system in
the begining..but that's enoug of my confusing thougts today..I'll tell next
time about it I guess..
mail me any comments..or whatever...
ALex
07/05/2000 - fastculling
05/15/2000 - parties&fpu fun
04/03/2000 - Tech File Update
02/11/2000 - Tech File Update
01/14/2000 - Tech File Update
06/28/1999 - Rendering Cities
06/24/1999 - Bones Animation
06/22/1999 - Introduction
This document may not be reproduced in any way without
explicit permission from the author and flipCode. All Rights Reserved.
Best viewed at a high resolution.
The views expressed in this document are the views of the author and NOT neccesarily of anyone
else associated with flipCode.
|
