
Hi,
The way to compute the OOB per bones is explained on page 2728. You must use Principal Component Analisis (PCA). This is a data processing technique that is used to find an efficient space to represent data. The core of the problem is to find the orientation of the OOB.
First you have to compute the covariance matrix using the given formula. It is a 3x3 matrix. It is computed with all your vertices. If you are not confident with the maths I'll explain you the notation : the Xi are all the vertices influenced by your bone. X with a bar on top is the average of all the Xis... just sum them all and divide the result by their number. The little t on the top right of the first term means transpose, that means that you will consider your vector as a line rather than a column. for example if one of your Xi is (5, 7, 3) and your averagi is (1, 2, 2) you have Xi  average = (4, 5, 1) then perform the vector multiply as follow :
4 5 1
4 / 4*4 4*5 4*1
5  5*4 5*5 5*1 
1 1*4 1*5 1*1 /

add all these matrices for each Xi and then invert all the coefficients. You got your covariance matrix. Now you have to compute the eigenvectors of that matrix (http://mathworld.wolfram.com/Eigenvector.html). You can find a lot of math libraries that will do that for you. Normalize these 3 vectors, Put them side to side in column and you have the rotation matrix of your OOB.
to compute the size find the min and max on the axis X, Y and Z of each vertices that you will have previously transformed by the inverse of your rotation matrix. Et voila...
As this computation is costly I would advise you to make it as a preprocessing stage.
I hope this helps,
Phil
