South African Game Development - Ultimate Quake Engine

Quite a few things have been happening since my last entry.
Yes I did work on the project through christmas and the new year's festivities.

Most of the code since my last entry got restructured and rewritten to accommodate features as I'm moving along.
Its interesting how one's line of thought evolves as you are working on a project.

What have happened?
Unlike 18 December, I have split the data structures now into two distinct parts: "disk" structures and "memory" structures.

The "disk" structures are only used to store data loaded from the Quake media files, in this case BSP data, from where it is used to build the more practical "memory" structures used by the game code.

Apart from all the rewriting some code the new features added to the game project is:

Changing Lists to Arrays
Lists are now only used within the "disk" structures where performance is not such a big deal, since the benefits of lists are in its ability to just add more elements as you are going about loading the data, especially when you are not sure how many pieces of information you are loading.

Once the loading of information is done the data captured in the lists are copied to the "memory" structures in the desired format in arrays which is allocated at "memory" structure load-time, sized to the exact loading capacity.
Arrays are about two 3rd's faster than using lists from what I've gathered while testing.

We have all the performance with no wastage.

Removing Co-linear Vertices
By removing co-linear vertices we get no visual benefit, its purely a performance benefit.
To make things clear, in Quake a "Face" is not a single triangle, its a collection of triangles that form a flat surface having a single texture assigned to it.

In this case we are using a "Triangle Fan" for each "Face".
By removing co-linear vertices we reduce the amount of "fans"/"triangles" each face would have saving us a few vertices, triangles and UV coords for textures and lightmaps to process each frame.

In the bigger picture most vertices are shared between faces, but when we talk about face data in "memory" we have to be able to deal with them individually because soon we will have PVS/VIS implemented that renders per-face, so the connection between faces doesn't matter in the vertex sense.

This means there is a lot of faces that will have odd vertices on their edges for no aparent reason.
Imagine it as knots in a string of wool under tension, or even better dots connected by lines, but when you are done it forms a straight line.. so why do we have five dots when two will do the same job just perfectly.

Basically what happens is we test the vectors of the previous, current and next vertice, if it appears to form a line, we remove current, then we move one vertex onwards and do the same test until we are finished with the number of vertices saved for the "face" in question.
When we are done, all vertices that will remain is those changing the "vector"/"direction" of the edges of the face.

Such a co-linear vertex is formed by two faces meeting on the edge of a third face forming t-junction vertex.

Quake Sky Rendering
To get Quake skies up we need to know which textures are sky textures. This is made easy since any sky texture in Quake starts with "sky".
Once we found the sky texture we should check its size which is supposed to be 256x128 in width and height.
The sky textures consist of two textures actually, the first (left) 128 block is the "alpha" sky texture and the second (right) 128 block is the "solid" texture.

The first order of action is loading the data into two seperate data containers (arrays) because there are a few operations that needs doing on the raw texture data before we upload the texture data to the graphics card.

First we need to calculate the average RGB color of the "solid" sky texture and store that temporarily.
Next we need to update the "alpha" texture, but before we continue I need to explain what was done to the alpha channel; We updated the alpha channel to a value of 0 for all pixels that has an RGB of 0 0 0 and an alpha channel value of 255 for anything else.

Now we update the color pixels on the alpha texture to our avarage color we calculated previously where the pixels has an RGB of 0 0 0.
Why do we do this? Thats because every dark cloud has an average color lining!

Did I say "average color lining"? I meant "silver lining".

Seriously...

We do this to aviod dark fringes when we blend the alpha texture over the solid one.
The fringes are formed because of linear blending of the textures on the edges of the alpha.
By doing this we just "hide" the fringes better, they are still there and the cake still remains a lie.

From here on its just blending the two together and scrolling them, but that stuff we will leave until a next time.

Animated Textures
The was probably one of the more easier things to get implemented.
If you were able to load all the texture data successfully then you are pretty much 90% there.

With quake textures we have a few identifying pieces of information we can derive directly from the texture name itself.
Textures starting with "sky" means its a sky texture (duh?); Starting with "*" means its a UV warping texture, like water; Starting with "+" means its part of an animated texture set.

We can derive how many frames an animation has by using the second character of the texture name out of a maximum of 10 frames normal and an optional maximum of 10 frames alternate.
They are identifiable as "+0" to "+9" for normal and "+a" to "+j" for alternate animations.
Alternate animations are typically used when you trigger something, like a switch, for example normally in Quake a button would flash at you red and lightred, this is the normal animation.. once you trigger it it goes green on you, this is the alternate animation which in this case only has one frame.

For this case we created links between the different frames each being an indice to the frame in the texture information as they are stored in the "memory" structures. All that needs doing now is attaching the animations to the game code's timing and there we have it.. animated textures!

Resampling of Non-Power2 Textures
This was quite a mission to sort out especially when you are not making the needed changes all around your code.
I noticed for some time that only some of the textures in a Quake BSP rendering is coming out all squashing in any of the given dimensions.
At the time I thought its probably some odd rendering bug with the way the wireframe gets built.. definitely not the texture loading code, since then all the textures should have some odd problem, right?

Wrong!

After some fishing for it, I finally uncovered the underlying issue.. some Quake textures are not Power of 2 textures!!
It really started irritating me when I loaded-up start.bsp and found that the coolest part (for me) of the game is messed-up, and that would be the introductory "quake" name.

The "quake" texture is non-pow2 at a size of 288x64 where it should be either 256x64 or 512x64.
But theres a trick involved here beyond the trickiness of writing a function that successfully resamples the texture; We need to store a "scaled" version of the texture width and height and rather use that everywhere where we are working directly with the texture data. This is because we still need the original texture width and height to calculate the UV texture mapping of it.

The texture will look stretched when you are sampling up from 288 for 512, but once you render it and properly UV map it again theres not much difference you can see since theres no loss that takes place other than a bit of extra texture memory.
You would rather always want to sample upwards opposed to downwards, because you will be loosing pixel information even if 256 is closer than 512.

Lets now get to the interesting business of viewing screenshots!

21 December 2007

E1M1. Perfect Quake-style sky rendering that matches the original Quake.

26 December 2007

E1M1. Pay attention to the brightness of the red coloring in the animated texture on the side of this slipgate.

26 December 2007

E1M1. Pay attention to the brightness of the red coloring in the animated texture on the side of this slipgate.

1 January 2008

START. Resampled "quake" texture, one amongst many non-power-of-2 resampled textures.

Next on my list is getting some of the client structures implemented, loading the entities into the "memory" structures and then get styled lightmaps working.
Now I need to do some more work to be ready to bring more good news on my next entry.

Lets start by saying that those two words used in my subject has unleashed quite some interesting responses.. in far more ways than I expected. Given the facts from all sides, this is what is happening and what is possible through this.

First question would probably be: "Why Quake?"
Its very simple actually. Quake is the most simple of the modern-day polygon-based FPS games out there, with the least or none extra logic that is not the gameplay itself in making a simple FPS game.

Your second question would probably be: "Why emulate/re-write Quake?"
To set the record straight, its an emulation of Quake "The Game" and not a re-write of Quake "The Engine". Infact, the old engine is so outdated right now and has such a lot of porting problems and design flaws in it, it will be just as hard fixing it as it will be re-writing it.

So why re-write it, emulate the game logic and media data and render it via a modern-day engine, whatever that engine will be.
Games are becoming so large these days, its easy to decide on which side you want to be, engine development, or game development.. I see myself in the latter.
Given the fact that lately I'm not all that much into C/C++ and being a C# convert I would much rather spend my time honing C# skills than keep on falling back to C development when it comes to Quake and general game development.

Why do I talk about falling back all the time?
The answer is twofold:
It's about doing work once and getting a double-payout from it.
Firstly I have started the UQE project (Unofficial Quake Engine) a few years back and as the months progressed until the current day the project generated quite some following and it will be a shame and a very personal failure to stop supporting it. I'm a very passionate person when it comes to people enjoying your work.

The first step is to get them to still enjoy the old Quake series of games, but in the same breath allow me to maintain less or easier to manage projects.
Once we reach full emulation of Quake (it would be cool to emulate Hexen2 and *maybe* even Quake2) we will be using our newly developed technology to create a small and very highly playable FPS title that we will be selling online either through our own portal or through another; obviously I'm leaning towards having an own portal even though there are people not agreeing with this approach.

Creating an emulated version of the Quake game will help much with regards to us writing technology that will fit and compliment a previous AAA title other than that it will help building some support for international and local people to see and appreciate South African produced technologies and hopefully generate enough interest to help sell the games we make.

With a fully emulated Quake game we have all the technology we can use to make a simple FPS while not competing directly with current AAA titles, then primarily focus on changes and fine-tuning our already developed technology enabling us to give more attention to the art side of the project.

For this project we are currently using Truevision3D version 6.5 unless we find something else along the line.
Changing engines will be fairly simple as most of the Quake interpretation is managed by pure C# code which gets passes to the engine in question in the way the engine expects it.

Now we get to the intresting part.. progress screenshots!!

Sofar the game engine project is in its second week of development.

8 December 2007

E1M1. The first successful rendering on Quake BSP level data loaded with pure C# code. The lower part of the image is the starting spawn point in the level.

8 December 2007

E1M1. Changed the vertex coordinate system from Y being depth to Z being depth and reversed the primitive edges. The shot was made in the first outdoor area with the bridge in view.

9 December 2007

E1M1. Shot made in the first room where the starting spawn point in the level is located. (where you start playing Quake)

9 December 2007

E1M1. After a little bit more work the texture information got loaded and converted from 8-bit paletted textures to true color RGB(A) textures (runtime) and mapped accordingly to the correct faces after having to calculate each face's UV coordinates.

9 December 2007

E1M1. A little bit deeper into the level (near the finish) another rendering of the newly textured environment. This shows the camera slightly "outside" the world. In the background also visible is unfiltered brush models which must still be managed.

9 December 2007

E1M1. The equivalent fullscreen rendering from the original Quake engine, here only lightsmaps are obviously working and a bloom effect I've added to the engine some months ago.

12 December 2007

E1M1. After much trouble getting lightmaps loaded, converting the data into blocklights and calculating the lightmap UV coordinates we have here some lightmapping results with some bugs.

13 December 2007

E1M1. After spending another day pulling out some hair we FINALLY have fully functional static lightmapping rendered here in fullscreen mode!! This excludes lightmaps with added styles and dynamic lightmapping.

13 December 2007

E1M1. The 10 lightmap textures generated. So, now that the lightmapping textures is sorted, heres the lightmap textures being generated (not easy) at runtime (level load) and blockmaps allocated within each lightmap texture (not easy) of which the lightmap coordinates needs to be calculated (not easy) before you get to load these into memory and render them. The lightmaps above are those of E1M1.

17 December 2007

NEWMAP. Lots of days have passed and last bits of remaining hair have been pulled. Showing here in fullscreen (not easy to spot) is fully functional water warping! Look closely at the 64x64 texture seams and you'll see the warping.

Onto some feedback on where I struggled with the tech up until now..
With the lightmapping there are such a lot of calculation work that needs to be done before you get to the point of being able to render the lightmaps. Its not as straightforward and basically a given like with the Quake III Arena BSP format.
What was needed here is to load the lightmap data into lightmap images roughly estmated by lightmapsize divided by 128x128 blocks, then you know how many lightmap images there are. Like you have seen, each lightmap image contains lots of pieces of lightmaps (for level surfaces). the lightmap's coordinates within the texture had to be calculated, once found you would also need to calculate the UV coordinates on the surfaces to map it correctly. It was quite hard to get all these calculated and working correctly.

The waterwarping was hard too.
When I started out I thought: "Ag lets quickly do the waterwarping, should be easy!"
How wrong I was!

Firstly, I knew already how to do proper UV warping since I've done it before (on the vertices) of a Quake2 level viewer I developed some years back.
With horror I realized that water surfaces in Quake1 wasn't pre-subdivided for you, like they were in Quake2.
Subdivision of water and sky surfaces was done at load-time and the surfaces was updated while the level loaded adding the missing information by calculating and subdividing! The reason why it wasn't already subdivided is simple, Quake1 and its technologies was developed and designed way before hardware 3D, so waterwarping was done very differently in the original Quake's DOS software renderer than the later OpenGL version.

So, after a day and a half of struggling I finally got the water and sky surfaces subdivided, recalculated the face extents and UV coords.. finally I was in a position to implement water warping! Warping is working now and my calculation renders a result that matches Quake's rendering by an estimated 95% accuracy!

Our implementation further uses a real Sine calculation opposed to the original Quake's pre-calculated Sine array, which is quicker but renders more jaggy results on faster/slower framerates.

Onto the next challenge: Getting perfect Quake-like sky rendering and writing my next entry..