Let the party begin

Okay, yes. Neglected blog. Badly neglected. Come into the warm den of my attention and I shall nurture you back to health.

*GlaDOS voice* It's been a long time...

There's far too much I've been working on to cover in a single blog post (GlaDOS voice is off now btw), so I've decided to have a week of BLOG POSTING EXTRAVAGANZA! Each day I'll make a post focusing on a particular area of activity.

This brings me to the troubling conundrum of where to begin. After some deliberation, it became obvious it didn't really matter, so I just picked whatever I felt like writing about right now. Thrilling, I know.

So let's take a look at Netherworld Odysseys! Though this had a brief limelight of one post in this blog, I did actually dedicate a considerable amount of time to this WarCraft 3 map. I know you don't know or remember what I'm talking about, so a quick refresher:

Netherworld Odysseys (or just Netherworld or even Neth if I'm feeling lazy enough) is a custom WarCraft 3 map that is basically a Diablo clone. I wanted to focus on party interaction, challenge and replayability. These goals eventually expanded out into the following feature list:
- Random loot and enemy placement
- Multiple difficulties that get hard. So hard that you call me a bastard.
- Difficulty also scales with number of players
- Large variety of settings and situations
- Classes with multiple strategies and unique roles among the party
- Saving and loading of characters for each player

So that's the gist of it. Where I left the map after the last post about it was miserable. The map was starting to crash the game after a certain amount of time playing. This hamstrung the majority of goals I set out with the map. The most sucky thing was it always crashed just as it was getting good. The party was establishing its roles and equipment. The levels were getting tough. You were just about to charge into the crazy boss fight tanked up with buffs, potions and an elaborate plan.

Then the game crashed.

Not cool. At that point in development I had no idea how to save or load characters. I was focused so heavily on using the built-in cache feature (which only works for single player) that I just couldn't see any other way.

Then my housemate Fox pointed out this saving system he'd seen in another War3 RPG map he'd played a while ago. It printed out a code that you wrote down and typed in later when you wanted to load the character again. NES style. I leapt at the idea. Perfect! Easy!

So after a little bit of time I had that up and running. The map was back on track!

As a side note, yes I agonisingly hunted for every possible cause of the crashing. I spent at least a week on it (I don't know if that's impressive or not). The crash ultimately was something I could not fix.

Establishing the saving and loading greatly reinvigorated my enthusiasm for the project. I started adding new classes, new areas, items, bosses, abilities. The stages started getting more elaborate. A boss fight could no longer be just a big dude with a lot of health. The battle needed to evolve as you wore him down.

Suddenly the boss would start busting out a brutal new ability (video game cliche #??). You've run out of potions... now you need to focus on the towers to get a healing rune drop, now there's minions pouring in from the sides.

Playing around with these ideas was insanely fun. It began to carry over into the level design. At the same time I was reading this D&D Dungeon Master's Guide that I had bought at this book market a while back (claimed the "find of the day" by Ben XD ). That was really fuelling my ideas for situations and elements in the environment.

Big horde of enemies ambushing you? No worries, just Shockwave that support beam and cause a cave in to crush them; or hit that barrel to set the area on fire; or anger the wildlife then cloak to cause a battle between the two forces!

For this reason, the most recent 2 dungeons, and especially the last one, took much more time to make than the starting areas. Ultimately I started to get burn out. There are 7 levels in the map as it stands, and I have 2 more planned but not implemented beyond basic concepts in my head. It would be very satisfying to return to this project (it has been dormant for... at least 9 months) and finish off those last two levels (including the EPIC final boss battle), and complete all the classes (Barbarian, Paladin, Mage and Druid are fully playable, Assassin and Ranger remain incomplete).

If I do so, I'll dedicate an entire post to its release, talk about all the classes and various strategies my friends and I have tried while play testing.

Oh the play testing. Funny stuff. Ben and David had a LAN at their place and gave a bugged version of the map a play... and were tortured for hours. The bug? Sometimes the 5th level boss would spawn as a regular enemy. They didn't know it was a bug... they just thought I was an evil, satanic being that feeds on suffering.

Then there was the LAN we had before that at my place. There was a particular combination of enemy types that spawned near the end of the most recently implemented stage that was ridiculously hard to counter. Ultimately David rage quit and told me to "fuck that map". Ah, good times.

So yes, blog post about shelfware War3 map. Like I said, I wrote about what I felt like, not necessary the PEAK OF THE EXTRAVAGANZA. More like the tip. Oh clever puns and metaphor interplay.

Return tomorrow for more incredible content.

Skylines

Been working with more traditional materials lately. Attempting to develop a solid field-sketching work flow. Getting used to working with an artist's grip on the pencil and drawing confident strokes with large sweeps. Will scan in my work when I figure out how to exactly.

In the mean time, some paltry offerings:

Concept art for a sprite idea I had. The idea primarily sprung from thinking about palette, especially the neon green. Awful anatomy :)


Concept sketch for a 3D model to be used in a Team Fortress 2 map I'm making. Learning Blender in the process, great app.

Random doodle I did after work one night. Mainly playing around with brushes.

Background concept for another project of mine which I'm flirting with resuming. Really happy with the direction of this.

Resuscitation

Right, where was I? It's been a while. What happened? Where have I been? What have I done, and why have I returned?

I needed time to consider what I'm doing in my life and where I want to go. I needed to think about the world I'm in and what role I play, as I felt that leaving this issue largely unattended was the cause of a lot of the difficulties I have been having in progressing.

It's a long story, but I believe I know the answers as best I can now, and am moving forward again. So, does art still play a role in my life? Certainly; that is to say of course it does. The direction and focus of my art has become clearer, though.

I doubt I've mentioned it on here before, but I'm very much interested in broad social issues. There's a lot of things about day to day life and society in general that have bothered me for a long time. I used to be very critical, negative and irritable about a lot of things. Over time, however, my frustration has given way to determination and clarity.

Over the last couple of years I've read things here and there on the net, and each item has gotten me closer to a big picture of society. Much of this culminated in the discovery of the Zeitgeist Movement and the Venus Project. To me, these now are the principle concern of my efforts. If you are unfamiliar with them, I urge you to look at them. There's movies, podcasts and books (just the tip of the iceberg). A wealth of information that will most assuredly broaden your perception.

I've actually been considering how to marry this blog to my Venus Project focus. Either a sister blog will be created, or this one will need to be reworked. Still deciding...

So back to my art. I will try to the best of my ability to use what artistic skill I have to help spread awareness of the Zeitgeist Movement and its core ideas. Where exactly that goes, I don't really know. I imagine a lot of it will be things like banners, posters and fliers. Though I did have a few chats with someone about making a Venus Project themed comic for a while. The possibilities are many, at any rate. This is the first real thing I've turned out:


It was originally conceived as part of a series of posters to be placed around university campuses. The idea I had was to have different posters for different schools within the institution, so concepts could be presented to people in ways that would appeal to what they study. For instance, you have issues about behaviourism in the Psychology buildings, issues about censorship and philosophy in the Arts building, and, as is the case with the above poster, issues of technology in the IT and Engineering buildings, etc. I have an idea for a business one which I hope to turn out soon.

Just for fun, here's a concept sketch I did for the Venus Project comic:

The protagonist within his gloomy industrial work center

Aside from that, I will continue my artistic study in order to improve my skills. Not a great deal of time has been committed to this, but progress is being made nonetheless. Here's how it's shaped up:

All posemaniacs references. These were mainly concerned with getting gesture and loose, confident strokes.

Photo reference. These were for just studying real people. The second woman in particular has a very interesting body.

Mainly posemaniacs stuff. Playing around with charcoal brushes. I really like the grittiness and ambiguity it gives pieces. I see myself using things like this a lot more.

Just a weird thing I did. Originally it was meant to be a concept sketch for a new pixelation avatar. I basically had this concept of a very concave face shape and glowing eyes, but it ended up as a goblin man painting, so, there you go...

Painting concept; something I'd love to do on a traditional medium at some point. Part of a series on the body and spirit. Part of a long, intricate vision I had while meditating.

Anatomy study needs to be returned to. Shading study could also use some redressing, but for the most part I'm feeling good about the direction I'm heading. Very happy with the palette I've developed, makes work very fast and easy. Now I just need to get a good selection of brushes going and I'll be set.

P.S. I cut my hair!

Artistic Transmutation

Well I suppose things haven't been as productive as is possible. I've seen myself essentially shift from the position of a producer to that of a consumer. Rather than draw, sing and write, I view, hear and read. This has resulted in a stagnation of inspiration as I lose impulsive association between what is perceived and what is expressed.

In steps to reverse this alarming position, I reformed my own opinion on the matter, transmuting the situation from indifference to concern. Once this critical realisation was manifest, events aligned to assist in an actuation of this concept to reality. I had a good talk with my old friend Jack, whose reassurances of my inspiration and abilities pushed my resolve and momentum towards action, further resulting in his gifting of a book to my possession. It's an addition to the paltry collection of art books I hold, and it eerily mirrors my current artistic visions and sensibilities.

Following this event was the reconsolidation of the artistic alliance between Nick and I, where we have undertaken careful study of human anatomy for the purposes of drawing. I've been doing a considerable number of sketches since that commencement, and improvement has been marked, though yet to be notably achieved. Prepare for the largest dump this blog has seen to date:

The first sketch, from reference

Another from reference, still on head study at this stage

More head study, from reference

Following from notes, head study

Neck study, this is totally messed up

More neck study, even more messed up

Neck study, following from notes

And finally, more neck study from notes, some serious errors on head, and extension of study to shoulders

Each section of study will need numerous reviews. I am currently looking over the torso and pelvis, and hope to delve into some sketches of those in the next couple of days.

Rejoice!

A Special Kind of Hell

I've spent the majority of today attempting to get Synergy to compile.

Basically, as far as I can tell, after I formatted my computer some time ago and reinstalled MSVC++ 2008, reinstated all the libraries that Synergy uses (Allegro, AllegroGL & SOIL), at some stage something went wrong.

I loaded up the .sln file, and hit Build, and I was confronted with one of the strangest errors I've seen in a while: something about an inline assembly code error, expected an opcode and found a data type, or something bizarre like that.

I started to systematically compile each individual .cpp file, and found that all of them except 1 compiled just fine. The troublemaker that caused the problem was gam_Entity.cpp, which is weird because that's a very simple source file, with very few dependencies and very straight-forward class-definition-style code.

So I kind of fiddled around in that file, commenting out sections of code and recompiling, trying to find a particular line that spouted the error, and I found that no line caused it, since I ended up commenting out the entire .cpp and .h files! By this point I was just downright flabbergasted, and I turned to google for help.

I found a couple of pages that mainly went through fiddling with project properties, cleaning out .obj itermediary files, that sort of thing. But none of those seemed to fix it. So then I started thinking maybe it was something about where Allegro was being included, so I started including allegro.h and alleggl.h at different points in the header file, commenting out includes, trying #include instead of #include "allegro.h", random trial and error things like that.

I don't remember exactly at what point it worked, but somewhere it worked. As far as I can tell, no one thing that I did fixed it, it just seemed to `come good', which is troubling.

Anyway, after that assembly error stopped coming up, I had a new set of errors coming up; a tonne of link errors. Welcome to the next nightmare. This was basically about 180 LNK2001 errors saying various identifiers had already been included in .obj files. I did more scanning of the net and went a little crazy, and then I realised I had actually gone through all the functions in m_common.h and removed the inline from the beginning of each non-template function definition in an attempt to fix the previous problem. Dur. So I replaced the inlines and bam! No more link errors.

But then I got a single error; something about P1 and P2 being different versions. Scanning the net more, I found out this relates to the compiler having incosistent versions of itself, or that libraries that I'm using have been compiled with different version compilers, something quite likely. So then I kind of started losing my mind, and tried bringing in different versions of each library to MSVC++ 8.0, downloading and installing MSVC++ 9.0 and installing the libraries for that, and even going back to MSVC++ 6.0 and trying there.

What I ended up with was uninstalling 8.0 sort of out of some irrational belief that it would help to quell the curse upon my computer, but also (more rationally) that I should try reinstalling it at some point and carefully version checking each library I bring into it.

MSVC++ 9.0 overcame the P1 != P2 error, but now SOIL.lib was spouting a couple of link errors to unresolved external symbols __alloca() and something else. So now I'm quite lost. My guess is some version inconsistency between the 9.0 compiler and the compiler that SOIL was written for, ie SOIL only compiles on 8.0 or earlier, or something like that. That's something I mean to test when I reinstall 8.0.

I then tried to comment out the SOIL code and I was left with a single link error about MSVCRT not being able to locate _main. So I followed some instructions on a website to fiddle with the MFC settings in the project settings, which just spawned more errors so I reverted back to EXACTLY how the settings were, and now I have 7 errors. It's absolute madness!

MSVC++ 6.0 finally managed to compile and execute an older version of the code once I commented out all reference to SOIL. This is basically the best result that I've achieved. At this point I'm looking at bringing in all the code that I've written since this old version, getting it to function, and then (hopefully!) compile and run. At that point I'll be back to where I was, except in 6.0 instead of 8.0, to which I say good riddance. This might also be a good time to rid myself of AllegroGL for good. One lesson I take from this is `the less dependencies, the better'.

So there it is, my special kind of hell. To be honest, this should never have happened. Part of the function of this blog is to act as an engineering journal, where I can record version changes and things I do to my projects which might end up leading to something like this current scenario in the future. I should have had logged which version all of my libraries, IDE and compiler were and the directory of my last known successful build of Synergy.

I won't make the same mistake again.

--------------------------------------------------

Synergy Version: 0.0.0
IDE Version: MSVC++ 6.0
Allegro Version: 4.2.2
AllegroGL Version: 0.4.2
SOIL Version: 2008/07/07

Current Status:

• Compiling & Executing
• SOIL producing two LNK2001 errors; all SOIL references have been commented out
• Thus, textures currently not loading
• Old demo version; multiple smoothed 3D objects floating in space

To Do:

• Eliminate all AllegroGL dependency
• Fix texture loading
• Implement multicore functionality
  
• Reintegrate following features:

1. multiple material meshes
2. the `Big Room' demo
3. lightmaps
4. unified lighting attenuation system

• Implement uniform lightmap patching system
• Implement physics library integration

Multi Yodel Scope

So, I've been on holidays for two months or so, and have been partaking in eclectic activities. Most of these are not significant to the focus of this blog, however I have managed to glean some constructive work from this time.

First up, I've given some attention to restructuring Synergy to support multicore processing. I had the design goals to allow the engine to scale with cores at run time, to have next to zero locking mechanisms in the code, and to have the multicore functionality as transparent as possible.

What I ended up with is largely a combination of two materials I read. The first is an introductory article to writing multicore games for the Xbox 360, which gives a good overall breakdown of an engine's structure into parallel processes. The second is Valve Corporation's presentation on Source Engine's multicore concepts.

Taking these two together, I basically have two `Delegator' threads that run through the structure outlined in the 360 article as 2 parallel processes. Every significant independently executable function within those threads is called with a function wrapper task(), which takes a pointer to the function to be run, and up to four parameters caste to void type. The task() function takes the information passed to it and builds a linked list with it, and then pushes that linked list onto a lock-less task queue.

All the while this is happening, there is a thread pool with some number of threads all executing a peon() function. As the name might suggest, peon functions continually check the task queue for a task, and if they can grab it they do so, break the linked list up into a function call, and then execute the function representing the task to be done. Upon completing the function, the thread returns to its peon() function to continue checking for more work to do. As you might guess, the number of peon threads in operation is related to how many CPU cores are visible to the engine. The more cores available, the more peons can be running.

Within all of this is a simple semaphore system whereby a task in the queue can be associated with a semaphore. This allows the delegator threads to wait until a certain cluster of tasks in the queue are completed before adding more tasks. This is a necessary evil simply due to the serialised structure of games themselves, and at the moment is the most logical solution I can think of. This is, in fact, a locking mechanism, but it's only operating when the actual logic of the program cannot proceed until the current tasks are completed, and it should minimise any kind of thrashing, since the peon threads never rely on the semaphores.

So that's Synergy's multicore structure at the moment. Nothing too sophisticated, and still yet to actually be tested! But at least in theory it meets all of my design goals for the moment.

Next up in my projects is a Team Fortress 2 map! I've been itching to do some mapping for a while, but struggled to settle upon a good idea until I was reading over the Valve Developer Community Team Fortress 2 pages, and there was an article about Circular Control Point map types. This is a game mode currently not implemented in any official map, but has the potential to be an extremely dynamic game type if implemented appropriately.

After reading about it, I had a rapid explosion of ideas for a map, and begun to sketch out every area in photoshop, paying careful attention to attack/defend dynamics, flanks, killzones and setup areas. As well as the obvious considerations for each class.

After about a day I had the whole map sketched out and detailed with all major routes, spawns, setup areas, rocket jump access, etc, and began to actually rough the map out in Hammer. As of writing this I have the entire map roughed out in seperate map files and am currently bringing them all together and playing a juggling act to get them to all fit together. Some points have considerably larger areas than others, and so alot of scale harmonising needs to be done, which is a little tedious.

I'm hoping to have a tight, playable alpha of the map ready by the next tf2mappers.net game day, so it can have some testing done by fellow mappers. Anyway, though it's hardly eye-candy material at this point, here's some screen shots:

Battle area below D

Attacker's far staging area at C

Primary indoor contesting area of C

Oh and here's some random sketch I half-finished up the other day:

What if the mountain defeats you?

Well, I've been largely preoccupied with the struggle between procrastination and obligation that is typical of my student career. For the most part, the only things productive I have been working on have been my Systems Programming assignment, which essentially involves developing a shell for UNIX.

Outside of that (and largely useless activities such as online gaming), I have tried to find time and motivation to work on some kind of creative activities. I'm trying to avoid diving into my large projects (such as Synergy) for a few reasons:

- it is mentally draining and involving to work on them, and I need to maintain my mental stamina for academic pursuits.
- I'm learning a lot of very pertinent programming methods in my study this semester. Methods which can apply directly to improving the operation of Synergy. As such, I want to wait until I have grasped this knowledge more fully before I apply it to my personal projects.

So my focus has been towards finding creative activities which can still contribute to the development of my overall skill set and ultimately the development of my portfolio of work, but are light-weight activities that I can pick up, work on for a set amount of time and then put down and return to my study.

Ultimately, this has lead me doing a lot more artistic activities, such as drawing. Nick introduced me to a very useful site called `Posemaniacs'. Head over there if you want to see what it's about. You don't want to? Fine, I'll explain: it is a collection of browser-embedded tools for human figure sketching and study. My link leads to the 30-second drawing section, which as you might suspect, involves the presentation of figures in poses for only 30 seconds at a time before changing, promoting rapid sketching and utmost line economy.

I've only attempted a couple of sessions, undertaking a more reasonable [for me] configuration of 60 seconds. Here's how they've turned out:

And a slightly less deplorable collection:


I also partook in a competition over at GeekFanGirl, where entrants had to either draw a scene or character from an active role play, or write short story about it/them. Turns out, I was the only one to do an art entry. I drew one of the main NPCs from my own RP I GM, the wild druid, Taech:


There was a deadline for entries, and mine made it with 40 seconds to spare. To put it another way, this entry was largely rushed. This was a positive thing, in my mind, because I learned a lot about being economical with my work flow. I'm irritated with a lot of things in the piece though, such as the messed up lines and artifacts around the silhouette, the skull structure being a complete deviation from what I wanted, and the completely schizophrenic rendering styles across the image.

Still, I'm pretty proud of the results, either way. I refuse to alter it, leaving it as a record of what I could accomplish in the time I had for working on it.

So that's about it. I'm hoping to make the 60 second sketches a regular, if not daily activity. If I can manage that, then I believe I am well on my way to becoming a competent artist.

Lightmaps cont.

Well, it's been a little while since there has been any progress of note on any of my primary projects, but today I finally felt motivated enough to jump back in to work on Synergy.

What have I been up to in the mean time? After having a game of a WarCraft III map called `Warchasers' at Ben's place, I felt compelled to make a much improved clone of its gameplay. I worked obsessively on this clone, titled Netherworld Odysseys. It has become quite sophisticated, with a generalised random loot system, random monster spawning, character saving, charged item stacking, multiple difficulty levels including difficulty scaling with number of players, door/key system and Diablo 2-style mercenaries. Here's a couple of screenshots just for fun:

Assaulting the Spider Warren

The derelict wastelands of the Prison Realm

Unfortunately the map was foremost designed as a multiplayer experience, but due to a number of technical shortcomings saving and load of characters became a necessary process to keep the map running, and saving/loading of data only works for single player. This in effect rendered the map a single-player only experience, killing my enthusiasm for it.

However, it's not a complete loss. The triggers I developed for it are highly modular in nature and much of the cool features of the map can be reused in other warcraft maps of mine. If I find time and enthusiasm I might return to it and see if I can improve some of the technical issues crippling it currently and bring it back to a playable state for multiple players, get it to a satisfying play length and release it. Until then it festers on my hard drive...

So! What did today bring? I admit that I had become horrificly bogged down with the lightmap functionality of Synergy and thus suffering a fetid case of Coder's Block. The lightmaps needed to be divided into correct patches over the level geometry to give an unstretched, unwarped surface for the lighting to exist over.

Such functionality is non-existent in Synergy in its current state, and my coder's block prevented me from proceeding further with the matter. Thus, today was spent researching methods for patching, and other methods of lighting a scene.

I ended up reading up on radiosity methods, and the literature revealed a great deal of insight to me. Such was the extent of my new-found understanding that I went back to the problem of patching with much clarity and began to sketch out a patching algorithm for Synergy (based largely upon this example).

I'm still going through it, and will continue work on it tomorrow, but as it stands currently it seems most satisfactory. Upon implementation I expect Synergy's light mapping to be devoid of artifacts, uniform and correctly aligned. From there it's a very simple extension to add in some brute radiosity.

Here's hoping there's no hitches.

Let There Be Lightmaps

Lightmaps are officially a nightmare to implement. They seem easy, but I have found otherwise. The last four days have been a battle, but I have made significant progress in implementing this feature in Synergy.

I began with the code outlined in this tutorial. A number of modifications needed to be made, including accounting for multiple light sources and using triangular surfaces instead of quads. I also factored lambertian calculation into the formula. Adhering to the tutorial strictly, plus the minor modifications I made to the formula, I ended up with this strange result:

Puzzle #1

The streaks were caused by a value overflow with my premature byte casting. Working in dwords and then casting to byte after the clamp got me to this (after adding a couple more lights):

Puzzle #2

So it's obvious to see here the mapping coordinates are wrong. After systematically trying every single combination of texture coordinates for each vertex I discovered the closest result was this:

I smell victory close at hand...

This implied the texture needed to be flipped on one axis, so I changed the order the data was input into the bitmap, and lo and behold I had this:

Victory is mine!

Although its close to a correct solution, it's easy to see there are a number of artefacts present in the lightmap, such as the lit borders and holes in corners. All of the problems become much more visible viewing the lightmap in isolation:

Oh god damnit

Higher resolution lightmaps such as 64x64 mitigate the problem, but cut deeply into the framerate. I'm still considering how I can overcome this problem, but I discovered a number of documents and papers on the subject of lightmaps, so I'll be taking a read of those and hopefully gain a bearing.

I needed to implement a basic light management system in order to support all of this process, and it has lead to some rather nice results for the lighting overall in the demo. The main improvement is the addition of attenuation constants for each lightsource which coincide with those used to calculate the lightmap, leading to nice consistent lighting:

Yes, the trees are still around

And now for something that isn't to do with Synergy engine! I was talking to Nick online and he suggested we have a go at doing some sketching together using opencanvas. I've had my head so deeply in coding territory the last few days that it was a little difficult jumping back into drawing mode, but after roughly 15-20 minutes I had this horrible looking sketch of a person that resembles a damn night elf:

Mr Cumbersome

It's ok in some respects, very mediocre in others. Working with the construction was extremely cumbersome, and has highlighted the fact that I need to engage in more studies and revision of facial construction and anatomy theory. Still it was a good process to work through, and having Nick on hand to point out issues as they manifested was a new and interesting experience. We've agreed to spot each other's work (presumably studies) from now on, which I'm looking forward to.

Bits and Pieces 2

More relatively unremarkable tasks undertaken for Synergy today. Most of it is restructuring of code, generalising demo code into specific classes. This results in a fleshing out of the mesh/model class family, including the 2 new classes SynStaticModel and SynDynamicModel.

I've also added some code in SynMesh in preparation for implementing dynamic light map functionality in meshes' rendering and an animation skeleton data member for loading in animation information from 3DS files.

I also created the almighty SynEntity class, of which many other classes are derived. It's just a position and orientation in space with some wrapper functions for doing rotations easily, but that's something that's used over and over and it has tidied up the code considerably.

I also generalised texture loading functionality into 2 functions in the GfxCore class. This has made setting up textures a breeze.

That's about it, I suppose. I'm eager to begin implementing light maps as soon as possible. Hopefully tomorrow night.

Bits and Pieces

I've been sucked into Diablo 2 and other distractions lately (damn you, Blizzard), so progress has been meagre. However, I was determined today to continue progress on Synergy. I have been deliberating over integrating a physics engine into the code, and the task is daunting.

I've decided to make the task as easy as possible by setting up everything the physics tests will eventually need and fleshing out the 3DS loading more. This entailed implementing some functionality that supports environments. There's alot of different features that fall under this umbrella, including lightmaps, materials, entity spawning, and many others.

I decided decent ground work for a number of these things was beginning work on extracting material information from the 3DS files. This task proved to be another work in cryptography, as only relatively vague information is at my disposal regarding the file format.

However, after a number of hours stumbling around in the dark, I was successful: the engine loaded in a model of a simple room that had separate materials for the walls and floor. I decided to manipulate the number in the demo code somewhat to make sure all the objects sit within this new room, in preparation for implementing the physics/collision detection. Here's how it all shaped up:

Almost starting to look like a game?!

There's a lot more material information in the 3DS file I can potentially tap into, but it is not relevant at this stage in the engine. Next I will be focusing on loading in lights from 3DS files, and maybe, just maybe take a look at calculating light maps...

I've also devised an algorithm for welding points within a certain distance threshold of each other. Big deal, right? Well as far as my searching on google went, the approach appears to be brute-force O(n^2) algorithms, from what I could find (excerpt from game programming book). My approach was very simple; apply a 3-dimensional closest-pair algorithm to the vertices, with one modification: when the algorithm reaches an input of only 2 vertices, test if their distance is below the given threshold, if it is, weld them and return infinity for the distance.

This should in effect weld all vertices within the threshold in O(n log n) time, but I have yet to implement it and test its correctness.

Final Battle!

Exams are done and dusted! It's similarly epic to defeating the final boss in a game. So far I have gotten back my mark for Software Engineering; ~80%, so a D. Quite chuffed. So, besides basking in many games of Team Fortress 2, I've been noodling around ideas for what I should focus on next with Synergy.

I did some light research into physics engine libraries that are out there, and the selection is extensive. I was impressed. At this point the prime candidate looks to be Bullet. Failing that, perhaps ODE. I made an attempt to compile some of the many MSVC++6 projects that come with Bullet's SDK, some worked, some didn't. There's no binary currently in existence for the library, so I don't know exactly how to proceed. Investigate the forums, I suppose...

I decided to leave that kind of expedition to a later date, primarily because I had a headache and just didn't feel like I had the energy for it. Instead I decided to perform some optimisation on the 3DS loader function, which is still employing the crutch of the brute force algorithm to calculate smoothed normals, resulting in something close to 5 minutes stall to load in a 50000 polygon, single smooth group (worst case problem) model.

After chewing over the problem for a little while, I sketched out a rough algorithm that would run in O(fv) time, where f = number of faces and v = number of vertices. After implementing it, I found the algorithm `almost' works. Satisfyingly, however, it took about 2 seconds to load in that same 50000 poly model! A very significant speed increase. Conceptually, what the algorithm `should' be doing is exactly the same as the brute force algorithm, but somewhere in the implementation there is a deviation which I have yet to isolate.

I thought the incorrectness may have stemmed from the fact that 3DS files export duplicate vertices for the same coordinate in space. To remove this potential threat to the algorithm, I set about implementing a welding algorithm that would merge duplicate vertices into the one vertex list entry, thus compressing the total number of vertices in the mesh.

This algorithm posed some very interesting challenges. The root challenge is that it could not be less efficient than the proposed smoothing algorithm (O(fv)). From this root challenge spawns many other challenges as a result. The algorithm I sketched out is quite straight-forward, but the implementation proved to be a little trickier. The main issue was `how do you compare vertices?'

I divined the answer to this question whilst going to the toilet (details, details...). Essentially the order you compare the components of a vertex depends upon the order you sort them in, which seems obvious upon reflection. With a newly crafted comparison function, I had myself a linear-time welding function.

However, this welding function did not fix the smoothing algorithm. I've decided to leave further debugging until I've had a night's sleep, nevertheless progress is promising!

EDIT (14:14, 27/06/2008): The algorithm is working! I was missing a test for the final pass on indices to assign new normal values...I wasn't testing if the face was part of the smoothing group! How obvious. Amazing what some sleep can do. The code for the linear-time normal smoothing is pretty decent.

I've also had a look back over the mesh rendering function, and split the function depending on which shade model is enabled, meaning that rendering in Flat mode will use the face normal, rather than the normal of the first index in the face, which gives incorrect results.

I'm very happy that I managed this speed increase! Now I can consider physics integration with a clear state of mind.

Give Me Some Skin!

After suffering an 08:30 exam and the apocalyptic effects it has on my sleeping patterns, I felt I might loosen the screws in my mind by taking another look at implementing texture mapping in Synergy.

My suspicions lay in the library I was using to load texture data into OpenGL. To test this, I downloaded and integrated SOIL. So it seemed I could load in textures, but they were not mapped at all to the meshes.

Which made me realise I had left the texture mapping aspect of the SynMesh class, well, hanging. So going back over that was a simple affair, though I had to make an educated guess as to how the .3DS file's texture mapping data is actually organised in respect to polygons. It is, of course, the obvious answer, which is for each vertex in the vertex list, the texture vertex at the same relative position in the texture vertex list is that vertex's texture coordinate.

A little tweaking of SynMesh.render(), and various other spots in the demo code, and I was in business:

Bless my flesh!

Note, the skin for the tree actually isn't finished. So that's a data thing, not a processing error. Very satisfying, after the endless battle with AllegroGL just to do basic texture mapping. Tsk tsk. Here's a lit screenshot, just for fun:

Trees of Bliss

Criminally Smooth

:). I am pleased. It was a long, hard battle, with much contemplation, testing, probing and trial and error, but Synergy now sports Smoothing Group functionality!

Loading in the smoothing group info was a small mission in itself, one which began with trying to find info on how to get it out of a .3DS file. Implementing this actually spurred a revolution in my coding for being very specific about my longs (`32 or 64 bit?'), which seems to have somehow increased the frame rate of Synergy's various demos by up to 6Hz?!

Building upon the code from that tutorial I linked to in the previous post, the code for reading smoothing groups is about as simple as it gets.

I pondered for most of the day on how the process would actually work, sewing threads of ideas through the problem in my head, considering different approaches. What I finally ended up with was a brute-force algorithm with slight optimisations.

The road to reach this code was treacherous, though, with some interesting results along the way:

A World With 1 Normal

This problem was isolated and corrected relatively quickly, basically I didn't have the indices' blank normals initialised to be blank normals. So fixing that brought me to this:

Revenge Of The Cubes

Great. So some things looking lovely, others looking completely screwed. This proved to be part of a bigger problem, it also created seams:

Almost Anatomically Correct!

And generally just lead to strange lighting behaviour on flat surfaces:

Colt 1911s Never Looked So... Jaded

Although for the most part that Colt 1911 model (made by Dylan) looks very very sexy, one can quickly pick out areas which are rendered incorrectly.

I found the problem to be that the first index in a smoothing group wasn't included in part of the normal calculation! So, fixing that Synergy could now do the Colts justice:

Quantum Sexiness

Not to mention the skulls!

Mission: Accomplished

So, what's next? Study! It's time to stop getting sucked into my projects and turn my attention towards trying to get HDs for my end of semester exams. So, until next week I guess Synergy will have to wait.

Meshin' Accomplished

Yeah I know, I'm the lamest person on Earth for using that title. But I don't care, I've succeeded too much today to feel anything but empirically chuffed. Despite cleaning up baby vomit at my Day Job, my time was devoted wholly to Synergy.

After working on the SynMesh and SynModel classes last night, I was keen to get a function running that could load in mesh data from some 3D model file format. After some research I eliminated the majority of options, including all formats without bone/keyframe animation, and all formats which aren't widely accessible. That is, not supported by a wide range of modelling programs. That left me with 3D Studio Max's .3DS format. Big surprise there.

Like all great things my quest for this functionality began with a google search, which lead me to a very useful tutorial. After some copying, pasting, and butchering of this code into a form suitable for my code, I began to, with breath guarded, integrate it into the cubetopia demo, replacing my hand-coded geometry information with SynMeshes loaded from a .3DS geosphere.

Some small tweaks here and there and BAM! Meshes loaded:

Trippin' Balls

I suppose it can't really be called `cubetopia' anymore. Perhaps gala would be more fitting. A number of issues were outstanding at this point. The foremost was that there were no normals! So I set about calculating normals upon loading each face. That lead to some interesting mistakes:

Really Trippin' Balls

After becoming friendlier with the concept of vector normalisation, I got this:

Find the Lightsource

After loading in a cube mesh, I discovered the reason for this bizarre lighting was that the meshes were inside out! So, setting glFrontFace(GL_CCW) I finally achieved a well rounded result:

Lo! 'Tis a Gala Night

Feeling very satisfied with myself, I was eager to push the code and see just how far I'd actually gotten it to work. Next was a model of a tree that I was working on once upon a time, and to my delight it worked straight away, giving a pretty funky result:

Styx

I began to notice a significant delay in loading the 3DS file, so I decided to see how much this delay scaled with input size. I needed a high poly model... and then I remembered a skull I was sculpting in zbrush once upon a time. This lead to a huge delay of almost 2 minutes to load the file. It was obvious where the hangup was: the obscene amount of debug info I was printing to the console and the log file. I commented those out and the delay was vanquished, which lead me to the final product for the evening:

Synergy's High Poly Flat Shading v0.00001

Each skull is ~50k polygons, so theres 400k polys being rendered there. It is both a milestone and a map of the steps yet to take. The most obvious next step is to implement Smoothing Groups. For instance, here is the exact same model rendered in Max with smoothing groups on:

The Next Mission

Big difference. So that's about it; excellent progress for 5 hours work, in my opinion. I hope to make the same leaps and bounds tomorrow, but we'll see...