Crown and Cutlass Blog

As I mentioned before, I recently made the switch to a mac. I have been pretty happy, although I still don’t quite feel at home in terms of my development environment. Xcode and CMake don’t quite seem to get along (and yes it is still broken with CMake 2.6.0, I just tried it), so I’m actually using the CMake Makefile generator and Eclipse as my IDE. I could probably still use Xcode, but I’m comfortable with Eclipse from linux. Also, it appears that the provided al.h has a pragma that is unrecognized by Apple’s gcc, so it generates a warning. I should report that…

Anyway, the other day, I shut my laptop so it went to sleep, put it in my bag, and ate lunch. After lunch, I opened the lid and found that my keyboard and trackpad no longer worked. I tried rebooting, etc. and even a workaround that I found on the apple support forums, but nothing made any difference. I was pretty sure it was hardware, since I couldn’t even get the a boot menu for bootcamp. I managed to find a USB keyboard and mouse on campus, and confirm that the system still worked otherwise.

I was near an apple store, so I just ran over there and had one of their “geniuses” look at it. That name kills me, but I think it’s just a part of the sort of smug apple culture. My “genius” was helpful, but all he did was confirm it didn’t work. He didn’t seem at all surprised so I don’t think this is a rare issue. Apparently, a keyboard firmware update was supposed to prevent this from happening, but I had it installed and it didn’t do me any good.

They ordered me a new keyboard and trackpad, and I kept my laptop until it came in, so I only had to be without my computer for a day once the part got there.  After that it worked fine for a few weeks.  However, it has stopped working again.  At first, I would be using the trackpad and somehow OS X would think I was trying to eject the CD drive so I’d see the eject icon on the screen.  A little while after that, I would get a kernel panic with a stack trace in the trackpad driver.  From there, it just kind of slowly degraded to not working at all.  For a little while, it worked in XP but not OS X, but now it doesn’t work in either OS.  That means I get to have another appointment to see a genius this week.

That is pretty inconvienient.  I am trying to do work on campus this summer for a research project, but I can’t when my keyboard isn’t working.  I also need to be able to VPN in to work in case an issue comes up, but again, I can’t do that without my laptop working.  I thought Macs were supposed to be wonderful pieces of hardware (and that was why I paid extra for it), but after talking to the Mac owners I know it seems like almost every one of them has had to have their system replaced at least once since they got it.  Maybe we are all witless users who screw up our systems so badly they can’t be salvaged.  However, I never had these sorts of issues with my old Dell.  I am pretty disappointed in Apple at the moment…

David has posted a couple times now about python. We certainly had a good time going to PyCon and had a good time doing PyWeek. I really am a fan of Python. I think it worked really well for us during PyWeek and the other times I’ve used it. I have other thoughts on both items, but I think they’re better left for another day. What I want to focus on is how is all this affects our work on Crown and Cutlass.  

If anyone is still with us, watching the blog or the forums for activity, you probably have noticed a considerable slowdown in work. This has been caused by a couple things. First, and I think most importantly, is that fact that David went back to school 8 months ago. Since we work together on Crown and Cutlass quite a bit, our motivation often ebbs and flows together. Since David has less time to work on it, I find myself with less motivation. The other issue I have, is that when I take time off from Crown and Cutlass, I have a very hard time jumping back into the code. Getting back into a C++ mindframe from Delphi (which I use at work) is really difficult.  

Based on that, we’re going to be writing as much of the game play code as possible in Python. By exposing our Protocce engine to Python, we’ll be taking advantage of the rewrite work we’ve been doing as well as moving to a more hospitable development environment. I think this will speed up development on Crown and Cutlass by removing accidental complexity.

We’ve long talked about scripting as a major component to our game engine, but have always put it off for another day. Now, by using Python, all (or most) of our actual game code will be done in script. It gains us the flexibility of Python, while maintaining the speed of C++ in the engine. Anything that needs to be sped up can be implemented in C++.

I think is an exciting development for Crown and Cutlass. It gives us more opportunity to think about the gameplay and avoid the problems we’ve had at times in C++. You should be seeing more exciting developments over the summer as we make more progress on Crown and Cutlass.

Collin and I participated in PyWeek6 (week long challenge to make a game with Python) with a mutual friend who has a Python game project he is working on in his spare time. He actually went to PyCon with us, so that was where this idea came up. It was interesting and we put in a lot of time on it. Unfortunately, I had a test and two projects due for school that week too, so I didn’t end up sleeping much… We did come up with a game (you can see it here), although I think we all wish we could have had another day or two. We are getting together this weekend to have a post-mortem. I think a lot of stuff went well (and we actually got something done in a week!), and of course I think a lot of stuff could have gone better. For example, I spent a good chunk of the week working on a quadtree for culling the world. The world ended up being much smaller than we originally planned so the overhead of the quadtree (implemented in Python) killed any potential performance gains. It also turns out we also unwittingly used some 3rd party libs that were released after the library deadline (a month before the start of the competition), so we technically could be disqualified. Oh well. I don’t think any of us really cares about winning the competition, we were more in it for the challenge. However, I am looking forward to hearing some feedback from the other contestants.

That was by far the largest Python project that I have worked on, so it was a real learning experience for me. Python has some great stuff and some really frustrating stuff. I learned that this is not the correct way to use assert:

assert(False, 'error')

That will never be triggered because assert is actually a statement, not a built in function. This is the correct way to use assert:

assert False, 'error'

Note the lack of parentheses. With the parentheses, it evaluates the tuple “(False, ‘error’)” as True since it is not an empty tuple, so it effectively becomes “assert True”. I’ve heard Python 3000 will change “print” to a function instead of a statement, but I haven’t heard anything about “assert”. Ugh. We also encountered some issues with performance doing “engine-level” stuff in Python. However, for a lot of what we required to make a game, Python let us do it quickly and easily. Integrating the different stuff we all worked on went well. Duck typing makes development much easier than having to use interfaces/templates/inheritance/whatever (although it seems like it really requires good automated testing to be safe).

Perhaps I’ll post more after our post-mortem, but try out our entry if you want. There are installation instructions included with the tar.gz file (also read the errata since there are a few errors in the readme). You will need Python 2.5, but beyond that setuputil will install the other dependencies automatically. If you try it, give us some feedback!

Collin and I are both signed up to go to PyCon 2008. While we don’t use Python in the Crown and Cutlass code itself, we have used it for several utilities to support our development. For example, we wrote a script to update CMake build numbers that gets used in the Protocce build (here is the CMake file that uses it). We also have a script that updates the year in the license text in all of our source code. None of that is too complicated and none of it took very long to develop, but Python has made it pretty easy. I’ve done a few other little odd scripts in Python, and I use it at work whenever I need to do quick one-time text munging for some reason. Basically, I use Python for everything I used to do with Perl except it doesn’t take me an hour to remember how to get started and when I return to the code a few weeks/months later it still makes some sense to me.

Anyway, we are excited about PyCon. I am looking forward to learning about Stackless and there is a talk about embedding Python in Counter-Strike: Source which should be really interesting. Plus there is a bunch of stuff about Jython (”Jython on the Joint Strike Fighter”, how cool is that?!?), IronPython, and a many other talks that look good. Unfortunately, it doesn’t look like you can link to descriptions of individual talks, but if you are interested you should be able to find what I have mentioned in the complete list of scheduled talks without much effort.

If anyone reading this wants to get together at the conference, let us know. I’d enjoy meeting some people who have interest in Crown and Cutlass, and you could give us a hard time about not making more progress on the game!

As you may have noticed, we recently had some issues with our wiki.  We were using a wiki system that we could install from the web host’s control panel.  However, it did not get automatically updated.  I guess someone exploited a security flaw in the wiki software and the host noticed unusual load on the server machine from our account.  Rather than continue to mess around with our own wiki, we decided to just transition over to a new Crown and Cutlass wiki hosted by SF.net.

The other day I got the content from the old site running on a local linux box so I started transitioning content to the new wiki.  I also wrote a PHP script to forward links from the old wiki to the new wiki pages.  I don’t have all the content there yet, but I think it will be fairly painless to move it over.  This is also a good opportunity for us to flush out some old stuff from the wiki and update the content.  If you notice a missing page or have questions about the wiki, leave a comment here or email me.

When we originally set up this blog, we had all comments go into the moderation queue so that we could catch spam or inappropriate comments.  After a few weeks, we started getting dozens of spam comments a day, so Collin installed the reCAPTCHA plugin.  We both liked the idea of helping to digitize books and get rid of our spam at the same time.  However, the spam comments kept appearing in our moderation queue so ever few months one of us would have to go clear out hundreds of spam comments (it even broke a thousand at least once).  That was annoying but not bad enough that either of us cared enough to figure it out.

The other day, I finally got tired of the spam and read some of the reCAPTCHA support forum posts and think we got the problem figured out.  It sounds like reCAPTCHA allows comments to go through that fail the CAPTCHA but it will mark the comment as spam automatically.  I guess Wordpress was putting the comments into the moderation queue before reCAPTCHA could mark them as spam though.  Anyway, I hope I really got it right and we won’t all of a sudden get flooded with spam comments.  So far so good…

I finished the first semester of my MS program.  It was interesting in some ways, frustrating in some ways, and overall just kind of weird to be back in school.  I may write more about my thoughts of coming back into academia (while also trying to stay in the business world and keep a small open-source project running) at some point.  But for now, I’m excited to finally have time to work on getting Protocce to build in osx.

I have finally done it, I bought a Mac. A few weekends ago, I bought a MacBook Pro and then last weekend I upgraded to Leopard. The computer was expensive (I got an academic discount and the upgrade to Leopard was basically free, but still…) but it sure is a nice machine. I now have a video card with decent shader support, so I am excited to play around with that. It is also nice to have a unix-based machine with decent driver support out of the box. The wireless card actually works! Amazing! However, I really feel lost trying to do development in OS X and in some ways I feel lost just trying to use the computer.

I really have not quite gotten used to the way the filesystem is arranged. I guess I am comfortable with it not being like Windows, but I want it to be like linux. In Windows, there is no “/usr/lib/” or “/usr/local/lib/” for libraries so you put them somewhere else (like “C:\OgreSDK” or in program files or something). In OS X there is a “/usr/lib/” and “/usr/local/lib/” but it looks like the preferred place to install libraries is into “/Libraries/” which is just weird. Which should I use? Should I treat it as a unix machine, or embrace the weird OS X stuff?

The keyboard also is a little annoying. I want a “home” and an “end” key. I want a “del” key that is separate from the backspace key. I guess you can do “command-right” to get to the end of the line, but that does not seem to work consistently in all applications. I also really think only putting one mouse button on the laptops is a poor decision. And no, I don’t buy the argument that using modifier keys for things like that or to get a right-click is somehow more efficient.

I am also trying to figure out how to get dependencies for Protocce installed. Ogre was really easy since they provide a framework. If I remember correctly from his blog, Steve Streeting got a Mac recently so OS X has a higher priority on the Ogre team. The ogg and vorbis packages came with XCode projects to make frameworks, so building and installing them was really easy. I couldn’t find a binary boost distribution, so I built/installed it just like I would have in linux (”./configure && make && make install”). That is what boost recommends, but I’m not sure that is the OS X way to do things. At this point though, I don’t think there is reason to go through the trouble of making a framework (and really figuring out what frameworks are all about, how they differ from bundles, what a bundle is, etc.). Unfortunately, OS X support in CEGUI seems to be lagging behind. CEGUI seems to be kind of quiet lately anyway (Crazy Eddie appears to have left the project, hopefully it can rise from the ashes and continue strongly eventually). I am going to try building it from SVN where I hope the OS X support is a little better.

XCode seems like it will be nice. I need to try a “hello world” project in it, but it looks like CMake can generate XCode projects. That will be save a lot of work, since it will shield me from having to set up a project manually until after I see how Xcode handles projects a little. I’m sure it’ll still take some getting used to, but I’m interested to see how it compares to Eclipse with the CDT and Visual Studio.

Anyway, if you have tips on getting started with development is OS X, let me know. I found Apple’s getting started page and it seems there could be some good stuff in there., but I have a lot to learn.

In my previous post about why I’m excited for our rewrite, I forgot to mention our unit test framework. It is really quick and dirty, and we don’t do anything fancy to get around the fact that certain things are just hard to deal with (e.g. “did that sound right?” or “did that draw correctly on screen?”). However, the unit test framework has really allowed us to be sure what we are building on is solid and behaves as we expect it. It has also great been really great for testing a change quickly. For example, the other day I gutted our LoadOgg helper function, but since we had tests written for it it was really easy to be sure I had not broken it. Before our rewrite, we would have to run the whole game, get to the place in-game where our change came into play, and then see if it was right. Now we can just write a test or two for the new change and if we want to we can disable all the other tests. That really speeds up debugging.

Also, because we are making heavy use of the event system, we can test the event sender and the receiver separately. For example, the SoundBuffer and SoundStream classes basically just send events to the sound system telling it to play/pause/stop a sound. In the test, I set my own listener for those events and verified that they are being sent correctly. Another place where the event system was useful was in the initialization of the input system. The input system needs some kind of window handle to give to OIS. Collin set it up so that the input system sends an event with a Variant window handle field. If it is initialized, the render subsystem receives that event and sets the window handle value. However, if you do not have the render system running for whatever reason (like in the input test), you can just register your own handler for that event. So in the input test, Collin creates a simple little window just to get input with raw X11 or Win32 code, and his test receives the event and returns the proper handle. The input system doesn’t care whether the render system created the window, or you did yourself. All it cares about is that something provided it with a window handle. That is pretty slick.

As I mentioned earlier, we aren’t doing anything fancy for user input/output tests for things like sound, graphics, or input. We basically do what we want and then ask the user “did that seem right?” That can get annoying if you are working on something like the sound system. Testing it is a lot of “did the music start?”, “did it stop?”, “did it start again?” kind of questions. It’s not perfect, but it is a simple way to verify that the code works. Compared to what we had it is a huge step forward.

I have mentioned that I am really excited about our rewrite several times, but I haven’t ever gone into much detail about why that is. I hope that we have learned from what worked and what didn’t work in the old Crown and Cutlass code, and applied those lessons to the new Protocce code. I’m sure there are still things that we will realize are not well done, but this is a much more stable base for future work. Here are several specific things I’m excited about:

Exception - We implemented the pcce::Exception class which we now use consistently for error reporting. Coupled with our greater focus on RAII (see below), I think we have made some serious progress. We used to use error codes sometimes and throw other times. Unfortunately, sometimes we threw strings, sometimes we threw char *’s, sometimes we threw ints, etc. This is much better. It does not descend from std::exception at the moment, but we are thinking about doing that.

Greater focus on RAII - RAII is really a great idiom for resource management in C++, especially when coupled with smart pointers like boost::shared_ptr. We have design for RAII from the start, so that should make things much more manageable. Again, I’m sure we have more to do in this area, but we have a good start.

Variant - At work, I have gotten used to using the Win32 Variant class in Delphi. While it certainly has its drawbacks, it sure can be helpful for doing things like reading text from a config file. Our variant class is cross platform, and based on boost::variant. I think it will change a little as we use it, but I’m pretty happy with it now. You wouldn’t want to use it when a statically typed variable will do, but for example the config system was really easy to do using Variant.

PropertyBag - Again, at work I got used to using “property bags” which are basically just a map from a string to a variant value. You wouldn’t want to replace all of your normal classes with a property bag (static typing can be nice!), but it can be really useful. For example, our configuration system is an INI file. That gets loaded into a property bag. When you initialize a subsystem, you pass its settings as a property bag. That way, you can change the config options around without changing the config object or the method signature. We have also used property bags to create “game objects” which are just property bags with a couple of predetermined fields for model name, texture, etc. The game programmer can put whatever else he wants in there.

Consistent use of shared_ptr - I really don’t mind managing my own memory, but once you start talking about exceptions and RAII, using some kind of smart pointers really makes sense. We are making pretty serious use of boost::shared_ptr, which has made a lot of things easy. No need to worry about who owns an object, as long as you have a shared_ptr to it, it will stay valid. Of course, we have to think about circular references, but overall I think using smart pointers is a big win.

OGRE - I think it’s pretty clear why I am excited about OGRE. Originally, the project was just an excuse to do OpenGL programming. I enjoyed that, but now I am more interested in other things. I’m a little sad not to get to try that from the ground up, but eventually we can revamp Protocce to use a custom engine if we want. Until then, using OGRE will simplify our code, standardize our models (our old model loading code was really picky), and open up the doors to easily implementing graphical improvements. It will also allow us to focus on the game itself, rather than the details of the graphics.

EventSystem - Our old code used to be very tightly coupled (see the SailingState for an example), so minor changes could not only cause a large number of units to need to be rebuilt, but also end up causing a large number of code changes. The event system is based on what is mentioned in Game Coding Complete and allows us to significantly decouple our code. Once an event is defined, you can register a boost::function to receive that event. Events can be dispatched immediately, or queued to be dispatched once per frame. I expect the way that will work is that game events will be queued, while internal “plumbing” type stuff will be sent immediately. As an example, the input subsystem needs a window handle to receive input. Rather than connecting it directly to the render subsystem (which would force you to initialize the entire render subsystem if you wanted to use the input subsystem), the input subsystem just sends an event that the render system receives. If you don’t want to use the render subsystem for some reason, you could just register a different handler for that event. Collin actually does that in the input test. As I mentioned in my previous post, the sound system is also event driven. It is running in its own thread, and all the main thread does is send an event that says “play sound X” or “stop sound Y”. That data is easily sent out to the other thread, where the actual sound code handles it. We can totally revamp the sound implementation and not touch client code at all. I think the event system is a much more flexible system and I think it will open up a lot of doors for us.

Improved resource management - I mentioned this a little in my last post too. Between the release of Alpha 1.4 and the decision to start the rewrite, I wrote a resource management system. It was pretty good. It would make sure that only one copy of a texture, sound file, or model was loaded at a time and used reference counting to automatically unload the model when it was finished. I have further refined it (partially based on the OGRE resource system) and posted a reply on gamedev explaining the Protocce resource system.

Anyway, there are other things but that is a decent explanation of the things that I am excited about.