Zarf Updates

I somewhat randomly stumbled across these two resources from a couple years back:

Internal Secrets of Infocom Games by Michael Ko

(PDF version; forum thread announcing the project.)

This is a detailed analysis of the classic Infocom parser. As we recall, the Infocom parser wasn't built into the Z-machine -- it was a pile of ZIL code which was copied into each game.

The parser was built to be flexible, but also had to be customized somewhat for each game. So there is no such thing as the Infocom parser. Rather, there was an evolving chain of versions. This document is based on Zork 1 but has copious notes on variations in later games.

ZILCH How-to by Roman Bartke

(Forum thread mentioning the site.)

This is a step-by-step guide to getting Infocom's original ZIL compiler working. It starts with the PDP-10 ZIL source, which was dug out of old ITS tapes by Lars Brinkhoff. It then runs through emulating a PDP-10, getting ITS to run, making Muddle work, and compiling and testing Zork 2.

(Bartke uses Zork 2 as a test case because both the source and the compiled ZAP assembly have been preserved.)

Enough with ZIL. What's been going on in Inform?

I must say up front that there's no news about Inform 7. As far as I know, Graham Nelson is still chugging away on it. No release date announced. (Sorry, but someone always asks.)

However, I have exciting news about Inform 6! Well, exciting to a few of us.

I have just completed an overhaul of the compiler's memory management. The old memory limitations are completely gone. If you've used I6, you will bitterly remember the old errors:

Fatal error: The memory setting MAX_PROP_TABLE_SIZE (which is 30000 at present) has been exceeded. Try running Inform again with $MAX_PROP_TABLE_SIZE=<some-larger-number> on the command line.

These even percolated out into Inform 7. It would sometimes choke and awkwardly suggest adding a line like this to your game:

Use MAX_PROP_TABLE_SIZE of 50000.

These all stem from Inform's history as an early-90s C program which was meant to run on DOS, classic MacOS, and other extremely ancient platforms. (Even VAX!) The compiler is careful to allocate all its memory at startup, to avoid the possibility of running out and crashing -- which might take down the entire OS in those non-protected days. The compiler also goes through some contortions to never malloc more than 32K at a time. (See the technical notes, section 11.2.)

All of this fuss is entirely unnecessary on an OS with virtual memory. Modern software ("modern" means "the past twenty years") just allocates as much memory as it needs. The understanding is that no matter how much memory the program needs, it won't crash. Rather, it will bog down ("swapping") until the user gets annoyed and buys a bigger computer. This sounds like a joke but it's actually a practical solution!

Anyhow, the memory usage of Inform is minuscule compared to your web browser. A quick test compiling Hadean Lands finds that the compiler needs to allocate about 12 megabytes of working space.

So I've spent the past couple of months rethwacking all of I6's memory handling to "allocate what you need, when you need it". (Thanks to David Kinder for maintaining the I6 source code and reviewing my changes.) I fixed a few obscure bugs while I was in there, too.

These changes have not yet made it to a release, but you can get the I6 source and compile your own binary. You can even install it into your I7 package and get the benefits immediately. (Although if you're using the Mac version of I7, you'll need to unsign it first... probably not worth the hassle.)

I was worried that this rewrite would have a performance penalty. After all, the compiler is now calling malloc and realloc more often. But no! You'll be happy to know that I6 now runs both faster and slimmer. In my HL test above, the new compiler used 11% less RAM and ran 10% faster than the original (the 6.35 release).

With that out of the way, I am considering the future of Inform 6.

There are three major compiler features which have, historically, almost never been used. It's worth considering whether they can be stripped out of the codebase.

The temporary file option (-F1). This tells the compiler to put some of its working space in a disk file, instead of keeping it all in RAM. In other words, it's a cheapass virtual memory feature for machines that didn't have any.

Again, this is quite irrelevant on any machine less than twenty years old. As far as I know, even people doing retro development on old machines aren't using this option. (See forum thread.)

Module linking (-M, -U). This allows the user to compile parts of an Inform game (e.g., the standard library). Then, when compiling their game, they can recompile only the parts that have changed. This is analogous to the C model of compiling object (.o) files and then linking them together.

The module feature was a heroic effort by Graham Nelson. He comments in the tech notes:

(I have by now designed and painfully got working three different versions of the linker, and would not go through all that again for all the tea in China.)

Sadly, this appears to have solved a problem that nobody had, even in 1996. Module linking only works for Z-code. (I skipped it when adding Glulx support, and have never seen a complaint.) It is irrelevant to I7 development, as the I7 compiler generates monolithic I6 files which cannot be split up.

Infix mode (-X). Infix is an inline debugging feature for games. Some of its code is in a library file (infix.h); some is added during compilation. Infix allows you to examine game variables and object properties, and change them, at any time during play. (See DM4 chapter 7.6.)

This is an ambitious feature, which, again, doesn't seem to have gotten much use. Like linking, I skipped over Infix support when implementing Glulx. Like linking, Infix is not very useful for I7 games. In both I6 and I7, authors have generally added custom debug actions where needed, and not bothered with the idea of a generalized debug tool.

Rearranging command-line options. (This is a footnote, not a major feature.) I'm considering rearranging Inform's blizzard of trace/stat/report options into a tidier set. See notes here.

So what do we think?

I like the idea of ripping out the temporary-file and module-linking options. These don't let you make better games. They just let you compile the game you've got in "more convenient" ways -- which are worthless on any modern machine. Removing these options would simplify the compiler codebase for no real cost.

Infix mode, despite its limitations, is a real feature. One of the goals of I6 is to be able to recompile old Inform games, and that includes old games which use debug features. So I am inclined to let Infix stand as-is. I'd even accept a patch to support Infix for Glulx, if someone wants to put in the effort.

(A debugging interpreter, using the copious symbol info generated by the -k option, seems like it would be more useful than updating Infix. I leave that discussion to another time!)

What do you think? (I hate blog posts that end like this, but it's a real question.) Have you ever used any of these features? Would you shed a tear if they were dropped? Let me know.