Friday, May 22, 2020

Compiling for the Z-machine version 3

The Inform 6 compiler has been pretty stable for the past several years. It's still in active use as part of the Inform 7 toolset, but the I6 compiler hasn't changed much.
However, I've put in a few I6 updates over the past week. Exciting news? Maybe not for most of us, but these changes are important to people who are trying to write really old-fashioned Inform games.
Let me go back to the old days. (Jangly harp transition...) In 1979, when Infocom ported Zork to personal computers, they designed the famous Z-machine platform. It went through a couple of iterations, but by 1982 the "version 3" Z-machine was firmly established as Infocom's workhorse.
The V3 machine was tightly constrained in some ways. For example, it could only support 255 objects (counting rooms, items, scenery, NPCs, and the player!) But this was deliberate; it was intended to run on some really tiny computers like the TRS-80 and Commodore 64.
Infocom games got larger and more sophisticated, but they kept on stuffing them into V3. The Zork trilogy, Enchanter trilogy, Hitchhiker's, Planetfall... it wasn't until AMFV in 1985 that they had to design a V4 Z-machine. And even then they kept using V3 for any game that fit.
Mind you, the Infocom people didn't say "V3" and "V4" back in the day. They referred to V3 as "ZIP". V4 was "EZIP", for "Extended ZIP". Then "XZIP" (V5) came along in 1987, and "YZIP" (V6) in 1989. These updates allowed more objects and more content. They also added a parade of new features: bold/italic text, expanded status window, sound, timed input, and finally graphics.
But this only underlines that V3 was Infocom's standard technology. You used V3 unless you had some specific need for one of the larger, fancier platforms.

Jump forward (jangle jingle) to the "modern" era of 1993. The Z-machine has been reverse-engineered; we have open-source interpreters that support all versions. Graham Nelson releases Inform, a compiler which can generate Z-machine game files.
Inform let you write games for any version, but in practice, your choice was between V3 and V5. (V1/2 were too antiquated to bother with. V6 was a headache for reasons I won't get into here. And V4 was like V5 minus a few features; if your game outgrew V3, you might as well go straight to V5.)
But among Inform users, unlike Infocom, it was V5 that emerged as the "standard platform". If you look at the games/zcode directory on the IF Archive, there are over 300 .z5 games and just five .z3 games!
The reasons are obvious. Everybody had modern Mac/PC machines which could run the largest Z-code games with ease. Authors felt free to put more scenery, more detail, more responses into their games. In that atmosphere, the V3 limit of 255 objects really pinched. And the other V5 features were nice to have around. Most games didn't need sound or timed input, but bold and italic text always look good. Why not build your game on V5 and have all the amenities available, just in case?
(Then, in 1995, Graham Nelson's Jigsaw overflowed V5 and he had to invent V7 and V8 in a hurry. But never mind that.)
So Inform's V3 compiler code was barely ever tested after the mid-90s. And we know what happens to untested code: it breaks. A couple of bugs crept in and nobody noticed.
That is, not until 2020 (jingle bloop). A couple of projects are now working on Z-machine tools for retro machines. MetroCenter '84 and PunyInform are Inform libraries optimized for size; Ozmoo and Pitch Dark are Z-machine interpreters which run on the C-64 and Apple 2.
Running on that classic metal means embracing all the memory limitations which we forgot about in the 90s. Every object and every byte counts. V3 is once again the order of the day. And presto -- the bug reports started rolling in.
Okay, only two bug reports. The fixes were a couple of lines each. Now Inform 6 can compile V3 games again!
While I was in there, I added a feature which could be of additional help. I6 games can now contain "static arrays", whose data goes into ROM rather than RAM. (Yeah, the Z-machine has ROM and RAM. I'm simplifying a bit but that's the idea.)
Static arrays may not be a lot of help. I first considered this idea when I was working on Hadean Lands -- a game which was originally planned for the "limited hardware" of the iPhone. (This was back when mobile phones didn't have gigabytes of memory.) I knew HL's alchemy system would require a lot of data and I thought that putting it into ROM might be worthwhile. But, long story short, it turned out not to be. So I didn't do it. Until now.
(Before you ask: yes, Hadean Lands is written in Inform 7, and it uses the Glulx VM rather than the Z-machine. The I6 compiler is still part of the toolchain and the concept of static arrays applies equally well to Glulx ROM and RAM.)
So there's your history lesson of the week. I could have tweeted "Inform 6 bugs fixed", but this is more fun to read, I hope.

6 comments:

  1. A minor correction: Z1 and Z2 aren't merely antiquated, but are something which, as far as I can tell, out-of-the-box Inform could never prooduce. They used a string encoding which was slightly different from Z3, and Inform's text-encoding routines didn't implement it at all.

    This is one of the few Inform/Z-Machine issues I'm particularly knowledgeable about, because roundabout 2002 I tweaked the Inform source code to produce Z1 and Z2 games (I never got around to actually creating a public patch, but I called it "golmac" provisionally). I used it on two occasions to produce proof-of-concept SpeedIFs. Later on when I tried to use it again, it was incompatible with the most recent Inform 6 library/veneer elements, which I think were broken for Z3 too. So I'd known for some time that Z3 was broken, but I assumed it had been intentionally left by thhe wayside.

    Well, now that version-3 functionality's back, maybe I'll see if I can dust off golmac and make it work with the most recent version of Inform 6.

    ReplyDelete
  2. Ah, you're right, I should have caught that. I6 lets you select Z-machine output from V3 through V8.

    The I6 *library* has intentionally left V3 behind. The compiler has not.

    The veneer code is implicated in this, though. The I6 library contains expressions like obj.prop(), and the veneer implementation of those uses non-V3 code. But if you compile a game with an I5 library (or one of the puny/metro84 ilk), it *doesn't* use those expressions, so those veneer routines never get compiled in.

    Anyhow, I'd be interested in adding V1/V2 support if it's relatively non-intrusive.

    ReplyDelete
    Replies
    1. If I can find the code for the golmac projejct, I'll happily share it; it's probably more or less the same set of changes necessary to the inform source code, even over a decade and a half later. I seem to recall it being a pretty straightforward and harmless tweak: some additions to the command-line option parser to recognize -v1 and -v2, changing the conditional "version_number=3" to "version_number<4" in a zillion places, and a lot of special-case code in text.c to use different shift codes for Z1 and Z2. A cleverer but more invasive code redesign would probably include some optimization for shift-lock characters to compress strings.

      In most respects, the established standards for Z1 and Z2 are very nearly the same as Z3. That's certainly the case as far as any post-ITF interpreter is concerned, even with respect to the deficiencies we know about (as a quick experiment, I took a Z3 version of Deadline and changed the first byte to 01, a sloppy and hackish way to create a Z1 file. The text was, as expected, completely mangled, but significantly the status line displayed a time rather than a score, which wasn't a capability Z1 historically had).

      Delete
  3. Can a compiled version of the Inform 6 installer be posted with the new additions for Windows? It would be great to experiment and compile to .z3 for the TRS-80 color computer. I remember a method described back in 2017 by John Linville. https://retrotinker.blogspot.com/2017/11/building-infocom-disk-images-for-coco.html

    ReplyDelete
    Replies
    1. David has uploaded a current Windows binary here: http://ifarchive.org/indexes/if-archive/infocom/compilers/inform6/executables/

      Delete