Back to Roots: xu4's Graphics, Ultima 5 Discoveries
Fearyourself has had another busy week working on Back to Roots, his Ultima-inspired game engine project.
Just under a week ago, he challenged himself to find a way to incorporate the recently-released high-resolution tileset for xu4 into his engine:
First step was to take their tileset which is a 1 column, 256 row tile set into my classical view of 16 * 16 tiles. That was an easy small tool I wrote of 100 lines that created a new version of this.
Once that was done, I had a minor issue with RGB vs BGR encoding but it wasn’t that difficult to get around and so I can now propose a new video showing the changes of tilesets that are available for the U4 version: the original, the improved EGA, the VGA version and this new tileset.
Update: Video goodness showing off the new tilesets:
Mmmm...tiles.
Following this accomplishment, he took some time to dig into the files for Ultima 5 again, and made a fairly significant discovery:
The OVL files contained one repetition which was 55 8B EC. These 3 values came over and over again in the different OVL files. Thinking that this might be a magic key for structure starts, I went on to looking to the rest of the values.
After a while, I thought, perhaps someone else has seen these values somewhere so I went to see my friend Google. It so happens that I wound up on a page explaining that 55 8B EC is actually the start of the ABI (Application Binary Interface) thus the start of a function!
It then hit me! The OVL files are actually code files! Then it got my eyes even more openned: U5’s executable code is not only in ULTIMA.EXE, they actually put the code in each OVL file…Basically, we have here a scripting language and a dynamic library system. Instead of loading everything from the start of the executable, they loaded the code on a need to execute basis.
I don’t know if this was ever something that was known about the Ultima 5 game data files; certainly, it’s the first I’ve heard of it. Regardless, it’s a fascinating find.
Most recently, he has been working on decrypting and decoding how merchants are handled in Ultima 5:
I’ve been working on figuring out how the merchants are decoded. It seems that SHOPPE.OVL contains all the code for handling an innkeeper. Iíve figured out the emplacement for the name of the merchants, what they sell and I know where most texts are saved. I also know how to detect if an NPC is a arms merchant, or anything else.
The last point that I’ve still got to handle is: I’m in city X, I have an arms merchant Y, how do I know which arms merchant he/she is ?
In Ultima 4, this was handled by a single array that gave you the index into the weapon’s dealers table. The array’s index was based on the cities. From what I can tell, there is no such array for Ultima 5…How the game linked its merchants to their data seems to remain a mystery… For now…
That is a cool post. Shit, sounds like lines from Lord of War with the talk of arms merchants. Instead of an array, they probably use flags somewhere that point to certain pieces of data (vague but it’s there…it is probably close to the simplest solution).
If there is indeed executable code, direct or scripted, in non-exe data files for the DOS version of Ultima V, that is a hell of a significant discovery considering the source code is lost and no one at Origin is talking (damn them, blind bastards not realizing we’re here).
That goes beyond the reverse engineering of data files and into the realm of finding the source code itself, which is awesome.
Assembly code is always accessible. Understanding it though is a whole other matter. Going from the first to the latter is a difficult task to sat the least.
I’m working my way back up to figure how it was done. The biggest issue is due to a difference of programming era:
Nowadays, you’d have a function that would return an index and with that index go through the various arrays. However, at the time they had to be quick so anything was possible.
From comparing things with the dosbox debugger and getting a CPU log, it seems that they stored in memory at a fixed spot the merchant’s name address. I’ve not figured out where they write that address because that will give me where that address is calculated which should give me the famous merchant index.
But that must wait another day 🙂
That’s some serious Sherlock Holmes stuff. I wonder what compiler they used.
There is no mention about which compiler was used. It seems like they stripped out every bit of extra information.
For those who are curious by the way, here is the video of the graphic render changing :
http://www.youtube.com/watch?v=XoixsX1in1w
Thanks for the link; I’ve added the video to the post…hopefully. I’m on my iPhone today, so it’s not as easy as when I’m in front of my laptop.
Yeap, it seems to be working for me :-). Every instant I get, I try to get a step closer to understanding how they did their internal plumbing.
Hopefully, I’ll get the answer soon so that I can get the U5 port as advanced as the U4 port. That would be nice and a good way to finish the month of July. But figuring these kind of things out is a grueling and slow pacing job and it really is not easy.
They used Borland C++ for U7 ,Turbo C for U6 and assembler for U5.
U6 and U7 can be run though a decompiler like disc dcc boomerang but the decompilers i tried need patching or dont support the binary format.
I found similar things out years ago as i disassembled the graphics renderer interface starting to port the 256 colours .ovl to U5.
OVL is shorthand for overlays
Back in those days, most PC just got 640KB of memory, the Ultima V game content has 54disk of 5.25″ disk, each disk has capacity of 1.2MB.
With 640KB of memory, the PC can not load all the code at once, so programming technic called overlay is used to load code only when needed (in modern days OS swaps memory for you)
more about it is:
http://en.wikipedia.org/wiki/Overlay_(programming)