ATRLoad - run full disk games as executables

[Rybags]

with prerequisites and limitations.

 

I've been working on this the last few days. Still in progress, but to the point where I've got a nice working example.

 

spelunk.xex

 

Spelunker as a file. Normally this game only runs from a full disk as it loads data for the subsequent stages. 192K RAM minimum required.

 

zork 2.xex

 

And Zork 2. Note that you won't be able to save games.

 

Requirement to use ATRLoad is an XL/XE with at least 128K RAM. 192K even better as that allows a full disk image to be kept in RAM.

 

How ATRLoad works:

The sector data from an ATR image is appended in stages to the loader/patch program. The disk image is kept in extended RAM.

Note of course that if you're running a RAMDisk then you'll likely lose the RAMDisk data.

If your DOS is using ext-RAM for buffers/workspace, then there's a good chance an ATRLoad program won't work from it. But if you setup a CONFIG that doesn't use extended RAM then it should work (haven't tested yet).

 

Once preloaded, ATRLoad copies the OS to RAM and uses a trick to get the system to reboot. There's a SIOV patch in place now so that all SIO is intercepted.

PBI devices can be left attached, it sets the PBI mask ($247) so that they won't respond.

 

Writes aren't supported yet, but in any case they'd only go to the RAM-based disk copy and be lost once you reboot.

 

I've just tried Spelunker with my IDE + 2, and it loads fairly quickly. Still some work to do with the program so some gains still to be had.

Note that if you press RESET and the system warmstarts, the patched OS doesn't re-establish itself (default is to coldstart anyway but some games might override that).

 

To do:

- improve loading screen with progress indicator

- speed up code, make memory moves etc more efficient

- possible compression or selective sector loading to fit more games into 128K systems.

- implement writes to RAM image.

- seperate "sync" utility to update XEX file with changed data resident in RAM.

- detect/use VBXE install, leaving normal system XRAM alone.

 

I've also tried a couple of the Zork games. I'll upload some more stuff tomorrow. The program I use to generate the final XEXs is pretty scrappy ATM, but once complete I'll just upload the required materials for people to do their own.

 

NOTE: The SIO patch generates it's own "psuedo SIO sound" to let you know it's active.

Edited July 13, 2011 by Rybags

[Fres]

Wow, very useful stuff. Thanks!

[GoodByteXL]

with prerequisites and limitations.

 

I've been working on this the last few days. Still in progress, but to the point where I've got a nice working example

....

This proofs it is possible to do. :thumbsup:

I was totally amazed to see Spelunker starting from RAM after having it loaded from the hard disk on my IDE V2. .

 

NOTE: The SIO patch generates it's own "psuedo SIO sound" to let you know it's active.

thanks for this note, otherwise it could be jugded as a failure.

Just a short pfeeep.

 

Is there a chance to convert copy protected originals with your concept into "image" files.

I still have some nice titles and hardware to regularily back them up on disk - 1050TurboIV. But this will not go on forever ...

[Rybags]

In theory protected stuff might be doable, but probably more trouble than it's worth since it'd be a case by case situation.

 

ATM I'm playing around with Super Packer to see if that helps. It's knocking the file size down but kind of defeating the purpose of the whole exercise by tripling the time from start of load to game running.

 

I'm probably going to just settle with a cheap RLE solution... keep an index of where all the sectors are and any that contain all the same data e.g. all zeroes don't get loaded.

The aim here is to get more stuff that'll run in only 128K but I've checked a few titles and most only have 80-150 blank sectors which still makes for 192K RAM.

 

The SIO sound. I changed it and it still sounds like a keyboard going crazy. So, it'll be changing again. I want it just so you know that something's happening.

 

In the meantime, here's Drol - colour version. I actually had this ready beforehand too but for some reason it's got a problem - the screen goes blank once the next level loads. But you can fix it by pressing any key twice (pause/unpause).

 

drolc.xex

 

Spelunker with Exomiser compression, for comparison to earlier one:

 

Spelunkp.xex

Edited July 14, 2011 by Rybags

[Heaven/TQA]

cool!

[phaeron]

In theory protected stuff might be doable, but probably more trouble than it's worth since it'd be a case by case situation.

 

I don't know if it's worth the work, but I don't think it'd have to be that specialized. Many copy protection schemes depend on one of two behaviors:

• Sectors that fail to read with a specific error.
  
• Multiple sectors with the same ID that appear with varied timing.
  

 

Many games simply call through DSKINV/SIOV to verify a protected disk. A game that does this and is OK with a soft-loaded OS (such as the Translator) would probably be supportable. The tricky part is supporting phantom sector load, which depends on the sector that is closest to the drive head at the time that the read request is sent. When the SIO patch is enabled, Altirra handles this for ATX/VAPI images by estimating a nominal rotational angle for the time needed to read and receive the last sector, adding in any needed track step delay, and then checking which phantom sector is next in order from that rotational position. A surprising number of games work with this and are able to load accelerated.

 

Things that I've found that would be problematic:

• Programs that go straight to SIO. I actually don't know of a specific one offhand, but I'd probably find one if I checked again.
  
• Programs that break on a soft loaded OS. Archon and MULE appear to be included.
  
• Programs that use RTCLOK to measure the read delay. You can probably adjust it on a case by case basis, but not generally (I broke Carina that way).
  
• Programs that issue disk reads over the OS database (Formula 1 Racing). Yes, someone actually did this. They depended on an SIO read auto-terminating when TIMFLG was overwritten.
  
• Programs that issue disk reads into ROM (Karateka). Again, another entry for the list of Stupid SIO Tricks. You could trap this and skip the sector copy, but CHKSUM needs to be set correctly.
  
• Programs that use weak sectors. This could be handled, but rarely used.
  
• Programs that rely on undocumented register or OS database state when SIO or DSKIN returns (A.E., Dimension X, Joyride part A).
  

[Rybags]

Yes, for a time I had an RTCLOK adjustment going as I thought maybe Spelunker used it (but doesn't) while tracking bugs down.

 

ATRLoad uses SIOV, I figured may as well use that, the calling mech is practically the same and you'll catch those odd titles that don't use $e453.

 

The other consideration is - is there actually much software out there that's not available cracked?

 

And from that lot, how much is only 1 disk/1 side.

 

I initially planned for this to do checksumming and also reflect a simulated SIO throughout a lot of the other OS aside from the DCB, but then, if a program's (remaining) protection is that rigorous, it probably sets PORTB to get rid of any patched OS anyway.

[Marius]

Would this work with Summergames (two sides disk) ?

 

That would be AWESOME.

[Rybags]

I suppose with enough RAM we could do multiple disks or just simulate one huge one with 5,000 or whatever sectors.

 

A hotkey system could be used to change floppies, e.g. SHIFT-CTRL-<number>

 

Might be worth looking into. Especially since plenty of people should have 1 Meg systems soon.

Edited July 14, 2011 by Rybags

[Marius]

I suppose with enough RAM we could do multiple disks or just simulate one huge one with 5,000 or whatever sectors.

 

A hotkey system could be used to change floppies, e.g. SHIFT-CTRL-<number>

 

Might be worth looking into. Especially since plenty of people should have 1 Meg systems soon.

 

Yeah indeed. I have 576KB which should be enough for almost any multiple disk game either.

 

Btw. Some games (like The brundles) do something very funy on the OS. (I think it is switched off) There won't "SHIFT-CTRL-<number>" work... unfortunately)

 

Greetz

M>

[Rybags]

Some games could be patched to leave the OS alone. Of course that's no use if they need the RAM.

 

Next revision I'll probably just add a load progress bar and VBXE detection. When VBXE found, use it's RAM in preference to system expanded RAM.

 

More complex stuff like compression and keeping multiple disks in RAM can come later.

[miker]

Cool stuff! I vote for making file of "Saracen" game. I've always wondered why it takes two sides of disk...

 

And maybe "Gemstone Warrior", too.

[Rybags]

I've not done any more yet because the whole system is still "inefficient".

 

It just puts the entire disk into the file with no compression. So rather than end up with people having to redo stuff I'll just hold off until the program's a bit more advanced.

 

The end product will just be a "builder" program that lets you do it all yourself. Supply it with image file/s or actual disks and it'll build the executable file.

 

Compression I'll just limit to eliminating sectors that have bytes all same value. When a game's active, we have to keep an index of where each sector is in RAM (ATM the whole disk is there so it's easy to calculate). Any more "efficient" compression just slows the load process down too much, plus makes the random access of the "virtual floppy" harder to implement.

 

Final system will probably keep 11 bits per sector of index + flag.

Flag set = RLE'd sector, with lowest 8 bits being the repeated byte.

Flag clear = normal sector, lowest 11 bits used to calculate the bank/address where it's in RAM.

Edited July 14, 2011 by Rybags

[WizWor]

I'm confused as to the point of this. I have always run these games from images. Is this just to speed up loads?

[+Philsan]

I think Gary would like to run those games from IDE+ or SIDE.

[Rybags]

The main point is so people with fast storage devices, ie mainly IDE, to be able to run ATRs.

 

The scope of this thing has grown now, IDE Plus 2 has the means to be able to run the system without needing to have a RAM-based patched OS.

 

That will mean that it could run later games that use 64K or alter PORTB, so long as they don't use the same parts of extended RAM.

 

I'll also be adding simple compression, so some games will gain an advantage if loaded from normal SIO devices.

[GoodByteXL]

... I'll also be adding simple compression, so some games will gain an advantage if loaded from normal SIO devices.

If possible, please make it optional. At first many (I guess most of the actual users do run 320KB machines or more) do have memory expansion in their machines, at second, loading even from SIO devices isn't that bad when using a speeder and often faster than unpacking routines on the A8.

[Rybags]

At this stage, the compression will be "all bytes the same" elimination on a full sector basis. Decompressing that is faster than raw read so it won't matter.

 

What will have to be optional though is if the compression is retained when it unpacks to RAM - since some games write to the disk, you need to have that memory allocated to hold the written data.

[+Larry]

Hi Gary-

 

Not trying to be negative, but I'm still unclear as to the "why" of this.

 

It works by selecting an ATR from those stored on a HD as a full file? (Header included?) And then you select from the ATR's and load it so that the system thinks a boot disk was loaded? So you might have a subdirectory of ATR's and select/load any of those? Is that the idea?

 

Or do the ATR's have to be converted to regular BLOAD (.XEX) files ahead of time? If so, why wouldn't you just run an XEX version of the disk/software? If there is no XEX, then it probably is because the protection was so bad that it couldn't be cracked (rare).

 

-Larry

 

The main point is so people with fast storage devices, ie mainly IDE, to be able to run ATRs.

 

The scope of this thing has grown now, IDE Plus 2 has the means to be able to run the system without needing to have a RAM-based patched OS.

 

That will mean that it could run later games that use 64K or alter PORTB, so long as they don't use the same parts of extended RAM.

 

I'll also be adding simple compression, so some games will gain an advantage if loaded from normal SIO devices.

[Creature XL]

Hi Gary-

 

Not trying to be negative, but I'm still unclear as to the "why" of this.

 

It works by selecting an ATR from those stored on a HD as a full file? (Header included?) And then you select from the ATR's and load it so that the system thinks a boot disk was loaded? So you might have a subdirectory of ATR's and select/load any of those? Is that the idea?

 

Or do the ATR's have to be converted to regular BLOAD (.XEX) files ahead of time? If so, why wouldn't you just run an XEX version of the disk/software? If there is no XEX, then it probably is because the protection was so bad that it couldn't be cracked (rare).

 

-Larry

 

The main point is so people with fast storage devices, ie mainly IDE, to be able to run ATRs.

 

The scope of this thing has grown now, IDE Plus 2 has the means to be able to run the system without needing to have a RAM-based patched OS.

 

That will mean that it could run later games that use 64K or alter PORTB, so long as they don't use the same parts of extended RAM.

 

I'll also be adding simple compression, so some games will gain an advantage if loaded from normal SIO devices.

 

OR the game needs to load additional data from disk.

[+Larry]

Yes, good point.

 

-Larry

 

OR the game needs to load additional data from disk.

[Rybags]

You can't do random access within a file very well without a DOS, or at least a read-only one. Fragmentation makes matters even worse.

By loading the entire image into RAM, we know where everything is and that problem goes away.

 

This method is a "best fit" for HDD users to have lots of "full disk only" type games without resorting to dozens of partitions, which would also mean having to run utilities to change disk mapping each time you rebooted.

 

Of course the downside is that you need plenty of RAM, 128K probably won't be enough in most cases, and it won't really benefit users of slower I/O devices very much.

 

But for IDE devices capable of > 30K/second loading, it'll be of much benefit.

[+remowilliams]

Do any of these work with 128K? I was just playing with the IDE+2.0 on a stock XE and none of them seem to load.

[Creature XL]

Do any of these work with 128K? I was just playing with the IDE+2.0 on a stock XE and none of them seem to load.

In the first post it is mentioned that at least Spelunker needs 192Kb.

I tried the two from that post and they seem to work with standard XL-OS. However, not with QMeg.

As far as I understood they will definitely not work with any DOS/ OS which uses buffers in the RAM under the OS-ROM.

[doomguy]

Sorry to resurrect an old topic. Any further progress made on this project? I'd love to be able to convert many of my ATRs to nice clean executables.