7 replies to this topic

[snicklin]

I'm no expert on the subject and have recently been wondering about Atari cassette-tapes and modern compression technologies.

Could we get .wav/.mp3 files of existing cassettes, then convert them back to binary data, use some modern compression algorithms and then output them back to cassette?

The idea is that we shorten the length of the load times. Why we me might do this, I don't know as most people are using SD cards, it's more of a proof of concept that it is possible.

Did games back in the day use much compression? That initial boot section which then changed the colour of the screen or displayed text or graphics, was there a decompression algorithm built into those typically?

Please feel free to discuss, I'm not sure where this topic will end up, I'd just like to know a bit more about the subject.

[Rybags]

The problem with decompression with tape loads is the short IRG.

Anything much more complex than simple RLE would probably overrun the time and you'd miss the next record.

Alternatively you can load a large block and decompress, probably stopping the motor. I don't think much software in the day used compression for the load process itself, even disk-based.

The archive MP3 thing - I guess so, but just the same you can use CAS files but I don't believe they retain the exact characteristics such as leader length or length of IRGs.
Also unsure if they can cope with non-standard record lengths, not that much commercial software ever deviated from the standards anyway.

If an "accurate" archive system was to be devised, then maybe some sort of extended CAS standard where extra detail was retained.

[snicklin]

Ahh, I see what you mean... you don't want too much data to fill memory and leave no decompression space.

Would it be possible to follow the following sequence of events?
1) Load compression lookup data / header
2) Load first block
3) Decompress previous block while loading next block.
4) If not at end of program, go to step 3.
5) Decompress final block.
6) Run game.

Is step 3 possible?

One thing I've always wondered also, are blank memory segments loaded from tape or are only segments with data in loaded?
i.e. Does everything from the first memory byte touched to the last byte get loaded (with potential blank areas of memory in the middle)? Or are only the used memory segments loaded into place?

Edited by snicklin, Sun Nov 20, 2011 7:02 AM.

[Rybags]

Decompress while loading - I don't see why not possible although overrun might be a problem and you'd need a custom SIO routine.

Typically the old systems, even Amiga/ST, would load big sections then decompress. You'll notice that modern day systems for our 6502 machines tend to do the same.

Traditionally, tape games tended to just load big blocks of memory. It's entirely possible to do binary load (DOS style) files from tape, so having segments all over the place, but again I don't think many games did so.

Realistically the best decompression system for a tape load would load as much data as possible, decompress, do your stuff, then repeat that again.

Probably the same again for disk. In doing so you minimise the number of spin-up delays, plus decompressing on the fly from disk might mean the sector reads get delayed and you don't get optimal read rates.

IMO in this day, the main reason for compressed executable files is to reduce load time - but if the decompress takes too long that advantage soon vanishes for fast SIO or IDE devices.

[snicklin]

Rybags, on Sun Nov 20, 2011 7:15 AM, said:

IMO in this day, the main reason for compressed executable files is to reduce load time

Absolutely! The one thing I hated about the Atari was the load times (and errors as you see in my avatar).

Now there were some devices to fit to your tape desk, Rambit(?). I got one of these attached to my XC12 by Derek Fern. Did they use some form of compression, albeit I guess in hardware?

[Rybags]

Haven't heard of that one.

The only things I heard of for tapes where the turbo hardware mods mainly from Eastern Europe, and the more modern technique of using one of those false tapes that people used to play CD walkmans through their car stereos.

[CharlieChaplin]

Well,

in the past or better in the 90s, they used compression in Poland, Czech Republic and Slovakia for lots of commercial games (which were then available on tape, turbo-tape and diskette). Packers used were Super-Packer, Code3-Cruncher, Magnus-Cruncher and others. This was typical for LK Avalon, ASF, Sonix, Mirage and many other polish companies. There were packers that de-compressed after the whole program had been loaded (Code3-Cruncher, Magnus-Cruncher) and packers that de-compressed after every data-segment (Super-Packer, Dj-Packer, etc.).

While I have lots of these polish programs on disk, I do not have any on tape - but I guess the same compression methods were used there. On diskettes these packers or compressors saved some diskspace and a few seconds of loading time, but on tape the packers saved many minutes of loading time (up to 70% less space for the packed program meant up to 15-20 minutes less loading time on tape; de-compression was done within seconds).

One can easily test this nowadays by using a packer, cruncher or compressor with a ML file and then saving this compressed ML file to tape. All you have to do then is add a tape bootloader (there are even old bootloaders for tapes available, so you needn`t write one yourself). Or download some polish games from Fandal`s site or atarionline.pl, look/test if they are compressed and then copy them to tape (add a bootloader in front of them, so you can boot the tape with Start+Option and the bootloader will then load the compressed ML file). Searching for a ML tape-bootloader ? Look for COS (Cassette Operating System) or BL/C (binary load cassette) or something similar. There are also special tools which will turn any ML-file (*.COM/*.XEX) into a boot-tape, so a tape-bootloader won`t be needed then...

Afaik, none of these professional compression methods (like RLE, Huffman, Crunch, Lempel-Ziv, Inflate, etc.) were used in the 80s on the A8 with automatic depacker. All I do remember, is that some programs did use simple "Bit/Byte compressors" (all zeros were removed from the ML file; the ML file could then mostly only be loaded from a gamedos and the ML file did have dozens or hundreds of segments). One example of this bit/byte compressing is the type-in game "Zand`s Labyrinth" that appeared in german magazine Happy Computer (it was written by S.Baucke, the author also coded + released a gamedos named NDOS and a compressor named File-Compactor). Most newer file-versions on the Homesoft compilation disks are packed with such a simple Bit/Byte compressor (and therefore have hundreds of segments)...

-Andreas Koch.

[Kr0tki]

Rybags, on Sun Nov 20, 2011 6:21 AM, said:

If an "accurate" archive system was to be devised, then maybe some sort of extended CAS standard where extra detail was retained.

Done already.

CharlieChaplin, on Sun Nov 20, 2011 8:36 AM, said:

While I have lots of these polish programs on disk, I do not have any on tape - but I guess the same compression methods were used there.

Yes, they were. Some of the tapes have been dumped to CAS format and are available for all to play.