23 replies to this topic

[ParanoidLittleMan]

Gaming from hard disk on Atari ST and compatibles

This is not something special - using modern storage
with old micros is 'in' in last decade. Spreading of Flash
storage medias as Compact Flash and SD cards just uplifted it.
So, we have now diverse interfaces for all popular old micros.
Myself designed first 8-bit IDE IF for Sinclair Spectrum, some
Atari ST hard disk interfaces and later CF version of 8-bit IF.
With Atari ST serial Satandisk and UltraSatan are most popular.
While US is not too cheap, cards are - you can buy for 10 Euros
4GB SD or CF card. Enough for all Atari ST games, believe me.

But above is just theory - most of games will not work from
your SD card. Well, situation is actually better: most of quality
games WILL work - because they are already fixed (adapted, patched)
for running from some hard disk.

Whole gaming from hard disk issue is pretty complex matter, and I
will try to give here some clues, for people interested about more
than just "is it work on my Atari" .

Why it is so complicated on Atari ST, TT, Falcon ?

Because most of games were coded for working from floppies
only. Because of changes in TOS versions, bad coding and even
bugs in games etc. TT, Falcon is special problem: they are not
100% compatible - CPU is main difference, but there are others too.

Little statistic: about 67 % of Atari ST games use TOS calls for
disk access and rest 33% use own code for as said floppy only
access. Some people claimed different ratio, but what I wrote is
based on examining about 600 games.

We could expect that by TOS calling games we can simple copy
all files on hard disk and run them. It works in couple % only.
Even after removing all copy protections. Usual reasons are:
files are prefixed with A:\ or similar. Code is not relocatible in
RAM, and will crash if there is hard disk driver (or just higher
TOS version, taking more low RAM) . So, we need to do some
measures. It can be changing gamecode self - what is hardest
and slowest way. Better is to give same environment to game,
as it is when runs from floppy. It includes freeing low RAM for
game, assign hard disk partitions to logical A, etc.
Floppy Image Runner is SW what can install floppy image (ST, MSA)
as RAMdisk assigned to A, so many games will then run from hard disk.
But it is not good for multifloppy games.

I developed more advanced solution, called GOS. It is actually TOS 1.04
core, modded and reassembled so that can run in High RAM, with hard
disk driver also in High RAM. It solves RAM conflict and TOS compatibility
problems. There is Falcon, TT version of it. Only real flaw is that no support
of 1GB Falcon partitions, as TOS 1.04 can max 512MB.

So much about TOS calling games. Games with direct floppy access need
more work in most of cases. Work from hard disk is possible only by changes
in gamecode. It may be relative easy, but may be pretty hard - I saw a lot
of very strange, inefficient, overcomplicated floppy loaders.
Then, copy protections, checksums make it even harder. And we have RAM
conflict too. TOS has hardcoded workspace for lowest RAM space - it is
about 40KB, but higher TOS versions take more. To achieve disk access during
gameplay we need to resolve RAM conflict and some other problems as interrupts,
MFP etc. I will not go here in more details and all possible solutions. All what can
say is that there are more ways. Every has its goods and bads.
More can read at - http://forum.8bitchip.info FAQ section.

Development is not finished: I working currently on HAGA (HArddisk Gaming Atari).
The goal is to give solution for most of people, configurations. In way somekind similar
to WHDLoad - by using few library files for common functions (by WHDLaad it is Master).
I started with it already by GOS. The benefits are smaller launchers, easier to code, and
easier update - you need only to replace 1 file in libDIR.
So far, you can play Space Ace, Dragon's Lair serial on your ST(E), Mega ST(E), TT,
Falcon, with any decent hard disk driver, 1GB Falcon partitions too. With fast loadings,
even with only 1MB RAM.
Future plans for HAGA are: TOS dep. game support, statesaves, and of course more games.

I could finish this here. However, experiences say that we can expect here some writings
about data loss, stolen code and similar crap. Or that my SW damaged some machine. But we know that SW can not damage HW. I can say to people involved in such: prove it, give concrete evidences. Improve your solutions, instead spreading crap and poisoning community.

We are most likely in last decade of using old Atari machines. They will stop working gradually. Or become unreliable (2 of my Ataris are already such). And we can not do much against. So, enjoy until can, and try to learn little while...

[Lynxpro]

Fascinating read.

Interesting - and disheartening - to read about how games are often dependent upon the TOS version you have to work. It's bad enough there's too much incompatibility between the 68000 and the 68030...

As for this being the last decade our machines will run, I hope that's not true...the 8-bits are still going and most of those models have more years on them than STs do!

As for using SD/Compact Flash, does anyone use anything beyond 4GB? From anecdotal evidence, that seems to be the largest card size I've seen used by ST/TT/Falcon and Amiga users...

[DarkLord]

I use up to 4 gig cards on my STacy with TOS 1.04 - but the higher end models, Falcon, etc, should
be able to do more, I believe.

Perhaps someone who actually has done it can post their experiences.

[DarkLord]

PPera, if you can get something like WHDLoad on the Amiga for Atari, that would be just awesome.

[Sauron]

ppera? Weren't you banned before?

[CyranoJ]

DarkLord, on Fri Jan 13, 2012 2:52 PM, said:

PPera, if you can get something like WHDLoad on the Amiga for Atari, that would be just awesome.

ULS did that, about 5 years ago, is more feature-rich than WHDLoad, is opensource, and is cross platform compatible (ST, STE, MST, MSTE, TT, Falcon)... But hey, wouldn't it be great if he did it again...

[ParanoidLittleMan]

There is no 4GB limit by any hard disk interface (adapter), or even by some driver. I just gave 4GB as example what is enough for some gamer (and not anly gamer).
I don't know exact limits of UltraSatan - in theory it should handle at least 8 GB HcSD cards. With more than 8GB we will encounter max partition count limit (14) for instance.
There are some common mistakes about limits: IDE has no HW limit. All depends from driver. If it is 48bit LBA aware, limit is in skies (some 130000 TB ) .
I connected 160GB IDE disk on STE and Falcon, and it worked well with mine and Hddriver 8.13 . Only that there were problems with Mint and very large FAT32 partitions. Other user had similar problems with very large Ext2 partitions and Mint. So, SW problems.
Compact Flash card of any size should work on Falcon with some new driver. But real problem is signal timing - old Falcon IDE port gives not correct timings by newer ATA specs. Therefore some cards work more less unreliable. Especially writing can be problem. http://atari.8bitchi...o/astideTP.html

That one decade expected is of course just rough estimation. Based on some experiences that consum electronic has about 20 years time, where no much failure happen. After it, more and more components break due to aging.

[ParanoidLittleMan]

DarkLord, on Fri Jan 13, 2012 2:51 PM, said:

I use up to 4 gig cards on my STacy with TOS 1.04 - but the higher end models, Falcon, etc, should
be able to do more, I believe.

Perhaps someone who actually has done it can post their experiences.

Yes, it would be good to know limits of UltraSatan.

@Sauron: I asked at one moment to delete my account here.

Considering WHDLoad porting for Atari ST and compatibles: I don't think that we can do it 100% . 2 machines are different. CPU is same, but many things differ. Amiga has no filesystem driver ( what handles hard disk file access too ) in ROM, as ST serial has. There may be other differences related with OS calls used by games, and how games are sensitive on TOS versions (Amiga OS versions), I really don't know much about Amiga. Then, WHDLOad is based on Master-Slave concept, where Master is some kind of function library for common tasks, what slaves (for every game special one) must call. Master is not free by Amiga. I don't think that anyone here wants commercial WHDLoad equ. for Ataris :-) But Master-Slave concept seems as very practical for me, especially as I already did someething similar with functions in later GOS versions - GOS5 handles hard disk access, reinstalling of hard disk drivers in High RAM and init of TOS 1.04 core in High RAM. I go now further, and make HAGA, what will have much more library functions, and diverse ways for accessing files from hard disk, during gameplay. Basically, it will be same Master-Slave concept, but I don't like to call it so. Game launcher will call library functions, lets say like that.

But loading files during gameplay is just part of problem. We have a lot of TOS function calling games (2/3). This is what needs support too. And we can not take something from WHDLoad for that. I don't want to start some new flame war here. Just few questions: in which way ULS supports TOS incompatibility problems ? Games using TOS calls regulary (and not only disk access), which run in very low RAM ?

Edited by ParanoidLittleMan, Sat Jan 14, 2012 7:10 AM.

[DarkLord]

CyranoJ, on Fri Jan 13, 2012 8:07 PM, said:

DarkLord, on Fri Jan 13, 2012 2:52 PM, said:

PPera, if you can get something like WHDLoad on the Amiga for Atari, that would be just awesome.

ULS did that, about 5 years ago, is more feature-rich than WHDLoad, is opensource, and is cross platform compatible (ST, STE, MST, MSTE, TT, Falcon)... But hey, wouldn't it be great if he did it again...

Hmm, can you post a link where I can download this software and try it on my Atari's? Thanks.

[CyranoJ]

DarkLord, on Sat Jan 14, 2012 8:14 AM, said:

Hmm, can you post a link where I can download this software and try it on my Atari's? Thanks.

OpenSource ULS Code: here
ULS Patches: here and here

In over 5 years of doing this, not a single report of a destroyed partition.

[DarkLord]

Ahhhh....I hadn't realized what you were talking about.

So all the Dbug releases are done using this system, I assume.

Nice.

[ParanoidLittleMan]

As expected, data loss is mentioned. I could talk here a lot what is possible and what not, that after least 10000 game adapt. starts I never had any data loss, even if 90% of it finished with crash, errors (during testings), etc. Most of people believe what they want. So, will be short: I think that I know why ULS is so reliable :
it uses lea someadr(pc),a2 ; move.l (a2),d3 instead straight move.l someadr(pc), d3 .

[Lynxpro]

It sounds like you need a virtual disk drive for TOS to write the commands to to get around all of this mess.

Then again, I'm not a programmer so I'm just talking speculation out of an orifice...



And how do the ST emulators on PC and Mac get around all these TOS commands to read/write from floppy?

Edited by Lynxpro, Mon Jan 16, 2012 10:33 AM.

[Lynxpro]

DarkLord, on Fri Jan 13, 2012 2:51 PM, said:

I use up to 4 gig cards on my STacy with TOS 1.04 - but the higher end models, Falcon, etc, should
be able to do more, I believe.

Perhaps someone who actually has done it can post their experiences.

Card(s)? Just how much ST platform software do you have that encompasses more than 4GB?

[DarkLord]

Lynxpro, on Mon Jan 16, 2012 10:32 AM, said:

Card(s)? Just how much ST platform software do you have that encompasses more than 4GB?

None, really.

Recently, I've actually been using 2gig cards in the Ultrasatan. Their fairly cheap and easy
to find these days, gives plenty of room, and its so awesome to be able to pop in a 2gig card
with Geneva/Neodesk in one session, then switch out for a card with MagiC/Jinnee in another
session, or pop in yet another that's set up for an optimal gaming environment.

Of course, you can do all that with one 4gig card with something like XBoot or equivalent, its
just a bit more complicated.

I love the Ultrasatan.

[Official Ninja]

DarkLord, on Mon Jan 16, 2012 4:01 PM, said:

I love the Ultrasatan.

I am patiently waiting for mine from Zaxon. 4-6 weeks seems like forever! (Almost week 5 now....)
I'm going to order TOS1.04 for my STFM. Also, what is the proper power supply I am going to need for the US?

[DarkLord]

Over at AtariForum, some people have just recently got their orders, so maybe yours will be there soon.

As far as a power supply goes, Jookie's website has a FAQ:

http://joo.kie.sk/ultrasatan/faq.html

Q: What voltage does UltraSatan need?
A: It should be at least 7V, because there is some voltage drop down in the device (due to rectifier). But it shouldn't be much higher than 7V, because higher voltage will result in higher heating of the device. So from 7V to 9V should be OK. You can use some sort of universal DC adapter. For input voltage to regulator temperature relation look at the table bellow.

Read the whole page (and website - its interesting!) though, lots of good info you'll want and need.

HTHs.

[ParanoidLittleMan]

Lynxpro, on Mon Jan 16, 2012 10:32 AM, said:

It sounds like you need a virtual disk drive for TOS to write the commands to to get around all of this mess.
Then again, I'm not a programmer so I'm just talking speculation out of an orifice...
And how do the ST emulators on PC and Mac get around all these TOS commands to read/write from floppy?

Virtual Disk Drive for TOS is actually my Floppy Image Runner.
Considering r/w from/on floppy images by emulators: by Steem and Hatari (2 best for ST serial) floppy is emulated at low level. No direct floppy related TOS command support (but is for hard disk file access) . Basically, floppy controller in Atari is emulated (WD1772) . User may set accurate emulation, when even speed is same, or faster one, when no full emulation (data will be loaded instantly, without waiting on virtual spin) . Accurate emulation is necessary because some SW is sensitive on loading speed. Then, we have even support for copy protections - Pasti . It works like: TOS command (by your words) will send parameters, commands to FDC chip, which will then seek for requested sectors and load data via DMA chip to RAM. It is what is emulated in emulators.

Considering WHDload port for Atari ST serial : I don't think that it is right idea, approach. But one thing we missing for sure from Amiga 'waters' : better support for all this by users, by forums. So, we could have more people testing fixed games and giving feedback, and not just writing 'Works great' and similar, or "Worked not on my Falcon" . We need more details in all this, people with some experience, who want to contribute and not just get everything on silver plate. Some FAQ sections on forums would help that more people understand basic things about all this.

So, I will write here some more basics:

Instead thinking about ULS port (what is ported in some extent, yes) we need to make solutions specific for Atari needs. To achieve that all games, or at least all good games can be run from hard drives. First step in it is understanding the problems, reasons why some game works not etc.

I did categorisation of games by way how it can be fixed for hard disk run:

1. The simplest case : just copy all files to some DIR on hard disk and play.
There is only few such - like Knights of Sky (but only latest edition), most of Sierra games ...
Here belong games with included hard disk installer too. However, it may not work on some configs, often can install only on specific partition, folder. And because of copy protection original must be in A .
Then, here may belong some cracks too - where load is with only relative path.
Beacuse of RAM allocation problems, game must be coded so, that can run at any RAM loc. Otherwise will fail on higher TOS versions and hard disk drivers taking more RAM (buffers). I did not make some statistic for it, but guess that only some 20% of games is coded to run at any RAM loc.
Finally, even if all above conditions are fullfilled, game may fail on Falcon, TT . Usually because: stackframe problem, some writings in video registers, bad code, often sensitive on TOS version - (Prince of Persia) , then even some bugs, which appear only on TT mostly , etc.

2: Singlepart game (or singleparteable) : after start no more disk access. So, we don't need to care about hard disk access. Only need to load all to proper RAM pos, set needed things for start and jump ... If game uses not TOS calls it works pretty well in almost all cases (except possible Falcon, TT problems - see above) . If such game uses TOS calls, we may be in trouble, especially if is not relocatible (runnable at any RAM loc) . It is long story, and I can not go here in all details. What is important is that problem can not be solved without : A: making game relocatible (too much work, almost impossible) . B: having some lover TOS version in RAM , what will then work with game occupying RAM area conflicting with higher TOS versions workspace. Or at least having all by game needed TOS functions in RAM - this is what Klapauzius did.

To be continued ....

[ParanoidLittleMan]

Continuing previous post:

3: Multipart games - which accessing disk during gameplay - usually to load some new stage or similar. Here we have 2 main sub categories :
Games using TOS calls, so accessing disk via them too, and games not using TOS calls for floppy access (there is no game which accessing hard disk with own code, normally) .

3A : Knighs of Skies belong here. And hard disk access works well. But most is not such, and we can have several diverse problems - like file access with absolute path A: , low RAM conflict, Timer C problem with some hard disk drivers, TOS incompatibility problems etc. Even mentioning all is too long. But everything can be fixed.
Worst problem is low RAM conflict, and happens if game is not relocatible.

3B: Direct floppy access, multifloppy games with it. As already said in first post here, some changes ic gamecode must be done, to redirect floppy calls to hard disk calls. It is the hardest part in most cases - understanding how loader works without sources may be time consuming. Fortunately, there are some common systems used, and for instance all games by UbiSoft where Markus Fritz did floppy part have prectically same floppy code. And of course, all mentioned above about TT, Falcon specific problems stays here too.

I will not go in further details about all possible situations.

The solutions:

First used was RAMdisk. And is still in use, although mostly from speed reasons. It is in fact very practical, fast and reliable + simple solution. Drawback is need for RAM, what may be lot by multifloppy games.
So, loading from hard disk during gameplay must be solved somehow. First who did it by Atari ST is crew Superior, some 20 years ago. It was FFLS driver. Little outdated, as works only with ACSI and max 32 MB partitions.

To be continued - it would be good in this posting, but how to edit it after some hours ?

[ParanoidLittleMan]

Continuing about solutions for case 3B (hard disk access by games which 'killing' TOS :
ULS is another solution - based on swapping low RAM area content when disk access from game is requested. Then game content goes in High RAM, before start saved TOS workspace, CPU vectors etc. + hard disk driver go low. After finishing disk access all mentioned must be swapped back - so game low, TOS WS++ high . It is good concept considering compatibility with hard disk drivers. But may be slow in case of many short disk accessings, so WHDLoad recommends to use always RAMdisk if there is space for it. And by Atari patches is so too .
GOS is my solution, similar to FFLS, but it can write to hard disk too, working with all hard disk types via installed drivers etc. Only real drawback is not suporting 1GB Falcon partitions.
New development by me is HAGA - RAM area swap concept too, but with some improvements as using PMMU on TT, Falcon, library files with functions etc. Will take couple months to finish it.

So much about hard disk access by TOS 'killing games' .

And I think that here finishes whole idea about porting WHDLoad to Atari ST :
Because now come Atari ST specific problems, mostly TOS related :

Example for start: there is game Millennium 2.2, using TOS calls, mouse. It even can not be started from AUTO, because then no mouse available. But is not relocatible, and placed to very low RAM (around $13000) . Conflict with some higher TOS v., hard disk driver is inevitable. There is alleged hard disk fixed version in Atari ST Gamebase. But it works only under emulators, GEMDOS hard disk emulation, which takes no RAM for hard disk drivers. So, it is not really hard disk fixed.
To achieve work of Millennium 2.2 on real Atari with hard disk driver installed we need to solve RAM conflict somehow . My first working solution was Hole - tricky solution what moved hard disk driver in high RAM . But it solves not problems by higher TOS versions (Falcon, TT) . Even if you achieve somehow that game works, and disk access is good, likely will be unable to name space ships - as it uses some TOS calls, which fail because RAM conflict.

Good solution for this case is TOS in high RAM, hard disk driver in high RAM . Then no RAM conflict, and game will work well . This is what GOS (Gaming OS) makes possible. And there is a lot of games where it is required. Basically, GOS is disassembled, modded TOS 1.04 GEMDOS part, for work in High RAM. And it solves other things too, as TOS version problems with badly coded games (Prince of Persia) . As it is TOS 1.04, there is 512MB partition limit - only real flaw, as already stated.

To resolve that GOS work with 1GB Falcon partitions, we need to use 2 things at once : GOS self, so GEMDOS in High RAM , + RAM swapping technique, fortunately working very fast on Falcon - and it will not cost us extra RAM, as TOS is already saved in purpose of Desktop exit. This is next step in HAGA development.

Thanx for your patience

[KLund1]

Paranoidlittlema, awesome read. Like the detailed, but easy to understand explanations.
Perhaps I missed it, (most likely) but where can I find your GOS? & and HAGA?

[ParanoidLittleMan]

KLund1, on Tue Jan 24, 2012 7:24 PM, said:

....
Perhaps I missed it, (most likely) but where can I find your GOS? & and HAGA?

http://atari.8bitchip.info/fromhd.php

There are games fixed for hard disk running, but you may find more explanations, and even some sources related (in ASM) .
Where can find GOS ? : GOS is in files D15R_*.FIC . There are diverse versions, all based on TOS 1.04 core, modded for specific needs. If you want more details, ask. I can add even source files of GOS self. GOS is just abbrev, for easier pointing on it in talking. = Gaming OS, running in RAM instead ROM, with some modifications mostly RAM usage related. But there is much more.
Same is with HAGA . But HAGA is new project, currently under development. So far there is support only for TOS 'killing' games, so those not using TOS calls. You will see files HAGA in some 10 latest hard disk fixes. It is actually library of common functions needed by fixing for harddisk run (+ Falcon, TT run specific needs). + functions for return to Desktop and most interesting: gamestate saving and restoring. I think that I solved it pretty well, and easy to use. May try even if have no Atari with hard disk - with Steem or Hatari. Btw. HAGA will use same GOS files in case of need and mine hard disk drivers.
When HAGA will be enough mature, and support all game types, I will publish sources together with usage guide how to call functions and when) - for those who maybe want to deal with fixing some games.

[ParanoidLittleMan]

HAGA development continuing: http://atari.8bitchi...SH/fireice.html

[tresas]

@ParanoidLittleMan: Thank you for great analysis about the hard disk issues for Atari machines gaming. It was very enlightening for me. You have done really admirable job...