10 replies to this topic

[drac030]

Hi

I always thought that in MyDOS the VTOC size is indicated by the first byte of the VTOC, according to such a formula: vtoc[0] = vtoc_size-1.

Today it occurred to me that it doesn't work that way: for example, a 720k disk has two VTOC sectors, so I'd expect $03 as the first VTOC byte, but MyDOS puts $04 there. A 16 MB disk contains 33 sectors of VTOC, and the first byte is $23 whereas I'd be expecting $22. Moreover, a disk which has 65535 sectors, 128 bytes each, contains 65 VTOC sectors, so the expected value is $42, but on a real disk it is $23 again.

So, before I waste probably several hours digging in the MyDOS source code, would someone, who has the knowledge, be so kind to answer there two questions:

1) what is the correct formula that allows to calculate the value of the first byte of MyDOS' VTOC? (or in other words, what's the true meaning of vtoc[0]?)

2) what is the correct method of finding out, what is the size of the VTOC on a MyDOS disk?

Thanks in advance.

[HiassofT]

Hi!

drac030, on Sun Apr 3, 2011 3:49 PM, said:

I always thought that in MyDOS the VTOC size is indicated by the first byte of the VTOC, according to such a formula: vtoc[0] = vtoc_size-1.

Not exactly, SD and DD formats differ. Here's a code snippet from Dos2xUtils.cpp (AtariSIO):

                // general MyDos format
                numVTOC = 1 + (numsec + 80) / (seclen * 8);

                if (seclen == e128BytesPerSector) {
                        if (numVTOC == 1) {
                                buf[0] = 2;
                        } else {
                                if (numVTOC & 1) {
                                        // mydos allocates VTOC in pairs of 2 in SD
                                        numVTOC++;
                                }
                                buf[0] = (numVTOC+1) / 2 + 2;
                                fUse16BitSectorLinks=true;
                        }
                } else {
                        if (numsec < 1024) {
                                buf[0] = 2;
                        } else {
                                buf[0] = 2 + numVTOC;
                                fUse16BitSectorLinks=true;
                        }
                }
                buf[1] = buf[3] = (lastsec - 11 - numVTOC) & 0xff;
                buf[2] = buf[4] = (lastsec - 11 - numVTOC) >> 8;

                fImage->WriteSector(eVTOCSector, buf, seclen);

                // clear additional VTOCs
                memset(buf, 0, seclen);
                for (s=1;s<numVTOC;s++) {
                        fImage->WriteSector(eVTOCSector - s, buf, seclen);
                }

"numsec" is the total number of sectors on disk, seclen is either 128 or 256.

I verified this code against MyDOS 4.53, using various disk sizes from 720 to 65535 sectors and it produced identical VTOCs, so I guess it should be correct.

MyDOS allocates the VTOCs in chunks of 256 bytes, meaning each chunk will use 2 sectors in SD. The first byte of the VTOC doesn't reflect the number of VTOCs in sectors, but in 256 byte chunks (plus 2..., and also with a special case for 90k DOS2.x compatibility where the value is just 2).

so long,

Hias

[1050]

1) There is no formula except this one in the comments from the source code:



BIT DLINK ; (SINGLE DENSITY?)
BMI MPNSD ; IF NOT, M-3
ASL A ; IF SO, M*2-5

M is supposed to be the DOS type byte I assume, which is the first byte of the first MyDOS VTOC sector at sector 360. I assume it's based on the total number of VTOC sectors used. Earlier they added 3 to MAP2 which is the number of VTOC sectors created but without comment.

They really don't say for sure what M is anywhere. It's never mentioned again that I know of. They do subtract two but never 5. I think they didn't bother to correct the formula that they finally did settle on. That one doesn't seem to work for me.

None of it works out correctly in many cases anyway like your example of the 720K disk. A 360K DSDD disk is also fully a type 3 disk, but has only one VTOC sector just like any 90k or 180k DOS 2.0 compatible disk would, but it has to forego file numbers and file ordering in order to access a higher sector count than can be conveyed by the 10 bits of the DOS 2.0 convention which has it's upper limit at sector number 1023. And that DSDD 360k disk surely won't fit any formula I can come up with.

As noted, a 16 meg partition should have for a type byte 22h but has 23h instead. My 1 meg ramdisk should have 0Ah but has instead 07h.

There are no utilities accessing this byte for any reason and it's not even used in MyDOS except to be tested as being larger than 2. So it's value really doesn't matter in the end. But on a possibly related note my 1 meg ramdisk overallocates it's own VTOC structure such that I have one sector that is not ever used for any purpose ever at the front of the VTOC sector string and that situation would be very much preferrable as to having one with an overwritten VTOC sector.


2) Since it's not documented anywhere, I assume we have to use it with a grain of salt. We all pretty much know why, how, and what it's there for. At least among coders, that's a fair statement.

In actual use the exact value doesn't seem to be too important. So neither does a formula for it? As long as we don't break the general rules for it's use, I'm fine with that.

[drac030]

Hias: thanks a lot.

1050: there is just only one problem, that while writing to a MyDOS disk, even MyDOS itself must be able to find out, where the VTOC's end is, because otherwise it could not reliably decide, if a disk is already full or not.

And, btw. both your ramdisk and 360k DSDD are congruent with Hias' formula: a 1 Meg ramdisk contains 8192 sectors, 128 bytes each, so the VTOC has to have 1024 + 10 bytes. If you divide this by 256 (coz according to Hias, the VTOC is measured in pages, not sectors) and round the result up, you'll get 5. Next you add 2, and voila: it is 7, as it is there.

Also the 360k disk example is okay: it has 1 VTOC sector consisting of 256 bytes. So the value must be 1 + 2 = $03. A 90k and a 180k disk, as Hias wrote, are special cases, where the value of $02 is reserved for.

A 16 MB disk: 65535 sectors, so 65615 bits of the VTOC = 8202 bytes = 33 pages, plus two = $23. And so on.

Thanks again guys for replies.

[Mathy]

Hello guys

The VTOC bug should be fixed in the current beta of MyDOS 4.55 (I guess that would be beta 5, which only Lee has). If I don't forget, I'll email Lee later this evening.

sincerely

Mathy

[drac030]

What VTOC bug? Does the fix affect the above calculations?

[Mathy]

Hello Drac030

I just sent Lee an email. I know MyDOS has some problems with the VTOC which sometimes do and sometimes don't occur. But I don't know enough about them to answer you question. I hope Lee responds soon.

sincerely

Mathy

[1050]

There is more than one problem but it's not the one you are pointing to. You are assuming the wrong thing here. MyDOS does NOT find the end of the VTOC string using the Type byte as you are suggesting.

Replace the type byte with any number larger than 2 after the disk/drive has been formatted and find absolutely no difference in ANY behavior at all. Ever. Doing anything anywhere. Outside of low capacity floppy disks of course.

Whether it uses just the running free sector count stored at offset 3 and 4 (zero based) of sector 360 and/or the total free sectors at 1 and 2 offset on the same sector, I don't know. But it's using something else other than the Type byte for the task at hand, I've done the above substitution and sat stunned in real disbelief. What's it for then if not used for exactly that (as you have suggested)? Been there, done that, bought the tee shirt, it shrank in the wash and won't fit anymore. Nothing else to say about that.


And the orginal concept is EXACTLY as you (and I) first assumed it to be:
The DOS Type byte value should be equal to the number of VTOC sectors actually used + 1 always but for the single DSDD 360K exception already discussed. Which can only mean MyDOS is NOT congruent with it's own orginal concept of the type 3 disk, except for that very same one exeptioned floppy disk and all those of lesser capacity. While MyDOS is exactly congruent with Hias's posted C code snipet (how could it NOT be? see derived from same...) for whatever that is actually worth, my ramdisk is still overallocated and missing a useful sector. It gets worse, I should have confessed all before this point was ever reached. When I reserve banks for other uses and then format it at that odd size, my ramdisk overwrites it's own VTOC sectors sometimes (or not) depending on the size as formatted. Apparently, the 'wrong' Type byte IS being used to allocate the VTOC sector string at format time! That's a very scary prospect if actually true.

But then the risk is real only to certain large SD disks, which would have to be ramdisks, since it's written in the manual that all large disks (under the H device) are assumed to be DD, and ramdisks can only be SD. Larger double density drives always have a type byte that is close enough (+1) to the orginal concept such as to operate without the errors inflicted upon ramdisks of one off odd sizes. In order to cause these same problems for large DD disks, the Type byte (during format time) would have to be +2 (overallocation) or -2 (overwritten last VTOC) of the original concept Type byte. All standard sized ramdisks just happen to have exactly allocated or overallocated VTOCs and no real problems are apparent at first glance. This is a MyDOS bug, good and proper, it should be fixed and be made congruent to it's own type 3 disk concept.

It's been fun, thanks.
How many college classes would one have to sit thru before he heard the word congruent?
Nice touch.

drac030, on Mon Apr 4, 2011 2:11 AM, said:

A 90k and a 180k disk, as Hias wrote, are special cases, where the value of $02 is reserved for.

I must take exception to that wording. Contrary to being a special case, MyDOS code entirely is based upon these two disk structures such that all other and larger disks remain compatible with and use their basic rules and conventions as much as possible. The 90K and 180K disks are then THE foundation stones of MyDOS and has as it's Type byte rule:
The number of VTOC sectors + 1

Which is exactly the same one that type 3 disks are supposed to follow. That's the whole idea. The 130K disk follows the rule exactly but the 360K disk can't possibly and at that point on, MyDOS is not congruent with it's own type 3 disk convention, which is what attracted your attention in the first place. That part IS broke and needs fixing.

[Mathy]

Hello guys

This is what Lee wrote:

Quote

1050 outlines the basic problem point on. The original authors hit the same snag at the 360k disk and just
munged the math a digit to make the 360K disk still work while adding the same extra bumped by one value to
all the rest of the larger disks. Which only causes any problems with larger ramdisks because the type
byte grows with the size of the disk times two only on single density disks. At some point in that process
then that number is just too far away from the correct one that needs to be used and VTOC problems begin.
They thought they could get away with it and as far as they tested it - they did, I suspect that they just
didn't fully test all possible ramdisk sizes. In order for the munge to work like this they had to put the
doubling of sector counts for single density in the wrong place which masks the ramdisk issue for a good
spell until a very large size is reached finally. It also allows 7 to work with one over allocation where
5 is the correct number before doubling to 10 VTOCs, very confusing.

The type byte is indeed used to allocate the VTOC sectors and must be close to correct at that time for
that process to pan out properly. Even DD is using double buffers for VTOC work and that's 0200h or 512
bytes for the record, which is also why close to correct can work. Once I found where the type byte came
from and started using it as is, instead of the current munge, and also placed the SD doubling routine
where it logically belongs, the ramdisk problem just evaporated. No more under or over allocation occurred
no matter what the size even with single bank resolutions, on big or small disks. That tells me I'm on the

right track and the bug is squished, but then the only disk that wouldn't work out right was the 360K one
(the original problem), so I had to dig a hole in the code right there, make room to handle it special and
got it to work out right too.

Yes the math is affected for the Type byte, it is now just the same as a 90K disk, also complying with the
type 3 disk as well, # VTOC sectors + 1 for all except 360K disks which get an extra + 1


The number sent to the VTOC allocation routine is the type byte - 2. So it's working with only the extra
VTOC sectors existing if any. If none, then a simple return is done and the disk is finalized with 8 blank
directory sectors. The 360K gets another exception here too, an extra -1 so that it returns with no extra
VTOCs allocated.

Sincerely

Mathy

[drac030]

Just to clear it up, I am not "suggesting" anything. I asked for the meaning of the first VTOC byte and Hias came up with a formula, that worked in my tests under MyDOS 4.50/4.53/4 and also looks consistent. Calculating the VTOC size out of it is simple and elegant:

1) if it is less than $03, then assume 1. stop.
2) else, if it is $03 or more, subtract 2
3) if it is a single-density disk, multiply by 2.

Question 1: If this value is flawed, as you say, then please give an example of a disk size, where MyDOS puts a wrong value here during formatting (definition of "wrong": such one, that does not match the number of VTOC pages). Note that I am only interested in "real" disks, not ramdisks, because I am only able to make a real disk (an ATR file) to be of arbitrary size to test that.

Now you seem to propose to change the math and return to some original idea behind it: vtoc[0]=vtoc_sector_count+1. Note however, that your formula does not allow to make the above calculation, because $03 is ambiguous: it may mean two sectors of VTOC or just one (in 360k).

Moreover, if we are given a MyDOS-formatted disk, there is no way to tell, if its vtoc[0] has been calculated according to the old or to the new formula: $23 may mean "33 pages of VTOC" (33 or 66 sectors, depending on the density, in MyDOS 4.50) or "34 sectors of VTOC" (in the "new" MyDOS). So the VTOC field in question becomes, again, ambiguous.

Plus, an 8 MB disk (65535 sectors, 128 bytes each) requires 65 sectors of VTOC, which expands the possible values of vtoc[0] to $42. So far, checking if the vtoc[0] is between $03 and $23 was a simple way to determine, if the sector links are 16-bit. Now these boudaries expand so that they include APE PC Mirror (vtoc[0]=$3C), which does NOT use 16-bit links. So, again, there is another point where the new system is (can be) ambiguous.

Question 2: what is, then, the correct way of determining MyDOS format unambiguously?
Question 3: what is the correct way to determine, if the vtoc[0] was calculated the new or the old way?

Note that if the vtoc[0] is so ambiguous ($03 may mean this or that, and any other value as well, plus $3C may mean 16-bit links or not), then calculating it according to a "formula" completely lacks sense, because the value does not mean anything useful anyway. So in the light of the information provided, you might as well keep it $03 for all formats extended (MyDOS-ish) formats.

Now let me get back to the original problem: how to calculate the VTOC size in sectors. You may say that I could calculate it out of the initial free sectors value stored in vtoc[1] and vtoc[2]. But this number, let me underline, is the number of FREE sectors, not the total number of sectors. And VTOC embraces all sectors, not only the initial free count. To calculate the VTOC size, I need the total count, and to make this from the free count, I have to add boot sector count, directory sector count, and VTOC sector count.

In other words, it seems that I must know the VTOC size to calculate the VTOC size. So:

Question 4 (as in post 1): what is the correct method of finding out, what is the VTOC size? I am interested in concrete formula to calculate it out of the values we find in the first sector of VTOC.

The only method I thought of so far is not simple nor elegant:

1) add 80
2) add 11 (3 boot + 8 dir)
3) divide by 8, round the result up
4) divide by sector size, round the result up
5) add the result to the free sector count
6) repeat steps 3-4

PS. It all this effort really worth the trouble, considering that the math change allows to avoid overallocation (by 128 bytes) in ramdisks and single density disks, which (I guess) are not anyone's primary storage anyway?

Edited by drac030, Wed Apr 6, 2011 6:12 AM.

[HiassofT]

Hi!

1050, on Sun Apr 3, 2011 9:25 PM, said:

1) There is no formula except this one in the comments from the source code:

BIT DLINK ; (SINGLE DENSITY?)
BMI MPNSD ; IF NOT, M-3
ASL A ; IF SO, M*2-5

You missed the important part before this, here's the whole code:

120D AD0809     010050   LDA MAPBUF
1210 38         010060   SEC
1211 E902       010070   SBC #2      ; GET # OF SECS
1213 2C1107     010080   BIT DLINK   ; SD?
1216 3001       010090   BMI MPNSD   ; IF NOT, M-3
1218 0A         010100   ASL A       ; IF SO, M*2-5

The comments about M are a little bit confusing, if I got it right M here means the number of additional VTOC sectors (excluding the main VTOC sector at $168). But it's better to look at the A register which contains (type-2) in case of DD or (type-2)*2 in case of SD. Very simple and neat :-)
The rest of the MyDOS code is also clearly calculating the "type" byte as 2+(number of 256-byte VTOC blocks). It could well be that there are some errors in other parts of MyDOS, but at least the formatting code works as expected :-)

Here's what I get when I format a 1440 sectors DD disk (aka 360k) with MyDOS 4.53:

command frame [ 32 50 68 01 ]  OK
[ put sector ] data block length = 256 bytes
03 94 05 94 05 00 00 00 00 00 0f ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff 00 7f ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff 80 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................

"type" is 3, meaning MyDOS is using 16-bit sector links and the disk has 1 VTOC at sector $168.

Now, formatting a 2880 sectors SD disk (also 360k):

command frame [ 32 50 68 01 ]  OK
[ put sector ] data block length = 128 bytes
04 31 0b 31 0b 00 00 00 00 00 0f ff ff ff ff ff   .1.1............
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff f8 00 7f ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
command frame [ 32 50 67 01 ]  OK
[ put sector ] data block length = 128 bytes
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
command frame [ 32 50 66 01 ]  OK
[ put sector ] data block length = 128 bytes
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
ff ff 80 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
command frame [ 32 50 65 01 ]  OK
[ put sector ] data block length = 128 bytes
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................

"type" is 4, which means 16bit links, and 4-2=2 pages (4 sectors) of VTOC. You also see the 4th VTOC sector (containing all zeroes) at $165 being written.

so long,

Hias