58 replies to this topic

[LX.NET]

On the Atari Lynx programming forum I mentioned one possible activity for the Atari Lynx community: continuing development of Handy.
Currently I am working on an .NET based emulator that follows the Handy implementation pretty closely. It is now able to run most single player games on Windows and Windows Phone. (For those interest: http://lynxemulator.codeplex.com)
Programming the emulator has given me insights in the Handy codebase. I think I can attempt to fix some known bugs and add new features.
This post is meant to build a list of known bugs and feature requests, then triage them and get working on it. I have heard and read about some bugs (such as incorrect off-screen drawing). My intention is to get detailed descriptions or hints and preferrably a way to reproduce or compare with an actual Lynx device.

One that I know of:

Joust.
Lava on play screen runs into platform too far.
 snap00.bmp   777.38K   106 downloads

Do you know of any other bugs out there?
Any functionality that you would like to see added to the Handy core?

[Ninjabba]

Nice progress you're making there

Haven't played on an emulator in a while, but I know there's quite some issues with the sound in general.

Also, Warbirds shows glitches in the landscape when flying around. Would be great to have that solved!

[LX.NET]

Ninjabba, on Mon Aug 22, 2011 5:13 AM, said:

Nice progress you're making there

Haven't played on an emulator in a while, but I know there's quite some issues with the sound in general.

Also, Warbirds shows glitches in the landscape when flying around. Would be great to have that solved!

Thanks. Are you making progress as well (either study or game)?

I think I was able to reproduce one glitch: a wrong horizon.


Can you expand on the sound issues a bit?
Thanks.

[sage]

From my memory...

Sprite drawing:
Real hardware is accepting "wrong" spritedata, emulation only the one specified in docs. (Bastian's Cube demo)

Out of Screen/End of Life Bit is not correctly set (sometimes). (have to look for my example code...)

Scaled sprites are sometimes off by one pixel (guess: rounding? wrong fractions?)

Multiply/Divide behaves differently, I think esp if signed math is used. (have to look for my example code...)

[Ninjabba]

LX.NET, on Mon Aug 22, 2011 8:25 AM, said:

Ninjabba, on Mon Aug 22, 2011 5:13 AM, said:

Nice progress you're making there

Haven't played on an emulator in a while, but I know there's quite some issues with the sound in general.

Also, Warbirds shows glitches in the landscape when flying around. Would be great to have that solved!

Thanks. Are you making progress as well (either study or game)?

I think I was able to reproduce one glitch: a wrong horizon.


Can you expand on the sound issues a bit?
Thanks.

That's the glitch I was recalling yes.

Study goes steady, I'm really close to finishing this (finally..). As for the game, I'm working on it on a bare minimum, but its definitely been worked on on a weekly base. Hope to give some update of that in a few weeks.

As for the sounds, the music of Shadow of the Beast couldn't be played normally on my PC. Can't check it though.. for some awkward reason Handy stopped working constantly asking for the boot image. Tried everything to get it working, but no dice. On Linux using wine I have no sound at all, but thats an issue caused by wine I think.

[LX.NET]

sage, on Mon Aug 22, 2011 9:32 AM, said:

From my memory...

Sprite drawing:
Real hardware is accepting "wrong" spritedata, emulation only the one specified in docs. (Bastian's Cube demo)

Out of Screen/End of Life Bit is not correctly set (sometimes). (have to look for my example code...)

Scaled sprites are sometimes off by one pixel (guess: rounding? wrong fractions?)

Multiply/Divide behaves differently, I think esp if signed math is used. (have to look for my example code...)

Sage, I would really, really appreciate if you could look up the code. It sounds isolated enough to be able to reproduce and fix.

I do not understand the End of Life Bit. Which register/address is that and what is its purpose?

Can you also name the exact demo name of Bastian?

Thanks in advance.

[LX.NET]

Ninjabba, on Mon Aug 22, 2011 10:20 AM, said:

LX.NET, on Mon Aug 22, 2011 8:25 AM, said:

Ninjabba, on Mon Aug 22, 2011 5:13 AM, said:

Nice progress you're making there

Haven't played on an emulator in a while, but I know there's quite some issues with the sound in general.

Also, Warbirds shows glitches in the landscape when flying around. Would be great to have that solved!

Thanks. Are you making progress as well (either study or game)?

I think I was able to reproduce one glitch: a wrong horizon.


Can you expand on the sound issues a bit?
Thanks.

That's the glitch I was recalling yes.

Study goes steady, I'm really close to finishing this (finally..). As for the game, I'm working on it on a bare minimum, but its definitely been worked on on a weekly base. Hope to give some update of that in a few weeks.

As for the sounds, the music of Shadow of the Beast couldn't be played normally on my PC. Can't check it though.. for some awkward reason Handy stopped working constantly asking for the boot image. Tried everything to get it working, but no dice. On Linux using wine I have no sound at all, but thats an issue caused by wine I think.

Yes, sound is sometimes a bit garbled, but I think that has to do with the stitching together of small pieces of sample. I haven't gotten around to implementing the sound in my own emulator and am a little less knowledgable on that topic, but as soon as I have my last major bug found (I broke multiplayer games fixing another bug) I will turn my attention to implementing sound.

[sage]

LX.NET, on Mon Aug 22, 2011 11:01 AM, said:

Yes, sound is sometimes a bit garbled, but I think that has to do with the stitching together of small pieces of sample. I haven't gotten around to implementing the sound in my own emulator and am a little less knowledgable on that topic, but as soon as I have my last major bug found (I broke multiplayer games fixing another bug) I will turn my attention to implementing sound.

The sound works better in mednafen, but as its using the same engine, its really a sound buffer problem.

[sage]

LX.NET, on Mon Aug 22, 2011 10:57 AM, said:

I do not understand the End of Life Bit. Which register/address is that and what is its purpose?

Can you also name the exact demo name of Bastian?

Thanks in advance.

The bit which is set, if a sprite is off-screen (EVERON). i think collision detection must be switched on for that, but i am too lazy to read the doc now. Keith tried to fix that in the lastest release, but i think the bug is still in.

Bastians demo is in the BLL kit as asm source code. either demo006.asm or drawtest.asm, both have the same problem.

Add: The problem comes from the file fpolygon.inc which is drawing the actual filles polygons.

Add: I tried to attach the sourcecode, but looks like this is not allowed.

Edited by sage, Mon Aug 22, 2011 2:49 PM.

[Songbird]

The timing emulation is off in certain circumstances. I know that's vague; let me try to provide a specific example.

Here's the behavior I used to notice on Windows XP: in Remnant, I use some simple C code to play the title music and do the timing (no interrupts). When I am scaling the Remnant logo graphic at the same time as the music, the music accelerates. Once the scaling stops and everything on the title animation reaches a "steady state", the music timing is correct.

On Windows 7, all of the title music on Remnant is fast regardless of the scaling. This is more curious since Champ Rally uses similar code, and yet that music plays at the correct speed.

[johnodd]

Awesome!!!!!

Hey if you could get this to work with wineskin you could have a pretty cool emulator that windows and mac users could use nice work for starters i didn't really have any issues with emulators in the past but you are right the horizon problem was common with a lot of emulators but keep up the amazing work

[LX.NET]

Songbird, on Mon Aug 22, 2011 9:00 PM, said:

The timing emulation is off in certain circumstances. I know that's vague; let me try to provide a specific example.

Here's the behavior I used to notice on Windows XP: in Remnant, I use some simple C code to play the title music and do the timing (no interrupts). When I am scaling the Remnant logo graphic at the same time as the music, the music accelerates. Once the scaling stops and everything on the title animation reaches a "steady state", the music timing is correct.

On Windows 7, all of the title music on Remnant is fast regardless of the scaling. This is more curious since Champ Rally uses similar code, and yet that music plays at the correct speed.

Hi Carl,

Thanks for thedetails on this bug. Strange indeed.

The first thing that came to mind is the prediction based implementation of the timers (including those for sound) in Handy. The CPU in the emulator will sometimes "miss" interrupts because it put itself to sleep (e.g. to give the bus to Suzy to start the sprite rendering). At the end of the sprite painting in Suzy the clock cycles are advanced to include the work done in Suzy. The emulator has predicted moments for the sound timers to expire, and executes these. Then, because the next one has expired as well, it will fire that one, and the next until it has caught up. So, I am not entirely sure if this is related (at all), since you mention not to use any interrupts. Another strange thing is that it does not occur for W7.

Is XP running on a slower machine?

Definitely needs some further investigation.

I also have sometheory on the CPU timings in Handy and how these might be different. I doveinto the actual fetch, read/write and parallelism of the 6502 processor (and65SC02) and came up with somewhat different numbers, some of them evendependent on whether a specific path (like in branches) was taken or not. Itwould certainly be interesting to take a closer look at the precise timings of the CPU ona piece of code that has a cycle precise implementation.

Once we get started on this, we could dive some deeper into the details. I'll put it on thelist of bugs for now. Thanks for the contribution.

Everyone, keep themcoming.

Edited by LX.NET, Wed Aug 24, 2011 1:53 PM.

[LX.NET]

sage, on Mon Aug 22, 2011 2:07 PM, said:

LX.NET, on Mon Aug 22, 2011 10:57 AM, said:

I do not understand the End of Life Bit. Which register/address is that and what is its purpose?

Can you also name the exact demo name of Bastian?

Thanks in advance.

The bit which is set, if a sprite is off-screen (EVERON). i think collision detection must be switched on for that, but i am too lazy to read the doc now. Keith tried to fix that in the lastest release, but i think the bug is still in.

Bastians demo is in the BLL kit as asm source code. either demo006.asm or drawtest.asm, both have the same problem.

Add: The problem comes from the file fpolygon.inc which is drawing the actual filles polygons.

Add: I tried to attach the sourcecode, but looks like this is not allowed.

Hi Sage,

Found the code samples that show the other sprite data structure (in a wrong way in Handy). I also came across other samples (Wuerfel3 and lite that exhibit the same problem. That is enough to get me going.

Your samples for the signed math and EVERON bit are certainly still welcome. I need a little more to get started with those two. Can you take some time to look those up?

Do you have any particular examples or moments where you saw the one-pixel-off behavior in the sprites? The sprite rendering has some default offsets (documented ones and other ones for the squashed look), and special algorithms for scaling, tilting and stretching. There might be edge cases where the rounding is not accurate or some other reason. Some extra info will certainly help.

[rdemming]

As far as I remember, in "Shadow of the Beast", the rasters (the color changes making up the background) are not stable. They jitter up and down.

Robert

[LX.NET]

sage, on Mon Aug 22, 2011 2:07 PM, said:

Bastians demo is in the BLL kit as asm source code. either demo006.asm or drawtest.asm, both have the same problem.

Add: I tried to attach the sourcecode, but looks like this is not allowed.

I thought it would be illustrative to include some screenshots showing the bugs.


Wuerfel demo show the wrong sprite data and the messed up polygons

[LX.NET]

rdemming, on Thu Aug 25, 2011 2:25 AM, said:

As far as I remember, in "Shadow of the Beast", the rasters (the color changes making up the background) are not stable. They jitter up and down.

Robert

Thanks for the input, Robert
Included a picture with the jittery background. Hard to see in the picture (as in: not at all), but the area that shows this defect is marked
The jittery bit is not present for the larger part of the purplish background. It is just the bottom part (if visible through the layers covering it).



The bug goes on the triage list.

Edited by LX.NET, Sun Aug 28, 2011 1:59 PM.

[sage]

LX.NET, on Wed Aug 24, 2011 2:23 PM, said:

Your samples for the signed math and EVERON bit are certainly still welcome.

-> PM

[ThomH]

Re: the sprite problems highlighted by LX.NET, if I recall from my investigations with BLL probably almost half a decade ago, it's related to this comment in the Lynx documentation:

Quote

Well, I finally found the bug that required a pad byte of 0 at the end of each scan line of data. But, It is actually 2 bugs. I have fixed one of them, but the other requires an extensive change. Too bad, I am out of time

Beyond that the specifics escape me, but suffice to say there's a Lynx hardware bug that BLL was benefiting from. Specifically, those polygons are drawn by stretch and tilting a one pixel sprite. The source data in the relevant BLL library says there should be two pixels. Handy obediently ends up with scan lines that are twice as long as they should be. The real Lynx's hardware bug ignores the second pixel and draws with only one.

I verified this at the time by correcting the BLL library method and testing against Handy. I will have posted the relevant facts to usenet, I expect, but probably since BLL was last updated.

Given that there's no documentation on exactly what the Lynx hardware bug is as far as I'm aware, I guess the solution would be to treat the last byte as a pad byte regardless of its actual contents.

As an emulator author, I'd guess that the Shadow of the Beast problem is probably very hard to solve with the Handy source code. There are a multitude of ways to structure an emulator that — broadly speaking — span a range from completely correct but horrendously processor intensive to 'acceptably' inaccurate but very zippy. It's a trade off, essentially, and Handy's placing will be based on the processing envelope of over a decade ago.

From a development point of view, if I recall correctly then Handy's hardware maths unit is, incorrectly, instantaneous. That likely gives some timing errors in commercial code and when you're trying to write optimised code that overlaps maths and CPU stuff it's a genuine hazard because you lack that extra check that you've had the CPU do enough things for the maths result to definitely be ready.

[lauwlauwlauw]

There was some bug reported on the SourceForge Handy site. See: http://sourceforge.n...313&atid=640866

Basically:
In Gates Of Zendocon the intro is not playing all the sounds (actually missing only one) and the floating brain' eyes are jittering.

[LX.NET]

ThomH, on Wed Nov 23, 2011 5:53 PM, said:

Re: the sprite problems highlighted by LX.NET, if I recall from my investigations with BLL probably almost half a decade ago, it's related to this comment in the Lynx documentation:

Quote

Well, I finally found the bug that required a pad byte of 0 at the end of each scan line of data. But, It is actually 2 bugs. I have fixed one of them, but the other requires an extensive change. Too bad, I am out of time

Beyond that the specifics escape me, but suffice to say there's a Lynx hardware bug that BLL was benefiting from. Specifically, those polygons are drawn by stretch and tilting a one pixel sprite. The source data in the relevant BLL library says there should be two pixels. Handy obediently ends up with scan lines that are twice as long as they should be. The real Lynx's hardware bug ignores the second pixel and draws with only one.

I verified this at the time by correcting the BLL library method and testing against Handy. I will have posted the relevant facts to usenet, I expect, but probably since BLL was last updated.

Thanks ThomH, this is really helpful. I will have a look at this particular bug mid December (bogged down with preparing for presentations before that time).

Quote

As an emulator author, I'd guess that the Shadow of the Beast problem is probably very hard to solve with the Handy source code. There are a multitude of ways to structure an emulator that — broadly speaking — span a range from completely correct but horrendously processor intensive to 'acceptably' inaccurate but very zippy. It's a trade off, essentially, and Handy's placing will be based on the processing envelope of over a decade ago.

So true.

Quote

From a development point of view, if I recall correctly then Handy's hardware maths unit is, incorrectly, instantaneous. That likely gives some timing errors in commercial code and when you're trying to write optimised code that overlaps maths and CPU stuff it's a genuine hazard because you lack that extra check that you've had the CPU do enough things for the maths result to definitely be ready.

I have thought of doing work in a different thread, to simulate the concurrent processing going on. There are other ways. At least, in my version I do set the MathInProgress when math calculations are starting (and clearing when done, instantly atm).

Do you have any knowledge of the other sprite rendering bug that seems to occur in the cube demo: the bad header in red pixels?

[LX.NET]

lauwlauwlauw, on Fri Nov 25, 2011 4:28 AM, said:

There was some bug reported on the SourceForge Handy site. See: http://sourceforge.n...313&atid=640866

Basically:
In Gates Of Zendocon the intro is not playing all the sounds (actually missing only one) and the floating brain' eyes are jittering.

Thanks, I had missed that one.

[ThomH]

LX.NET, on Sat Nov 26, 2011 2:41 PM, said:

Thanks ThomH, this is really helpful. I will have a look at this particular bug mid December (bogged down with preparing for presentations before that time).

Further on this, the BLL bug that is nullified by the Lynx hardware bug that Handy doesn't reproduce is recorded on line 330 of BLL's fpolygon.inc:

.triimage       dc.b 2,$11,0

That's unpacked data, so if the Lynx hardware were perfect it'd be two pixels and Handy renders it as such. On the real hardware it's only one pixel. Switching the $11 constant to $10 fixes that code in Handy.

I guess that, based on the description in the hardware guide that the problem is just the final bit, switching the $11 to $22 should produce double-width scan lines on both Handy and a real Lynx. I don't currently have any tools set up and things are a bit complicated because I'm about to emigrate but I do have one of Lynxman's flash carts, so if you're unable to test that and think it would be helpful then I can probably do that but not necessarily very soon.

LX.NET, on Sat Nov 26, 2011 2:41 PM, said:

Quote

From a development point of view, if I recall correctly then Handy's hardware maths unit is, incorrectly, instantaneous. That likely gives some timing errors in commercial code and when you're trying to write optimised code that overlaps maths and CPU stuff it's a genuine hazard because you lack that extra check that you've had the CPU do enough things for the maths result to definitely be ready.

I have thought of doing work in a different thread, to simulate the concurrent processing going on. There are other ways. At least, in my version I do set the MathInProgress when math calculations are starting (and clearing when done, instantly atm).

I obviously don't know much the structure of your project but if there's any sort of global notion of time and you can do processing on each port read, couldn't you just check the time on read and see if it's been sufficiently much time that the result should be ready?

LX.NET, on Sat Nov 26, 2011 2:41 PM, said:

Do you have any knowledge of the other sprite rendering bug that seems to occur in the cube demo: the bad header in red pixels?

Not offhand but I've surprised myself by being able to remember my diagnosis (which I highly recommend you treat critically by the way, memory being what it is) of the double-width scan line bug at least nine years after the fact so I'll have a poke around.

[LX.NET]

LX.NET, on Sat Nov 26, 2011 2:41 PM, said:

Thanks ThomH, this is really helpful. I will have a look at this particular bug mid December (bogged down with preparing for presentations before that time).

I guess I lied and took a short look the same evening.

ThomH, on Sun Nov 27, 2011 6:25 AM, said:

Further on this, the BLL bug that is nullified by the Lynx hardware bug that Handy doesn't reproduce is recorded on line 330 of BLL's fpolygon.inc:

.triimage	   dc.b 2,$11,0

Funny, I was debugging and tried the exact same thing. It did render the polygons correctly after that.

ThomH, on Sun Nov 27, 2011 6:25 AM, said:

That's unpacked data, so if the Lynx hardware were perfect it'd be two pixels and Handy renders it as such. On the real hardware it's only one pixel. Switching the $11 constant to $10 fixes that code in Handy.

I guess that, based on the description in the hardware guide that the problem is just the final bit, switching the $11 to $22 should produce double-width scan lines on both Handy and a real Lynx. I don't currently have any tools set up and things are a bit complicated because I'm about to emigrate but I do have one of Lynxman's flash carts, so if you're unable to test that and think it would be helpful then I can probably do that but not necessarily very soon.

I'm not sure what it is exactly. My guess is that if the last pixel ends exactly at the end of the last bit in the bit shifter, it will not output the bit. In my code I have patch this bug (or rather enabled this hardware bug) with this line:

[color=blue]if[/color] (register.BitsLeft == 0) [color=blue]yield[/color] [color=blue]break[/color];

As it turned out, the too-correct emulation was also causing the Joust lava bug. Pictures of the Wuerfel demo and the Joust screen with fixed code included in next post.

ThomH, on Sun Nov 27, 2011 6:25 AM, said:

I obviously don't know much the structure of your project but if there's any sort of global notion of time and you can do processing on each port read, couldn't you just check the time on read and see if it's been sufficiently much time that the result should be ready?

My general notion of time is (just like in Handy) the cycle count of the 65SC02 cpu. So, another way of implementing the non-instantaneous math behavior would be to set up some flag for the cycle count at the end of the math calculation and then flip the MathInProgress bit back off.

ThomH, on Sun Nov 27, 2011 6:25 AM, said:

Not offhand but I've surprised myself by being able to remember my diagnosis (which I highly recommend you treat critically by the way, memory being what it is) of the double-width scan line bug at least nine years after the fact so I'll have a poke around.

Keep them coming. Everything was perfectly true so far. Thanks again.

Next will be the fixing of the Handy source code, which might be at the initial time mentioned earlier. Shouldn't be too hard. After that, we will need to do some regression testing to find out whether the fix did not break things.

[LX.NET]

Fixed imagery for Joust and Wuerfel demo that show the effect of not returning the last pixel in the bit shifter when the last bit is the used in the last pixel.

[LX.NET]

Just a quick update on the progress on fixing the bugs in the Handy emulator codebase:

• Everon detection:
  Thanks to sage for providing me with details on the EVERON detection. I found and fixed the bug. It turned out that the Handy implementation initialized the boolean that detects the Sprite Control Block everon screen state only once per chain of SCBs (as in SCB linked together by SCBNEXT value). The effect is that the first SCB in a chain is on-screen, all linked SCBs after that are also regarded as on-screen. Moving the initialization of the everonscreen variable inside the do-while loop fixed it. The Pong1K demo by sage now works without a workaround (hardware version).
  
  Required code change in Susie.cpp around line 487
  
  cycles_used=0;
  do
  {
  // Moved inside of do while loop
  everonscreen=0;
  
• Hardware bug emulation of SCB data ending exactly at last bit of shiftregister
  Again thanks to sage for reporting and to ThomH for the exact reason.
  The previous posts show some of the details. Here the problem with the hardware is that when the last bits in the shift register accidently are also the last bits of the SCB data for a pixel, that pixel is not rendered. The workaround for Lynx developers would be to pad with an additional byte of data. I gues this is pretty hard for literal encoded SCB data, as this would result in additional pixels being rendered.
  The fix for the Handy code involves checking that the number of bits left in the shift register is not 0 after having read the last pixel data bits. If this turns out to be true, the code for end of line (0x0) is returned instead of the bits retrieved for the shifter.
  Code change to Handy source (including an additional = (equals sign) as part of the comparison) in line 1474 of Susie.cpp:
  
  
  // Only return data IF there is enought bits left in the packet
  if(mLinePacketBitsLeft<=bits) return 0;

@sage: if you have more details on the math signed divide bug, please let me know. I'm ready to fix more bugs.
@everyoneelse: will take a look at the other reported bugs as well, but sound related issues are somewhat lower on the list (i.e. when I get around to properly implementing in my own emulator).

Edited by LX.NET, Fri Dec 23, 2011 2:08 PM.