[opcode]

While designing the Opgrade Module IDE interface, I started to wonder if the interface was fast enough. So I tried to think about what would be the most demanding application possible in terms of throughput, and the answer was full motion video. I made some calculations, the maximum possible throughput for the CV VDP is around 140KB/s (though 120KB/s is a more realistic number). Well, the OM IDE can do that, so I got my answer (the OM IDE can get near 160KB/s).
However during the process I thought of Dragon's Lair, a game that is basically an interactive video. So I started to wonder if Dragon's Lair would be done with the OM. Well, here is what I found out:

- Dragon's Lair isn't really full motion in the sense it uses 24 frames/s or more. The actual footage is around 12 frames/s. From the CV VDP max throughput, and I am going to use 120KB/s here, and considering that a full “graphics 2” screen requires 12KB, we can say that the CV VDP can produce 10 frames/s of full screen video in the best case scenario.
- There are many sources of Dragon's Lair footage available, however the best one is the recent Blu-ray release. Previous sources had very faded colors, while the new source is very vibrant and has better contrast. Also, aspect ratio for the new transfer is 1.78:1. That would mean a video resolution of 256x144 when using the CV video. Now, using that resolution would reduce throughput to 108KB/s for a 12frames/s video. Looking good...
- So we still have around 40~50KB/s left. Of course image with no sound would be boring, so we also need to add sound in the math. From the MSX scene I know that 15kHz sound is possible with this kind of video using IDE.
- And of course some CPU time is required to control game logic, which in Dragon's Lair's case is very simple.

So from a theoretical point-of-view everything looks fine. But a very important question remains: how about audio and video quality?

- I can tell that 15kHz sampling is pretty descent.
- Video is the big question, so here are some samples I produced... While some would argue that quality is pretty bad, from what I saw from the MSX video player, quality gets a lot better once the video gets into full motion. The motion masks a lot of the color clashing. But of course a demo needs to be produced in order to prove that.

Considering that the whole game has 19 minutes of footage, I believe that around 165MB would be required for audio and video. Not a problem considering that we would be using CF.
Here is an interesting project to be considered in the future...

Attached thumbnail(s)

• 
• 
• 

[Pixelboy]

Those shots look pretty good! I would have expected worse from the good ol' ColecoVision. However, how would one go about converting all that video footage into screen mode 2 graphics? Because of CPU versus VRAM performance issues, I would expect that the video and sound couldn't be compressed to save space...

I know there are encoders for still pictures, but I can't imagine anyone going through the trouble of encoding every animation frame of Dragon's Lair one by one. If there was a proper tool out there, with some training I could do the video conversion myself and pass the material along to you. Then all you'd need to do is convert the sound/voice samples, and program the video player and game logic that binds everything together.

Of course, we'd need a functional prototype of the Opgrade Module to make all this work. But then how would the video streaming work? Would there be a way to send data from the CF directly to VRAM very rapidly? Otherwise, you'd need to upload 500K chunks to the MegaRAM component, and then go from MegaRAM to VRAM. With a throughput of 108K per second, 500K won't last very long...

Interesting project indeed, but only with a good video re-encoder. It would make the ADAM version look like a totally obsolete dinosaur!

EDIT: Could you post PNG versions of those comparison pics? With JPEGs, there's too much pixel blurring to really appreciate the conversion.

This post has been edited by Pixelboy: Sat Dec 6, 2008 12:44 PM

[opcode]

Pixelboy, on Sat Dec 6, 2008 1:28 PM, said:

Those shots look pretty good! I would have expected worse from the good ol' ColecoVision. However, how would one go about converting all that video footage into screen mode 2 graphics? Because of CPU versus VRAM performance issues, I would expect that the video and sound couldn't be compressed to save space...

No, we don't have CPU time for compression. It needs to be all raw data, audio and video.

Quote

I know there are encoders for still pictures, but I can't imagine anyone going through the trouble of encoding every animation frame of Dragon's Lair one by one. If there was a proper tool out there, with some training I could do the video conversion myself and pass the material along to you. Then all you'd need to do is convert the sound/voice samples, and program the video player and game logic that binds everything together.

It seems like it is possible to create scripts to automatize the process of encoding stills. The problem is how to get a good source. I know that the Blu-ray is the best version out there, but Blu-ray cannot be directly accessed yet. So one would need to capture the Blu-ray film frame by frame, or maybe capture it in very slow motion.

Quote

Of course, we'd need a functional prototype of the Opgrade Module to make all this work. But then how would the video streaming work? Would there be a way to send data from the CF directly to VRAM very rapidly? Otherwise, you'd need to upload 500K chunks to the MegaRAM component, and then go from MegaRAM to VRAM. With a throughput of 108K per second, 500K won't last very long...

We can send data to the VRAM directly from the CF, the question is if we can use a filesystem or not. Probably not. So maybe this game would need its own CF.

Quote

Interesting project indeed, but only with a good video re-encoder. It would make the ADAM version look like a totally obsolete dinosaur!

EDIT: Could you post PNG versions of those comparison pics? With JPEGs, there's too much pixel blurring to really appreciate the conversion.

Sure.

Attached thumbnail(s)

• 
• 
• 

[Pixelboy]

opcode, on Sat Dec 6, 2008 2:26 PM, said:

No, we don't have CPU time for compression. It needs to be all raw data, audio and video.

I expected as much.

Quote

It seems like it is possible to create scripts to automatize the process of encoding stills. The problem is how to get a good source. I know that the Blu-ray is the best version out there, but Blu-ray cannot be directly accessed yet. So one would need to capture the Blu-ray film frame by frame, or maybe capture it in very slow motion.

I have no idea how to do that, but perhaps someone else out there with some knowledge and equipment could get us that huge lot of stills.

Quote

We can send data to the VRAM directly from the CF, the question is if we can use a filesystem or not. Probably not. So maybe this game would need its own CF.

So we'd need to program a CF cart somewhat like a CV cart? That sounds cool! Would there be special routines to include in the OM BIOS to facilitate this setup?

Quote

Quote

EDIT: Could you post PNG versions of those comparison pics? With JPEGs, there's too much pixel blurring to really appreciate the conversion.

Sure.

Looks very good, considering the palette limitations. Thanks for the PNGs.

[nonner242]

looks good! not really that much worse than say the Sega CD version...lol

but....how about makeing it look better...maybe run smoother in....B&W!

All the older FMV games did it. and they Always looked better than the color stuff on the limited systems.

[Murph74]

Another thing that can make that video clean up quite a bit would be to eliminate the 'dithering'. From my experience, the dithering gives it that speckled look.

Here's a pair of screen caps using a still from DL that I found accompanying a blue ray article. This is by no means scientific, just an example. The image was dropped to an optimised 16 color pallete, then screen-shotted at fullscreen dithered and undithered (1024x768 res) then reduced 50% for posting here.

But I hope you can see what I'm referring to on the speckled look...

As usual Eduardo, you've blown my mind just thinking about this!

Attached thumbnail(s)

• 

[opcode]

Murph74, on Sat Dec 6, 2008 4:01 PM, said:

Another thing that can make that video clean up quite a bit would be to eliminate the 'dithering'. From my experience, the dithering gives it that speckled look.

Here's a pair of screen caps using a still from DL that I found accompanying a blue ray article. This is by no means scientific, just an example. The image was dropped to an optimised 16 color pallete, then screen-shotted at fullscreen dithered and undithered (1024x768 res) then reduced 50% for posting here.

But I hope you can see what I'm referring to on the speckled look...

As usual Eduardo, you've blown my mind just thinking about this!

Nothing like a couple of heads thinking together...
The problem with your example is that you are using 16 custom colors, and you didn't take color clash into consideration.
However you were right about too much dithering, so I created a few alternate versions...

Below are the same 3 samples as before. However now in addition to the original still and original conversion (top two images), I added (from top to bottom, left to right):

- No dithering at all, no color diffusion (or whatever they call it): image is dark, colors are too wrong.
- Dithering but no color diffusion: a little bit better color, but still too dark.
- Dithering and color diffusion (0.20, original conversion was using 0.43): much better colors, contrast is almost spot on.
- Dithering and color diffusion (0.22): slightly better contrast, color is almost the same as cf 0.20.

While the original conversions had better colors (compared to the original), dithering is much better in the bottommost right conversion, and contrast is better too.
There are still a few different dithering patterns that I want to try but Murph74 was right, images look a lot better now with less dithering.

Attached thumbnail(s)

• 
• 
• 

[Pixelboy]

opcode, on Sat Dec 6, 2008 5:33 PM, said:

Below are the same 3 samples as before. However now in addition to the original still and original conversion (top two images), I added (from top to bottom, left to right):

- No dithering at all, no color diffusion (or whatever they call it): image is dark, colors are too wrong.
- Dithering but no color diffusion: a little bit better color, but still too dark.
- Dithering and color diffusion (0.20, original conversion was using 0.43): much better colors, contrast is almost spot on.
- Dithering and color diffusion (0.22): slightly better contrast, color is almost the same as cf 0.20.

While the original conversions had better colors (compared to the original), dithering is much better in the bottommost right conversion, and contrast is better too.
There are still a few different dithering patterns that I want to try but Murph74 was right, images look a lot better now with less dithering.

Hmm... I dunno... I'm all for less dithering, but there are visual side-effects do doing this kind of adjustment. For example, in sample_6.png, the colors might look better, but those green lines in the top-center half of the screen create a sort of vertical dither that completely destroys the backgrounds behind Dirk.

On the other hand, if you look at the middle set of stills in sample_4.png, the walls might be incorrectly green, but there's something to be said for the nice texture of the wall, which the vertical dithering (again) completely destroys in the bottom pics.

Wouldn't it be possible to preserve the textures of backgrounds, even if it means a slight reduction of colors, while still trying to get the best colors for Dirk and other objects in the foreground? I think that would look the best, IMHO.

[Murph74]

Eh, truth is the dithering will be much less noticable in motion... some good info to read up on (which I did AFTER I posted earlier! lol) is at Wikipedia...

http://en.wikipedia.org/wiki/Dithering

Hard to make a judgment after reading that until I see it in motion. But still like where this is going.

[PressureCooker2600]

Makes me think my Colecovision would explode after loading that game.

But, besides that, the idea is awesome and would be awesome to see running.

[joyrex]

I think the MSX Dragon's Lair had the same approach as you probably have in mind; converting every frame with its own tileset. Looking at the screenshots, it looks like you also changed the palette (v99x8?).

Another interesting approach is done by Arturo Ragozini. He created a video encoder that does a search for the optimum tileset for a couple of frames, rather than an optimum tileset for a single frame of video. You don't really require that if you take the flashcard as a medium, but it can provide you more cpu time during playback (and dedicate that to eg. sound).

Interesting reads about his approach;

http://www.msx.org/forumtopic7309.html
http://ragozini.googlepages.com/vdpenc

[opcode]

joyrex, on Sun Dec 7, 2008 6:06 PM, said:

I think the MSX Dragon's Lair had the same approach as you probably have in mind; converting every frame with its own tileset. Looking at the screenshots, it looks like you also changed the palette (v99x8?).

Another interesting approach is done by Arturo Ragozini. He created a video encoder that does a search for the optimum tileset for a couple of frames, rather than an optimum tileset for a single frame of video. You don't really require that if you take the flashcard as a medium, but it can provide you more cpu time during playback (and dedicate that to eg. sound).

Interesting reads about his approach;

http://www.msx.org/forumtopic7309.html
http://ragozini.googlepages.com/vdpenc

Thanks a lot, joyrex. I am going to check that. In the meantime, I created this animated gif, just to see how the final video would look. It's just 5s, but probably enough to show the quality of the video (which isn't that great, btw). I suggest to watch it full screen, from some distance. Perhaps I should try to increase error diffusion, from .25 to .43... More dithering but better colors...

Attached thumbnail(s)

• 

[Murph74]

Very nice again Eduardo! I gotta think just based on squinting a little bit, a SD TV will be more forgiving than my PC monitor too.

[nonner242]

Ok, how bout this.....but, Some work involved here...
I used this process back in the day on my Amiga for FMV.
Make a static background.
Only have the parts that are acutually animated move.
This will take alot of work on something like this, unless you can find some ani cells online or on DVD.
Or you can redraw the BG as stills and insert the animation over it.
Or go with posterize instead of dither....
I still think B&W will work best...you could even add color to dirk....just red/yellow. That will leave you with 14 shades of grey, which is more than enough. to make the video look smoother.

[opcode]

joyrex, on Sun Dec 7, 2008 6:06 PM, said:

I think the MSX Dragon's Lair had the same approach as you probably have in mind; converting every frame with its own tileset. Looking at the screenshots, it looks like you also changed the palette (v99x8?).

Actually I used BMP2MSX, which probably doesn't have the best rendition of the TMS9918 palette.

Quote

Another interesting approach is done by Arturo Ragozini. He created a video encoder that does a search for the optimum tileset for a couple of frames, rather than an optimum tileset for a single frame of video. You don't really require that if you take the flashcard as a medium, but it can provide you more cpu time during playback (and dedicate that to eg. sound).

I checked Arturo page and found a piece where he says that his encoder isn't the best for Dragon's Lair. There is a small Dragon's Lair video there, and it looks worst than mine.
What I was thinking is that if I transfer 2K every NMI to VRAM, then I could produce the video as described above. Considering the aspect ratio, the video requires 9KB per frame, 12 frames/s. That is feasible, I think. Also, sound would be produced at 15kHz because the CPU spend a lot of cycles waiting for the VDP. 15kHz sound would require a sound byte every 64 video bytes.
The remaing CPU time is for joystick reading and game control.

[opcode]

nonner242, on Sun Dec 7, 2008 9:41 PM, said:

Ok, how bout this.....but, Some work involved here...
I used this process back in the day on my Amiga for FMV.
Make a static background.
Only have the parts that are acutually animated move.
This will take alot of work on something like this, unless you can find some ani cells online or on DVD.

The problem is the CV graphic limitations. The CV can produce only two colors per pixel octet. So even if the background is static, animated parts will interfere with the FG graphics, creating color clashes.

Quote

Or you can redraw the BG as stills and insert the animation over it.
Or go with posterize instead of dither....
I still think B&W will work best...you could even add color to dirk....just red/yellow. That will leave you with 14 shades of grey, which is more than enough. to make the video look smoother.

14 shades of gray? We just have 1 shade of gray, the CV color palette is fixed...

[Pixelboy]

opcode, on Sun Dec 7, 2008 10:32 PM, said:

14 shades of gray? We just have 1 shade of gray, the CV color palette is fixed...

Indeed, but this makes me wonder if the video would look smoother with color shades, such as:

- Black, dark blue, medium blue, light blue, white
- Black, dark red, medium red, light red, dark yellow
- Black, dark green, medium green, light green, grey

That's probably the best black-&-white style one could do with the CV. Major loss of color, but perhaps there wouldn't be as much color-clash artifacting as the one seen in your animated GIF (which looks really cool BTW).

So the process would be to turn each frame to black-and-white, then alter the image to replace shades of grey with color shades (red, green or blue, whatever suits the current level best), then convert the image to the CV.

This post has been edited by Pixelboy: Mon Dec 8, 2008 5:38 AM

[Pixelboy]

(continued from previous post)

But then again, the trade-off would be less distinct animated objects. For example, in the part where Dirk picks up the sword from the crystal, if shades of red were used to reproduce this sequence, the dragon would appear red just like the rest of the picture, and with the two-color-per-pixel-octet limitation, that might look pretty bad...

So I guess what I'm saying is that I still believe shapes and textures should be priviledged over precise color reproduction. I mean, look at the ugly dithering on the stone wall behind Dirk when Eduardo's GIF animation begins. There's gotta be a way to improve that...

This post has been edited by Pixelboy: Mon Dec 8, 2008 5:53 AM

[nonner242]

Sorry, Im not trying to be a CV expert, I'm just dropping a few tricks that I used for other platforms I dabbled in years ago.
Very interesting post to me, I love this stuff, makes me wanna get back into the game field!
Just throwing some stuff out there to see if it might get somebodies gears rolling!
I would love to see what ppl can do with older systems, its truly amazing sometimes.
I love the Dragons Lair/Space Ace games too! But my ultimate would be TRON!
I will die a happy man if I ever get to see a version of Tron come out for Atari (or any older system) that is close to the Arcade.
I did a mock up here. Posted it as a 2600 game but really should of had it as a 7800.
http://www.atariage....mp;hl=tron+2600
One day I hope!
Good luck with this one guys!

[Pixelboy]

nonner242, on Mon Dec 8, 2008 9:45 AM, said:

I love the Dragons Lair/Space Ace games too! But my ultimate would be TRON!
I will die a happy man if I ever get to see a version of Tron come out for Atari (or any older system) that is close to the Arcade.
I did a mock up here. Posted it as a 2600 game but really should of had it as a 7800.
http://www.atariage....mp;hl=tron+2600
One day I hope!
Good luck with this one guys!

Eduardo is also a huge Tron fan, and the only thing that's keeping him from doing an adaptation of the arcade game on the ColecoVision is the bulk of other CV projects that have a higher priority.

[opcode]

Pixelboy, on Mon Dec 8, 2008 6:52 AM, said:

(continued from previous post)

But then again, the trade-off would be less distinct animated objects. For example, in the part where Dirk picks up the sword from the crystal, if shades of red were used to reproduce this sequence, the dragon would appear red just like the rest of the picture, and with the two-color-per-pixel-octet limitation, that might look pretty bad...

So I guess what I'm saying is that I still believe shapes and textures should be priviledged over precise color reproduction. I mean, look at the ugly dithering on the stone wall behind Dirk when Eduardo's GIF animation begins. There's gotta be a way to improve that...

Ok, I see what you are trying to mean. I tried many, many combinations here, monochromatic, shades of the same color, etc, none worked. However I noticed something about the gif you mentioned, with the stone wall. I noticed that it uses a lot of green when in fact there is no real green in the original image. I know that probably the encoder is trying to compensate for something, create the illusion of some color that isn't available in the CV palette...
Anyway, I removed all greens, reduced error littering a little and voila, better definition (but colors are bad).
Check the new animated gif...
The problem, how to create an automatic process for that? You cannot simply remove green from all images, in some cases there are real green in the original…

Attached thumbnail(s)

• 

[monzamess]

What would that look like on a composite screen? A lot of that dithering will blur together--for better or for worse?

[ScottHuggins]

Man, Eduardo... when you come back to the "scene" you come back hard. Sheez... take it easy man!

[opcode]

monzamess, on Mon Dec 8, 2008 2:49 PM, said:

What would that look like on a composite screen? A lot of that dithering will blur together--for better or for worse?

Actually RF screen...
Good question. I would also say that the colors shown in the screenshots aren't the correct CV colors. The real CV palette is more muted than that.
Here is another test:

test1 is the original image
test2 is the original CV encoded image (same as the first animated gif posted here)
test3 has all greens removed. Definition is much better, but the dragon is red...
test4 is a mix of test2 and test3. Excellent definition (for the CV), and ok colors
test5 is the same as test4 but with some blur applied, trying to simulate the CV RF. I really like this final image. If just the whole movie looked that good...

Attached File(s)

•  test4.zip (179.43K)
  Number of downloads: 43

[Pixelboy]

opcode, on Mon Dec 8, 2008 2:39 PM, said:

Ok, I see what you are trying to mean. I tried many, many combinations here, monochromatic, shades of the same color, etc, none worked.

Okay, thanks for trying.

Quote

However I noticed something about the gif you mentioned, with the stone wall. I noticed that it uses a lot of green when in fact there is no real green in the original image. I know that probably the encoder is trying to compensate for something, create the illusion of some color that isn't available in the CV palette...
Anyway, I removed all greens, reduced error littering a little and voila, better definition (but colors are bad).
Check the new animated gif...

Aaaaaaah! Much, much better! The lack of green littering does improve the picture a lot, and I prefer it that way, even though there's some color loss on Dirk.

Quote

The problem, how to create an automatic process for that? You cannot simply remove green from all images, in some cases there are real green in the original…

Well, if this is the kind of result we would be aiming for, we'd need to find just the right balance of color adjustments, so that the encoder does the best job possible without human intervention. Not easy to do, but it would be worth the effort, I think. I believe gimp can be programmed with scripts to automate this kind of adjustment on multiple files.