My suggestion would have been a 1200XL, the best, bar none, if you are going to upgrade anyway. But you say it has cartridge problems. I haven't used many cartridges since I've owned a 1200XL, what cartridges don't fit? After looking closer I see that the cartridge slots on my 600/800XL's are about an 1/8 of an inch wider, never noticed before...the 1200XL has such a large accessable port, I didn't realize it was a bit narrower up in there. That part of the case, which is a seperate piece from the rest, could be easily modified.
As for OS's, the only real incompatibilities are between the 4/800's and the XL/XE's. So for full compatibility, OS wise across all Atari 8-bits you need a minimum of 2 OS's; the 4/800 and any XL/XE OS after the 1200XL (which is on two chips and wouldn't work anyway). Or, you can just use a translator disk, except for those few 800 cartridges that won't work.
If you have the money, the easiest thing to do is get an Ape 32-in-1 OS, or find one of the older OS expansion boards from back in the day. The only other option is to build your own from scratch which requires an eprom burner too, unless you know someone who can burn the OS's for you.
As for memory, you might as well jump on the Ultimat 1MB bandwagon that's happening right now, though there were and are other user or dealer options too. I personally recommend the original Rambo XL upgrade, which can be further upgraded to 512K, if you can't find the Rambo, AtariMax sells an unadvertized Rambo compatible upgrade, also upgradable to 512K, And of course building your own Claus Bucholz or other more modern memory mod.
If you want upgraded video, on any Atari XL/XE, I recommend the SV 2.1, at least as a starting point, some have made modifications to it that have worked best for their monitor. There are no ready-made installable video upgrades except the RGB one included in the VBXE video board.
If the 1200XL didn't exist, I would recommend the 800XL or an 800 with the user-made XE board in it.
I've heard mention here that MetalGuy66 might do contract work for repairs and upgrades, but not from him personally. There are others. I do all these upgrades myself, and repairs, but I must have the ability to do them at my liesure, so I only do it with machines I aquire on my own. I am currenlty in possesion of a 600XL and 800XL that are my current repair and upgrade projects which will be sold when done...days...weeks...months...if you want to send a computer with the knowledge it might take up to a year and be suprised to recieve it back sooner, I'll consider it.
But I have a question for you; what about your current 800XL is "dying?" It may be that whatever is going bad can be repaired easily, or may even repair itself in the course of upgrading it.
Edited by Gunstar, Mon Feb 6, 2012 12:12 PM.