ST-412/ST-506/ESDI hard drive emulation?

Discussion of development and patch submission.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Sat 04 Mar, 2017 8:49 am

I must say that I've been thrilled at how far PCem has come in just a couple of years. For the early-era enthusiasts like me, it's especially gratifying to have a near-perfect emulation solution for emulating almost all of the major basic PC hardware elements from throughout the 1980s. The single most notable exception now is in the hard drive arena, with no support at all for anything pre-IDE.

Now, I can see why some would be skeptical as to the need for this, given that XTIDE does more or less get the job done. I would argue that there are several good reasons to get the major pre-IDE technologies into PCem, however. First and foremost, I have yet to find a version of XTIDE that works perfectly at all times. Back when I used it with actual, physical XT clones, it was always a given that sooner or later, one of XTIDE's many bugs would result in data corruption. That has indeed continued to happen as I run emulated systems in PCem. It's not model-dependent or configuration-dependent, as these are issues with XTIDE itself. The issues I run into are rare and totally unpredictable/non-reproducible, so unfortunately I can't say more than that, but it is an issue and I have lost data during PCem sessions. It's a real problem for those of us who spend long hours working on stuff using the '80s-era models in PCem. And aside from all that, there's also the "historical realism" angle, of course, and the sheer satisfaction of reaching as high a level of accuracy and completeness as possible.

Ideally, I think PCem's early-era disk support should eventually be rounded out with ST-412, ST-506, and ESDI emulation, at minimum. The ST-506 might be the best place to start, given that MESS has already implemented it quite nicely; I don't believe the licenses allow us to actually take or port the code, but it's at least available to be consulted, which should make the process of re-developing it for PCem relatively easy. I've had a look at their implementation, and it doesn't seem like it'd be much of a headache at all. I'm not sure how different the ST-412 and ESDI specs are, but I wouldn't think they'd be all that much more difficult to get up and running.

I'd like to know what everyone thinks about this--whether there's much interest, whether anyone can see themselves taking on any of the work, etc. Of course, most of all, I'd like to hear from Sarah and any others who are more intimately familiar with the PCem codebase than I am. Due to other commitments, I won't have the ability to take the lead on implementing this anytime soon, but if there's anyone here who can, I think it's something that should be considered. If there are any particular hazards or pitfalls that one might anticipate in trying to add this to PCem, I'd be interested to hear about them. Thanks, everyone. :)
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sat 04 Mar, 2017 9:51 am

ST-412 and ST-506 are the same interface, often referred to as just 'MFM'.

In principle basic MFM support wouldn't be too difficult to implement. You would have to decide which controller to implement though - unlike with IDE there's no 'standard' controller, and the various MFM boards are basically incompatible from an emulation point of view. The main issue though is trying to integrate all of this neatly into the existing emulator, given that hard drive support is heavily tied to IDE emulation at present. Either it's hacked so that MFM hard drives are completely separate from IDE hard drives, or the hard drive emulation needs to be 'genericised' so that it can serve either, which is quite a bit of work.

I haven't really looked into ESDI too much, but it's going to have most of the same issues I think.
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sat 04 Mar, 2017 10:06 am

Actually, I'm exaggerating here a bit I think - the interfacing work shouldn't be _too_ difficult, assuming you only want one controller emulated at a time. Still need to decide what controller(s) to emulate though.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Sat 04 Mar, 2017 7:39 pm

Interesting--I hadn't realized that ST-412 and ST-506 were both totally reducible to just "generic MFM" at the controller level. One controller emulated at a time seems like a perfectly reasonable limitation, of course (assuming you mean "one controller running at any given time", and not "one controller available to PCem's codebase and selectable by models for any given compile" here).

As for which controllers to emulate, there are a few that come to mind that would be nice to have for either symbolic or practical reasons. For example, any controller that came standard with one of the existing models already in PCem would contribute directly to the faithful emulation of that model, which is an obvious plus. As such, the Xebec 1210 controller in the IBM XT might be a good place to start, especially given how extensively documented it must be. Another consideration in the controller selection would be ease and practicality of implementation. I'm no expert here, but I'd think that some controllers might be dramatically more complicated to emulate than others, even if functionally similar in the end, and it might make sense to prioritize the ones that are easier to get working and less likely to be bug-prone. A bit more research is probably needed to determine which ones those might be, however.
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sun 05 Mar, 2017 9:28 am

From what I can see, the Xebec controller is a bit limited in what drives are supported - the first two variants only support 10 MB drives, the later (AT) variant also supports 20. Probably a more flexible controller would be the best one to start with I think.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Sun 05 Mar, 2017 10:11 am

Good point. Maximizing the flexibility and range of supported drives is the obvious thing to do at the outset. Presumably that means looking at controllers from the last major wave of MFM cards (1986-87 or so, IIRC), yes? Some quick research suggests that there is fairly broad agreement that the Western Digitals are the most versatile. This looks like a good overview of the major WD offerings from that period, also covering RLL and ESDI: http://www.minuszerodegrees.net/manuals/Western%20Digital/wdcontr.txt

Haven't gone through it in much detail yet, but it seems like a good starting point.
Zup
Posts: 40
Joined: Mon 30 Nov, 2015 10:47 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby Zup » Sun 05 Mar, 2017 10:31 am

SarahWalker wrote:From what I can see, the Xebec controller is a bit limited in what drives are supported - the first two variants only support 10 MB drives, the later (AT) variant also supports 20. Probably a more flexible controller would be the best one to start with I think.



Keep on mind that some computers (i.e.: Compaq Deskpro) have their own HDD BIOS included on the "main" BIOS. I don't really know if ports and protocols are different between implementations, but if they are different the ports in the emulated HDD controllers will have to match those used on the on-board HDD BIOS. If the ports are equal, you should have to emulate an HDD controller that allows the connection of the HDDs used (shipped or offered as options) in every XT computer. There is a list of those disks somewhere on the forums, they vary between 10 and 40 Mb (except ESDI disks that can be bigger). Or at least, that HDD controller should allow the connection of disk types defined on IBM AT BIOS.

If they don't match, the MFM solution is as good as the actual XT-IDE (= is not the onboard controller but an emulated additional ISA card).
User avatar
gen_angry
Posts: 48
Joined: Sat 25 Feb, 2017 6:48 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby gen_angry » Sun 05 Mar, 2017 12:25 pm

Following this a bit since I am interested in making an AT machine at some point.

Zup wrote:There is a list of those disks somewhere on the forums, they vary between 10 and 40 Mb (except ESDI disks that can be bigger). Or at least, that HDD controller should allow the connection of disk types defined on IBM AT BIOS.


I had saved a few lists locally at some point, any of these help?:

http://pastebin.com/8J52V72W

http://pastebin.com/v552Vmeb
JohnElliott
Posts: 46
Joined: Sun 31 Jan, 2016 7:29 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby JohnElliott » Mon 06 Mar, 2017 11:03 am

For flexibility, perhaps SCSI is the way to go -- the Future Domain TMC-850 was quite a common 8-bit SCSI card. The 8.2 BIOS is the best one to use on an XT as later versions don't hook INT 19h and rely on the motherboard BIOS to know about booting from hard drives.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Mon 06 Mar, 2017 9:48 pm

SCSI support would be nice to have, but wouldn't really serve as a substitute for proper MFM emulation. There's a fair amount of early '80s productivity software that, for whatever reason, doesn't play well with SCSI.
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Mon 06 Mar, 2017 10:23 pm

I've done some digging, and I think probably the best board to start with is the IBM AT fixed disk adapter (which I think is basically a Western Digital board?). It's essentially a subset of IDE - or more correctly, IDE is a superset of it. It also seems to be widely supported by AT models, some of which I think are incorrectly marked in PCem currently as IDE.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Mon 06 Mar, 2017 11:08 pm

Was just looking into that myself--yep, it was developed by Western Digital and was indeed later used as the foundation for IDE. Seems like an excellent choice.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Mon 06 Mar, 2017 11:20 pm

Seems there were two versions or "generations" of the card, both by WD. The second version was also used in the XT 286. Lots of good info here: http://minuszerodegrees.net/5170/cards/5170_cards.htm#fddda_1. Full documentation from IBM for the first-gen version here: http://minuszerodegrees.net/oa/OA%20-%20IBM%20PC%20AT%20Fixed%20Disk%20and%20Diskette%20Drive%20Adapter.pdf and second-gen version here: http://minuszerodegrees.net/oa/OA%20-%20IBM%20PC%20AT%20Fixed%20Disk%20and%20Diskette%20Drive%20Adapter%20-%202.pdf. Worth noting, sources that say this is the same board as the WD1002-WA2 are wrong.
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Tue 14 Mar, 2017 9:33 pm

Rev 656 adds support for the AT Fixed Disk Adapter.
User avatar
omarsis81
Posts: 544
Joined: Thu 17 Dec, 2015 6:20 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby omarsis81 » Wed 15 Mar, 2017 12:51 pm

I'm having trouble running the latest build:

pclog.txt

Code: Select all

Arg 0 - C:\PCem12\PCem.exe
Raw input registered!
Preparing scan code map list...
Done preparing!
executable_name = C:\PCem12\PCem.exe
path = C:\PCem12\
Keyboard initialized!
joystick_init: joysticks_present=0
Config data :
[]
gameblaster = 0
gus = 0
ssi2001 = 0
voodoo = 0
model = 430vx
cpu_manufacturer = 0
cpu = 0
cpu_use_dynarec = 1
cpu_waitstates = 0
gfxcard = px_trio64
video_speed = 3
sndcard = 0
cpu_speed = 9
has_fpu = 1
disc_a =
disc_b =
mem_size = 65536
cdrom_drive = 0
cdrom_enabled = 0
cdrom_channel = 2
cdrom_path =
vid_resize = 0
vid_api = 0
video_fullscreen_scale = 0
video_fullscreen_first = 1
hdc_sectors = 0
hdc_heads = 0
hdc_cylinders = 0
hdc_fn =
hdd_sectors = 0
hdd_heads = 0
hdd_cylinders = 0
hdd_fn =
hde_sectors = 0
hde_heads = 0
hde_cylinders = 0
hde_fn =
hdf_sectors = 0
hdf_heads = 0
hdf_cylinders = 0
hdf_fn =
drive_a_type = 0
drive_b_type = 0
window_w = 0
window_h = 0
window_x = 0
window_y = 0
window_remember = 0
joystick_type = 0
mouse_type = 0
enable_sync = 1
[Joysticks]
joystick_0_nr = 0
joystick_1_nr = 0
Config loaded
Starting with romset 36
timer_reset
Loading :0 
Loading :1 
Path is \\.\
DDRAW_INIT complete
Starting with romset 0
romset 0 - 1
Starting with romset 1
romset 1 - 1
Starting with romset 2
romset 2 - 1
Starting with romset 3
romset 3 - 1
Starting with romset 4
romset 4 - 1
Starting with romset 5
romset 5 - 1
Starting with romset 6
romset 6 - 1
Starting with romset 7
romset 7 - 1
Starting with romset 8
romset 8 - 1
Starting with romset 9
romset 9 - 1
Starting with romset 10
romset 10 - 1
Starting with romset 11
Loading BIOS
FF FF FF
romset 11 - 1
Starting with romset 12
romset 12 - 1
Starting with romset 13
Failed to load ROM!
romset 13 - 0
Starting with romset 14
Failed to load ROM!
romset 14 - 0
Starting with romset 15
Failed to load ROM!
romset 15 - 0
Starting with romset 16
Failed to load ROM!
romset 16 - 0
Starting with romset 17
Failed to load ROM!
romset 17 - 0
Starting with romset 18
Failed to load ROM!
romset 18 - 0
Starting with romset 19
Failed to load ROM!
romset 19 - 0
Starting with romset 20
romset 20 - 1
Starting with romset 21
Failed to load ROM!
romset 21 - 0
Starting with romset 22
romset 22 - 1
Starting with romset 23
romset 23 - 1
Starting with romset 24
romset 24 - 1
Starting with romset 25
Failed to load ROM!
romset 25 - 0
Starting with romset 26
romset 26 - 1
Starting with romset 27
Failed to load ROM!
romset 27 - 0
Starting with romset 28
romset 28 - 1
Starting with romset 29
romset 29 - 1
Starting with romset 30
romset 30 - 1
Starting with romset 31
romset 31 - 1
Starting with romset 32
romset 32 - 1
Starting with romset 33
romset 33 - 1
Starting with romset 34
romset 34 - 1
Starting with romset 35
Load SIS496 ea ff
romset 35 - 1
Starting with romset 36
romset 36 - 1
Starting with romset 37
Failed to load ROM!
romset 37 - 0
Starting with romset 38
Failed to load ROM!
romset 38 - 0
Starting with romset 39
romset 39 - 1
Starting with romset 40
Failed to load ROM!
romset 40 - 0
Starting with romset 41
Failed to load ROM!
romset 41 - 0
Starting with romset 42
Failed to load ROM!
romset 42 - 0
Starting with romset 43
Failed to load ROM!
romset 43 - 0
Starting with romset 44
Failed to load ROM!
romset 44 - 0
Starting with romset 36
Starting with romset 36
timer_reset
Initting as Award 430VX PCI
io_init
Video_init 36 19
Could not find hdd_controller


I tried deleting pcem.cfg, same.
It would seem is something ROM related, do I need to change the ROM folder or anything?
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Wed 15 Mar, 2017 6:24 pm

Should be fixed in rev 657. A bit of debugging code that I forgot to remove.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Wed 15 Mar, 2017 11:49 pm

Whoa, that was fast! Sarah, you are awesome. Looks like a pitch-perfect implementation from what I see so far. :)

BTW, did you base this on the first or second-gen version?
User avatar
omarsis81
Posts: 544
Joined: Thu 17 Dec, 2015 6:20 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby omarsis81 » Thu 16 Mar, 2017 3:32 pm

Excellent job Sarah! And very fast indeed. Only minor bug I found was that when I have another machine selected other than the IBM AT and I go to the Configure menu and select the IBM AT the HDD option stays grayed out. One must select the IBM AT, exit PCem, and relaunch PCem in order to change the HDD controller

ibm_at_hdd.png
ibm_at_hdd.png (37.77 KiB) Viewed 2617 times
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Thu 16 Mar, 2017 7:34 pm

Fixed in rev 658.

This is based on 2nd gen, though I couldn't see any obvious differences between 1st and 2nd gen from an emulator perspective.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Fri 17 Mar, 2017 5:28 am

One other question about the AT Fixed Disk Adapter: it doesn't seem to have any onboard ROM, does it? As such, I assume the standard AT ROM BIOS includes extensions that handle everything that an MFM card's own onboard ROM would typically handle, right? But if that's the case, how is it that the emulation of this board works with other systems, such as the XT clones? Is it just that the architecture is simple enough that a standard XT BIOS provides all of the necessary functionality, whereas other MFM cards with onboard BIOS are catering for more sophisticated capabilities that the AT Fixed Disk Adapter never offered?
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Fri 17 Mar, 2017 9:22 pm

It doesn't work with XT systems - it needs BIOS support (which seems to be in all AT models I tried), uses IRQ 14 and uses a 16-bit IO bus.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Fri 17 Mar, 2017 11:41 pm

Oh, hah--that's what I get for not proofreading! I meant AT clones of course, not XT clones. So this board does seem to be compatible with the various non-IBM-brand AT BIOSes, then? There was some doubt about that on some other forums. A couple of people seemed to think it had some dependency on IBM-specific AT BIOS features or data, but it wasn't entirely clear what the issue was, if any. Worth keeping in mind in any case, if there are problems down the road.

With respect to XT systems, we're still limited to XTIDE until an 8-bit MFM card is implemented, right? Luckily, it seems the AT Fixed Disk Adapter itself is a pretty straightforward 16-bit expansion of Western Digital's previous 8-bit cards, so it should be a simple enough job to port the code over at some point, I'd think. Some interesting info here: https://web.archive.org/web/20120331081552/http://oldibmpc.sitesled.com/Expansion%20cards/IBM%20Corp.html. Apparently the AT Fixed Disk Adapter is just a slightly customized WD1003 series controller. The WD1002 series would be the 8-bit equivalent. The WD1002S-WX2 model seems to have good documentation, available here: http://www.minuszerodegrees.net/manuals/Western%20Digital/WD1002S-WX2%20-%20OEM%20Manual.pdf. A dump of the onboard BIOS for that particular board is also available. Looking at the manual, at a glance it looks similar enough to allow for reasonably easy porting from the mfm_at code. A lot of the same registers, basic chipset logic and such, AFAICT, though I have yet to really dig into it. This could then serve as an "mfm_xt" device to provide MFM support to the remaining models--am I on the right track here?
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sat 18 Mar, 2017 12:02 pm

The AT BIOSes I tried all seemed to support it - it appears to be reasonably standard.

The AT Fixed Disk Adapter is _completely_ different from the Western Digital 8-bit cards, which are essentially compatible with the XT's Fixed Disk Adapter. I do have emulation of the latter in progress.
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sat 18 Mar, 2017 5:34 pm

Rev 659 emulates the XT Fixed Disk Adapter. This isn't as well tested as the AT Fixed Disk Adapter - there are quite a few holes in functionality (eg non-DMA transfers aren't supported) - but seems to work okay with IBM's BIOS.
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Sat 18 Mar, 2017 6:50 pm

Oh, awesome! This is the Xebec 1210 that came standard with the original XT, right? Hadn't realized you were working on it, thank you for this!
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Sun 19 Mar, 2017 10:12 pm

Rev 661 adds emulation of the DTC 5150X. This is probably a better controller to use for the XT, as it supports more than 4 drive types.

Configuring this card is a bit odd - to enter setup you have to run DEBUG.EXE, and type 'g=c800:5'. From there you can configure and format drives. I'd suggest when selecting drive type, to use type 15 - 'Free Format' - which allows you to just enter cylinder and head count.

I think that's probably enough hard drive controllers for the moment...
ecksemmess
Posts: 122
Joined: Wed 18 Mar, 2015 5:27 am

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby ecksemmess » Mon 20 Mar, 2017 11:06 am

Agreed. These are fantastic, and much appreciated. Whenever you (or any other contributors) do add another controller, I'd say a late-80s ESDI board would be perfect, to allow for larger drives and faster/easier interfacing on machines like the Deskpro 386. But with these three MFM controllers, we're already leaps and bounds beyond anything I expected to see anytime soon, and well beyond anything I've seen so far in any other emulators. Thanks again!
User avatar
omarsis81
Posts: 544
Joined: Thu 17 Dec, 2015 6:20 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby omarsis81 » Mon 20 Mar, 2017 2:16 pm

Are the DTC 5150X and the Xebec enabled to be used? :roll:
SarahWalker
Site Admin
Posts: 1327
Joined: Thu 24 Apr, 2014 4:18 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby SarahWalker » Mon 20 Mar, 2017 5:59 pm

Yes. You do need the relevant BIOS ROMs though.
User avatar
omarsis81
Posts: 544
Joined: Thu 17 Dec, 2015 6:20 pm

Re: ST-412/ST-506/ESDI hard drive emulation?

Postby omarsis81 » Mon 20 Mar, 2017 7:23 pm

Oh silly me! I got a BIOS and tested with the DTK XT clone. Works wonderful so far, great job!

I really like the drop-down list with all the hard disks types.
Do you think something similar could be done with like a dozen of real IDE drives?
For instance:
- Quantum Maverick C=1049 H=16 S=63 Size=540 MB
- Western Digital Caviar 2200 C=3876 H=16 S=63 Size=2000 MB

Also, for historical resemblance it would be nice if in the BIOS POST it would show a real hard disk model, like in the picture, instead of PcemHD, or rather, only display
PCemHD if a Custom HD is used, like now.

Let me know what you think

mobo_award.jpg
mobo_award.jpg (26.21 KiB) Viewed 2229 times

Return to “Development”

Who is online

Users browsing this forum: No registered users and 3 guests