[Bug] IDE HDD stuttering emulation

Discussion of development and patch submission.
User avatar
ruthan
Posts: 43
Joined: Fri 27 Apr, 2018 5:07 pm

Re: [Bug] IDE HDD stuttering emulation

Post by ruthan » Mon 03 Feb, 2020 6:34 pm

I have this issue too, see this thread for details:
https://pcem-emulator.co.uk/phpBB3/viewtopic.php?f=2&t=3376&p=13162#p13162

For me its connected even with often DOS FAT32 disk image corruptions.. sometimes same game become unplayable at the moment when is usually fine.. for example Q1Dos level loading is taking minutes instead of seconds.. Q2DOS first loading take few minutes, instead of few seconds on real machine.. and assets loading take forever.

Im never fully understood it, but its used disk geometry with - 16 Heads, 63-Sectors.. and zillion of cylinder optimal from performance point of view?

Is possible to capture PCem ouput on ASM level and do the same on real HW and just compare these 2 codes?

I did some testing with proven Rayers - RAWSPEED.EXE (http://rayer.g6.cz/programm/programe.htm) and FIC 503 MB, and its reporting nonsenses - very high writing speeds - 38 MB/s.. without any enhancers, even without smartdrive etc.. it can copy 256 MB file within 12 second (~22 MB/S).. but its recording speed for 6-7 second(38 MB/s).. Rayer told me that its probably something wrong on emulated machine timer. Reading speed seems to calculate fine.
Other strange thing is that even when i disable UDMA in bios, speed is the same, if im not wrong PIO mode theoretical max is 16 MB/s, there has to be something wrong..

Here is some video (i know that emulation speed is bad, but it doesnt means that disc speed such report nonsenses), you can also see big delays at the start and end of disk operations, they are also not here when im using Dos within real machine, or even Vmware / Virtual Dos machine:
https://www.dropbox.com/s/tcwi7s6nshik14q/20200203135245-3.mkv?dl=0

I dont thing that it primary problem, but to be sure that problem is not part of code which is handle hdd operation on host OS.. it would be nice to add option to load all disk image (they are quite small in from todays point of view, i can give emulator lot of ram whole 2Gb, or 8GB disk could be in ram easily even without some quick compression.. within some inteligent to load unused disc space it could be even better ) to ram and make write operations in other asynch thread.. to not block emulation - it will at least show that everything is fine.

Update: I just checked QEMU just for info, it has block I/O on own thread too.. I thing that it make sense, because IO on MBs is handled by own microchip(s).. and from point of view own microchip is ideal place to make it own thread, especially if some operations are handled asynchronously with other operations - it this is IDE and SCSI specs if yes, own thread would make sense?

User avatar
James-F
Posts: 87
Joined: Tue 30 May, 2017 10:26 am

Re: [Bug] IDE HDD stuttering emulation

Post by James-F » Sat 02 May, 2020 4:08 pm

Bumping for v16.
Still same slow and stuttery IDE HDD.

Duke Nukem 3D.
Start the game and watch the demo,, you will see stuttering when the HDD is accessed, plus the game takes a super long time to load.
This does not happen on a real IDE HDD on my Pentium machine (or DOSBox for that matter).
I'm testing this each new release for years now, and it's still not fixed.
ruthan wrote:
Mon 03 Feb, 2020 6:34 pm
you can also see big delays at the start and end of disk operations, they are also not here when im using Dos within real machine, or even Vmware / Virtual Dos machine:
https://www.dropbox.com/s/tcwi7s6nshik14q/20200203135245-3.mkv?dl=0
Exactly.
That translated to stuttering in games each time the HDD is accessed.
Unplayable if you ask me.

User avatar
James-F
Posts: 87
Joined: Tue 30 May, 2017 10:26 am

Re: [Bug] IDE HDD stuttering emulation

Post by James-F » Sun 03 May, 2020 6:25 pm

I think I found what's causing the stuttering.

I created a new IDE HDD with 63 Sectors, 16 Head, and 1024 Cylinders,, and now Duke 3D plays smoothly like from a RAM drive.
The old IMG I used to test for years now, had 8000 cylinders for 3937MB and it stutters like hell.

Lowering or increasing Cylinders from 1024 creates various degrees of stutter.... very strange. :shock:
I have tried many combinations, but 63, 16, 1024 works the best.

* Yes, I've configured the IDE HDD in the bios properly each try.
* I'm using DOS v7.1 for boot C (it supports large HDDs), and testing with various Cylinder numbers.

User avatar
te_lanus
Posts: 129
Joined: Tue 28 Jul, 2015 4:47 am

Re: [Bug] IDE HDD stuttering emulation

Post by te_lanus » Sun 03 May, 2020 6:49 pm

wonder if that old hdd.img could be heavily fragmented O_o. I know that I had a 500mb (1015cyl,63sec,16heads) hdd, that tended to be slow. "Installed" it into my Win95 system, ran defrag on it, and it was fragmented a lot (since I was using it a lot in testing. Easiest fix was to create a new hdd and copy everything over.

User avatar
James-F
Posts: 87
Joined: Tue 30 May, 2017 10:26 am

Re: [Bug] IDE HDD stuttering emulation

Post by James-F » Sun 03 May, 2020 6:55 pm

I specifically said I tested multiple NEW ide hdd images, they all stutter differently, nothing to do with fragmentation.
And I verified that indeed the same amount of cylinders produced the same stuttering with each new hdd image.

I have no idea how PCem works, but Duke3D reveals some inconsistency with HDD drive speed emulation depending on the number of Cylinders.

My settings:
Image

User avatar
ruthan
Posts: 43
Joined: Fri 27 Apr, 2018 5:07 pm

Re: [Bug] IDE HDD stuttering emulation

Post by ruthan » Wed 20 May, 2020 8:28 pm

This is need some bugfixing or some HDD fixing / migration utility, my image where quite new and there was this problem from the start.

User avatar
James-F
Posts: 87
Joined: Tue 30 May, 2017 10:26 am

Re: [Bug] IDE HDD stuttering emulation

Post by James-F » Sun 24 May, 2020 10:23 am

@Sarah
Does changing the number of Cylinders tell you something about HDD read speed?

User avatar
SarahWalker
Site Admin
Posts: 1867
Joined: Thu 24 Apr, 2014 4:18 pm

Re: [Bug] IDE HDD stuttering emulation

Post by SarahWalker » Sun 24 May, 2020 10:52 am

No.

User avatar
leilei
Posts: 777
Joined: Fri 25 Apr, 2014 4:47 pm

Re: [Bug] IDE HDD stuttering emulation

Post by leilei » Mon 25 May, 2020 3:56 am

For Duke3D's case, the long load & stuttering is an uncached-FAT32-under-pure-dos problem. 1024 cylinders just happens to have OSR2+/W98+ FDISK not offer FAT32 support for partitioning (FAT16 only at that point). 1024 cylinders is officially too small for any FAT32 business.

Still in doubt?

- Try making a slightly larger FAT16/32able drive with 1536 cylinders, say no to all large disk support for FAT16 formatting, throw Win98 and Duke3D on there and see Duke3d run okay in pure DOS. Convert it to FAT32 using Win98's tool for that and watch it be slow in DOS!

- And now for your FAT32 drive, load SMARTDRV while in DOS and that super long load + stutter goes away. I've already mentioned SMARTDRV the previous time this was brought up.

User avatar
James-F
Posts: 87
Joined: Tue 30 May, 2017 10:26 am

Re: [Bug] IDE HDD stuttering emulation

Post by James-F » Mon 25 May, 2020 6:13 pm

First, thank you for the information leilei, I will try.
I started this post exactly 3 years ago, and I think I've missed all the posts about SMARTDRV.

User avatar
ruben_balea
Posts: 120
Joined: Mon 08 May, 2017 11:24 pm
Location: Spain

Re: [Bug] IDE HDD stuttering emulation

Post by ruben_balea » Wed 27 May, 2020 8:56 pm

:o Since late 80s I have never seen an MS-DOS installation without SMARTDRV and definitelly all systems capable to run Duke Nukem 3D had SMARTDRV loaded by default...
:arrow: For those too young (Windows 95+ users) installing MS-DOS isn't just taking a Windosws 9x boot floppy and running fdisk and format c:/s
:idea: Install MS-DOS from original floppies and it will be installed properly.

Once I forgot to load SMARTDRV when booting from a floppy :oops: :roll: and a task that would require 3 or 4 hours took over 50 hours :shock: on what I thought was a dying hard disk, I was afraid of its heads flying out of the case... after that I classified that hard disk as reliable but with a peculiar sound :D

User avatar
ruben_balea
Posts: 120
Joined: Mon 08 May, 2017 11:24 pm
Location: Spain

Re: [Bug] IDE HDD stuttering emulation

Post by ruben_balea » Thu 28 May, 2020 1:17 am

:arrow: Correction: In MS-DOS it is the Windows 3.x setup that enables SMARTDRV, not the MS-DOS setup. I just tested it with Windows 3.1 and MS-DOS 5.00 but I guess that Windows 3.0 and MS-DOS 4.0/4.01 did the same.

User avatar
leilei
Posts: 777
Joined: Fri 25 Apr, 2014 4:47 pm

Re: [Bug] IDE HDD stuttering emulation

Post by leilei » Thu 28 May, 2020 4:52 am

Duke3D's DN3DHELP also notes it has a fast internal caching system that doesn't need SMARTDRV. This was written much before FAT32 was a thing

Maybe there could be a sticky thread that covers emulated real behavior that may be frequently perceived as bugs that could include this, Soundblaster init pops, 3dfx not doing <512 or reporting 4mb, S3 Trio64 not offering VESA2 modes by default, choppy mouse rates, AudioPCI's FM being crap/poppy SB emulation, not enough EMS, no Voodoo D3D HAL on XP, Virge/Mystique can't blend things, etc...

User avatar
ruben_balea
Posts: 120
Joined: Mon 08 May, 2017 11:24 pm
Location: Spain

Re: [Bug] IDE HDD stuttering emulation

Post by ruben_balea » Thu 28 May, 2020 6:35 am

I never liked FPS so I don't know much about Duke Nukem 3D and I killed my first Windows 98 installation with FAT32 in a few days, then to not have to deal again with data corruption due to some incompatible disk utilities I installed a secondary hard disk with MS-DOS 6.22 and Windows 3.1 on my computer and had it until emulation/virtualization was good enough for me, maybe around 2007.
On real machines under pure DOS without SMARTDRV loaded and things like directories with thousands of files a simple DIR command can take 30 seconds or more to display used & free space and return to DOS prompt, even on a Pentium II with an UDMA drive.

Post Reply