Page 2 of 2

Re: [Bug] IDE HDD stuttering emulation

Posted: Mon 03 Feb, 2020 6:34 pm
by ruthan
I have this issue too, see this thread for details:
https://pcem-emulator.co.uk/phpBB3/viewtopic.php?f=2&t=3376&p=13162#p13162

For me its connected even with often DOS FAT32 disk image corruptions.. sometimes same game become unplayable at the moment when is usually fine.. for example Q1Dos level loading is taking minutes instead of seconds.. Q2DOS first loading take few minutes, instead of few seconds on real machine.. and assets loading take forever.

Im never fully understood it, but its used disk geometry with - 16 Heads, 63-Sectors.. and zillion of cylinder optimal from performance point of view?

Is possible to capture PCem ouput on ASM level and do the same on real HW and just compare these 2 codes?

I did some testing with proven Rayers - RAWSPEED.EXE (http://rayer.g6.cz/programm/programe.htm) and FIC 503 MB, and its reporting nonsenses - very high writing speeds - 38 MB/s.. without any enhancers, even without smartdrive etc.. it can copy 256 MB file within 12 second (~22 MB/S).. but its recording speed for 6-7 second(38 MB/s).. Rayer told me that its probably something wrong on emulated machine timer. Reading speed seems to calculate fine.
Other strange thing is that even when i disable UDMA in bios, speed is the same, if im not wrong PIO mode theoretical max is 16 MB/s, there has to be something wrong..

Here is some video (i know that emulation speed is bad, but it doesnt means that disc speed such report nonsenses), you can also see big delays at the start and end of disk operations, they are also not here when im using Dos within real machine, or even Vmware / Virtual Dos machine:
https://www.dropbox.com/s/tcwi7s6nshik14q/20200203135245-3.mkv?dl=0

I dont thing that it primary problem, but to be sure that problem is not part of code which is handle hdd operation on host OS.. it would be nice to add option to load all disk image (they are quite small in from todays point of view, i can give emulator lot of ram whole 2Gb, or 8GB disk could be in ram easily even without some quick compression.. within some inteligent to load unused disc space it could be even better ) to ram and make write operations in other asynch thread.. to not block emulation - it will at least show that everything is fine.

Update: I just checked QEMU just for info, it has block I/O on own thread too.. I thing that it make sense, because IO on MBs is handled by own microchip(s).. and from point of view own microchip is ideal place to make it own thread, especially if some operations are handled asynchronously with other operations - it this is IDE and SCSI specs if yes, own thread would make sense?

Re: [Bug] IDE HDD stuttering emulation

Posted: Sat 02 May, 2020 4:08 pm
by James-F
Bumping for v16.
Still same slow and stuttery IDE HDD.

Duke Nukem 3D.
Start the game and watch the demo,, you will see stuttering when the HDD is accessed, plus the game takes a super long time to load.
This does not happen on a real IDE HDD on my Pentium machine (or DOSBox for that matter).
I'm testing this each new release for years now, and it's still not fixed.
ruthan wrote: Mon 03 Feb, 2020 6:34 pm you can also see big delays at the start and end of disk operations, they are also not here when im using Dos within real machine, or even Vmware / Virtual Dos machine:
https://www.dropbox.com/s/tcwi7s6nshik14q/20200203135245-3.mkv?dl=0
Exactly.
That translated to stuttering in games each time the HDD is accessed.
Unplayable if you ask me.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 03 May, 2020 6:25 pm
by James-F
I think I found what's causing the stuttering.

I created a new IDE HDD with 63 Sectors, 16 Head, and 1024 Cylinders,, and now Duke 3D plays smoothly like from a RAM drive.
The old IMG I used to test for years now, had 8000 cylinders for 3937MB and it stutters like hell.

Lowering or increasing Cylinders from 1024 creates various degrees of stutter.... very strange. :shock:
I have tried many combinations, but 63, 16, 1024 works the best.

* Yes, I've configured the IDE HDD in the bios properly each try.
* I'm using DOS v7.1 for boot C (it supports large HDDs), and testing with various Cylinder numbers.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 03 May, 2020 6:49 pm
by te_lanus
wonder if that old hdd.img could be heavily fragmented O_o. I know that I had a 500mb (1015cyl,63sec,16heads) hdd, that tended to be slow. "Installed" it into my Win95 system, ran defrag on it, and it was fragmented a lot (since I was using it a lot in testing. Easiest fix was to create a new hdd and copy everything over.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 03 May, 2020 6:55 pm
by James-F
I specifically said I tested multiple NEW ide hdd images, they all stutter differently, nothing to do with fragmentation.
And I verified that indeed the same amount of cylinders produced the same stuttering with each new hdd image.

I have no idea how PCem works, but Duke3D reveals some inconsistency with HDD drive speed emulation depending on the number of Cylinders.

My settings:
Image

Re: [Bug] IDE HDD stuttering emulation

Posted: Wed 20 May, 2020 8:28 pm
by ruthan
This is need some bugfixing or some HDD fixing / migration utility, my image where quite new and there was this problem from the start.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 24 May, 2020 10:23 am
by James-F
@Sarah
Does changing the number of Cylinders tell you something about HDD read speed?

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 24 May, 2020 10:52 am
by SarahWalker
No.

Re: [Bug] IDE HDD stuttering emulation

Posted: Mon 25 May, 2020 3:56 am
by leilei
For Duke3D's case, the long load & stuttering is an uncached-FAT32-under-pure-dos problem. 1024 cylinders just happens to have OSR2+/W98+ FDISK not offer FAT32 support for partitioning (FAT16 only at that point). 1024 cylinders is officially too small for any FAT32 business.

Still in doubt?

- Try making a slightly larger FAT16/32able drive with 1536 cylinders, say no to all large disk support for FAT16 formatting, throw Win98 and Duke3D on there and see Duke3d run okay in pure DOS. Convert it to FAT32 using Win98's tool for that and watch it be slow in DOS!

- And now for your FAT32 drive, load SMARTDRV while in DOS and that super long load + stutter goes away. I've already mentioned SMARTDRV the previous time this was brought up.

Re: [Bug] IDE HDD stuttering emulation

Posted: Mon 25 May, 2020 6:13 pm
by James-F
First, thank you for the information leilei, I will try.
I started this post exactly 3 years ago, and I think I've missed all the posts about SMARTDRV.

Re: [Bug] IDE HDD stuttering emulation

Posted: Wed 27 May, 2020 8:56 pm
by ruben_balea
:o Since late 80s I have never seen an MS-DOS installation without SMARTDRV and definitelly all systems capable to run Duke Nukem 3D had SMARTDRV loaded by default...
:arrow: For those too young (Windows 95+ users) installing MS-DOS isn't just taking a Windosws 9x boot floppy and running fdisk and format c:/s
:idea: Install MS-DOS from original floppies and it will be installed properly.

Once I forgot to load SMARTDRV when booting from a floppy :oops: :roll: and a task that would require 3 or 4 hours took over 50 hours :shock: on what I thought was a dying hard disk, I was afraid of its heads flying out of the case... after that I classified that hard disk as reliable but with a peculiar sound :D

Re: [Bug] IDE HDD stuttering emulation

Posted: Thu 28 May, 2020 1:17 am
by ruben_balea
:arrow: Correction: In MS-DOS it is the Windows 3.x setup that enables SMARTDRV, not the MS-DOS setup. I just tested it with Windows 3.1 and MS-DOS 5.00 but I guess that Windows 3.0 and MS-DOS 4.0/4.01 did the same.

Re: [Bug] IDE HDD stuttering emulation

Posted: Thu 28 May, 2020 4:52 am
by leilei
Duke3D's DN3DHELP also notes it has a fast internal caching system that doesn't need SMARTDRV. This was written much before FAT32 was a thing

Maybe there could be a sticky thread that covers emulated real behavior that may be frequently perceived as bugs that could include this, Soundblaster init pops, 3dfx not doing <512 or reporting 4mb, S3 Trio64 not offering VESA2 modes by default, choppy mouse rates, AudioPCI's FM being crap/poppy SB emulation, not enough EMS, no Voodoo D3D HAL on XP, Virge/Mystique can't blend things, etc...

Re: [Bug] IDE HDD stuttering emulation

Posted: Thu 28 May, 2020 6:35 am
by ruben_balea
I never liked FPS so I don't know much about Duke Nukem 3D and I killed my first Windows 98 installation with FAT32 in a few days, then to not have to deal again with data corruption due to some incompatible disk utilities I installed a secondary hard disk with MS-DOS 6.22 and Windows 3.1 on my computer and had it until emulation/virtualization was good enough for me, maybe around 2007.
On real machines under pure DOS without SMARTDRV loaded and things like directories with thousands of files a simple DIR command can take 30 seconds or more to display used & free space and return to DOS prompt, even on a Pentium II with an UDMA drive.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sat 30 May, 2020 7:49 am
by James-F
I want to say huge thank you.
After 3 years all I had to do is add LH SMARTDRV /X into autoexec.bat.
Plenty of conventional memory left 624KB for all my DOS games.

I think DOSBox already have some kind of HDD caching without smartdrv.
Never needed SMARTDRV on the real Pentium MMX 233MHz either, I assume the modern IDE HDD already has some kind of caching.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sat 30 May, 2020 5:51 pm
by ruben_balea
I guess that DOSBox can throw data to guest programs almost as fast as the host OS can read it from disk drive while PCem has to emulate the BIOS calls, interrupts, DMA transfers and all those things at speeds acording to the emulated machine.

Re: [Bug] IDE HDD stuttering emulation

Posted: Sun 31 May, 2020 12:21 am
by leilei
It's real behavior. Even Windows 2000 install warns about not having SMARTDRV loaded before installation for these issues if you were to boot from DOS and install it there.

I know i've had Duke3D stutters and long loads on my 486 long ago but never to the extent to the FAT32-related thing which this was about. This also reminds me of why some of the new OEM machines in early '97 that still had FAT16 on multiple GB drives (Micron's well-reviewed MXE line, which also included PMMX233s)...