Dynarec execution rate plummeting drastically at DOS prompt

Support and general discussion.
Post Reply
ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Dynarec execution rate plummeting drastically at DOS prompt

Post by ecksemmess » Tue 22 Sep, 2015 3:02 am

In general, I've been getting great performance out of the dynarec, but I've noticed that the execution percentage plummets strangely while idling at the DOS prompt, in ways that suggest that something other than the usual issue of an idle loop hitting one of the dynarec's weak points may be the culprit. This doesn't seem particularly sensitive to the configuration of the emulated system, but as a typical example, I was just running the Advanced/EV with a Pentium 100 and a Trio64, booted to Windows 95 DOS via a standard boot disk, and noticed it happening consistently. I have a fast host system, and was easily maintaining 100% continuously while running even the most demanding games, but every time I'd return to the DOS prompt, the execution rate would plunge pretty ridiculously. At first I thought this might be normal behavior, but a brief glance at Tom's numbers in the dynarec development thread made it clear that something else was going on (assuming that there's no important difference between DOS 6.22, which I believe was used for those numbers, and DOS 7, which I've been using). Importantly, the plunge of execution rate doesn't happen all at once and then stabilize; instead, it just keeps dropping gradually lower and lower as I sit and wait at the DOS prompt. After a few minutes, it gets below 20%! I've no idea where it will bottom out, as no matter how long I wait at the prompt, it doesn't seem to fully stabilize. Doing anything whatsoever resets things, and the execution rate will then begin its steady drop from the top once again. This can't be normal behavior. Is anyone else experiencing anything like this? Unfortunately, I can't really test this properly or answer too many questions about it, as I'm in the midst of wiping out and re-doing my emulation directory, which will take some time, but the information I've already given here will probably be sufficient for this to either be reproduced or disregarded as some quirk specific to my own setup.

User avatar
te_lanus
Posts: 93
Joined: Tue 28 Jul, 2015 4:47 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by te_lanus » Tue 22 Sep, 2015 3:58 am

started pcem, and running Dos 6.22 (on Award 430VX PCI, Tseng ET4000AX, Mobile Pentium MMX 233, Sound Blaster Pro v2 & 128mb ram) the Percentage hover about 29-28%. At Grey Cat Linux prompt it's full speed.

EDIT: tested with BasicLinux 3.5 and it runs at 100% at the Linux Prompt

neozeed
Posts: 176
Joined: Tue 08 Jul, 2014 4:41 am
Location: Hong Kong SAR
Contact:

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by neozeed » Tue 22 Sep, 2015 8:47 am

yeah its the way dos is. Run idle.com from virtual pc.

Also its the same thing that makes dBASE 3 / clipper run terrible on multiuser systems.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by SarahWalker » Tue 22 Sep, 2015 4:44 pm

ecksemess, could you post your CONFIG.SYS and AUTOEXEC.BAT files here? There may be something there causing slowdown, most of my benchmarking was done on a clean boot. In general though, idling (on pretty much any operating system) hits all the weak points in the current recompiler and hence doesn't perform very well.

User avatar
ppgrainbow
Posts: 467
Joined: Thu 04 Sep, 2014 7:03 am
Contact:

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ppgrainbow » Tue 22 Sep, 2015 4:54 pm

Try doing a clean boot without any of the drivers loaded. If the Dynamic recompile (Dynarec) execution rate continues to plummet after a clean boot, then there could be a bug in the Dynarec execution rate.

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Wed 23 Sep, 2015 5:24 am

TomWalker wrote:ecksemess, could you post your CONFIG.SYS and AUTOEXEC.BAT files here? There may be something there causing slowdown, most of my benchmarking was done on a clean boot. In general though, idling (on pretty much any operating system) hits all the weak points in the current recompiler and hence doesn't perform very well.
I've been running a very minimal system, booting a standard, unremarkable Win95A boot disk. Bare-bones AUTOEXEC.BAT:

Code: Select all

@echo off
Bare-bones CONFIG.SYS:

Code: Select all

DEVICE=HIMEM.SYS /testmem:off
FILES=30
BUFFERS=20
That's it! Now, my knee-jerk assumption was that this was just the usual issue of the idle loop hitting the dynarec's weak points, as you say. The only reason I bothered posting this was that you gave some very good numbers for "DOS prompt" in your most recent performance benchmark in the dynarec development thread. 90-something%, as I recall. That, combined with the rather odd gradual descent of the execution rate during idling, had me thinking there must be something wrong. However, now that everyone here is saying they're getting similarly dismal numbers when idling at the DOS prompt, it seems that those impressively high numbers you cited are the odd ones out. Are you still able to reproduce those high numbers at the DOS prompt now? If so, are you also able to get the 20-something% that we all seem to be getting?

neozeed
Posts: 176
Joined: Tue 08 Jul, 2014 4:41 am
Location: Hong Kong SAR
Contact:

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by neozeed » Wed 23 Sep, 2015 1:15 pm

Image

You can totally idle @ 100%. The trick is to find a CPU level you can actually emulate.

But MS-DOS has no 'idle' loop like NT or Linux. This is why laptops from back then either came with utilities to calm the CPU down which didn't start to appear until the 386SXL processor.

Battler
Posts: 793
Joined: Sun 06 Jul, 2014 7:05 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by Battler » Wed 23 Sep, 2015 2:59 pm

Even the best pieces of software slow down when running idle DOS. Virtual PC 2007, for example, uses ~100% host CPU and slows down a lot when DOS is idle, unless IDLE.COM is running, and VMWare is not much better. So PCem slowing down that much is nothing unusual.

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Sun 27 Sep, 2015 7:07 am

Yeah, I get it. Check out this post, though:

viewtopic.php?f=3&t=85&start=90#p1077

That's Tom's most recent dynarec performance summary. Note, right off the bat:

Code: Select all

DOS prompt         -  96% faster (55% / 108%)
108% at the DOS prompt! And I believe that Tom was running the fastest possible CPU in the emulated system when doing that benchmarking. I'd still love to know what accounts for the huge difference between that 108% and the 20-something% we all seem to be getting now... any idea, Tom? :)

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by SarahWalker » Sun 27 Sep, 2015 7:33 am

That was a clean boot of DOS 6.22, on a Winchip 240 (the fastest CPU emulated when I started the recompiler, somewhat slower than the Pentium MMXes you can emulate now), on a 3.6 GHz Ivy Bridge Core i5. If your setup differs from this at all then I'd expect the performance to drop somewhat.

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Mon 28 Sep, 2015 5:05 am

TomWalker wrote:That was a clean boot of DOS 6.22, on a Winchip 240 (the fastest CPU emulated when I started the recompiler, somewhat slower than the Pentium MMXes you can emulate now), on a 3.6 GHz Ivy Bridge Core i5. If your setup differs from this at all then I'd expect the performance to drop somewhat.
Thanks for that clarification. Given that, it continues to strike me as odd that I (and others here, I think?) are pulling numbers in the 20-30% range, even when emulating Pentium CPUs far slower than the WinChip 240 that you used for benchmarking. I'm running an Ivy Bridge i7 host and, as I said, I get ~20% at a clean-boot Win95A DOS prompt when emulating a Pentium 100. That seems like quite a departure from what you'd probably get! What do you think might account for that? If it's not too much trouble, I'd love to know what kind of performance you get at the DOS prompt on a clean Win95A boot disk emulating a Pentium 100. I can provide the boot disk if it will help. (If this request seems too trivial/petty/annoying, please do feel free to disregard it entirely; it just seems to me that there might be something interesting going on here.)

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Thu 05 Nov, 2015 6:17 am

Sorry to bump this, but I've now noticed something else of interest in relation to this. Regardless of how the emulated system is configured, when DOS is loaded "low"/"noumb", the dynarec doesn't choke at the DOS prompt at all. The execution rate never dips below 100% at all, even when emulating the fastest possible Pentiums. On the other hand, again regardless of how the emulated system is configured, when DOS is loaded "high" (in the UMB/HMA), the dynarec exhibits the strange plummeting execution rate behavior I described earlier. The difference in the execution rate is kind of ridiculous--I'd estimate offhand that it's literally more than ten times as fast at the DOS prompt with DOS loaded low, vs. with DOS loaded high. Can someone please take a moment to explain briefly why this should be expected behavior, if in fact it is? Furthermore, the smooth, gradual, exponential-like decay in the execution rate at the DOS prompt that is seen when DOS is loaded high still doesn't quite make sense to me; admittedly, my understanding of the dynarec is sorely lacking, but I don't see any legitimate reason why the execution rate should fall according to a gentle and non-linear curve. Looking at the status window, I see that the culprit seems to be that "new blocks" and "reused" both trend upward without limit when idling at the DOS prompt (though, again, this only happens with DOS loaded high). Should that be happening when DOS is loaded high, but not when loaded low? And is the gradual and non-linear fall of the execution rate simply a natural consequence of this expected runaway increase of one or both of those values? I really would greatly appreciate it if anyone who has the necessary familiarity with the dynarec could briefly answer these questions. Thanks!

Battler
Posts: 793
Joined: Sun 06 Jul, 2014 7:05 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by Battler » Thu 05 Nov, 2015 10:24 am

- ecksemmess: This really makes me wonder if graphics rendering is affected by the same thing. Would certainly explain why 256 color modes are so slow (to the point that I need to emulate a 486 at 16 MHz for Wolf3D to run at 100%) while 32-bit color modes are fast (102% on Pentium MMX 233), as 256 color (and 16-color too) modes predominantly use A0000-BFFFF, while 32-bit color modes predominantly use linear memory read/write. It could be that all memory on A0000-FFFFF, except for B0000-BFFFF, is for some reason badly cached by the emulator, and therefore slows down a lot on access.

Edit: The DOS prompt dip down could be due to the HMA too, no idea how that is emulated, as in, how it's assigned at all.

Edit #2: I seem to have gotten the dip to disappeared. The changes are in mem.c:

Code: Select all

        mem_set_mem_state(0x0c0000, 0x40000, MEM_READ_EXTERNAL | MEM_WRITE_EXTERNAL);
        mem_set_mem_state(0x100000, (mem_size - 1) - 65536, MEM_READ_INTERNAL | MEM_WRITE_INTERNAL);
To:

Code: Select all

        mem_set_mem_state(0x0c0000, 0x50000, MEM_READ_EXTERNAL | MEM_WRITE_EXTERNAL);
        mem_set_mem_state(0x100000, ((mem_size - 1) * 1024 * 1024) - 65536, MEM_READ_INTERNAL | MEM_WRITE_INTERNAL);
And:

Code: Select all

        if (mem_size > 1)
                mem_mapping_add(&ram_high_mapping, 0x100000, (mem_size - 1) * 1024 * 1024, mem_read_ram,    mem_read_ramw,    mem_read_raml,    mem_write_ram, mem_write_ramw, mem_write_raml,   ram + 0x100000, MEM_MAPPING_INTERNAL, NULL);
        mem_mapping_add(&ram_mid_mapping,   0xc0000, 0x40000, mem_read_ram,    mem_read_ramw,    mem_read_raml,    mem_write_ram, mem_write_ramw, mem_write_raml,   ram + 0xc0000,  MEM_MAPPING_INTERNAL, NULL);
To:

Code: Select all

        if (mem_size > 1)
                mem_mapping_add(&ram_high_mapping, 0x110000, ((mem_size - 1) * 1024 * 1024) - 65536, mem_read_ram,    mem_read_ramw,    mem_read_raml,    mem_write_ram, mem_write_ramw, mem_write_raml,   ram + 0x100000, MEM_MAPPING_INTERNAL, NULL);
        mem_mapping_add(&ram_mid_mapping,   0xc0000, 0x50000, mem_read_ram,    mem_read_ramw,    mem_read_raml,    mem_write_ram, mem_write_ramw, mem_write_raml,   ram + 0xc0000,  MEM_MAPPING_INTERNAL, NULL);
In both mem_init() and mem_resize(). Now I get DOS prompt at 100% with Pentium MMX 233 and DOS loaded high, without IDLE.COM. Before, I needed IDLE.COM to achieve same. What this basically does, is assigns that small segment of memory above 1 MB that is the HMA to RAM mid mapping instead of high mapping.

Edit: No, for some reason, even without the change, I get the DOS prompt at 100-105% with Pentium MMX 233, and DOS loaded high, without IDLE.COM. I need to investigate further, as I certainly do remember the dip. Could even be a GCC thing... I distinctly remember the dip when I compiled with GCC 4.x, but now that I use GCC 5.2, it seems to be gone.

Edit #2: It's the cache! With cache set to infinite, the DOS prompt takes a dip to 19-32% without IDLE.COM, while with cache set to anything else, it stays comfortably at 100+%.

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Thu 05 Nov, 2015 11:07 am

For me it's definitely not related to the cache setting. There's something else going on. Are you using EMM386 or similar? If so, try disabling it. I've found that EMM386 triggers another entirely unrelated bug in PCem that drastically slows down the emulated PC under certain conditions, thereby artificially increasing the execution rate, which masks the problem we're talking about now. Messing with the cache setting probably has the potential to do something similar, which may explain why you were able to make the problem appear to vanish by doing that, but if you watch the numbers in the status window, you'll find that it's an illusion. Watch "new blocks" and "reused" for a while at an idling DOS prompt, making sure not to touch the keyboard. I think you'll find that, regardless of your cache setting, they trend upwards over time when DOS is loaded high. With DOS loaded low, they hold steady indefinitely. That's the only reliable way to tell whether the "dip" is actually occurring; it doesn't always actually affect the overall execution rate.

Battler
Posts: 793
Joined: Sun 06 Jul, 2014 7:05 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by Battler » Thu 05 Nov, 2015 1:19 pm

Both clock counters go up but also down. It's like a sine wave. They increase then decrease, then increase again, then decrease again, and so on.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by SarahWalker » Thu 05 Nov, 2015 6:11 pm

Most likely this is a result of HIMEM/EMM386 toggling the A20 gate on/off, which will force a bunch of internal buffers to be flushed. There is some code to attempt to stop this from throwing everything out, but this may not be working very well...

ecksemmess
Posts: 155
Joined: Wed 18 Mar, 2015 5:27 am

Re: Dynarec execution rate plummeting drastically at DOS pro

Post by ecksemmess » Fri 06 Nov, 2015 12:03 am

Good to know. I had been confusing the HMA with one of the memory areas just below the 1 MB boundary. Realizing now that it's just above that boundary, what you say makes sense, and probably does explain what's going on here. It's not a big deal, but it would be nice to get that code sorted that's supposed to be preventing the excessive block turnover.

Post Reply