Dynamic recompiler

[startmenu]

http://kikyoulinux.info/misc/20141202230932.jpg
Installing Windows 98, it turned out BDoD after logo shown... The first stage of installation (coping files,extracting cabs,etc) was successful.

[SarahWalker]

That BSOD is the result of a self-modifying code bug. Unfortunately it's not the only bug with Win98 setup...

[startmenu]

That BSOD is the result of a self-modifying code bug. Unfortunately it's not the only bug with Win98 setup...

Reinstalling Win98 each time (on recompiler enabled builds), I get the same BSoD...

[SarahWalker]

Yes, I know. It hasn't been fixed yet.

[SarahWalker]

Should be fixed as of rev 188. There were originally two bugs relating to self modifying code, and fixing them revealed a third where PCem couldn't handle a code block modifying itself. This may fix other stuff as well; one of the bugs related to code crossing pages, which could account for variation between different setups and different versions of Windows, etc.

[SarahWalker]

First part of code generation committed at rev 189. This adds generation for immediate->register and register->register versions of MOV, XCHG, ADD, SUB, AND, OR, XOR, CMP, TEST, INC and DEC. Benchmarks vs rev 188 :

Code: Select all

DOS prompt         -  8% faster (60% /  65%)
Doom               - 13% faster (64% /  72%)
Duke 3D            -  9% faster (43% /  47%)
Quake              - 10% faster (96% / 106%)
Transport Tycoon   -  5% faster (99% / 104%)
Tyrian             -  3% faster (90% /  93%)
Terminal Velocity  - 24% faster (91% / 112%)
Worms              -  4% faster (95% /  99%)
Blake Stone        -  9% faster (90% /  98%)

Windows 95 OSR 2 :
Win 95 desktop     - 13% faster (32% /  36%)
Jazz Jackrabbit 2  - 18% faster (89% / 105%)
GLQuake            -  6% faster (68% /  72%)
Croc (SW)          - 11% faster (76% /  84%)
Croc (Virge)       -  6% faster (66% /  70%)
Quake II           - 13% faster (88% /  99%)
GTA                -  5% faster (75% /  79%)

Note that the self-modifying code fixes in rev 188 caused some performance regressions.

[Alegend45]

I would hope you could get the dynarec working on Linux. My fastest computer runs Linux, so this would make it easier to test. Also, do you have any idea what might be holding Windows XP SP3 up if it doesn't run already?

[SarahWalker]

Also, do you have any idea what might be holding Windows XP SP3 up if it doesn't run already?

No idea, I haven't tested it. Has anyone?

[nerd73]

Also, do you have any idea what might be holding Windows XP SP3 up if it doesn't run already?

No idea, I haven't tested it. Has anyone?

I haven't, but I will try POSReady 2009.

[SarahWalker]

Committed some changes to memory system to improve self modifying code. Normal performance numbers below, some losses, but the gains are generally bigger. Some of the biggest gains aren't in the normal test set - Screamer is more than twice as fast now, and Screamer Rally is nearly three times faster than before the commit.

Code: Select all

Normal build :
DOS prompt         - 38% faster ( 65% /  90%)
Doom               - 17% faster ( 72% /  84%)
Duke 3D            - 19% faster ( 47% /  56%)
Quake              -  2% slower (106% / 104%)
Transport Tycoon   -  1% slower (104% / 103%)
Tyrian             -  6% slower ( 93% /  87%)
Terminal Velocity  -  2% slower (112% / 110%)
Worms              -  5% slower ( 99% /  94%)
Blake Stone        - 22% faster ( 98% / 120%)

Windows 95 OSR 2 :
Win 95 desktop     -  6% faster ( 36% /  38%)
Jazz Jackrabbit 2  -  2% faster (105% / 107%)
GLQuake            -  1% slower ( 72% /  71%)
Croc (SW)          -  5% slower ( 84% /  80%)
Croc (Virge)       -  0% faster ( 70% /  70%)
Quake II           -  2% slower ( 99% /  97%)
GTA                -  5% slower ( 79% /  75%)

Overall gain of 4.7%.

Compared to interpreter :

Code: Select all

Normal build :
DOS prompt         -  64% faster (55% /  90%)
Doom               - 133% faster (36% /  84%)
Duke 3D            -  81% faster (31% /  56%)
Quake              - 112% faster (49% / 104%)
Transport Tycoon   -  84% faster (56% / 103%)
Tyrian             - 102% faster (43% /  87%)
Terminal Velocity  - 182% faster (39% / 110%)
Worms              - 109% faster (45% /  94%)
Blake Stone        - 155% faster (47% / 120%)

Windows 95 OSR 2 :
Win 95 desktop     -  12% faster (34% /  38%)
Jazz 2             - 149% faster (43% / 107%)
GLQuake            -  51% faster (47% /  71%)
Croc (SW)          - 166% faster (30% /  80%)
Croc (Virge)       -  63% faster (43% /  70%)
Quake II           - 155% faster (38% /  97%)
GTA                - 108% faster (36% /  75%)

Average gain over interpreter is now 108%.

[SarahWalker]

More instructions recompiled now. Gain vs last major change :

Code: Select all

DOS prompt         -   7% faster  (90% /  96%)
Doom               -  10% faster  (84% /  92%)
Duke 3D            -   4% faster  (56% /  58%)
Quake              -  10% faster (104% / 114%)
Transport Tycoon   -   4% faster (103% / 107%)
Tyrian             -   6% faster  (87% /  92%)
Terminal Velocity  -  22% faster (110% / 134%)
Worms              -  32% faster  (94% / 124%)
Blake Stone        -  18% faster (120% / 142%)

Win 95 desktop     - same         (38% /  38%)
Jazz 2             -  13% faster (107% / 121%)
GLQuake            -   7% faster  (71% /  76%)
Croc (SW)          -  10% faster  (80% /  88%)
Croc (Virge)       -   3% faster  (70% /  72%)
Quake II           -   8% faster  (97% / 105%)
GTA                -  16% faster  (75% /  87%)

Average gain of 10.6%.

Verses interpreter :

Code: Select all

DOS prompt         -  75% faster (55% /  96%)
Doom               - 156% faster (36% /  92%)
Duke 3D            -  87% faster (31% /  58%)
Quake              - 133% faster (49% / 114%)
Transport Tycoon   -  91% faster (56% / 107%)
Tyrian             - 114% faster (43% /  92%)
Terminal Velocity  - 244% faster (39% / 134%)
Worms              - 176% faster (45% / 124%)
Blake Stone        - 202% faster (47% / 142%)

Win 95 desktop     -  12% faster (34% /  38%)
Jazz 2             - 181% faster (43% / 121%)
GLQuake            -  62% faster (47% /  76%)
Croc (SW)          - 193% faster (30% /  88%)
Croc (Virge)       -  67% faster (43% /  72%)
Quake II           - 176% faster (38% / 105%)
GTA                - 142% faster (36% /  87%)

Average gain of 132%.

[leilei]

Warbirds 2.5 is very playable now a few revisions earlier it froze before rendering the scene.

[insidious611]

Hey! I'm a relatively new user of PCem but an old hat at emulation in general. I got an emulated 286, 386, and 486 running and decided to compile myself a copy of the latest source code and try a Pentium 133.

I noticed some odd issues, which I'd like to report (Hopefully these aren't already known, but without a bug tracker I have to go by what's in this thread and I'm not seeing them)

All of these issues only seem to happen in Dynarec mode. I haven't tried a Dynarec'd 486 to see if they happen on a 486 yet, but they've been tested to happen frequently running a Pentium of any frequency.

First off, floppy images are being "mangled" on reboot on occasion. It's hard to describe what I mean, but basically, when doing a Hard Reset, sometimes images in the floppy drive seem to get mangled to the point that they read as empty in WinImage and PCem and become no longer bootable if they were bpptable to start. I had this happen to DR-DOS images and a Windows 95 boot disk.

Secondly, DR-DOS 7.03 installs fine, but after install it occasionally freezes or crashes, filling pclog.txt to several gigabytes. If you want said log, I can try to take a look at it, but for obvious reasons at that size it won't load in say, notepad.

As far as the crashes, they seem to happen during disk access. They consistently happen in the default DR-DOS install a few seconds after NWCACHE is loaded. If NWCACHE is removed, you can get to a prompt, but randomly while doing disk accesses (even as simple as DIR), it will crash again.

When it crashes, CPU percentage drops to 2%, stays there, and PCem fills its pclog.txt file to several gigabytes. PCem becomes completely unresponsive at this point and is eventually killed by windows (your usual graying-of-the-window-then-reporting-to-microsoft crash box).

I realize this is very much in-progress code, and I'm simply reporting these issues so that they can be known and eventually solved . If you need any further testing from me, feel free. I'm afraid, however, that I find the actual code somewhat intimidating and will not be submitting my own patches.

For now, I'm installing Windows 95 in an emulated WinChip 133 (so as to avoid the floppy-killing) and plan on switching it to a Pentium 133 after install to see if there's any other issues.

[leilei]

First off, floppy images are being "mangled" on reboot on occasion.

I just tried it with a Win95B boot disk in Award430VX PCI w/ P90 dynarec and can confirm this happens. I diffed the two images and the mangling happens right at the beginning of the disk. (Was trying to copy DOS cdrom drivers to load to use with Windows since i have issues getting Windows 95/98 to detect the drive)

Unrelated, there's a Intel Advanced/EV bug where it'll test memory into infinity on occasion

[rflego]

Okay, I've had similar problems, I noticed that most of them were related when they were used hard drives larger than 2GB, even using FAT32, causes corrupted files, many problems with boot floppy images and crashes in emulation.

May be the BIOS images, do not know, but I tested with 386 SX, DX 486 (AMD), Pentium 133, Pentium MMX 233 and several WinBIOS ...

I tried very DOS games will soon post the results and the problems they had in the PCem.

[insidious611]

Hmm. My issues were with a 1GB hard drive image.

[SarahWalker]

Can anyone confirm a case where this disc corruption happens, and exactly what configuration they were using and what they were doing with the emulator at the time? There is a bug where hard reset causes discs to be ejected, but I've yet to see it actually corrupt anything.

Secondly, DR-DOS 7.03 installs fine, but after install it occasionally freezes or crashes, filling pclog.txt to several gigabytes. If you want said log, I can try to take a look at it, but for obvious reasons at that size it won't load in say, notepad.

Was this after a hard reset? There are bugs with resetting the dynamic recompiler that can cause strange problems like this.

[rflego]

Normally I use the WinBIOS with AMD 486DX4, 100Mhz (I use this setting because, I have a motherboard old vesa, with the same processor and bios) the problem occurs while you are running with the system and run a game or make a copy file eventually starts giving errors, and many files are corrupted, not away that this happens, I could go days running the emulator smoothly until occurred, or, just a few minutes after you have installed the system.

I tried to use images created by the Bochs and DOSBox SVN Daum, the problems continue.

Tested with Windows 95, 98 and ME, MS-DOS 6.22 and FreeDOS 1.1, FAT16, FAT32 and NTFS.

[insidious611]

The disk corruption happens during the BIOS after a hard reset. Not immediately, but prior to the disk being accessed at boot time. It happens relatively consistently on an Award 430VX PCI motherboard, Intel Pentium (any MHz) or IDT WinChip (any MHz). It happens regardless of RAM settings but my current testbed has 16MB. I'm using a Tseng ET4000AX as the GPU, and a Sound Blaster 16 as the sound card, I haven't tested on different versions of either of those because after working around this issue through swapping floppy disks while the BIOS was on its "NO SYSTEM DISK PRESS ENTER TO RETRY" screen and installing Windows 95 I don't want to muck up the installation by changing hardware. Voodoo Graphics is enabled at the moment, though as I haven't even gotten around to installing the drivers yet, I tried with it disabled as well and the same thing happened. My hard drive is configured as 2048 cylinders, 16 heads, 63 SPT, for 1008mb.

I'm not having to "do" anything in particular in the emulator to make this happen. I can literally put in a disk, go to the BIOS setup screen, hard reboot, and it will happen. Same if the hard drive is nonbootable and it's a nonbootable disk and I hard reset while it's waiting for me to strike the key to try again. Etc. In short, absolutely nothing should be happening to the FDC while this is going on.

This, if I may use very old terminology, may be a bit of a heisenbug.

My CFG is below:

Code: Select all

gameblaster = 0
gus = 0
ssi2001 = 0
voodoo = 1
model = 25
cpu_manufacturer = 0
cpu = 4
gfxcard = 5
video_speed = 5
sndcard = 7
cpu_speed = 12
has_fpu = 1
slow_video = 1
cache = 3
cga_composite = 0
disc_a = 
disc_b = 
mem_size = 16
cdrom_drive = 69
cdrom_enabled = 0
vid_resize = 0
vid_api = 0
video_fullscreen_scale = 0
video_fullscreen_first = 1
hdc_sectors = 63
hdc_heads = 16
hdc_cylinders = 2048
hdc_fn = D:\Downloads\pentium-w95.img
hdd_sectors = 0
hdd_heads = 0
hdd_cylinders = 0
hdd_fn = 

If necessary I can easily provide an original image and a mangled image for comparison.

[insidious611]

As to the DR-DOS thing, it was indeed after a hard reset. I'll try installing it while closing and opening the emulator between fdisking, formatting, and beginning the install to see if that's the issue.

[insidious611]

As a quick test, I removed my hard drive (so as to make sure I wasn't booting into Windows 95), booted my DR-DOS 7.03 floppy #1 image to the point where it was asking me to run FDISK but wasn't doing anything with the drive or really in general (sitting at a choice in a batch file basically), hit hard reset, and after rebooting was greeted with DISK BOOT FAILURE, INSERT SYSTEM DISK AND PRESS ENTER. Mindful of what you said about disks being ejected, I tried "putting it back in", and pressed enter, same message. I then closed PCem and opened both a backup copy of the original I had made and the mangled copy in WinImage. The backup had all the files and etc, the version PCem mangled appeared blank. In a hex editor, you can clearly see that all the data is there but the boot sector and FAT have been altered. I can provide these on request, DR-DOS 7.03 is freeware afaik. If you're not comfortable with that, I can probably do the same with a FreeDOS bootdisk or any number of publically available DOS/9x/ME/2k/XP bootdisks and provide that.

[SarahWalker]

Rev 218 fixes a major bug with resetting the recompiler, this may fix some of the issues you're seeing. I'll continue looking at the disc corruption though.

[insidious611]

As I said in PM, compiling and testing Rev 218, it definitely seems to fix the floppy corruption bug.

I'm trying out the DR-DOS 7.03 bugs again now to see if they're still present, since installing it is much easier without having to avoid floppy corruption.

EDIT: Unfortunately, DR-DOS 7.03 still hangs on bootup after a successful install. It seems to hang in EMM386.EXE instead of NWCACHE now.

[rflego]

I took the test with this new revision, created a HD 8GB, not got problems, no file was corrupted, even "forced" to make it happen (restarting the emulator several times without closing the windows) and even then, the files were intact (though would often happens with this method, even in a real machine) even filled the hard disk, and everything was intact.

I tested the DR-DOS 7.03, I got no problem, the floppy disks read normal, and after installing the system, it usually works without error.

My config:

Code: Select all

gameblaster = 0
gus = 0
ssi2001 = 0
voodoo = 0
model = 25
cpu_manufacturer = 0
cpu = 10
gfxcard = 19
video_speed = 5
sndcard = 7
cpu_speed = 12
has_fpu = 1
slow_video = 1
cache = 1
cga_composite = 0
disc_a = 
disc_b = 
mem_size = 64
cdrom_drive = 0
cdrom_enabled = 0
vid_resize = 0
vid_api = 0
video_fullscreen_scale = 0
video_fullscreen_first = 1
hdc_sectors = 63
hdc_heads = 16
hdc_cylinders = 4096
hdc_fn = D:\EmuPC\PCem-1f993cc6e0be\HDD\DRDOS2GB.img
hdd_sectors = 0
hdd_heads = 0
hdd_cylinders = 0
hdd_fn = 

[Phoenix S3 Trio64]
memory = 4

I used a 2GB image in DR-DOS, the same installed normal, and not getting problems in the boot.

I downloaded the DR-DOS in Winwolrd site, do not know if it is the same version that you use.

In order, I noticed another problem with the windows 95 and 98, as I tested many things only in DOS (even though only installed systems in order to use the FAT32) I noticed just now that windows is too slow, to cause lag even in sound, this is "normal" in the emulator or may have something wrong?

EDIT: The problem occurs when you start Windows, but I noticed that if you replace the processor in the config with it running, the system returns to normal, even using hard or soft reset. Unfortunately, when closing and opening the PCEm, the problem returns having to replace the processor again, has anyone had this problem? I'm using the version Dynarec like many here, and seeing the problem that insidious611 are having with DR-DOS and I do not, I'm wondering a little strange ...

PS: Sorry my english.

[SarahWalker]

Glad to see the corruption has gone.

rflego: The Windows desktop does run quite slowly on PCem at present, with the Pentium MMX 233 you're trying to emulate you have basically no chance of getting anywhere near 100%. I'd suggest something a little more modest; on my machine (Core i5 3something at 3.6 GHz) I can just about get 100% at the Windows desktop emulating a Pentium 75. Actual applications will run faster, it's the idling process the recompiler currently struggles with.

[SarahWalker]

Rev 219 fixes the disc eject on reset bug.

[rflego]

Yes, I expected that there would be 100% (My PC is an AMD FX-6100, six-core, running at 4.4Ghz overclocked, with 8GB Ram) however, the problem to which I refer is that the windows emulated is extremely slow, the mouse is "delayed" and the sound "failing" but when change the processor running with the windows, it is normal, the mouse is smooth and the sound too, even making the soft reset or hard reset, it is normal, the problem back to close the PCEm and open it again, regardless of the processor to which it is, I believe that only I am having this problem, it may be something from my PC.

By the way, I use Kaspesky antivirus, I'm lately needing it off because he accuses the PCEm.exe is a trojan, it occurs right after compilation.

[SarahWalker]

Sorry, just to be clear, are you saying the speed indicator in the title bar is actually saying 100% while Windows is being slow?

[rflego]

No, it is about 30% ~ 33% at idle, I just realized, doing other tests, it is not necessary to change the CPU him back to "normal", just opening and closing the config (settings - config) while the emulator is running, it returns in 99% ~ 101% at idle.

[SarahWalker]

I can't reproduce this, but it looks like you've hit some obscure bug of some kind. In any case, your machine can not emulate a 233 MMX at full speed, so I suggest you select something a bit more modest!

Just as a final quick test, what does the status window (Misc -> Status) show for 'effective clockspeed' before and after opening the config window?