Dynamic recompiler

Discussion of development and patch submission.
User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Dynamic recompiler

Post by SarahWalker » Wed 05 Nov, 2014 9:32 pm

As of rev 173 I've checked in an early version of the recompiler. It's in very early stages - it doesn't really do much actual compilation (instead effectively chaining calls to the existing opcode routines together), and there are probably some compability issues, but there's a noticeable improvement on most stuff. Some stuff will show performance loss though, due to weaknesses in self-modifying code handling and a few other issues.

To use, add -DDYNAREC to CFLAGS in Makefile.mingw and recompile. Please do NOT upload binaries built with this anywhere, it's a bit too early for that.

It will probably only work in Windows at the moment, and certainly only on 32-bit x86. It only recompiles when internal cache is enabled, so it's a bit pointless running it on 386 or earlier.

A few benchmarks, all on Winchip 240 / 64 meg / ViRGE/DX / Fast VLB/PCI video / no sound card (interpreter % / recompiler %) :

Code: Select all

Normal build :

DOS 6.22, clean boot :
DOS prompt         - 12% faster (52% / 58%)
Doom               - 74% faster (35% / 61%)
Duke 3D            - 50% faster (32% / 48%)
Quake              - 64% faster (45% / 74%)
Transport Tycoon   - 41% faster (56% / 79%)
Tyrian             - 74% faster (43% / 75%)
Terminal Velocity  - 82% faster (39% / 71%)
Worms              - 62% faster (45% / 73%)
Blake Stone        - 74% faster (47% / 82%)

Windows 95 OSR 2 :
Win 95 desktop     - 24% slower (34% / 26%)
Jazz Jackrabbit 2  - 70% faster (43% / 73%)
GLQuake            - 25% faster (44% / 55%)
Croc (SW)          - 83% faster (30% / 55%)
Croc (Virge)       - 26% faster (43% / 54%)
Quake II           - 76% faster (38% / 67%)
GTA                - 69% faster (36% / 61%)


PGO build (-fprofile-generate/-fprofile-use) :

DOS 6.22, clean boot :
DOS prompt         -  7% faster (60% / 64%)
Doom               - 48% faster (50% / 74%)
Duke 3D            - 34% faster (41% / 55%)
Quake              - 47% faster (58% / 85%)
Transport Tycoon   - 21% faster (80% / 97%)
Tyrian             - 24% faster (63% / 78%)
Terminal Velocity  - 57% faster (51% / 80%)
Worms              - 15% faster (72% / 83%)
Blake Stone        - 59% faster (56% / 89%)

Windows 95 OSR 2 :
Win 95 desktop     - 22% slower (36% / 28%)
Jazz Jackrabbit 2  - 50% faster (56% / 84%)
GLQuake            - 20% faster (50% / 60%)
Croc (SW)          - 51% faster (43% / 65%)
Croc (Virge)       - 16% faster (51% / 59%)
Quake II           - 32% faster (54% / 74%)
GTA                - 16% faster (63% / 73%)

Battler
Posts: 793
Joined: Sun 06 Jul, 2014 7:05 pm

Re: Dynamic recompiler

Post by Battler » Thu 06 Nov, 2014 1:32 am

Previously, LBA 2 on PCem was so slow that it was barely playable. Now, after compiling the latest revision with the dynamic recompiler, it's perfectly playable.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Thu 06 Nov, 2014 2:57 am

Ok ive compiled and used pcem and pcemn (one with networking) both work fine... but when ever i compile the code (your hg repo) with the new recompiler it crashes after memory testing or straight away when i change the machine value in this case it crashes right on launch of the exe...

Picture of the crash: http://goput.it/rowl.png

pclog.txt has one line in it !PC on init

truth
Posts: 26
Joined: Thu 06 Nov, 2014 3:15 am

Re: Dynamic recompiler

Post by truth » Thu 06 Nov, 2014 3:19 am

That's incredible! Thank you for all the work in creating a dynarec for pcem! Given the compilation part is under development, those performance numbers look impressive.

Is the dynarec x86-specific in design? :)

User avatar
leilei
Posts: 686
Joined: Fri 25 Apr, 2014 4:47 pm

Re: Dynamic recompiler

Post by leilei » Thu 06 Nov, 2014 7:49 am

I'm guessing my Mingw/Msys is a bit outdated for it

Code: Select all

$ make -j4 -f Makefile.mingw
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c 386.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c 386_dynarec.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c 386_dynarec_ops.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c 808x.c
In file included from 386_ops.h:1:0,
                 from 386_dynarec_ops.c:138:
x86_ops.h:1:15: error: redefinition of typedef 'OpFn'
x86_ops.h:1:15: note: previous declaration of 'OpFn' was here
In file included from 386_ops.h:1:0,
                 from 386_dynarec.c:1156:
x86_ops.h:1:15: error: redefinition of typedef 'OpFn'
x86_ops.h:1:15: note: previous declaration of 'OpFn' was here
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c acer386sx.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c ali1429.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c amstrad.c
gcc.exe -O3 -march=i686 -fomit-frame-pointer -DDYNAREC -c cdrom-ioctl.c
make: *** [386_dynarec_ops.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [386_dynarec.o] Error 1
Compiles fine without Dynarec however

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Thu 06 Nov, 2014 8:26 am

therock247uk wrote:Ok ive compiled and used pcem and pcemn (one with networking) both work fine... but when ever i compile the code (your hg repo) with the new recompiler it crashes after memory testing or straight away when i change the machine value in this case it crashes right on launch of the exe...

Picture of the crash: http://goput.it/rowl.png

pclog.txt has one line in it !PC on init
What configuration are you using? If you can't get that far then I'd suggest using another build to select a 486 config and then copy the pcem.cfg file across.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Thu 06 Nov, 2014 8:27 am

leilei wrote:I'm guessing my Mingw/Msys is a bit outdated for it
For the record I'm using GCC 4.7.2.

User avatar
leilei
Posts: 686
Joined: Fri 25 Apr, 2014 4:47 pm

Re: Dynamic recompiler

Post by leilei » Thu 06 Nov, 2014 11:35 am

I was using GCC 4.5.2. I just updated only the GCC to 4.7.2 and it compiles now.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Thu 06 Nov, 2014 4:26 pm

No matter if i let it gen a new cfg or use a one from another build it still crashes on all machines.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Thu 06 Nov, 2014 6:42 pm

A new generated config probably won't work - I think that defaults to a 286. Can you post the config you're using from another build?

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Thu 06 Nov, 2014 7:34 pm

Just says http://goput.it/h81x.png and crashes a few mins later...

Code: Select all

gameblaster = 0
gus = 0
ssi2001 = 0
model = 20
cpu_manufacturer = 0
cpu = 1
gfxcard = 0
video_speed = 3
sndcard = 4
cpu_speed = 1
has_fpu = 0
slow_video = 1
cache = 1
cga_composite = 0
disc_a = 
disc_b = 
mem_size = 4
cdrom_drive = 0
cdrom_enabled = 0
vid_resize = 0
vid_api = 0
video_fullscreen_scale = 0
video_fullscreen_first = 1
hdc_sectors = 0
hdc_heads = 0
hdc_cylinders = 0
hdc_fn = 
hdd_sectors = 0
hdd_heads = 0
hdd_cylinders = 0
hdd_fn = 

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Thu 06 Nov, 2014 8:43 pm

I don't recognise that BIOS. Please try replacing it with the one on the download page.

SA1988
Posts: 246
Joined: Wed 30 Apr, 2014 9:38 am

Re: Dynamic recompiler

Post by SA1988 » Thu 06 Nov, 2014 8:43 pm

well the fault isn't bios related as that AMI acer bios works perfectly fine here.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Thu 06 Nov, 2014 8:49 pm

Replaced with yours and still get the same... http://goput.it/ie6k.png few mins later it crashes http://goput.it/stu7.png

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Thu 06 Nov, 2014 9:24 pm

What's your build environment? What version of GCC are you using?

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Thu 06 Nov, 2014 9:34 pm

Ive tried two versions one is gcc version 4.8.1 (GCC) and the other gcc version 4.8.1 (tdm-2)

Battler
Posts: 793
Joined: Sun 06 Jul, 2014 7:05 pm

Re: Dynamic recompiler

Post by Battler » Thu 06 Nov, 2014 10:48 pm

TomWalker wrote:What's your build environment? What version of GCC are you using?
He even tried the exact same build environment I use (I uploaded it for him), and the resulting executable works for me but crashes for him.

User avatar
leilei
Posts: 686
Joined: Fri 25 Apr, 2014 4:47 pm

Re: Dynamic recompiler

Post by leilei » Fri 07 Nov, 2014 12:03 am

I've noticed it's unstable on AM5x86 w/ Win95B but works okay on WinChip w/ Win98SE. ViRGE for both.

I do notice the speed improvement in Quake :) the only game I tried, actually just running the Team Fortress v2.5 intro video which I thought was a cool benchmark then since music sync is tough for the slower CPUs

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Sun 09 Nov, 2014 9:44 pm

Tried a recompile today (after the commit Reorganised FPU emulation) and still get crashes on open could it be cause ive got a intel i7 cpu which has lots of cores? or due to the fact that i have two grahpics cards the dedicated card only gets activated while gaming... There the only reasons I can think off...

nerd73
Posts: 110
Joined: Wed 24 Sep, 2014 11:16 pm

Re: Dynamic recompiler

Post by nerd73 » Sun 09 Nov, 2014 10:04 pm

therock247uk wrote:Tried a recompile today (after the commit Reorganised FPU emulation) and still get crashes on open could it be cause ive got a intel i7 cpu which has lots of cores? or due to the fact that i have two grahpics cards the dedicated card only gets activated while gaming... There the only reasons I can think off...
Probably not, but on my side (with AMD Athlon II X2 3.1 GHz and single Radeon HD 7750) it works OK.

truth
Posts: 26
Joined: Thu 06 Nov, 2014 3:15 am

Re: Dynamic recompiler

Post by truth » Sun 09 Nov, 2014 10:27 pm

Thank you, Tom, for the easy to compile PCem. The code is very portable and the dynarec (work-in-progress) is working perfectly in all software I tested (used your recommended configuration). I'm anticipating the recompiler portion of the dynarec and its definite effect on the emulation speed.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Sun 16 Nov, 2014 7:32 pm

Just tried the latest code it seems to actually log stuff but still crashes...

Code: Select all

Raw input registered!
Preparing scan code map list...
Done preparing!
executable_name = C:\Users\therock247uk\Desktop\PCem-a2b392697602\PCem-a2b392697602\src\PCem.exe
path = C:\Users\therock247uk\Desktop\PCem-a2b392697602\PCem-a2b392697602\src\
Keyboard initialized!
joystick_init: joysticks_present=0
Config data :
[]
gameblaster = 0
gus = 0
ssi2001 = 0
model = 20
cpu_manufacturer = 0
cpu = 0
gfxcard = 0
video_speed = 3
sndcard = 4
cpu_speed = 0
has_fpu = 0
slow_video = 1
cache = 3
cga_composite = 0
disc_a = 
disc_b = 
mem_size = 4
cdrom_drive = 0
cdrom_enabled = 0
vid_resize = 0
vid_api = 0
video_fullscreen_scale = 0
video_fullscreen_first = 1
hdc_sectors = 0
hdc_heads = 0
hdc_cylinders = 0
hdc_fn = 
hdd_sectors = 0
hdd_heads = 0
hdd_cylinders = 0
hdd_fn = 
Config loaded
Codegen is 00000000
Starting with romset 22
timer_reset
Path is \\.\
Initting as AMI 486 clone
io_init
Video_init 22 0
x86 reset
znp8 b1 = 4 04
znp16 b1 = 4 04
znp16 65b1 = 4 04
hasfpu - 0
is486 - 1  7
Recalc - 0 0 0
Set fullspeed - 1 1 0
Recalc - 0 0 0
DDRAW_INIT complete
Starting with romset 0
romset 0 - 1
Starting with romset 1
romset 1 - 1
Starting with romset 2
romset 2 - 1
Starting with romset 3
romset 3 - 1
Starting with romset 4
romset 4 - 1
Starting with romset 5
romset 5 - 1
Starting with romset 6
romset 6 - 1
Starting with romset 7
romset 7 - 1
Starting with romset 8
romset 8 - 1
Starting with romset 9
romset 9 - 1
Starting with romset 10
romset 10 - 1
Starting with romset 11
Loading BIOS
EA 5B 20
romset 11 - 1
Starting with romset 12
romset 12 - 1
Starting with romset 13
romset 13 - 1
Starting with romset 14
romset 14 - 1
Starting with romset 15
romset 15 - 1
Starting with romset 16
romset 16 - 1
Starting with romset 17
Failed to load ROM!
romset 17 - 0
Starting with romset 18
romset 18 - 1
Starting with romset 19
romset 19 - 1
Starting with romset 20
romset 20 - 1
Starting with romset 21
romset 21 - 1
Starting with romset 22
romset 22 - 1
Starting with romset 23
romset 23 - 1
Starting with romset 24
romset 24 - 1
Starting with romset 25
Load SIS496 ea ff
romset 25 - 1
Starting with romset 26
romset 26 - 1
Starting with romset 22
Starting with romset 22
Recalc - 56 0 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
Recalc - 56 40 0
onesec
onesec
onesec
INITDIALOG : 0 0 1
INITDIALOG : 1 1 1
INITDIALOG : 2 2 1
INITDIALOG : 3 3 1
INITDIALOG : 4 4 1
INITDIALOG : 5 7 1
INITDIALOG : 6 8 1
INITDIALOG : 7 9 1
INITDIALOG : 8 5 1
INITDIALOG : 9 6 1
INITDIALOG : 10 10 1
INITDIALOG : 11 11 1
INITDIALOG : 12 12 1
INITDIALOG : 13 13 1
INITDIALOG : 14 14 1
INITDIALOG : 15 15 1
INITDIALOG : 16 16 1
INITDIALOG : 17 19 1
INITDIALOG : 18 20 1
INITDIALOG : 19 21 1
INITDIALOG : 20 22 1
INITDIALOG : 21 23 1
INITDIALOG : 22 25 1
INITDIALOG : 23 26 1
Starting with romset 22
timer_reset
Initting as AMI 486 clone
io_init
Video_init 22 15
x86 reset
znp8 b1 = 4 04
znp16 b1 = 4 04
znp16 65b1 = 4 04
hasfpu - 0
is486 - 1  7
hasfpu - 0
is486 - 1  7
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
INITDIALOG : 0 0 1
INITDIALOG : 1 1 1
INITDIALOG : 2 2 1
INITDIALOG : 3 3 1
INITDIALOG : 4 4 1
INITDIALOG : 5 7 1
INITDIALOG : 6 8 1
INITDIALOG : 7 9 1
INITDIALOG : 8 5 1
INITDIALOG : 9 6 1
INITDIALOG : 10 10 1
INITDIALOG : 11 11 1
INITDIALOG : 12 12 1
INITDIALOG : 13 13 1
INITDIALOG : 14 14 1
INITDIALOG : 15 15 1
INITDIALOG : 16 16 1
INITDIALOG : 17 19 1
INITDIALOG : 18 20 1
INITDIALOG : 19 21 1
INITDIALOG : 20 22 1
INITDIALOG : 21 23 1
INITDIALOG : 22 25 1
INITDIALOG : 23 26 1
hasfpu - 0
is486 - 1  7
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onesec
onese

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Mon 17 Nov, 2014 8:55 pm

Added Pentium emulation for recompiler only at rev 180. I'd suggest at least resetting the emulator when switching between Pentium, Pentium MMX and Winchip, to clear out the recompiler cache.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Mon 17 Nov, 2014 8:56 pm

therock247uk: Try uncommenting the fflush() call in pclog() in pc.c, and see if anything more useful is reported in the log.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Mon 17 Nov, 2014 9:15 pm

line 78? uncommented it. crashes on open...

Code: Select all

Raw input registered!
Preparing scan code map list...
Done preparing!
executable_name = C:\Users\therock247uk\Desktop\PCem-cd10495ca29a\PCem-cd10495ca29a\src\PCem.exe
path = C:\Users\therock247uk\Desktop\PCem-cd10495ca29a\PCem-cd10495ca29a\src\
Keyboard initialized!
joystick_init: joysticks_present=0
Config loaded
Codegen is 00000000
Starting with romset 14
timer_reset
Path is \\.\
Initting as Commodore PC 30 III
io_init
Video_init 14 0
hasfpu - 0
is486 - 0  2
x86 reset
znp8 b1 = 4 04
znp16 b1 = 4 04
znp16 65b1 = 4 04
Recalc - 0 0 0
Set fullspeed - 0 1 0
Recalc - 0 0 0
DDRAW_INIT complete
Starting with romset 0
romset 0 - 1
Starting with romset 1
romset 1 - 1
Starting with romset 2
romset 2 - 1
Starting with romset 3
romset 3 - 1
Starting with romset 4
romset 4 - 1
Starting with romset 5
romset 5 - 1
Starting with romset 6
romset 6 - 1
Starting with romset 7
romset 7 - 1
Starting with romset 8
romset 8 - 1
Starting with romset 9
romset 9 - 1
Starting with romset 10
romset 10 - 1
Starting with romset 11
Loading BIOS
EA 5B 20
romset 11 - 1
Starting with romset 12
romset 12 - 1
Starting with romset 13
romset 13 - 1
Starting with romset 14
romset 14 - 1
Starting with romset 15
romset 15 - 1
Starting with romset 16
romset 16 - 1
Starting with romset 17
Failed to load ROM!
romset 17 - 0
Starting with romset 18
romset 18 - 1
Starting with romset 19
romset 19 - 1
Starting with romset 20
romset 20 - 1
Starting with romset 21
romset 21 - 1
Starting with romset 22
romset 22 - 1
Starting with romset 23
romset 23 - 1
Starting with romset 24
romset 24 - 1
Starting with romset 25
Load SIS496 ea ff
romset 25 - 1
Starting with romset 26
romset 26 - 1
Starting with romset 14
Starting with romset 14

nerd73
Posts: 110
Joined: Wed 24 Sep, 2014 11:16 pm

Re: Dynamic recompiler

Post by nerd73 » Mon 17 Nov, 2014 11:52 pm

Unable to enable dynamic recompiling.
On 64-bit windows.
I add -DDYNAREC to makefile.mingw, and it compiles successfully, but it doesn't seem to pick it up.

EDIT: Working now.

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Tue 18 Nov, 2014 7:09 pm

therock247uk wrote:line 78? uncommented it. crashes on open...
Hmm. Maybe try running it through gdb, and seeing if you can get a meaningful backtrace? (with CFLAGS=-g3 -DDYNAREC).

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Tue 18 Nov, 2014 7:10 pm

gdb can debug windows programs?

User avatar
SarahWalker
Site Admin
Posts: 1719
Joined: Thu 24 Apr, 2014 4:18 pm

Re: Dynamic recompiler

Post by SarahWalker » Tue 18 Nov, 2014 7:29 pm

Wouldn't have suggested it if it couldn't.

therock247uk
Posts: 29
Joined: Fri 25 Apr, 2014 2:55 pm

Re: Dynamic recompiler

Post by therock247uk » Tue 18 Nov, 2014 7:35 pm

kk will let you know how it goes pcem grew by about 10 mb so hopefully i can debug it properly now.

Post Reply