[SOLVED] Data corruption issues in PCem v9 r304

[ppgrainbow]

Have you installed a lot of software on PCem lately?

For me, I have been running a lot of software under PCem and for some reason, when it tries to write a lot of data on the virtual hard disk, data corruption can occur once in a while and the MS-DOS utility, CHKDSK can at some point report a lot of errors.

I decided to take a look at the issue when I mounted the first hard disk image (drv_80.img) as drive G (using IMDisk), I typed in CHKDSK G: and found that at least 24 files out of 5,300 are corrupt!

Here's a CHKDSK log:

The type of the file system is FAT.
Windows is verifying files and folders...
0 percent completed.
13 percent completed..
14 percent completed...
15 percent completed....
16 percent completed.....
17 percent completed......
18 percent completed.
19 percent completed..
20 percent completed...
21 percent completed....
Windows found errors on the disk, but will not fix them
because disk checking was run without the /F (fix) parameter.
The size of the \WP\MCV.EXE entry is not valid.
22 percent completed.....
23 percent completed......
24 percent completed.
25 percent completed..
26 percent completed...
The size of the \WINDOWS\HELPHLPR.DLL entry is not valid.
42 percent completed....
43 percent completed.....
44 percent completed......
The size of the \WINDOWS\SYSTEM\CTL3D32.DLL entry is not valid.
45 percent completed.
The size of the \WINDOWS\SYSTEM\msvcrt20.dll entry is not valid.
The size of the \WINDOWS\SYSTEM\OLE2.DLL entry is not valid.
The size of the \WINDOWS\SYSTEM\OLE2.130 entry is not valid.
46 percent completed..
The size of the \WINDOWS\SYSTEM\VBRUN100.DLL entry is not valid.
The size of the \WINDOWS\SYSTEM\VBRUN200.DLL entry is not valid.
47 percent completed...
48 percent completed....
The size of the \WINDOWS\SYSTEM\WIN32S\W32SCOMB.DLL entry is not valid.
The size of the \VP\VP.EXE entry is not valid.
The size of the \VP\VP386.HLP entry is not valid.
The size of the \VP\EXT\GRCALC.VEX entry is not valid.
49 percent completed.....
50 percent completed......
51 percent completed.
52 percent completed..
53 percent completed...
54 percent completed....
55 percent completed.....
56 percent completed......
57 percent completed.
58 percent completed..
59 percent completed...
60 percent completed....
The size of the \PKZIP\MANUAL.TXT entry is not valid.
61 percent completed.....
62 percent completed......
63 percent completed.
64 percent completed..
65 percent completed...
The size of the \NV\NV.HLP entry is not valid.
The size of the \NV\NVGER.HLP entry is not valid.
The size of the \NV\NVIEW.PNG entry is not valid.
The size of the \NV\README.NV entry is not valid.
The size of the \NV\VIEWER.EXE entry is not valid.
66 percent completed....
67 percent completed.....
68 percent completed......
69 percent completed.
70 percent completed..
The size of the \NC\123VIEW.EXE entry is not valid.
The size of the \NC\NCEDIT.EXE entry is not valid.
71 percent completed...
The size of the \NC\NCFF.EXE entry is not valid.
The size of the \NC\NCMAIN.EXE entry is not valid.
The size of the \NC\PARAVIEW.EXE entry is not valid.
72 percent completed....
The size of the \GVFM\RTM.EXE entry is not valid.
73 percent completed.....
74 percent completed......
75 percent completed.
76 percent completed..
77 percent completed...
78 percent completed....
79 percent completed.....
80 percent completed......
81 percent completed.
82 percent completed..
83 percent completed...
84 percent completed....
85 percent completed.....
86 percent completed......
87 percent completed.
88 percent completed..
89 percent completed...
90 percent completed....
91 percent completed.....
92 percent completed......
93 percent completed.
94 percent completed..
95 percent completed...
96 percent completed....
97 percent completed.....
98 percent completed......
99 percent completed.
100 percent completed..
File and folder verification is complete.
Windows found problems with the file system.
Run CHKDSK with the /F (fix) option to correct these.

2,142,502,912 bytes total disk space.
68,026,368 bytes in 8 hidden files.
5,832,704 bytes in 178 folders.
458,162,176 bytes in 5,300 files.
1,610,481,664 bytes available on disk.

32,768 bytes in each allocation unit.
65,384 total allocation units on disk.
49,148 allocation units available on disk.

This is being run on the Award SiS 496/497 driver (80486 DX2 @ 66 MHz with 64 MB memory, Number Nine FX graphics card with 2 MB VRAM). The machine is currently running MS-DOS 6.22 with four 2 GB hard disk images and no CD-ROM.

I'm wondering if the data corruption issue has to do with the way how PCem has been handling the writing of data correctly or not or if there are any bugs with the Award SiS 496/497 driver not handling hard disks over 1.97 GB correctly.

If you are using the latest revision of PCem, can anyone install software (either via floppy disk image or CD-ROM) to see if it can at some point trigger possible data corruption or corruption of one of the executable files that cause it to not run correctly?

Incase data corruption occurs, please make a backup of the hard disk image!

Update: Looking at the system configuration at bootup, the PIO mode of the four hard disks images is set at Mode 2. I could be wrong, but I read somewhere that early versions of the mainboard chipset might have a nasty bug that causes massive data corruption. I'm wondering if lowering the PIO mode setting to 1 or 0 fixes the problem or not.

[win2kgamer]

I have had this happen on various emulated systems as well (If I remove SMARTDRV.EXE from AUTOEXEC.BAT, the issue doesn't seem to happen.)

[ppgrainbow]

I have had this happen on various emulated systems as well (If I remove SMARTDRV.EXE from AUTOEXEC.BAT, the issue doesn't seem to happen.)

I see that you're on the same boat as I am.

If I go into the CMOS Setup and in Chipset Features, lower the transfer mode to 0, set IDE Prefetch Read Buffer to Both and enable IDE HDD Block Mode, that could help reduce data corruption. It turns out that some motherboards that use an early version of the SiS 496/497 chipset have a nasty bug that results in data corruption on PIO modes faster than 2 or 11.1 Mb/s.

[te_lanus]

I've also got the corruption in some of my virtual hdds. Recently had to reinstall win98se (after an user error) and after installing it would throw a blue screen during bootup, complaining of corruption of the hdd. Also got errors installing dos 6.22.

Both on the ami486 and the sis486(or whatever its called, not near my pc to check).

Whipped out (onto my tablet) the pc engineers' reference book vol 3, got a config for a 1.5gb hdd and has yet to get any corruption.

So could be that the bioses could be picky about the geometry of the hdd.

One that didn't work:
Sect:63
Heads:32
Cylinders:2048

One that works for me:
Sect:63
Heads:16
Cylinders:3152 (if I remember correctly)

[ppgrainbow]

I've also got the corruption in some of my virtual hdds. Recently had to reinstall win98se (after an user error) and after installing it would throw a blue screen during bootup, complaining of corruption of the hdd. Also got errors installing dos 6.22.

Both on the ami486 and the sis486(or whatever its called, not near my pc to check).

Whipped out (onto my tablet) the pc engineers' reference book vol 3, got a config for a 1.5gb hdd and has yet to get any corruption.

So could be that the bioses could be picky about the geometry of the hdd.

One that didn't work:
Sect:63
Heads:32
Cylinders:2048

One that works for me:
Sect:63
Heads:16
Cylinders:3152 (if I remember correctly)

That's what I've been thinking. Most BIOSes will not accept more than 16 heads per track and old BIOSes will not even accept more than 1,024 cylinders.

I update the IBMulator to version 0.5 and the BIOS will only accept hard disks up to 1,024 cylinders, 16 heads and 62 sectors per track for a maximum capacity of 496 MB. The PS/1 Model 2011 BIOS has a bug that prevents the successful formatting of a partition with 63 sectors per track.

[SarahWalker]

Heads > 16 will not work. I'm fairly certain I've said this in the past at some point.

[ppgrainbow]

Heads > 16 will not work. I'm fairly certain I've said this in the past at some point.

That's true. In the Award SiS 496/497 CMOS setup, there is a IDE detection utility that has three options:
1. LBA
2. NORMAL
3. LARGE.

Some oddball OSes such as SCO-UNIX have to use "NORMAL" for installation. Option 2 is selected as default. A 2 GB hard disk image with 4,161 cylinders, 16 heads and 63 sectors per track when used in LBA mode will be reported as 520 heads, 128 cylinders and 63 sectors per track. If LARGE is selected, then it will translate to 2,080 cylinders, 32 heads and 63 sectors per track.

When LBA mode is selected, something could be causing a nasty bug in the hard drive emulation that could be causing data corruption.

[ppgrainbow]

I updated to r310 and the data corruption issue is still occurring. CHKDSK continues to report file allocation errors, causing the adjustment in the size of the file. When this happens, the executable files, DLLs, configuration files, etc. no longer work properly. Here's the attachment:

BUG.TXT

(2.74 KiB) Downloaded 391 times

Emulated machines will not work properly with more than 16 heads in the CHS geometry. The 2 GB hard drive that I'm using in the Award SiS 496/497 machine is using LBA mode which translate the CHS geometry cylinder count to less than 1,024 cylinders and having more than 16 heads.

The translated CHS geometry in LBA mode reports drives as the following:

32 heads: 504 MB to 1,008 MB
64 heads: 1,008 MB to 2,016 MB
128 heads: 2,016 MB to 4,032 MB
255 heads: 4,032 MB to 8,064 MB

When the drive is operating in LBA mode on the Award SiS 496/497 machine, either the hard drive emulation or the BIOS isn't handling the reading and writing properly...especially when it comes to exiting Windows 3.1 or rebooting the emulated machine. This is what results in possible data corruption.

Update 1: PCem just crashed with the error message in pclog.txt, "Bad getpccache 000BE5B3" and ends up corrupting data on drive C and it has to do with the QEMM memory manager causing this problem. I suspect that this is NOT suppose to happen on real hardware.

Something is seriously wrong with the latest revision of PCem here!

Update 2: Got rid of DriveSpace and DOS-DATA.SYS from the CONFIG.SYS file. DriveSpace seems to be useless on hard disks images larger than 512 MB. Even if that's the case, this may not get rid of the data corruption issue. I'm gonna need to do some more testing and backup the hard disk image incase anything goes wrong.

The data corruption issues if I launch Windows 3.11 from the Microsoft MS-DOS Shell, run Tristan Pinball in Windows and end up with a gray screen and a corrupt BSOD text message. I'm wondering if replacing QEMM with HIMEM.SYS or removing Sound Blaster parameters from the AUTOEXEC.BAT fixes this issue or not.

[leilei]

I've never used more than 16 heads in PCEM ever. If I wanted more space, it's always the cylinders that are increased. Can't say i've had serious data corruption...... yet.

[ppgrainbow]

I've never used more than 16 heads in PCEM ever. If I wanted more space, it's always the cylinders that are increased. Can't say i've had serious data corruption...... yet.

Me too. I too never used more than 16 heads in PCem too. However, BIOSes with buggy LBA support or poorly written software aren't playing well with large hard disks.

[SarahWalker]

Sorry, should have been more specific - the 16 head limit is a hardware limitation. The LBA BIOS can claim to set a drive to more than 16 heads without issue; those settings are only used on the INT 13 interface, and the BIOS translates those correctly to LBA. So the LBA BIOS stating a 16 head drive has more than 16 heads isn't a bug.

Rev 313 should prevent creating or loading a drive image with more than 16 heads.

[ppgrainbow]

Sorry, should have been more specific - the 16 head limit is a hardware limitation. The LBA BIOS can claim to set a drive to more than 16 heads without issue; those settings are only used on the INT 13 interface, and the BIOS translates those correctly to LBA. So the LBA BIOS stating a 16 head drive has more than 16 heads isn't a bug.

Rev 313 should prevent creating or loading a drive image with more than 16 heads.

Thanks for limiting the hard drive head count to 16. Where it says "MessageBox(ghwnd, "Drive has too many heads (maximum is 128)", "PCem error", MB_OK);" in win-hdconf.c, this needs to be changed to 16 also.

[ppgrainbow]

I would also like to confirm that when I try to run the MS-DOS prompt inside Windows 3.1 on the Award SiS 496/497 machine, CHKDSK is misreporting errors as a result of data corruption on drive C.

You'll eventually notice the 8.3 character filenames where false allocation errors are reported are not being completely shown. Here's a screenshot:

CHKDSK misreporting data corruption.png (55.15 KiB) Viewed 11959 times

The situation when checking Drive D is much worse! CHKDSK ends up reporting massive data corruption.

6,585 lost allocation united found in 2,282 chains.
215,777,280 bytes disk space would be freed.

I'm suspecting that the problem might be either an issue with PCem, or with the Award SiS 496/497 machine not handling large hard drives correctly.

This issue does not occur when running Windows 3.1 under Virtual PC or Qemu though it hasn't been tested under Qemu right now.

Update 1: I think that I may have solved the problem, but it's hard to easily correct. It turned out that I used an expanded memory manager such as HIMEM and QEMM386 which may cause data corruption issues causing CHKDSK to misreport errors. I'll do more research and try to isolate the problem more.

Okay, I think that I found the problem. It's a utility called HIDE87! It causes CHKDSK to falsely report errors regardless of how much data is stored on the hard drive!

Update 2: I finally solved the problem and it has to do with the inclusion of memory address range in the expanded memory manager, QEMM386 with HIDE87.

I had to modify the settings in line 7 of the CONFIG.SYS to get things finally right!

Code: Select all

DEVICE=C:\QEMM\QEMM386.SYS RAM BE:N ARAM=C900-DFFF I=B000-B7FF I=E000-EFFF R:1

Including address below E000 and using HIDE87 caused the CHKDSK to misreport errors while I was running MS-DOS inside Windows 3.1.