VHD support

Support and general discussion.
shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sat 29 Dec, 2018 7:58 pm

SA1988 wrote:
Sat 29 Dec, 2018 12:01 pm
Nice patch! However, I found a bug in this patch. The bug is about when win95's first phase setup is finished, opening a dynamic/sparse vhd to view files with winimage results in a corrupt/blank vhd (with winimage).
Does this bug happen with Windows created dynamic VHD's as well?

SA1988
Posts: 243
Joined: Wed 30 Apr, 2014 9:38 am

Re: VHD support

Post by SA1988 » Sat 29 Dec, 2018 8:48 pm

Well, it happens when I create the dynamic vhd's from pcem. Also, to note, I'm using Win7 x64.

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sat 29 Dec, 2018 8:54 pm

Just trying to narrow down whether it's a bug with my VHD code, or a bug with Winimage.

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sat 29 Dec, 2018 9:31 pm

I'm going to have a closer look at this. There may be some funny business going on with Winimage. I say may, because the first thing I tried with Winimage had a problem, but subsequent uses have behaved themselves.

Will try with a blank fresh VHD image to see what winimage is doing, hopefully I can spot something in a hex editor that may shine some light on any issues.

User avatar
leilei
Posts: 681
Joined: Fri 25 Apr, 2014 4:47 pm

Re: VHD support

Post by leilei » Sat 29 Dec, 2018 11:18 pm

I've only had WinImage ever working with VHDs if I shut down the guest fully or let them run scandisk, and if the file is not locked by something else (PCem must be fully terminated and Windows/anything else must not mount it).

I don't see any problems accessing the patch-created dynamic VHD with Win98SE in Winimage, though I do recall WinImage having an issue with FAT16 images at one point...

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sun 30 Dec, 2018 1:44 am

I think I can see what a potential issue MIGHT be. Unlike Windows (10) and MiniVHD, Winimage actually appears to use the sector bitmap for sparse images. This COULD lead to a potential issue where Injecting a file into the image via Winimage creates a new block with some sectors marked clean. MiniVHD could then write data to one of the "clean" sectors, and not update the sector bitmap accordingly. This would then cause winimage to not read that data (because it thinks the sector is still "clean"), and therefore shows as a corrupted file in Winimage.

Basically, MiniVHD (and therefore PCem) assumes (incorrectly, let's be honest) that new blocks are created with a fully "dirty" sector bitmap, and basically ignores it. I am going to have to solve that problem, probably by overwriting the sector bitmap on the first write to a block.

The reason I don't want to introduce "proper" sector bitmap handling for sparse disks is because I think it would overall harm performance due to modern hard disks and SSDs using 4K sectors, and performing fseek() on a file opened in read/write mode can be expensive.

Although, I suppose I could try and adapt the differencing sector bitmap handling, but I don't even know if I've got THAT working properly yet...

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sun 30 Dec, 2018 10:44 am

So I went ahead and implemented some sector bitmap handling. It basically does nothing on reads, but updates the sector bitmap if required on writes. At least that's what I INTEND it to do... Whether I've done it correctly or not...

I've attached a test patch, which can be applied on top of my V4 patch. Beware, minimal testing has been done. I just made sure Windows 95 booted and operated correctly. Also, Windows 10 mounted it, and I was able to open my lovely (*cough*) artwork I saved in Windows 95 Paint.

@SA1988 could you please test if this solves your Winimage issue?

EDIT: I haven't actually committed the changes to mercurial yet, so i don't know if that affects the import process. Just thought It'd mention it.

EDIT2: Oops, forgot a lone of code. Patch updated.
Attachments
vhd_test1_1.patch
(13.88 KiB) Downloaded 81 times

SA1988
Posts: 243
Joined: Wed 30 Apr, 2014 9:38 am

Re: VHD support

Post by SA1988 » Sun 30 Dec, 2018 12:14 pm

Your latest patch solves my issue, thanks!
Edit: At least, just partially, attempting to add files from winimage or mounted through imdisk results in missing other files or corruption again.

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Sun 30 Dec, 2018 7:04 pm

SA1988 wrote:
Sun 30 Dec, 2018 12:14 pm
Your latest patch solves my issue, thanks!
Edit: At least, just partially, attempting to add files from winimage or mounted through imdisk results in missing other files or corruption again.
Siigh... maybe something a little less subtle is required.

Are you able to provide a detailed step-by-step of what you do that results in corruption or missing files? Also, which versions of winimage etc do you use? I want to see if I can reproduce the problem myself.

Finally, I assume that you are stopping PCem emulation before accessing the image with other tools. If this is not the case, that would be the source of your problems, as MiniVHD is designed to have exclusive access to a VHD. This is especially important for sparse images, where it keeps a lot of state about the VHD in memory, and won't detect another program changing things (like adding new blocks for example...)

SA1988
Posts: 243
Joined: Wed 30 Apr, 2014 9:38 am

Re: VHD support

Post by SA1988 » Sun 30 Dec, 2018 10:59 pm

I mean, well, once it has installed win95 and rebooted (after the whole setup routine), for now, it's fine, but once adding one file (as little as 32kb for example) in the vhd (after closing PCem), it would result in some files (not all) missing like vwin32.vxd, etc, thus, corrupting and making win95 fail to boot fine.
Note: this issue doesn't happen when adding files from within win95 when run freshly and without opening winimage.

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Mon 31 Dec, 2018 1:09 am

SA1988 wrote:
Sun 30 Dec, 2018 10:59 pm
I mean, well, once it has installed win95 and rebooted (after the whole setup routine), for now, it's fine, but once adding one file (as little as 32kb for example) in the vhd (after closing PCem), it would result in some files (not all) missing like vwin32.vxd, etc, thus, corrupting and making win95 fail to boot fine.
Note: this issue doesn't happen when adding files from within win95 when run freshly and without opening winimage.
Well... I'm not sure what to try next, as I have been unable to reproduce this problem.

My test procedure was as follows:
  1. Create a 1024MB sparse VHD from my Windows 95 PCem config, which is a 430VX based machine
  2. Boot the machine from a Win 98SE boot disk, partition and format the VHD with FAT16
  3. Proceed to install Windows 95 OSR2 to the VHD.
  4. The machine boots to a fresh Windows 95 install
  5. Made copies of the original VHD, and injected files into them using Winimage 8.0, 8.1, 8.5, 9.0
  6. Booted all of the injected VHD files. They all booted just fine with no signs of corruption, and the injected files were where and what they were supposed to be.
My host system is a Windows 10 64-bit (1809) system. My compiler toolchain is MinGW-w64 (32-bit), with GCC 7.2. I'm not using MSYS2.

I simply don't know what might be causing that corruption you're seeing.

:? :? :? :?

SA1988
Posts: 243
Joined: Wed 30 Apr, 2014 9:38 am

Re: VHD support

Post by SA1988 » Mon 31 Dec, 2018 2:37 pm

Here I'm using Win7 x64 SP1 with MSYS2. After injecting some files, vmm32.vxd would corrupt itself here. I don't know what's causing this, anyways, nice fix :).
Edit: this doesn't happen with the Win7 disk management tools, for now.

shermanp
Posts: 122
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp » Wed 02 Jan, 2019 2:47 am

EDIT: I deleted the contents of this post, because I realized I had made a colossal arse of myself. Turns out, the bitmap macros I was using were incorrect for VHD's (the order was reversed).

This mistake will also explain why I've been having so much trouble with differencing VHD's, as they rely extensively on the sector bitmap.

:oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops:

A lesson to oneself... If in doubt about bitwise operations, write it out manually in binary to be sure.

Post Reply