VHD support

Support and general discussion.
User avatar
resle
Posts: 64
Joined: Mon 23 Feb, 2015 8:00 am

Re: VHD support

Post by resle »

shermanp wrote: That's alright, I've found a way of doing it using the method suggested here. Simple, and doesn't require compile time or run time checking. And since the only purpose is to read/write a one-time footer, performance is hardly an issue either.

I think I've got the VHD footer reading working. The next step is to figure out what I need to store where to get the info into the config file etc. That looks like it's part of the HdSizeDlg stuff. I shall persevere with it...

If I want to implement creating VHD's (which I do), I've discovered another little spanner in the works -- UUID. For whatever reasons, most implementations of UUID generation (in C) appear to use OS calls. So what the heck to do with MinGW? Where's a nice portable C (not C++) library for generating a valid UUID? Or do we just fake it for the purposes of creating a VHD file?

But to those interested, yes, I am making progress. I'm just trying to wrap my head around the wxWidgets stuff.

(As an aside, working with C makes me that much more appreciative of higher level languages!)
Sherman, Sarah, thanks a lot.

I didn't even actually intend this as a "request" (I don't really get the idea of requests, within the scope of something that is being given away for free), just a feature suggestion from some one who uses PCem literally every single day.

I wish I was more familiar with the toolchain and libraries used by you guys so that I could contribute myself, this is definitely one of those things that I would have enjoyed developing!

r.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

I've been thinking about VHD support for PCem for a while. Having never contributed any code to PCem before, it looked like adding VHD support wouldn't be too difficult, so I finally decided to have a go.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

A new patch has been posted, now with more VHD creation capabilities.
User avatar
resle
Posts: 64
Joined: Mon 23 Feb, 2015 8:00 am

Re: VHD support

Post by resle »

I have tested the newly implemented VHD support in how many scenarios as possible without a hitch. Old VHDs created under Virtualbox and Hyper-V were mounted and operated normally, and vice-versa PCem-made VHDs mount just fine everywhere else.

Great thanks to Sherman, Sarah, and everyone who was involved.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

resle wrote: Mon 11 Jun, 2018 10:01 pm I have tested the newly implemented VHD support in how many scenarios as possible without a hitch. Old VHDs created under Virtualbox and Hyper-V were mounted and operated normally, and vice-versa PCem-made VHDs mount just fine everywhere else.

Great thanks to Sherman, Sarah, and everyone who was involved.
Thanks for testing. No promises, but I've been thinking about how one might implement support for dynamic VHD's. The format itself doesn't look to difficult, it's whether or not I have the skills to implement it that I'm not too sure about...

The other option is to use an external library, but there doesn't appear to be much in the way of a standalone C library. There's a library called libvhdi, but that is labeled 'alpha', and I don't really know how stable and feature complete it is. Also, attempting my own implementation sounds like it could be an interesting problem to solve.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Just thought I'd post a quick update.

I have got dynamic VHD support semi-working. Current status is that I can attach a pre-formatted image to PCem as a secondary drive, and copy some files to/from Windows 98.

I have just tried installing Win 98 to a fresh (dynamic) VHD (including formatting).... It did not go well.

Obviously have more work to do yet!

I'll probably release a patch for testing purposes once I've got to the point of being able to install and run an OS to a dynamic VHD.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Coming soon to a PCem near you! (Hopefully... If I don't stuff anything ELSE up...)

DynamicVHD.PNG
DynamicVHD.PNG (139.79 KiB) Viewed 30655 times

Oh, and BTW, the VHD file was created by PCem, and Windows 10 mounts it as well.

Up next will be to release a patch in the next day or two, where I can hopefully get some victims—er—"kind volunteers" to help test it.
User avatar
xXLuckyXx
Posts: 66
Joined: Tue 06 Feb, 2018 3:52 pm

Re: VHD support

Post by xXLuckyXx »

What is the difference between VHD hard disk and PCem hard disk that we already have?
JosepMa
Posts: 202
Joined: Tue 20 Jun, 2017 6:25 pm

Re: VHD support

Post by JosepMa »

@xXLuckyXx : The answer is actually in that screenshot. 20GB disk (as seen by Windows 98 in PCem), sizing around 250MB on physical disk.

Shermanp did already implement fixed-size VHD support (which is almost a raw image with some headers), but he is currently implementing, with success, dynamic-sized disks.

What it means to all of us is that we will have have X amount of virtual hard drives on your disk, and the file sizes will be a lot less than the physical size (assuming a common disk usage, where you don't fill it up)
User avatar
xXLuckyXx
Posts: 66
Joined: Tue 06 Feb, 2018 3:52 pm

Re: VHD support

Post by xXLuckyXx »

So it's like VirtualBox. Virtualbox has Fixed size (if it is 20gb it will take 20gb immediately) and Dynamically allocated (it starts with 0mb and it will take more space when more files/applications are added but it won't shrink the size if those files/applications get deleted but it won't take more than allocated - 20gb)

Am I right?
JosepMa
Posts: 202
Joined: Tue 20 Jun, 2017 6:25 pm

Re: VHD support

Post by JosepMa »

Correct. VirtualBox supports different disk formats that support dynamic size allocation, including VHD and VDI
User avatar
xXLuckyXx
Posts: 66
Joined: Tue 06 Feb, 2018 3:52 pm

Re: VHD support

Post by xXLuckyXx »

But OS in VirtualBox works faster if it has Fixed size. Does that mean PCem will lose some of it's performance when having Dynamically allocated hard disk?
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

xXLuckyXx wrote: Wed 27 Jun, 2018 8:52 pm But OS in VirtualBox works faster if it has Fixed size. Does that mean PCem will lose some of it's performance when having Dynamically allocated hard disk?
Yes, potentially. But I don't think there will be too much in it. The format stores data in (typically) 2MB blocks. When a read or write occurs, it will, in most cases, be constrained to a single block, and performance should be almost as good as a raw image. There are two areas when performance gets degraded: 1. Read/write across data blocks 2. Write to a sparse (unallocated) block.

I haven't done any benchmarks yet, but my gut feeling is that modern hard disks are faster than any HDD emulated by PCem would have been anyway.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

So I decided to run ATTO under Windows 98. I don't know how representative it is etc etc, but it hopefully gives some indication of any performance differences between fixed and dynamic disks.

The setup was as follows:
  • All disk images are stored on my Crucial MX100 512GB SSD.
  • The guest OS (Windows 98SE) was running as drive C: using a raw disk image.
  • P166, 128MB RAM
  • Each image was created by PCem, and partitioned and formatted using the tools on the Windows 98 setup CD
  • Each image is an identical size of 4095MB
  • Each test was performed after a guest OS restart. The image to benchmark was attached to Secondary IDE: Master
The results of the two benchmarks are in the image below.
ATTO_Benchmarks.PNG
ATTO_Benchmarks.PNG (90.25 KiB) Viewed 30594 times
As can be seen, there appears to be negligible differences between the image formats, at least in the context of this setup and benchmark.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

In case anyone in this thread missed it, I have updated my patch in the patches subforum to add support for dynamic VHD's
User avatar
resle
Posts: 64
Joined: Mon 23 Feb, 2015 8:00 am

Re: VHD support

Post by resle »

shermanp wrote: Mon 09 Jul, 2018 5:34 am In case anyone in this thread missed it, I have updated my patch in the patches subforum to add support for dynamic VHD's
Tested as much as I could, and encountered no issues. Fiddled with mounting the pcem-made VHD in Hyper-V machines, or even directly under Windows, and viceversa (VHDs made in Hyper-V mounted under PCem), and all works flawlessly.
Thanks!
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

resle wrote: Fri 03 Aug, 2018 3:25 am Tested as much as I could, and encountered no issues. Fiddled with mounting the pcem-made VHD in Hyper-V machines, or even directly under Windows, and viceversa (VHDs made in Hyper-V mounted under PCem), and all works flawlessly.
Thanks!
Thanks for testing it. Glad to hear that you haven't found any issues so far.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Patch has been updated see this post for details => viewtopic.php?f=4&t=3056&p=11503#p11503

Main user facing change is better GUI integration with PCem.
User avatar
resle
Posts: 64
Joined: Mon 23 Feb, 2015 8:00 am

Re: VHD support

Post by resle »

shermanp wrote: Mon 26 Nov, 2018 5:42 am Patch has been updated see this post for details => viewtopic.php?f=4&t=3056&p=11503#p11503

Main user facing change is better GUI integration with PCem.
Once again thanks, and eager to test it if a build is available.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

resle wrote: Sat 01 Dec, 2018 7:27 pm Once again thanks, and eager to test it if a build is available.
PM sent
User avatar
leilei
Posts: 1039
Joined: Fri 25 Apr, 2014 4:47 pm

Re: VHD support

Post by leilei »

I'm starting to try it and it's been working good so far!

Couldn't mount my larger VHDs beyond 32gb though :( but i'm surprised at the dynamic vhd support in this early, which is very handy for fresh setup.
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

leilei wrote: Sun 02 Dec, 2018 11:37 am I'm starting to try it and it's been working good so far!

Couldn't mount my larger VHDs beyond 32gb though :( but i'm surprised at the dynamic vhd support in this early, which is very handy for fresh setup.
Yeah, unfortunately, ~32GB is the maximum size limit. More specifically, it's 65535 cylinders * 16 heads * 63 sectors/track. Basically, the limitation is because the VHD format can't support more than 65535 cylinders (it's stored as a 16 bit int), and PCem doesn't support anything higher than 63 sectors/track. I figure for most setups, that 30GB will be plenty.

The only thing that MIGHT work is to cheat. Store more than 63 spt in the VHD, and convert them to cylinders at runtime. I don't know enough about hard disk geometry to know it if that would go horribly wrong. Especially since one of the main points to adding VHD support in the first place is interoperability with other software (eg. Windows)
User avatar
ppgrainbow
Posts: 479
Joined: Thu 04 Sep, 2014 7:03 am
Contact:

Re: VHD support

Post by ppgrainbow »

Let's see, 65,535 cylinders, 16 heads and 63 sectors per track, that's actually, 66,059,280 sectors for a maximum total of 33,822,351,360 bytes or almost 31.5 GB which is plenty for a successful Windows 95 OSR2 installation. :)
Azzoz2005
Posts: 2
Joined: Fri 21 Dec, 2018 9:50 pm

Re: VHD support

Post by Azzoz2005 »

The biggest advantage of using a VHD in PCem to me is that I can mount the VHD in Windows and I can easily copy dos software that I downloaded from the internet to the PCem machines VHD, making transferring DOS software and games an absolute breeze, that makes getting software simple and easy as mounting VHD and copying programs to it, :D :D, Your mileage may vary
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Azzoz2005 wrote: Sat 22 Dec, 2018 12:01 pm The biggest advantage of using a VHD in PCem to me is that I can mount the VHD in Windows and I can easily copy dos software that I downloaded from the internet to the PCem machines VHD, making transferring DOS software and games an absolute breeze, that makes getting software simple and easy as mounting VHD and copying programs to it, :D :D, Your mileage may vary
That was the idea :D

Especially, since unlike Linux, Windows has no (in-built) way to mount raw disk images.

(Although it is possible to convert raw images to fixed VHD and back again).
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Patch updated again. See this post for details => viewtopic.php?f=4&t=3056&p=11503#p11618

As mentioned in that post, if anybody wants to assist me to try and get differencing support working, please feel free! I've basically slammed into a brick wall on that one otherwise :(
User avatar
Araripe
Posts: 12
Joined: Tue 16 Oct, 2018 11:07 am

Re: VHD support

Post by Araripe »

How do I apply the patch? I wanna test the VHD feature but I don't have a single clue on where to begin. Do I need a compiler or something?
shermanp
Posts: 175
Joined: Sat 18 Feb, 2017 2:09 am

Re: VHD support

Post by shermanp »

Araripe wrote: Tue 25 Dec, 2018 5:16 pm How do I apply the patch? I wanna test the VHD feature but I don't have a single clue on where to begin. Do I need a compiler or something?
PM sent.
User avatar
Araripe
Posts: 12
Joined: Tue 16 Oct, 2018 11:07 am

Re: VHD support

Post by Araripe »

Thank you so much!
SA1988
Posts: 274
Joined: Wed 30 Apr, 2014 9:38 am

Re: VHD support

Post by SA1988 »

Nice patch! However, I found a bug in this patch. The bug is about when win95's first phase setup is finished, opening a dynamic/sparse vhd to view files with winimage results in a corrupt/blank vhd (with winimage).
Post Reply