These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
tests/Makefile.am assumes the test image is located in
tests/syslinux/, but we actually created it in tests/. Fix the
location, and also ensure the test will clean it up after running
successfully.
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
It might be left over from a previous failed run. Best to unlink the
old file before starting qemu-nbd, so there's no possibility of
getting confused later when we wait for the file to appear.
This test fails for reasons I have not fully understood yet. However
one thing I noticed is that the Unix domain socket and PID file used
the tests are placed in the tests/ directory, not the tests/nbd/
subdirectory, so let's fix that:
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -p 63668 ...
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -p 60684 ...
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -k /home/rjones/d/libguestfs-rhel-9.2/tests/unix.sock ...
Fixes: commit 6d32773e81
Test fails as it cannot find .xml file. Turns out $pwd=./tests and
and not ./tests/regressions as supposed in script.
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@openvz.org>
Test fails as it cannot find .in file. Turns out $srcdir=. (which is ./tests) and
and not ./tests/regressions as supposed in script. Same apply to
$abs_srcdir.
Also put .out file in ./tests/regressions too.
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@openvz.org>
This was a feature that allowed you to add drives to the appliance
after launching it. It was complicated to implement, and only worked
for the libvirt backend (not "direct", which is the default backend).
It also turned out to be a bad idea. The original concept was that
appliance creation was slow, so to examine multiple guests you should
launch the handle once then hot-add the disks from each guest in turn
to manipulate them. However this is terrible from a security point of
view, especially for multi-tenant, because the drives from one guest
might compromise the appliance and thus the filesystems/drives from
subsequent guests.
It also turns out that hotplugging is very slow. Nowadays appliance
creation should be faster than hotplugging.
The main use case for this was virt-df, but virt-df no longer uses it
after we discovered the problems outlined above.
These APIs were an experimental feature for passing through 9p
filesystems from the host to the libguestfs appliance. It was never
possible to use this without hacking the qemu command line of the
appliance to add such drives by hand. It also didn't fit the
libguestfs model very well. And 9p is generally deprecated in
upstream qemu.
Note that for ABI reasons these APIs are not actually removed, they
have been changed so that they always return an error. These APIs
were actually hard-removed from all versions of RHEL.
See-also: https://bugzilla.redhat.com/921710
User-Mode Linux was an alternative hypervisor that could run the
appliance, instead of using qemu. It had many limitations including
lack of network, and UML support in Linux has been semi-broken for a
long time. It was also slower than KVM on baremeal in general and had
various corner cases which were much slower including the emulated
serial port which made bulk uploads and downloads painful. Also of
course it lacked qemu-specific features like qcow2 or any
network-backed disk, so many disk images could not be opened this way.
This was never supported in RHEL.
See-also: https://bugzilla.redhat.com/1144197
This experimental feature allowed you (in theory) to connect to an
existing instance of the libguestfs daemon. (Again, in theory) it
allowed you to attach to running guests. This didn't work well in
practice. If you want to do this, install qemu-guest-agent inside
your guest instead.
This also disables the --live options in guestfish and guestmount.
(The option now prints an error).
This was never supported in RHEL.
The daemon tests relied on this connection method to perform tests on
a bare daemon, so this removes those tests. They were not especially
valuable.
See-also: https://bugzilla.redhat.com/798980
Commit 6d32773e81 ("tests: Run the tests in parallel.") makes all of
the tests run in the same directory. Previously tests expected to be
run in their own subdirectories and so were freer about using generic
filenames. When run in parallel these filenames now clash. Fix
another case.
Fixes: commit 6d32773e81
Commit 6d32773e81 ("tests: Run the tests in parallel.") combined
multiple Makefiles into one. During this conversion I accidentally
made several mistakes - duplicating the EXTRA_DIST entry for
mountable/test-mountable-inspect.sh and omitting the backslash
continuation of the TESTS line.
Fixes: commit 6d32773e81
Create a new (fake) Fedora disk image with two partitions. /dev/sda1 is
the boot partition as usual, /dev/sda2 is used as an LVM PV. The VG has
four LVs, Root and LV1 through LV3.
Each LV holds a LUKS device (with a different key). Each decrypted LUKS
device holds an ext2 filesystem, with "/dev/mapper/Root-luks" holding the
root filesystem.
Each filesystem has a dedicated label (ROOT, LV1, LV2, LV3).
In the test case, run guestfish in inspector mode, twice.
In the first invocation, provide the LUKS passphrases by LV name. Also
specific to the first invocation, fetch the LUKS UUIDs by LV name.
In the second invocation, provide the LUKS passphrases by UUID.
In both invocations, after decryption, check the filesystem labels, the
/dev/mapper/* names generated for the decrypted LUKS block devices, and
the existence of "/etc/fedora-release" on the root filesystem.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1658126
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220223162120.16729-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Another test was removing all disk_*.img files, which removed the file
used by this test. This was ultimately caused by us using parallel
tests.
Put the disk image files back into the tests/gdisk/ subdirectory to
avoid this race.
Fixes: commit 6d32773e81
"my" variable $output masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 73.
"my" variable $end_sectors masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 78.
The "test-md-and-lvm-devices" test case creates, among other things, a
RAID0 array (md127) that spans two *differently sized* block devices
(sda1: 20MB, lv0: 16MB).
In Linux v3.14, the layout of such arrays was changed incompatibly and
undetectably. If an array were created with a pre-v3.14 kernel and
assembled on a v3.14+ kernel, or vice versa, data could be corrupted.
In Linux v5.4, a mitigation was added, requiring the user to specify the
layout version of such RAID0 arrays explicitly, as a module parameter. If
the user fails to specify a layout version, the v5.4+ kernel refuses to
assemble such arrays. This is why "test-md-and-lvm-devices" currently
fails, with any v5.4+ appliance kernel.
Until we implement a more general solution (see the bugzilla link below),
work around the issue by sizing sda1 and lv0 identically. For this,
increase the size of sdb1 to 24MB: when one 4MB extent is spent on LVM
metadata, the resultant lv0 size (20MB) will precisely match the size of
sda1.
This workaround only affects sizes, and does not interfere with the
original purpose of this test case, which is to test various *stackings*
between disk partitions, software RAID (md), and LVM logical volumes.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=2005485
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: remove stray empty line]
In commit 6d32773e81 ("tests: Run the tests in parallel.", 2021-03-18),
the "abs_srcdir" macro value that the 9p test would see changed from
".../tests/9p" to just ".../tests" -- the last component got dropped.
(Said commit updated some "abs_srcdir"-based references accordingly, for
example under "tests/disks", but "tests/9p/test-9p.sh" was missed.)
Therefore, the guest-visible location of the "/test-9p.sh" file changed to
"/9p/test-9p.sh", and a non-recursive listing of the guest-visible root
directory would not return the file. Thus, the test fails now.
Restore the host-side base directory to ".../tests/9p".
Fixes: 6d32773e81
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
In commit 6d32773e81 ("tests: Run the tests in parallel.", 2021-03-18),
the working directory relative to which "test-parallel-mount-local" would
be launched (by the test machinery) changed from "tests/mount-local" to
just "tests".
While the relative pathname of the "guestunmount" executable was updated
inside "test-parallel-mount-local" accordingly, the relative pathname of
the FUSE client ("test-parallel-mount-local" itself, just invoked with
"--test") was not. This issue guarantees that the exec call fails in the
child, and so the test case always hangs.
Because we had removed "mount-local" from the end of the working
directory, prepend it now to the relative pathname of the FUSE client
executable.
Fixes: 6d32773e81
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210902135124.15191-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Each worker thread of "test-parallel-mount-local" performs the following
steps (among others):
(1) it starts an appliance dedicated to that thread, using a private
scratch disk image,
(2) exports a dedicated FUSE mount point on the host, exposing the file
system on the appliance's disk,
(3) launches a child process for manipulating the particular FUSE mount
point on the host,
(4) enters a FUSE request processing loop, translating requests between
the host kernel (coming in via the FUSE mount point) and the
appliance.
Items to note:
- The child process from step (3) consists of a single thread of execution
(see fork() in POSIX): a duplicate of the parent process's respective
worker thread.
- The child process from step (3) blocks on any FUSE mount point access on
the host until the worker thread in the parent process starts processing
FUSE requests, in step (4).
- The FUSE request processing in step (4), in the worker thread living in
the parent process, terminates if and only if the child process unmounts
the FUSE mount point originating from (2).
Should the exec call in step (3) fail for any reason, the child currently
jumps to the "error" label. This is wrong: under the error label, we call
guestfs_close() on the appliance -- but the appliance is owned by the
parent process's worker thread, not the child. What happens is that the
child kills off the appliance while the parent's worker thread is in the
FUSE request processing loop (4).
The "error" label was never meant to be reached by the child process -- if
exec fails for any reason, exit the child immediately. The parent will
remain in the FUSE request processing loop (4) forever, but no state will
be corrupted. For example, using another (interactive) session on the
host, the FUSE mount points can be interacted with, and if all of them are
manually unmounted, the FUSE request processing (4) completes in every
worker thread.
This patch does not fix the primary issue with
"test-parallel-mount-local", but removes "chaos" from the symptoms. The
next patch will fix the actual regression in this test case.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210902135124.15191-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Use qemu-nbd --pid-file option so we don't have to use an arbitrary
sleep.
Enable all parts of the test, since everything should work now with
various upstream bugs having been fixed in the 8 years since the test
was originally written.
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
This test was correctly copied into the new guestfs-tools repository
when virt-resize was moved there along with the other tools. However
it was never removed from libguestfs. We were effectively running the
test on the installed virt-resize.
Fixes: commit 733d2182b6
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Although it's highly unlikely in normal use, while testing device name
translation patches it did happen and caused the test to segfault
instead of exiting with an error.
Nowadays there are hard drives and operating systems which support
"4K native" sector size. In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.
GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes. libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.
In order to fix the issue we need a way to provide correct logical block
size for attached disks. Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.
After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important. Thus both physical and logical
block size will be set to the same value.
In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method. Valid values are 512 and 4096.
add_drive_scratch has the same optional parameter for a consistency and
testing purpose.
add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
Libvirt 6.0 now requires that every disk in the backing chain has an
explicit backing format. For example this will be rejected by
libvirt:
qemu-img create -f qcow2 -b backing-disk disk.qcow2
with the error:
Original error from libvirt: Requested operation is not valid:
format of backing image 'backing-disk' of image 'disk.qcow2' was not
specified in the image metadata (See
https://libvirt.org/kbase/backing_chains.html for troubleshooting)
[code=55 int1=-1]
Instead you have to use the -F option to specify the format, eg:
qemu-img create -f qcow2 -b backing-disk -F raw disk.qcow2
