Commit f7a24b2ea8 ("lib/qemu.c: Use machine type none when inspecting
QMP properties") changed the number of command line parameters used
before the '-qmp stdio' option when libguestfs queries qemu features.
This broke some tests which rely on the exact order of parameters.
Fixes: commit f7a24b2ea8
Updates: commit 5da8102f5f
On RHEL 10.2 aarch64 (only) we see:
$ echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-kvm" }' '{ "execute": "quit" }' | QEMU_AUDIO_DRV=none "/usr/libexec/qemu-kvm" -display none -machine "virt,accel=kvm:hvf:tcg" -qmp stdio
qemu-kvm: invalid accelerator hvf
qemu-kvm: falling back to KVM
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 1, "major": 9}, "package": "qemu-kvm-9.1.0-15.el10_0.4"}, "capabilities": ["oob"]}}
qemu-kvm: warning: CPU model cortex-a57-arm-cpu is deprecated -- use 'host' / 'max'
qemu-kvm: kvm_init_vcpu: kvm_arch_init_vcpu failed (0): Invalid argument
Unfortunately we cannot use guestfs_int_get_cpu_model (as that
requires us to already know if KVM is supported), so we just have to
guess that -cpu max will work, at least enough for QMP to work.
Fixes: https://issues.redhat.com/browse/RHEL-121076
Reported-by: Xiang Hua Chen
Previously we tested if KVM was available, and cached that for the
qemu binary. I think this was actually wrong. For example, if the
machine restarts, then the cache is still around, but KVM might be
enabled or disabled because of a new host kernel.
In any case, let's radically simplify this.
Test for KVM on each run. Consequently we can remove all the qemu
test caching stuff as it is no longer used anywhere.
I also tightened up the code that runs the QMP query-kvm command, so
now any unexpected output will cause a runtime failure. This command
ought to work, and if it breaks we ought to know about it and fix it.
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Although it's highly unlikely in normal use, while testing device name
translation patches it did happen and caused the test to segfault
instead of exiting with an error.
Nowadays there are hard drives and operating systems which support
"4K native" sector size. In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.
GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes. libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.
In order to fix the issue we need a way to provide correct logical block
size for attached disks. Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.
After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important. Thus both physical and logical
block size will be set to the same value.
In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method. Valid values are 512 and 4096.
add_drive_scratch has the same optional parameter for a consistency and
testing purpose.
add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
Latest libvirt (3.8, I think) made a very small adjustment to the
format of the generated -drive parameter when using iscsi, from:
file=iscsi://1.2.3.4:1234/iqn.2003-01.org.linux-iscsi.fedora,
to:
file=iscsi://1.2.3.4:1234/iqn.2003-01.org.linux-iscsi.fedora/0,
This makes a corresponding change to the test so that both patterns
can be recognized.
Replace the monolithic 'test-max-disks.pl' script with a test program
written in C. The program is completely equivalent to the old script,
except for the enhancement that it is able to detect if disks are
added to the appliance in the wrong order.
The tests themselves are split out into some shell scripts:
- test-27-disks: Fully tests 27 disks.
This is the minimum supported configuration for all backends.
- test-255-disks: Fully tests 255 disks.
This is the most disks that libguestfs up to 1.36 supported.
- test-add-lots-of-disks: Add ‘a lot’ of disks and exit (without
any further testing). This is meant to try to exercise > 255
disk case but without being as slow as a test of the max number
of disks (which takes ages, even for as few as 4000 disks).
- test-max-disks: Test the maximum possible number of disks.
The maximum depends on several factors, notably which backend
is in use, and the limit on the number of open files.
‘test-max-disks’ is a slow test.
Apply this change across all the shell scripts containing tests.
Additionally this defines the environment variables $abs_srcdir,
$abs_builddir, $top_srcdir, $top_builddir, $abs_top_srcdir and
$abs_top_builddir which can now be used throughout test scripts.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
A disk of type 'volume' is stored as
<source pool='pool_name' volume='volume_name'/>
and its real location is inside the 'volume_name', as 'pool_name': in
this case, query libvirt for the actual path of the specified volume in
the specified pool.
Adjust the code so that:
- for_each_disk gets the virConnectPtr, needed to do operations with
libvirt
- when extracting the disk filename depending on the type, the code
snippet doing it can directly set 'filename', without setting an XPath
result variable
Only file-based volumes are supported for now; more types can be added
(with proper testing) later on.
Instead of hardcoding the location of perl (assuming it is installed in
/usr), use /usr/bin/env to run it, and thus picking it from $PATH.
This makes it possible to run these scripts also on installations with
perl in a different prefix than /usr.
Also, given that we want enable warnings on scripts, turn the -w
previously in shebang to explicit "use warnings;" in scripts which
didn't have it before.
All tests run under the ./run binary. For a long time the ./run
binary has set the $PATH environment variable to contain all of the
directories with binaries in them.
Therefore there is no reason to use ../fish/guestfish instead of just
plain guestfish (and the same applies to other built binaries).
This fixes the handling of paths for the protocols named above, with
respect to leading '/' (or not) on the volume/export name.
See previous commits which did the same fixes for Ceph:
commit 53a3ff9c00
commit 992a6b2970
This tests that libvirt domain XML (ie. guestfish -d option) is parsed
correctly and results in the correct qemu command line when using the
direct backend.
This is a good end-to-end test of various layers.
The path at the protocol level is:
pool/disk
(with no leading '/' character). This is now what you have to pass to
guestfs_add_drive_opts.
Also Ceph can be called with no explicit servers (it uses the contents
of /etc/ceph/ceph.conf instead). So allow zero servers to be used.
Although this test only specifically tests the direct backend, the
same code is shared with the libvirt backend in some circumstances
(eg. creating readonly drives). So this is testing a bit of both.
Replaces code such as:
fd = open "test1.img"
ftruncate fd, size
close fd
g.add_drive "test1.img"
with the shorter and simpler:
g.add_drive_scratch size
By using the once_had_no_optargs flag, this change is backwards
compatible for callers (except Haskell, PHP and GObject as discussed
in earlier commit).
The new API splits orderly close into a two-step process:
if (guestfs_shutdown (g) == -1) {
/* handle the error, eg. qemu error */
}
guestfs_close (g);
Note that the explicit shutdown step is only necessary in the case
where you have made changes to the disk image and want to handle write
errors. Read the documentation for further information.
This change also:
- deprecates guestfs_kill_subprocess
- turns guestfs_kill_subprocess into the same as guestfs_shutdown
- changes guestfish and other tools to call shutdown + close
where necessary (not for read-only tools)
- updates documentation
- updates examples
MALLOC_PERTURB_ is a glibc feature which causes malloc to wipe memory
before and after it is used, allowing both use-after-free and
uninitialized reads to be detected with relatively little performance
penalty:
http://udrepper.livejournal.com/11429.html?nojs=1
Modify the ./run script so that it always sets this.
We were already using MALLOC_PERTURB_ in most tests. Since ./run is
now setting this, we can remove it from individual Makefiles. Most
TESTS_ENVIRONMENT will now simply look like this:
TESTS_ENVIRONMENT = $(top_builddir)/run --test
This option, when added via
TESTS_ENVIRONMENT = [...] $(top_builddir)/run --test
allows us to run the tests and only print the full output (including
debugging etc) when the test fails.
