Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
User-Mode Linux was an alternative hypervisor that could run the
appliance, instead of using qemu. It had many limitations including
lack of network, and UML support in Linux has been semi-broken for a
long time. It was also slower than KVM on baremeal in general and had
various corner cases which were much slower including the emulated
serial port which made bulk uploads and downloads painful. Also of
course it lacked qemu-specific features like qcow2 or any
network-backed disk, so many disk images could not be opened this way.
This was never supported in RHEL.
See-also: https://bugzilla.redhat.com/1144197
The "test-md-and-lvm-devices" test case creates, among other things, a
RAID0 array (md127) that spans two *differently sized* block devices
(sda1: 20MB, lv0: 16MB).
In Linux v3.14, the layout of such arrays was changed incompatibly and
undetectably. If an array were created with a pre-v3.14 kernel and
assembled on a v3.14+ kernel, or vice versa, data could be corrupted.
In Linux v5.4, a mitigation was added, requiring the user to specify the
layout version of such RAID0 arrays explicitly, as a module parameter. If
the user fails to specify a layout version, the v5.4+ kernel refuses to
assemble such arrays. This is why "test-md-and-lvm-devices" currently
fails, with any v5.4+ appliance kernel.
Until we implement a more general solution (see the bugzilla link below),
work around the issue by sizing sda1 and lv0 identically. For this,
increase the size of sdb1 to 24MB: when one 4MB extent is spent on LVM
metadata, the resultant lv0 size (20MB) will precisely match the size of
sda1.
This workaround only affects sizes, and does not interfere with the
original purpose of this test case, which is to test various *stackings*
between disk partitions, software RAID (md), and LVM logical volumes.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=2005485
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: remove stray empty line]
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
Commit c11a92751e chose some strangely
sized partitions for testing Linux MD devices. The disks were 100M
however the four partitions only covered the first 10M of disk space
(with the rest being unallocated and unused in the test).
This worked until Linux 4.20 where the ~2M-sized partitions become too
small for a Linux MD device.
This commit changes the test to use four 20M-sized partitions which is
sufficient space to create the MD device with recent kernels. I also
modified the partition allocation code to use explicit calculations
and variables, making it considerably clearer.
Test guestfish list-filesystems command finds file system on partitioned md device and doesn't take into account md device itself (similar to as physical devices are filtered out if they are partitioned).
Test guestfish finds:
1. md device created from physical block device and LV,
2. md device created from LVs
3. LV created on md device
raid0 is used for md device because it is inoperable if one of its components is inaccessible so it is easy observable that md device is missing (raid1 in this case will be operable but in degraded state).
mdadm 4.0 no longer lets you create devices called literally "boot" or
"root", giving a very obscure error message:
https://bugzilla.redhat.com/show_bug.cgi?id=1433575
Work around this by calling them something else.
Apply this change across all the shell scripts containing tests.
Additionally this defines the environment variables $abs_srcdir,
$abs_builddir, $top_srcdir, $top_builddir, $abs_top_srcdir and
$abs_top_builddir which can now be used throughout test scripts.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
When possible, make the disk image format explicit when invoking tools
or using add-drive. This avoids warnings from qemu about the unspecified
format for the image, and also makes qemu slightly faster (skipping the
disk image probing).
Tests checking the image probing are not touched.
This changes also:
- old-style invocations of tools (`$tool $filename`) into new style
(`$tool -a $filename`)
- add-drive-ro/add-drive-with-if guestfish commands into add/add-drive
with explicit readonly/iface arguments
There should be no change in the tests results.
Create a new top-level directory called test-data, which will carry
all the test data which is large and/or shared between multiple tests.
There are actually several new subdirectories created:
test-data/binaries: The pre-built binary and library files for random
architectures that we use to test various architecture detection
features (was part of tests/data).
test-data/blank-disks: The blank disks which are used for disk format
detection (was part of tests/data).
test-data/files: Other miscellaneous test files from tests/data that
are not included in the above.
test-data/phony-guests: The phony guests (was tests/guests).
test-data: The top-level directory builds the 'test.iso' image file
that is used for testing the C API and in miscellaneous other tests.
All tests run under the ./run binary. For a long time the ./run
binary has set the $PATH environment variable to contain all of the
directories with binaries in them.
Therefore there is no reason to use ../fish/guestfish instead of just
plain guestfish (and the same applies to other built binaries).
Review every test(!) to ensure that it:
- Doesn't use a generic name (eg. "test1.img", "test.out") for any
temporary file it needs.
- Does instead use a unique name or a temporary name (eg. a name like
"name-of-the-test.img", or a scratch disk).
- Does not use 'rm -f' to clean up its temporary files (so we can
detect errors if the wrong temporary file is created or removed).
This allows tests to be run in parallel, so they don't stomp on each
other's temporary files.
Since our minimum supported version is now 1.16 and mount was fixed in
1.13.16, it is now safe to replace mount-options + empty options with
mount wherever it occurs.
By using the once_had_no_optargs flag, this change is backwards
compatible for callers (except Haskell, PHP and GObject as discussed
in earlier commit).
Replace:
cp tests/guests/fedora.img test.img
with the longer but possibly more space-efficient equivalent:
qemu-img create -F raw -b tests/guests/fedora.img -f qcow2 test.qcow2
MALLOC_PERTURB_ is a glibc feature which causes malloc to wipe memory
before and after it is used, allowing both use-after-free and
uninitialized reads to be detected with relatively little performance
penalty:
http://udrepper.livejournal.com/11429.html?nojs=1
Modify the ./run script so that it always sets this.
We were already using MALLOC_PERTURB_ in most tests. Since ./run is
now setting this, we can remove it from individual Makefiles. Most
TESTS_ENVIRONMENT will now simply look like this:
TESTS_ENVIRONMENT = $(top_builddir)/run --test
This option, when added via
TESTS_ENVIRONMENT = [...] $(top_builddir)/run --test
allows us to run the tests and only print the full output (including
debugging etc) when the test fails.
I noticed some uses of ${srcdir} in shell scripts.
That is almost always better written as $srcdir.
The patch below converts most such variable references.
Here are the few remaining candidates:
$ git grep -i -E '\$\{[a-zA-Z_0-9]+\}'|grep -v Makefile.in.in
configure.ac: JAR_INSTALL_DIR=\${prefix}/share/java
configure.ac: JNI_INSTALL_DIR=\${libdir}
debian/rules: for TEST in ${DEBIAN_SKIP_TEST}; do \
debian/rules:# mv $${mod} $$(dirname $${mod})/libguestfsmod.so; \
java/Makefile.am:libguestfs_jar_DATA = libguestfs-${VERSION}.jar
java/Makefile.am:libguestfs-${VERSION}.jar: $(libguestfs_jar_class_files)
perl/lib/Sys/Guestfs/Lib.pm: "-f", '${Package} ${Version} ${Architecture} ${Status}\n',
perl/typemap: croak (\"${Package}::$func_name(): called on a closed handle\");
perl/typemap: croak (\"${Package}::$func_name(): $var is not a blessed HV reference\");
tests/data/Makefile.am: echo "$${i}abcdefghijklmnopqrstuvwxyz"; \
We could change all of those, too, except the ones in configure.ac
and Makefile.am, since they refer to Make variables. Even those
should be changed, but to use the preferred Makefile notation:
$(prefix), $(libdir), $(VERSION).
>From a86770ecd45666232a94d76c8725c8f9b1c76e3a Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 23 Jan 2012 11:15:12 +0100
Subject: [PATCH libguestfs] maint: use $var notation rather than ${var} when
possible
The only case to avoid in a shell script is when the byte after the
"}" is word-constituent, and concatenating it would thus change the
name of the variable.
These changes were induced by running this command:
git grep -l -i -E '\$\{(srcdir|md)' \
|xargs perl -pi -e 's/\$\{(srcdir|md)\}($|\w)/\$$1$2/gi'
The "g" was needed because there was one line with two instances.
The "i" is to handle ${SRCDIR}. The ($|\w) ensures that concatenating
whatever follows the "}" won't change semantics.
* gobject/run-bindtests: Use "$srcdir", not "${srcdir}".
* haskell/run-bindtests: Likewise.
* java/run-bindtests: Likewise.
* ocaml/run-bindtests: Likewise.
* perl/run-bindtests: Likewise.
* python/run-bindtests: Likewise.
* ruby/run-bindtests: Likewise.
* tests/guests/guest-aux/make-debian-img.sh: Likewise, but $SRCDIR.
* tests/guests/guest-aux/make-ubuntu-img.sh: Likewise.
* tests/guests/guest-aux/make-windows-img.sh: Likewise.
* tests/md/test-mdadm.sh: Likewise, but $md.
