Apply this change across all the shell scripts containing tests.
Additionally this defines the environment variables $abs_srcdir,
$abs_builddir, $top_srcdir, $top_builddir, $abs_top_srcdir and
$abs_top_builddir which can now be used throughout test scripts.
This macro(?) expands to some shell script to source the
tests/test-functions.sh file from its correct location. The intention
is to use this in all tests, but in this commit only the existing
tests which already include test-functions.sh are modified.
Daemon 'proc_nr's have to be assigned monotonically and uniquely to
each daemon function. However in practice it can be difficult to work
out which is the next free proc_nr. Placing all of them into a single
table in a new file (proc_nr.ml) should make this easier.
Group the APIs logically and move them into new modules:
Actions_core:
Core APIs and anything that doesn't fit into another group, eg. launch.
(With some more effort this could be split further.)
Actions_augeas:
Augeas APIs, eg. aug-init.
Actions_debug:
Debug APIs.
Actions_hivex:
Hivex APIs, eg. hivex-open.
Actions_inspection:
Inspection APIs, eg. inspect-get-type.
Actions_properties:
Handle properties, eg. set-hv, get-hv.
Actions_tsk:
SleuthKit APIs, eg. filesystem-walk.
*_deprecated:
All of the above modules have deprecated variants, where we
place the deprecated actions.
Sort the functions so the output is stable.
This changes the order in which the C API tests run. Previously we
ran the newest tests first, which was useful when we were frequently
adding new APIs. Now we run them in sorted order.
Also rename the function 'uuidgen ()' (conflicts with a same-named
function in Common_utils) to 'stable_uuid'. Notice that the UUID is
now only computed once per run of the generator, whereas previously
the same value was computed over and over again.
Sadly, there are elements (hello "gentoo"!) that, during the cleanup.d
phase, wipe out almost everything in the /tmp of the guest, including
the /tmp/aux where virt-dib mounts the auxiliary data. Since that
removal excludes things starting with "in_target" (mostly to avoid
wiping the "in_target.d" that disk-image-create itself sets up, then
rename our "aux" subdirectory to "in_target.aux" to work it around.
Hopefully, the "gentoo" element will be fixed upstream:
https://review.openstack.org/#/c/436101/
Commit 8ee51ee396 re-scans for the
available scripts when running certain phases; OTOH, some of them may be
missing, usually due to the lack of scripts for that phase in the
selected set of elements.
Indeed, if there is no directory for a phase, safely raise Not_found so
the case is handled as if the phase was missing in the final_hooks
Hashtbl.
The real sudo does it as well, and leaving them when preserving the
environment (-E) maybe breaks the applications, as e.g. chroot will have
a TMPDIR path pointing outside of it.
The inspection code already computed the paths of the software hive
(twice!) and the system hive, plus we also recompute the same paths
elsewhere, in virt-v2v for example. Therefore it makes sense to store
the paths from the inspection code and make them available through two
new APIs.
Add a convenient tuple Registry.t for the currently open hive. It
contains the guestfs handle and the root node of a registry.
The functions with_hive_readonly and with_hive_write are modified to
pass this tuple to their callbacks.
For reasons unknown virt-v2v recomputed the CurrentControlSet from
first principles, and passed %systemroot% around to all functions.
However that data is available from the libguestfs inspection APIs.
Move the functions decode_utf16le, encode_utf16le, get_node,
with_hive_readonly and with_hive_write to common code in a new module
called Registry.
This also defines types for nodes and values, instead of using int64
directly.
Just code motion.
Adding a new optional parameter to hivex_open in
commit 1f99251223 reveals a subtle bug
in the virt-p2v slow test.
Because we didn't pass LIBGUESTFS_PATH through to the instance of
virt-v2v, it was running with the new binary and library code, but
with the installed appliance (or would have failed if libguestfs
wasn't installed on the host when running the test). In particular
this bug was revealed when the new virt-v2v binary passed the
GUESTFS_HIVEX_OPEN_UNSAFE flag to the guestfs_hivex_open call, which
the (old, installed) daemon did not recognize and rejected with an
error.
For the same reason we also have to pass in LIBGUESTFS_CACHEDIR to
make sure that supermin doesn't reuse the cached appliance from
/var/tmp.
When extracting the content of the guest as tar, save also the extended
attributes and the SELinux attributes. This makes sure guests exported
as tar, tgz, and as docker image will work fine afterwards.
This is what disk-image-create does as well.
Pass the HIVEX_OPEN_UNSAFE flag when opening hives for reading.
Do NOT pass it when opening hives for writing.
This should make inspection, virt-win-reg and virt-v2v more tolerant
about handling Windows Registry corruption, without increasing the
risk of causing new corruption in hives.
In hivex >= 1.3.14, there is a new HIVEX_OPEN_UNSAFE flag allowing
heuristics to be used to deal with corrupted hives. Map this flag
into the libguestfs API.
If the flag is not supported (because libguestfs was compiled with
hivex < 1.3.14) then the flag is ignored. This is safe behaviour:
opening corrupted hives will give an error, as happened previously.
Because v2v/test-harness is a subdirectory of v2v, and because both
paths are listed in $(DIST_SUBDIRS), using find $(DIST_SUBDIRS) will
list files in v2v/test-harness twice. (This probably happens in other
directories too, but I noticed it here.) The easiest fix for this is
simply to use 'sort -u' to remove the duplicates.
Implement the "squash" output format, i.e. a squashfs compressed
filesystem.
This was implemented in diskimage-builder upstream as
commit 9d13084c4183b63587e1f5e4b03395a8df6538f6.
Introduce a new API to create a new squashfs filesystem out of a path
in the guest. It can be configured to exclude paths based on patterns,
and to select which compression use for the filesystem.
The advantage of running mksquashfs directly in the appliance is that
ownerships are properly saved, as opposed to tar_out + local untar.
It will be useful also for APIs different than tar-out, so move it to
guestfsd.c, and add it a parameter to specify the function name that
invoked it.
This is mostly code motion.
Recently, diskimage-builder moved the log cleanup from the 'base'
element to disk-image-create proper; the cleanup done is:
- truncate any file in /var/log
- remove *.log files in /root
This was implemented in diskimage-builder upstream as
commit 022d93ee822e71245af52c4cf8f8a8e82f599af3.
Scripts (especially root.d and extra-data.d) may add new scripts to the
existing set during their execution: in this case, re-read the list of
available scripts, so we do not miss any new addition.
It looks like scripts (in particular in the extra-data.d phase) may
in-flight add new scripts to the existing ones. As first step to
handle this situation, change the path of hooks passed for extra-data.d
scripts: instead of the local temporary directory, use the in-guest
location that is visible as result of the local fuse-based mount of the
guest.
Source the "die" script, part of the diskimage-builder library, when
running hooks that run outside the guest chroot. This is what
disk-image-create does, and scripts expect to use the "die" function
without sourcing the "die" script containing it.
Query awk for the list of environment variables, instead of trying to
extract the list from the output of `env`: the old approach breaks when
any of the environment variable contains more than one line.
GCC 7.0.1 can determine if there is likely to be sufficient space in
the output buffer when using sprintf/snprintf, based on the format
string.
The errors were all either of this form:
bindtests.c:717:29: error: '%zu' directive output may be truncated writing between 1 and 19 bytes into a region of size 16 [-Werror=format-truncation=]
snprintf (strs[i], 16, "%zu", i);
^~~
bindtests.c:717:28: note: directive argument in the range [0, 2305843009213693951]
snprintf (strs[i], 16, "%zu", i);
^~~~~
or this form:
sync.c: In function 'fsync_devices':
sync.c:108:50: error: '%s' directive output may be truncated writing up to 255 bytes into a region of size 251 [-Werror=format-truncation=]
snprintf (dev_path, sizeof dev_path, "/dev/%s", d->d_name);
^~
Fixed by converting these into dynamic allocation, or making the
output buffer larger, whichever was easier.
There is a gnulib macro we can use to make this simpler for integers.
It requires a new gnulib module (intprops), but it turns out that we
were already pulling that in through dependencies, so the change to
bootstrap is a no-op. (thanks: Dan Berrange)
This happens with GCC 7.0.1. The errors were all of the form:
qemu-speed-test.c: In function 'main':
qemu-speed-test.c:153:7: error: this statement may fall through [-Werror=implicit-fallthrough=]
usage (EXIT_SUCCESS);
^~~~~~~~~~~~~~~~~~~~
qemu-speed-test.c:155:5: note: here
default:
^~~~~~~
