The old btrfs-fsck API used "btrfs check" which appears to be broken
or deprecated. The real tool you should use is "btrfs scrub". We
have already implemented that API, but it is very awkward to use from
libguestfs. In particular there's no existing way to run the scrub
and wait for it to finish.
Fix this by deprecating btrfs-fsck. Implement a new API
btrfs-scrub-full which runs btrfs scrub in the foreground, waits for
it to finish, and handles errors. It's much more like fsck tools in
other filesystems.
Thanks: Eric Sandeen
Fixes: https://issues.redhat.com/browse/RHEL-91936
This pulls in the commits below, requiring us to replace all uses of
String.is_prefix and String.is_suffix.
Mostly done with Perl like this, and carefully checked by hand
afterwards since this doesn't get everything right:
$ perl -pi.bak -e 's/String.is_prefix ([^[:space:]\)]+) ([^[:space:]\)]+)/String.starts_with \2 \1/g' -- `git ls-files`
Richard W.M. Jones (3):
mlstdutils: Fix comment that still referred to the old function names
mldrivers: Link to gettext-stub if ocaml-gettext is enabled
mlstdutils: Rename String.is_prefix -> starts_with, is_suffix -> ends_with
Pull in these commits which require minor changes:
Richard W.M. Jones (3):
mlstdutils: Remove Std_utils.identity
mlstdutils: Remove Std_utils.protect
mlstdutils: Remove List.filter_map
A customer case was found where /etc/fstab contained multiple root
mountpoints, something like:
LABEL=System / xfs ...
LABEL=Boot /boot ext2 ...
LABEL=System / xfs ...
This causes libguestfs and virt-v2v to fail. Either (on RHEL 9) we
try to mount the second instance of / which gives an error. Or (on
upstream kernels) we are able to mount the second instance but then
libguestfs gets confused when trying to unmount them.
In this case as the mounted devices are the same we can just delete
the duplicate. It's also possible that there could be multiple
non-identical root mountpoints, in which case we have to pick one, and
this code arbitrarily picks the first[*] (but emits a warning).
We don't do anything for non-root mountpoints.
Update common submodule to add 'List.same' function from mlstdutils.
[*] Which one is "the first" depends on what version of ocaml-augeas
we are using. ocaml-augeas version 0.6 Augeas.matches function
returns entries in reverse order (compared to augeas itself). This is
fixed in version 0.7:
http://git.annexia.org/?p=ocaml-augeas.git;a=commitdiff;h=b703b92e3d26690aa6f7b822132049ce5435983e
Fixes: https://issues.redhat.com/browse/RHEL-90168
Linux + LVM supports device names like /dev/disk/by-id/dm-uuid-LVM-
followed by two concatenated UUIDs, firstly for the volume group and
secondly for the logical volume. We can reverse those to get the
device name (/dev/VG/LV).
fstab entries look like:
# / was on /dev/vg0/lv-0 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YOOycQ533186x288FdU6UubU3OlnWJz6D / ext4 defaults 0 1
# /usr was on /dev/vg0/lv-1 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YZu53m4ZssZ8Jeb3I14RAJwIj5YlHIb9P /usr ext4 defaults 0 1
The upshot of this fix is that we are now able to correctly inspect
and run virt-v2v on Ubuntu 22+ guests with split /usr. In particular,
we correctly map /etc/fstab entries like the above to LV device names,
which means that /usr merging now works correctly.
Reported-by: Jaroslav Spanko
Thanks: Daniel Berrange
Fixes: https://issues.redhat.com/browse/RHEL-87493
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
Ubuntu 22= uses /dev/disk/by-uuid/ followed by a filesystem UUID in
fstab entries. Resolve these to mountables.
A typical fstab entry looks like this:
# /boot was on /dev/vda2 during curtin installation
/dev/disk/by-uuid/b4e56462-5a64-4272-b76d-f5e58bd8f128 /boot ext4 defaults 0 1
The comment is generated by the installer and appears in the fstab.
This entry would be translated to /dev/sda2.
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
OCaml 4.08.0 was released on 2019-06-14, over 5 years ago. By
requiring a slightly later OCaml version, we can drop more
compatibility code which was only used by older versions.
Consistent with qemu & libvirt, this drops support for compiling
upstream libguestfs on RHEL 8 (ocaml-4.07.0-4.el8.x86_64).
Qemu policy:
https://www.qemu.org/docs/master/about/build-platforms.html
Libvirt policy:
https://libvirt.org/platforms.html
Update the common submodule, pulling in:
Richard W.M. Jones (4):
qemuopts: Add ability to add raw, unquoted output to qemu scripts
qemuopts: Fix missing break statement
mlstdutils: Remove Option module
Remove test for caml_alloc_initialized_string
All recent compilers support this (except MS compilers which we don't
care about). Assume it is supported. We test it in ./configure and
hard fail if it doesn't work.
We still define HAVE_ATTRIBUTE_CLEANUP but you can now assume it is
always defined and don't have to check it.
openEuler is simliar to CentOS, but the ID is not lower-case string,
as below:
NAME="openEuler"
VERSION="24.03 (LTS)"
ID="openEuler"
VERSION_ID="24.03"
PRETTY_NAME="openEuler 24.03 (LTS)"
ANSI_COLOR="0;31"
Signed-off-by: Wang Guoquan <wangguoquan03@foxmail.com>
The actual output of sfdisk --part-attrs is bizarre and doesn't match
the documentation. After looking at the source from util-linux, fix
the parsing to match what sfdisk produces.
Reported-by: Yongkui Guo
Fixes: commit c6c266a85d
Fixes: https://issues.redhat.com/browse/RHEL-35998
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
This "new" parameter was added in 2014:
commit 8eab3194ce1737a167812d5e84d83b0dfc253fac
Author: Karel Zak <kzak@redhat.com>
Date: Mon Sep 15 12:37:52 2014 +0200
sfdisk: add --parttype
The patch also makes --{id,change-id,print-id} deprecated in favour
of --parttype. The original --id is too generic option name and the
--print-id and --change-id are unnecessary and inconsistent with
another sfdisk options (e.g. we don't have --change-bootable)
Also remove an extraneous / incorrect comment about parted. As
history has played out, sfdisk proves to be the better tool and parted
is a PITA.
OCaml 5.1 changes the names of these libraries for some reason.
Also in OCaml 5.1, if using those libraries you must link with -lzstd.
Since zstd was already described as "required" (although we only used
it in the appliance), there is no official change to the requirements,
but I have added a configure time check for the library.
Thanks: Jerry James <loganjerry@gmail.com>
Intermittent test failures in virt-filesystems showed that when using
the pvs_full API, the pv_name field in the returned list of structures
was not being reverse translated. As a result internal partition
names could appear in the output of virt-filesystems.
See: https://listman.redhat.com/archives/libguestfs/2023-July/032058.html
Since OCaml 4.07 (released 2018-07-10) the always-loaded standard
library module has been called Stdlib. The old Pervasives module was
finally removed in OCaml 5.
$ perl -pi.bak -e 's/Pervasives\./Stdlib./g' -- `git ls-files`
OCaml >= 4.07 is now required.
Also update the common submodule with:
commit d61cd820b49e403848d15c5deaccbf8dd7045370
Author: Jürgen Hötzel
Date: Sat May 20 18:16:40 2023 +0200
Add support for OCaml 5.0
Since SELinux userspace v3.4 [1], setfiles command supports "-T nthreads"
option, which allows parallel execution. "-T 0" allows using as many
threads as there're available CPU cores. This might speed up the process
of filesystem relabeling in case the appliance is being run with multiple
vCPUs. The latter is true for at least v2v starting from d2b64ecc67
("v2v: Set the number of vCPUs to same as host number of pCPUs.").
For instance, when running virt-v2v-in-place on my 12-core Xeon host
with SSD, with appliance being run with 8 vCPUs (the upper limit specified
in d2b64ecc67), and on the ~150GiB disk VM (physical size on the host),
I get the following results:
./in-place/virt-v2v-in-place -i libvirt fedora37-vm -v -x
Without this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 17.94 secs
...
With this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -T 0 -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 5.88 secs
...
So in my scenario it's getting 3 times faster.
[1] https://github.com/SELinuxProject/selinux/releases/tag/3.4
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
'X' in the setiles' stderr doesn't necessarily mean that option 'X'
doesn't exist. For instance, when passing '-T' we get: "setfiles:
option requires an argument -- 'T'".
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Since RHBZ#726528, filesystem.rpm doesn't include /selinux. setfiles
then gives us the warning: "Can't stat exclude path "/sysroot/selinux",
No such file or directory - ignoring."
Though the warning is harmless, let's get rid of it by checking the
existence of /selinux directory.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
