Linux + LVM supports device names like /dev/disk/by-id/dm-uuid-LVM-
followed by two concatenated UUIDs, firstly for the volume group and
secondly for the logical volume. We can reverse those to get the
device name (/dev/VG/LV).
fstab entries look like:
# / was on /dev/vg0/lv-0 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YOOycQ533186x288FdU6UubU3OlnWJz6D / ext4 defaults 0 1
# /usr was on /dev/vg0/lv-1 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YZu53m4ZssZ8Jeb3I14RAJwIj5YlHIb9P /usr ext4 defaults 0 1
The upshot of this fix is that we are now able to correctly inspect
and run virt-v2v on Ubuntu 22+ guests with split /usr. In particular,
we correctly map /etc/fstab entries like the above to LV device names,
which means that /usr merging now works correctly.
Reported-by: Jaroslav Spanko
Thanks: Daniel Berrange
Fixes: https://issues.redhat.com/browse/RHEL-87493
(cherry picked from commit e43ca19129)
(cherry picked from commit 180293338e)
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
(cherry picked from commit a73f248369)
(cherry picked from commit 80b2fcb243)
This was implemented wrongly. In the XDR protocol, UUIDs are fixed
buffers of length 32. We can just use memcpy to copy from the OCaml
string to the UUID, but we have to ensure the string length returned
by OCaml is correct (if not we just assert, it's an internal error).
(It didn't even compile before, so we know it was never used).
(cherry picked from commit bcd6b3ec3a)
(cherry picked from commit 1b64c54b8a)
Ubuntu 22= uses /dev/disk/by-uuid/ followed by a filesystem UUID in
fstab entries. Resolve these to mountables.
A typical fstab entry looks like this:
# /boot was on /dev/vda2 during curtin installation
/dev/disk/by-uuid/b4e56462-5a64-4272-b76d-f5e58bd8f128 /boot ext4 defaults 0 1
The comment is generated by the installer and appears in the fstab.
This entry would be translated to /dev/sda2.
(cherry picked from commit 7a1ffd744b)
(cherry picked from commit 0d466cb25e)
Remove the code which did explicit reverse device name translation,
and use the generator's code instead.
(cherry picked from commit 5a16d1120f)
(cherry picked from commit b306532e7a)
This acts just like FString except that we do reverse device name
translation on it. The only use is in the 'pvs-full' API where we
will use it (in a subsequent commit) to reverse translate the pv_name
field (a device name) before returning it from the daemon.
Compare this to the 'pvs' API which also returns a list of device
names, but using the generator's 'RStructList (RDevice,...)' return
type, where RDevice is similarly reverse translated.
Note in the library-side bindings, because the name has already been
translated in the daemon, we just treat it exactly the same as
FString. The vast majority of this patch is this mechanical change.
(cherry picked from commit 0ff73a42c7)
(cherry picked from commit 4e27b259c1)
Useful for debugging problems caused by the host kernel. In
particular we were looking at a problem with passt creating a user
namespace but didn't know what exact kernel was being used.
(cherry picked from commit 31fa712aa0)
(cherry picked from commit 19c4d1c8b9)
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
(cherry picked from commit 47ac4871b2)
(cherry picked from commit 42ae34115f)
The previous code split it on ',' which was completely wrong.
(It reveals the lack of testing however).
Fixes: commit c08032ebe2
Reported-by: Yongkui Guo
(cherry picked from commit e9a728bb22)
The list of patches is below. The one which matters for guestfish is
addition of --key all:... selector.
Andrey Drobyshev (1):
mldrivers: look for bootloader config in /boot/grub/grub.cfg in case of UEFI
Richard W.M. Jones (5):
mlxml: Include <libxml/parser.h> for xmlReadMemory
options/keys.c: Rewrite confusing match statement
options: Rewrite --key documentation fragment
options: Allow --key all:SELECTOR to be used to match any device
mltools/libosinfo-c.c: Fix off-by-one error
Fixes: https://issues.redhat.com/browse/RHEL-19367
(cherry picked from commit 7fd41b5a02)
Since OCaml 5.1.1, changes to custom blocks caused C finalizers that
call caml_enter_blocking_section to stop working (if they ever did
before). They are relatively inflexible compared to registering an
OCaml finalizer (Gc.finalise) to call Guestfs.close, so use that
instead.
Suggested-by: Guillaume Munch-Maccagnoni
See: https://github.com/ocaml/ocaml/issues/12820
See: db48794fa8
(cherry picked from commit 61418535ad)
Using 'virt-customize --tar-in some.tar:/dir -a disk.img' will unpack
'some.tar' into '/dir' in the guest. Note that this will not work for
compressed tar files as written since the underlying guestfs_tar_in
function requires the compression type to be set explicitly and
defaults to no compression (it does not auto-detect or default to
compression).
(cherry picked from commit b5f7b0ec18)
Intermittent test failures in virt-filesystems showed that when using
the pvs_full API, the pv_name field in the returned list of structures
was not being reverse translated. As a result internal partition
names could appear in the output of virt-filesystems.
See: https://listman.redhat.com/archives/libguestfs/2023-July/032058.html
(cherry picked from commit 32cb5b45cf)
The just added --chown option previously used StringPair, splitting
the argument as ‘UID.GID:FILENAME’. However this will not work if we
ever extend this with the ability to use user or group names, since
they may contain dot (but not colon). Add a new StringTriplet type
and split the argument string three ways. The new option becomes:
virt-customize ... --chown UID:GID:FILENAME
Include the following commit from the common submodule:
commit e70d89a58dae068be2e19c7c21558707261af96a
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Sat Jul 15 16:42:06 2023 +0100
customize: Update generated files for --chown with StringTriplet
Updates: commit d8e48bff21
(cherry picked from commit c08032ebe2)
Consider the following inverted call tree (effectively a dependency tree
-- callees are at the top and near the left margin):
lazy_make_tmpdir() [lib/tmpdirs.c]
guestfs_int_lazy_make_tmpdir() [lib/tmpdirs.c]
guestfs_int_make_temp_path() [lib/tmpdirs.c]
guestfs_int_lazy_make_sockdir() [lib/tmpdirs.c]
guestfs_int_create_socketname() [lib/launch.c]
lazy_make_tmpdir() is our common workhorse / helper function that
centralizes the mkdtemp() function call.
guestfs_int_lazy_make_tmpdir() and guestfs_int_lazy_make_sockdir() are the
next level functions, both calling lazy_make_tmpdir(), just feeding it
different dirname generator functions, and different "is_runtime_dir"
qualifications. These functions create temp dirs for various, more
specific, purposes (see the manual and "lib/guestfs-internal.h" for more
details).
On a yet higher level are guestfs_int_make_temp_path() and
guestfs_int_create_socketname() -- they serve for creating *entries* in
those specific temp directories.
The discrepancy here is that, although all the other functions live in
"lib/tmpdirs.c", guestfs_int_create_socketname() is defined in
"lib/launch.c". That makes for a confusing code reading; move the function
to "lib/tmpdirs.c", just below its sibling function
guestfs_int_make_temp_path().
While at it, correct the leading comment on
guestfs_int_create_socketname() -- the socket pathname is created in the
socket directory, not in the temporary directory.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20230714132213.96616-6-lersek@redhat.com>
(cherry picked from commit 0b2ad40a09)
The last (only?) caller of guestfs_int_cmd_clear_close_files() disappeared
in commit e4c3968880 ("lib/info: Remove /dev/fd hacking and pass a true
filename to qemu-img info.", 2018-01-23), part of v1.37.36.
Simplify the code by removing guestfs_int_cmd_clear_close_files().
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20230711113906.107340-1-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 13c7052ff9)
When finalizing the handle we call guestfs_close. This function could
be long-running (eg. it may have to shut down the qemu subprocess), so
release the runtime lock.
(cherry picked from commit 4a79c023e5)
Since OCaml 4 the old and confusing caml_enter_blocking_section and
caml_leave_blocking_section calls have been replaced with
caml_release_runtime_system and caml_acquire_runtime_system (in that
order). Use the new names.
(cherry picked from commit 7e1d7c1330)
Various errors like this:
In function ‘test_fuse’,
inlined from ‘main’ at test-fuse.c:133:11:
test-fuse.c:274:5: error: argument 1 null where non-null expected [-Werror=nonnull]
274 | fclose (fp);
| ^~~~~~~~~~~
In file included from test-fuse.c:26:
/usr/include/stdio.h: In function ‘main’:
/usr/include/stdio.h:183:12: note: in a call to function ‘fclose’ declared ‘nonnull’
183 | extern int fclose (FILE *__stream) __nonnull ((1));
| ^~~~~~
(cherry picked from commit ca20f27cb0)
The tool only supports an older version of the diskimage-builder
metadata, and we do not have the time or inclination to update it to a
newer version.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1910039
(cherry picked from commit 7503eeebed)
This disables support for unsupported remote drive protocols:
* ftp
* ftps
* http
* https
* tftp
* gluster
* iscsi
* sheepdog
* ssh
Note 'nbd' is not disabled, and of course 'file' works.
We hope to gradually add some of these back over the lifetime of RHEL.
Since SELinux userspace v3.4 [1], setfiles command supports "-T nthreads"
option, which allows parallel execution. "-T 0" allows using as many
threads as there're available CPU cores. This might speed up the process
of filesystem relabeling in case the appliance is being run with multiple
vCPUs. The latter is true for at least v2v starting from d2b64ecc67
("v2v: Set the number of vCPUs to same as host number of pCPUs.").
For instance, when running virt-v2v-in-place on my 12-core Xeon host
with SSD, with appliance being run with 8 vCPUs (the upper limit specified
in d2b64ecc67), and on the ~150GiB disk VM (physical size on the host),
I get the following results:
./in-place/virt-v2v-in-place -i libvirt fedora37-vm -v -x
Without this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 17.94 secs
...
With this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -T 0 -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 5.88 secs
...
So in my scenario it's getting 3 times faster.
[1] https://github.com/SELinuxProject/selinux/releases/tag/3.4
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit d0d8e67384)
'X' in the setiles' stderr doesn't necessarily mean that option 'X'
doesn't exist. For instance, when passing '-T' we get: "setfiles:
option requires an argument -- 'T'".
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 152d6e4bdf)
Since RHBZ#726528, filesystem.rpm doesn't include /selinux. setfiles
then gives us the warning: "Can't stat exclude path "/sysroot/selinux",
No such file or directory - ignoring."
Though the warning is harmless, let's get rid of it by checking the
existence of /selinux directory.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 9ced5fac8c)
