Commit f7a24b2ea8 ("lib/qemu.c: Use machine type none when inspecting
QMP properties") changed the number of command line parameters used
before the '-qmp stdio' option when libguestfs queries qemu features.
This broke some tests which rely on the exact order of parameters.
Fixes: commit f7a24b2ea8
Updates: commit 5da8102f5f
Reimplement xfs_info by returning a hash table of values (rather than
a limited struct), and by writing it in OCaml with PCRE which makes
string parsing a lot simpler. This will now flexibly return all the
fields from the underlying xfs_info command, even (hopefully) future
fields.
Note the field values are returned as strings, because the actual
fields in xfs_info output are fairly random and free-form. There is a
trade off here between returning as much information as we can, and
requiring the user to do a bit of (simple) field parsing.
Fixes: https://issues.redhat.com/browse/RHEL-143673
Dan mentioned that there is a special machine type ("none") we can use
when querying for KVM. It has no CPU, memory, etc and does not run,
but you can still enable KVM for it.
Note we have to remove the -cpu parameter, otherwise qemu prints this
error:
qemu-kvm: apic-id property was not initialized properly
Updates: commit 5da8102f5f
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
/usr/bin/ld: /tmp/ccvSGq6E.ltrans7.ltrans.o:(.data.rel.ro+0x1f8): undefined reference to `optgroup_selinuxrelabel_available'
The reason is that we didn't include optgroup_selinuxrelabel_available
on the fallback / no libselinux code path.
Reported-by: David Runge
Thanks: Toolybird
Fixes: https://github.com/libguestfs/libguestfs/issues/290
Fixes: commit ed40333a23
Add a new optional boolean argument 'keepdirlink' to tar_in that passes
--keep-directory-symlink to tar. This preserves existing symlinks to
directories when extracting, which is important for usrmerge systems
where /lib is a symlink to /usr/lib.
Without this option, extracting a tarball containing lib/modules/...
to / would replace the /lib symlink with a real directory, breaking
the system.
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
When checking for QMP properties, we run a qemu command and interact
with the QMP console. However we also start the guest running (there
is no actual guest in this situation). This occasionally causes this
line to be printed:
libguestfs: generic_qmp_test: 3: {"timestamp": {"seconds": 1768823946, "microseconds": 898287}, "event": "GUEST_PANICKED", "data": {"action": "poweroff", "info": {"core": 0, "psw-addr": 0, "reason": "disabled-wait", "psw-mask": 562956395872256, "type": "s390"}}}\r\n
which confuses our parser.
As there is no reason to start the non-existent guest, add the -S
option which causes qemu to start up in a paused state.
For some reason this only happens on s390x but I think it could happen
on all architectures, so it may be a timing issue or something
particular about s390x firmware.
Commit 669eda1e24 ("lib/launch-direct.c: Simplify test for KVM, remove
qemu caching") made it so that failure of generic_qmp_test() will
cause launch to fail. However we still used debug messages to print
the error, so unless you have debugging enabled you wouldn't see any
error message if this function fails.
Updates: commit 669eda1e24
Devices associated with btrfs volumes are not reverse-translated
(e.g., btrfsvol:/dev/sdX to sdY).
Forward translation occurs, creating a path mismatch. This causes
errors in subsequent btrfs commands.
Thanks: Arye Yurkovsky
lvm returns logical volumes even if they're broken, for instance when a
physical volume is missing in their volume group.
In such cases, stat would fail to resolve the provided path.
Handle such cases by skipping such failures when finding the matching
lvm device.
In C23, strchr or strrchr on a const parameter now returns a const
pointer. We saw this error:
destpaths.c: In function ‘complete_dest_paths_generator’:
destpaths.c:165:9: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
165 | p = strrchr (text, '/');
| ^
The existing code had a bug which you can demonstrate by doing:
$ guestfish -N fs:btrfs:10G -m /dev/sda1 \
btrfs-subvolume-create /sub :
btrfs-subvolume-snapshot /sub /snap1 : \
btrfs-subvolume-snapshot /sub /snap123 : \
btrfs-subvolume-snapshot /sub /snap123456 : \
btrfs-subvolume-show /sub
...
libguestfs: error: appliance closed the connection unexpectedly.
This usually means the libguestfs appliance crashed.
As the code for parsing the output and creating the comma-separated
list of snapshots was unncessarily complicated in the first place,
simplify it. This also fixes the bug.
This also adds a regression test.
Thanks: Arye Yurkovsky
Link: https://lists.libguestfs.org/archives/list/guestfs@lists.libguestfs.org/thread/QV5VDHIH7WRUNAE54K6OEOKJMWL6M7EM/
Python is broken and requires that we include <Python.h> before all
other headers so it can make inadvisable definitions of
_POSIX_C_SOURCE and other things. This wasn't a problem before, but a
recent change to glibc makes this necessary now.
See also a similar commit in nbdkit:
f924c3c34b
I also removed the -Wcast-align suppression as that is no longer
needed.
It tests a corner case that we don't experience in real life (although
it can happen). Moreover the test often hangs, so we're not proving
anything except that there's a very obscure bug that we haven't fixed
and probably won't ever fix.
Part of this test uses `qemu-nbd -k SOCKPATH`, and passes
that socket to libvirt. SOCKPATH will have label user_home_t,
which svirt_t can't access, so running with selinux enforcing this
test will always fail.
Manually change the socket label to svirt_image_t which makes
selinux happy.
Note: libvirt does not relabel most (all?) externally managed socket
paths. I think this is correct, since in theory this could be a
socket shared with other processes, so its up to the user to ensure
DAC/MAC permissions are acceptable for their usecase.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
really what we need is sgdisk in the appliance, but for test suite
purposes this check is enough IMO
Signed-off-by: Cole Robinson <crobinso@redhat.com>
To match other non-libtool tests.
Before: PASS: /path/to/libguestfs.git/tests/c-api/.libs/test-config
After: PASS: c-api/.libs/test-config
Signed-off-by: Cole Robinson <crobinso@redhat.com>
When iterating on commit 3c1554e7f2 ("lib: Add new app2_class field
for classifying applications") in early versions I was accidentally
passing NULL to match(). Unexpectedly this returned 1 (ie. matching).
This happened because we did not correctly handle other errors apart
from PCRE2_ERROR_NOMATCH from pcre2_match.
Emit a debug message when this happens, and return no match.
To test this I ran these commands, and there was no output:
$ make && LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 make check
$ find -name '*.log' | xargs grep 'match.*unexpected error'
Windows group policy objects (GPOs) are restrictions that can be added
by an administrator to Windows to lock down various operations. From
our point of view the ones that matter involve restricting the ability
to inject device drivers.
Previously virt-v2v detected group policy here:
9bb2e7d470/convert/convert_windows.ml (L69)
We would like to report group policy through the libguestfs API and
tools such as virt-inspector, so move the code that is used to detect
group policy to libguestfs. A new API is introduced that returns
whether group policy was found (only for Windows guests) during
inspection of the software registry.
Fixes: https://issues.redhat.com/browse/RHEL-125846
For McAfee VirusScan, the name is bogus and the display name is the
actual name:
<application>
<name>{CE15D1B6-19B6-4D4D-8F43-CF5D2C3356FF}</name>
<display_name>McAfee VirusScan Enterprise</display_name>
<version>8.8.04001</version>
<install_path>C:Program Files (x86)McAfeeVirusScan Enterprise\</install_path>
<publisher>McAfee, Inc.</publisher>
<url>http://www.mcafeesecurity.com/</url>
</application>
Simplify the existing ad hoc code so we just match name, display name
and publisher (if present) against all the virus scanning strings.
Existing virt-v2v code uses some simple heuristics for detecting
Windows anti-virus software:
7520185504/convert/windows.ml
Replicate exactly this code as a new field in the struct returned by
guestfs_inspect_get_applications2. Because of limitations with the
API, we must use one of the existing spare fields in the struct, and
it must have the same type (a string), so we are limited in the design
of this new API. I chose to return a primary classification for the
application, with the only classification possible so far being
"antivirus" (or "" if not). This allows the possibility of future
expansion of use of this field if we need to in future.
Fixes: https://issues.redhat.com/browse/RHEL-125846
This is currently broken, and gobject bits are slated to be
removed entirely, so just disable this test.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Convert to TypedData_Get_Struct which has been in ruby since 2009
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
It was added in d4a1c3a778 for gnulib, but since then gnulib
is no longer a submodule, and it does not seem to be required
for the build anylonger.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This reverts "m4: Add junk required to make 'AM_GNU_GETTEXT' work"
commit b9f75ca5b8
And replaces AM_GNU_GETTEXT call with a simpler LIBINTL check
(recommended by claude).
For linux builds on glibc it doesn't appear we need anything
from autotools here, since libintl.h is always available.
For some other cases like macos I think this new minimal check
covers us in the common case.
There are definitely more obscure corner cases that gettext
m4 magic covers but I suspect none of them really matter for
any platforms libguestfs already functions on.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This tests adding a running libvirt domain to libguestfs.
This was never really a safe real-world example AFAICT because it
creates qemu overlays on top of disks that are opened writeable
by the source guest.
Nowadays qemu uses write locks to reject this type of behavior,
and the test fails.
Adjust things so the source VM opens its storage readonly, but
we are still confirming that add_libvirt_dom doesn't mess up
selinux labels.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
If a user passes a custom hv, and uses add_libvirt_dom (like
in test820RHBZ912499.py) , this hits a bogus libvirt validation
check which rejects disabling svirt at both domain and disk level.
Work around it by skipping the redundant disk override when
svirt is disabled at the domain level.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
On fedora, ./configure will set QEMU=qemu-kvm, but libvirt
domcaps will default to qemu-system-x86_64. The former is a symlink
to the latter, but libguestfs doesn't know that, so determines
the user requested a non-default hv override and changes some
settings as a result (like disabling svirt).
Check for the symlink case and consider it a non-custom config
if realpath matches libvirt's default
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Add debugging about is_custom_hv result.
cache the initial call so we are only hitting the debug() once.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
libvirt has DAC relabeled sockets for us for a decade, so we don't
need to do chowning here anymore
The chmod calls are still required, otherwise our created sockets
may be too permissive.
Update the comment to try and preserve the still relevant info,
though now it's a bit awkwardly placed.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
https://bugzilla.redhat.com/2082806 is a private kernel bug (sigh) for
RHEL 8, where some missing backports caused 5-level page tables to
fail when qemu emulated them. This was fixed back in 2023, so let's
remove this workaround now.
On RHEL 10.2 aarch64 (only) we see:
$ echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-kvm" }' '{ "execute": "quit" }' | QEMU_AUDIO_DRV=none "/usr/libexec/qemu-kvm" -display none -machine "virt,accel=kvm:hvf:tcg" -qmp stdio
qemu-kvm: invalid accelerator hvf
qemu-kvm: falling back to KVM
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 1, "major": 9}, "package": "qemu-kvm-9.1.0-15.el10_0.4"}, "capabilities": ["oob"]}}
qemu-kvm: warning: CPU model cortex-a57-arm-cpu is deprecated -- use 'host' / 'max'
qemu-kvm: kvm_init_vcpu: kvm_arch_init_vcpu failed (0): Invalid argument
Unfortunately we cannot use guestfs_int_get_cpu_model (as that
requires us to already know if KVM is supported), so we just have to
guess that -cpu max will work, at least enough for QMP to work.
Fixes: https://issues.redhat.com/browse/RHEL-121076
Reported-by: Xiang Hua Chen
virt-customize and friends have the following options:
--sm-register
--sm-unregister
--sm-attach
--sm-remove
--sm-credentials
These run `subscription-manager` commands inside the VM, which
is often necessary to get working package install for RHEL VMs.
These were added in 2015ish. Conceptually I understand why we
might want native support in the tools: virt-customize and
virt-builder have package management knowledge, and
for some distros subscription-manager is necessary.
But this support doesn't add much convenience over calling the commands
directly with `--run-command`. And on RHEL10, subscription-manager was
reworked and complete drops the backing commands for `--sm-attach` and
`--sm-remove`.
We _could_ make this code smarter, try to detect that situation,
and not error when subscription-manager is new enough. But if
subscription-manager itself doesn't care about maintaining that kind
of compat, I'm not sure why we should care either.
Instead we decided to just rip it all out and document the
alternatives.
common module is updated to sync related changes:
Cole Robinson (1):
mlcustomize: deprecate and remove --sm-* options
Richard W.M. Jones (3):
mlstdutils: Export List.find_opt
daemon, generator: Use power of 2 for initial size of Hashtbl.create
mlcustomize/inject_virtio_win.ml: Use viostor.inf instead of guestor
https://issues.redhat.com/browse/RHEL-113397https://issues.redhat.com/browse/RHEL-113398
Signed-off-by: Cole Robinson <crobinso@redhat.com>
test-parallel-mount-local reliably fails when my machines are
under load (like running the test suite in parallel). The
issue is /usr/libexec/gvfs-udisks2-volume-monitor fiddling with
the fuse mounts, preventing them from being unmounted.
./fuse/guestunmount defaults to 5 unmount retries with increasing
time backoff, but apparently that isn't enough. Bumping it to
use 10 retries makes things more reliable for me.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
On macOS this fails with:
guestunmount.c:170:5: error: call to undeclared function 'sigaction'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
170 | sigaction (SIGINT, &sa, NULL);
| ^
1 error generated.
Reported-by: Mohamed Akram
Related: https://github.com/libguestfs/libguestfs/pull/188
macOS macfuse has an extra options parameter for the setxattr and
getxattr FUSE callbacks. The possible options are documented below.
However the underlying libguestfs APIs don't allow us to act on these
flags, so we ignore them for now.
(from https://manp.gs/mac/2/setxattr)
XATTR_NOFOLLOW
do not follow symbolic links. setxattr() normally sets attributes
on the target of path if it is a symbolic link. With this option,
setxattr() will act on the link itself.
XATTR_NOFOLLOW_ANY
do not follow any symbolic links encountered during pathname
resolution. An error is returned if a symlink is encountered
before the last component of path.
XATTR_CREATE
fail if the named attribute already exists.
XATTR_REPLACE
fail if the named attribute does not exist. Failure to specify
XATTR_REPLACE or XATTR_CREATE allows creation and replacement.
Reported-by: Mohamed Akram
Fixes: https://github.com/libguestfs/libguestfs/issues/180
Related: https://github.com/macfuse/macfuse/issues/1065
Previously we tested if KVM was available, and cached that for the
qemu binary. I think this was actually wrong. For example, if the
machine restarts, then the cache is still around, but KVM might be
enabled or disabled because of a new host kernel.
In any case, let's radically simplify this.
Test for KVM on each run. Consequently we can remove all the qemu
test caching stuff as it is no longer used anywhere.
I also tightened up the code that runs the QMP query-kvm command, so
now any unexpected output will cause a runtime failure. This command
ought to work, and if it breaks we ought to know about it and fix it.
Previously we ran 'qemu -device ?' to list devices, using the output
to test if qemu supported particular devices. This was extensively
used in the past, but after recent commits there are no further users
of this code, so we can remove it all completely.
It's always a good idea to provide proper entropy to the libguestfs
appliance (eg. for any cryptographic operations). We already assume
virtio, so we can assume virtio-rng is present.
We no longer use the libvirt version anywhere, except when reporting
the version. Remove this from the handle.
Simplify the remaining code. In particular:
* don't bother parsing the libvirt version, just print what
virGetVersion gives us
* guestfs_int_version_from_libvirt is dead code, so it can be removed
Libvirt 9.0.0 was released in January 2023, and it seems safe to
assume that if you're enabling the non-default backend, you can at
least use a new version of libvirt.
If you're using new libvirt, might as well also assume passt is
available.
Previously we parsed qemu -help output to get the version of qemu and
to test for other features. After prior commits, this is no longer
done, so we can remove the qemu -help invocation and associated code
completely.
This function is now only used in one place, to print the version of
qemu in direct mode, when debugging is enabled.
Remove this function and replace with a direct command invocation
('qemu --version'). We only need to run this command when debugging
is enabled, and we copy all of the output to the debug channel.
I have made the assumption here that qemu -version does not try to
create a display device. (The previous invocation of qemu -help
actually ran 'qemu -display none -help' indicating that this was not
always the case.)
This is actually an improvement on before, since now we get to see the
full output of 'qemu --version'. The new output looks like:
libguestfs: begin testing qemu features
libguestfs: command: run: /usr/bin/qemu-kvm
libguestfs: command: run: \ -version
libguestfs: qemu: QEMU emulator version 10.1.0 (qemu-10.1.0-8.fc44)
libguestfs: qemu: Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers
We already effectively assume that qemu is much newer than the 1.3.0
baseline previously documented. As one example, commit 47857751a7
("lib: direct: Remove test for qemu mandatory locking") assumes that
qemu >= 2.10. Because passt support is desirable in general, let's
assume that qemu is at least version 7.2.0, and document it.
qemu 7.2.0 was released in December 2022, nearly 3 years ago, and RHEL
9 is currently on qemu 9.1.0.
This keeps the SLIRP fallback path in case passt isn't installed, but
we should remove that fallback in future too.
We can safely assume that qemu supports -nodefaults and
-no-user-config, since these have been supported since forever.
-no-hpet was deprecated in qemu 8.0 and the option removed in early
2024, replaced with -machine hpet=off. HPET defaults to 'on' in
upstream qemu, and to 'off' in downstream RHEL rebuilds.
Since (for libguestfs) we can assume an up to date Linux kernel is
running inside the guest, and that the kernel will do the right thing
with regards to timers, we don't need to mess with qemu defaults. In
practice, Linux chooses kvm-clock.
Thanks: Thomas Huth, Daniel Berrange
We used to read the QMP schema so we could see if the binary supported
qemu mandatory locking. However after commit 47857751a7 ("lib:
direct: Remove test for qemu mandatory locking") this is dead code, so
remove it.
Updates: commit 47857751a7
This was a test for a 13 year old problem to do with problems
launching libguestfs appliances in parallel. That problem does not
affect current code.
Testing build/launch of libguestfs appliances is a valuable thing to
test. However we start libguestfs in parallel in both the current
test suite and in higher level tools, and this test was dubious (and
breaks with recent changes). Therefore remove the test.
Before 2011 it was recommended to use a prime number for the initial
size. In 2011 the OCaml hash table was reimplemented using a hash
function based on Murmur 3. Hashtbl.create now adjusts the initial
size to the next power of 2 (minimum 16). So replace obsolete
'Hashtbl.create 13' with 'Hashtbl.create 16'.
In filesystems that have many millions of files, the default behaviour
of setfiles is to build a huge hash table containing every filename.
This uses up lots of memory which prevents relabelling from happening
in the reduced memory environment of the libguestfs appliance.
I added the setfiles -A option to change this default behaviour. If
setfiles has the option then use it.
Fixes: https://issues.redhat.com/browse/RHEL-114292
Related: https://issues.redhat.com/browse/RHEL-111165
Related: https://issues.redhat.com/browse/RHEL-111505
It turns out that ubuntu-latest is the same as 24.04, so we're just
testing on the same version twice. Remove the duplication.
Thanks: Stephen Gallagher
Fixes: commit e218dd73cc
Add an API to do the equivalent of `chmod [-r] MODE PATH` for
NTFS filesystems.
Files created on a linux ntfs-3g mount can not change permissions
directly. New files and directories are created with rough windows
equivalent of `chmod 777`. These wide open permissions can generate
security warnings on windows after virt-v2v installs bits into
`Program Files\Guestfs`.
Behind the scenes we use `ntfssecaudit(8)` from `ntfsprogs`
which is already part of the appliance. We only expose the chmod-style
feature; the rest of `ntfssecaudit` is concerned reporting and
managing fine grained windows security info which is way more than
we need.
Also note, `ntfssecaudit` needs to run on an unmounted partition
so using this is more complicated than a traditional `chmod` call.
Related: https://issues.redhat.com/browse/RHEL-104352
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This builds libguestfs and runs the in-tree tests, ie. the basic
'./configure && make && make check'.
We're using the "free for public repositories" plan. We run the tests
directly on an Ubuntu VM, versions 24.04 & latest. Also on Fedora 42,
43 in a container (running on the Ubuntu VM).
The tests are run on pushes to the master branch, and on pull
requests to the master branch.
GObject is not built, since the tests fail and anyway there are some
deep problems with the GObject bindings and they probably will be
removed.
Thanks: Stephen Gallagher, Cole Robinson
This change was done almost entirely automatically using the script
below. This uses the OCaml lexer to read the source files and extract
the strings and locations. Strings which are "candidates" (in this
case, longer than 3 lines) are replaced in the output with quoted
string literals.
Since the OCaml lexer is used, it already substitutes all escape
sequences correctly. I diffed the output of the generator and it is
identical after this change, except for UUIDs, which change because of
how Utils.stable_uuid is implemented.
Thanks: Nicolas Ojeda Bar
$ ocamlfind opt -package unix,compiler-libs.common find_strings.ml \
-o find_strings.opt -linkpkg
$ for f in $( git ls-files -- \*.ml ) ; do ./find_strings.opt $f ; done
open Printf
let read_whole_file path =
let buf = Buffer.create 16384 in
let chan = open_in path in
let maxlen = 16384 in
let b = Bytes.create maxlen in
let rec loop () =
let r = input chan b 0 maxlen in
if r > 0 then (
Buffer.add_substring buf (Bytes.to_string b) 0 r;
loop ()
)
in
loop ();
close_in chan;
Buffer.contents buf
let count_chars c str =
let count = ref 0 in
for i = 0 to String.length str - 1 do
if c = String.unsafe_get str i then incr count
done;
!count
let subs = ref []
let consider_string str loc =
let nr_lines = count_chars '\n' str in
if nr_lines > 3 then
subs := (str, loc) :: !subs
let () =
Lexer.init ();
let filename = Sys.argv.(1) in
let content = read_whole_file filename in
let lexbuf = Lexing.from_string content in
let rec loop () =
let token = Lexer.token lexbuf in
(match token with
| Parser.EOF -> ();
| STRING (s, loc, sopt) ->
consider_string s loc; (* sopt? *)
loop ();
| token ->
loop ();
)
in
loop ();
(* The list of subs is already reversed, which is convenient
* because we must the file substitutions in reverse order.
*)
let subs = !subs in
let new_content = ref content in
List.iter (
fun (str, loc) ->
let { Location.loc_start = { pos_cnum = p1 };
loc_end = { pos_cnum = p2 } } = loc in
let len = String.length !new_content in
let before = String.sub !new_content 0 (p1-1) in
let after = String.sub !new_content (p2+1) (len - p2 - 1) in
new_content := before ^ "{|" ^ str ^ "|}" ^ after
) subs;
let new_content = !new_content in
if content <> new_content then (
(* Update the file in place. *)
let new_filename = filename ^ ".new"
and backup_filename = filename ^ ".bak" in
let chan = open_out new_filename in
fprintf chan "%s" new_content;
close_out chan;
Unix.rename filename backup_filename;
Unix.rename new_filename filename
)
Pulls in the commits listed below. This has no effect as all changes
are confined to the common/mlcustomize subdirectory which we do not
use or ship.
Richard W.M. Jones (4):
mlcustomize/SELinux_relabel.ml: Add comment
mlcustomize/SELinux_relabel.ml: Use new guestfs_setfiles API
mlcustomize/SELinux_relabel.ml: Relabel every mountpoint
mlcustomize/firstboot.ml: Use quoted string literals for firstboot
Vadim Rozenfeld (1):
Modify the firstboot script to check the scripts execution return status
If HKLM\System\MountedDevices references a blank disk, then when we
try to search for the actual backing device we will get an error from
parted:
parted: /dev/sdb: parted exited with status 1: Error: /dev/sdb: unrecognised disk label: Invalid argument
Just ignore these errors instead of failing inspection.
Fixes: https://issues.redhat.com/browse/RHEL-108803
Reported-by: Ameen Barakat
Thanks: Ming Xie
The function 'map_registry_disk_blob_gpt' immediately below this one
has a debugging statement. Add the equivalent to the function
'map_registry_disk_blob_mbr'.
The output looks like:
map_registry_disk_blob_mbr: searching for MBR disk ID 31 32 33 34
map_registry_disk_blob_mbr: searching for MBR partition offset 00 00 00 10 00 00 00 00
The guestfs_selinux_relabel function was very hard to use. In
particular it didn't just do an SELinux relabel as you might expect.
Instead you have to write a whole bunch of code around it (example[1])
to make it useful.
Another problem is that it doesn't let you pass multiple paths to the
setfiles command, but the command itself does permit that (and, as it
turns out, will require it). There is no backwards compatible way to
extend the existing definition to allow a list parameter without
breaking API.
So deprecate guestfs_selinux_relabel. Reimplement it as
guestfs_setfiles. The new function is basically the same as the old
one, but allows you to pass a list of paths. The old function calls
the new function with a single path parameter.
[1] https://github.com/libguestfs/libguestfs-common/blob/master/mlcustomize/SELinux_relabel.ml
No existing OCaml functions have a StringList parameter, but we would
like to add one.
The original plan seems to have been to map these to 'string array'
types, but 'string list' is more natural, albeit marginally less
efficient. The implementation here just has to convert the 'char **'
into the OCaml linked list of values.
Previously calling 'sysroot_path "/dev"' for example would return the
string "/sysroot//dev". While this is not wrong, it confuses some
external programs (hello, setfiles), and it's not very "clean". Be a
bit more careful to avoid doubling the '/' character in the common case.
Encrypted root fs on SUSE distros will present itself like so:
```
/dev/mapper/cr_root / btrfs defaults 0 0
UUID=588905f9-bfa4-47b5-9fe8-893cb8ad4a0b /var btrfs subvol=/@/var 0 0
... more subvols here ...
UUID=8a278363-3042-4dea-a878-592f5e1b7381 swap btrfs defaults 0 0
/dev/mapper/cr_root /.snapshots btrfs subvol=/@/.snapshots 0 0
cr_root UUID=5289379a-a707-41b5-994c-c383f7ed54cc none x-initrd.attach
```
This breaks `-i` inspection, since libguestfs doesn't know what
/dev/mapper/cr_root is supposed to be, and nothing in the appliance
will autopopulate that path. This isn't a problem on Fedora, where
it uses UUID= instead of a /dev/mapper path.
Currently when we see /dev/mapper as a mount prefix, we only attempt
to do some LVM name mapping. This extends libguestfs to check
/etc/crypttab first. If we find an entry for the mapper path, and it
points to the encrypted luks UUID, we use that UUID to build the
associated /dev/disk/by-id/dm-uuid-CRYPT-* path, which is a symlink
to the unencrypted /dev/dm-X path
Resolves: https://issues.redhat.com/browse/RHEL-93584
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Also some mdadm configuration files. This is useful for debugging.
The output looks like this:
info: /etc/fstab in /dev/VG/Root
LABEL=BOOT /boot ext2 default 0 0$
LABEL=ROOT / ext2 default 0 0$
Fixes: https://issues.redhat.com/browse/RHEL-106490
This saves us going into a loop if virDomainDestroyFlags keeps
returning -EBUSY quickly, which apparenrly can happen in containers.
The equivalent 'direct' backend code sleeps for 2 seconds in this case.
This is just code movement.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
RWMJ: Renamed and moved the function for consistency with surrounding
code.
Many years ago we used to pass acpi=off on the Linux kernel command
line. In commit db1f811b2 we stopped doing that (around 2016).
However unless you also use:
<features>
<acpi/>
</features>
then it turns out that libvirt disables ACPI generation at the qemu
level. None of this mattered until SeaBIOS 1.17 changed its
behaviour, causing ACPI to be required for virtio devices to work.
Updates: commit db1f811b29
Related: https://bugzilla.redhat.com/show_bug.cgi?id=2372329
Thanks: Gerd Hoffmann
This test has never been run. It was originally added in
commit 36d6df671e ("tests/http: Add a test of HTTP protocol.", 2013).
However even when it was added, it was commented out.
Updates: commit 36d6df671e
We removed the final uses of custom printf formatters in
commit 0b3c6cc0c0 ("daemon: Remove remaining uses of custom printf %Q
and %R", 2022).
Updates: commit 0b3c6cc0c0
libosinfo changes the naming scheme it uses for SUSE starting with
major version 15. Previously it used names like "sles12" (or
"sles12sp1"), "sled12" for Server and Desktop variants. In 15+ it
uses "sle15" as there are no variants any longer (instead the
installer asks you what variant you want to install). We're only
interested in the Server variant. Change the name that we return to
"sle15" or "sle15sp1".
See: b0fa386699
Fixes: https://issues.redhat.com/browse/RHEL-95791
Thanks: Ming Xie, Victor Toso
Related: https://issues.redhat.com/browse/RHEL-95540
They will be removed in libguestfs 1.58 (the next but one version).
Currently they don't actually compile. The larger problem is that
they don't handle 64 bit quantities properly (using floats instead),
meaning that any disk size or offset above a certain size will be
improperly passed through the API, usually rounded to the nearest
53 bits.
In SLES guests in particular, btrfs snapshots seem to be used to allow
rollback of changes made to the filesystem. Dozens of snapshots may
be present. Technically therefore these are multi-boot guests. The
libguestfs concept of "root" of an operating system does not map well
to this, causing problems in virt-inspector and virt-v2v.
In this commit we ignore these duplicates. The test is quite narrow
to avoid false positives: We only remove a duplicate if it is a member
of a parent device, both are btrfs, both the snapshot and parent have
a root role, and the roles are otherwise very similar.
There may be a case for reporting this information separately in
future, although it's also easy to find this out now. For example,
when you see a btrfs root device returned by inspect_os, you could
call btrfs_subvolume_list on the root device to list the snapshots.
Fixes: https://issues.redhat.com/browse/RHEL-93109
Back in commit 8289aa1ad6 ("New APIs for guest inspection.", 2010)
when inspection was first added, we did inspection in the library, so
it was accurate to say that inspection information was stored "in the
handle". Much later, in commit 394d11be49 and commit 3a00c4d179
(2017) we moved inspection to the daemon, but left the comment the
same.
Fixes: commit 3a00c4d179
This function is used from other parts of the daemon, especially for
example with inspection. However it was difficult to follow exactly
what filesystems it was returning because of insufficient debugging
information.
create.c: In function 'disk_create_qcow2':
create.c:372:5: error: format not a string literal and no format arguments [-Werror=format-security]
372 | debug (g, cmd_stdout);
| ^~~~~
Fixes: commit 606aa1d182
https://issues.redhat.com/browse/RHEL-92239
After this, output looks like
$ ./run guestfish --ro --format=qcow2 -a test.img
libguestfs: error: qemu-img: qemu-img: /home/crobinso/src/libguestfs/tmp/libguestfsFlxnb0/overlay1.qcow2: Image is not in qcow2 format Could not open backing image. : qemu-img exited with error status 1.
To see full error messages you may need to enable debugging.
...
Signed-off-by: Cole Robinson <crobinso@redhat.com>
The old btrfs-fsck API used "btrfs check" which appears to be broken
or deprecated. The real tool you should use is "btrfs scrub". We
have already implemented that API, but it is very awkward to use from
libguestfs. In particular there's no existing way to run the scrub
and wait for it to finish.
Fix this by deprecating btrfs-fsck. Implement a new API
btrfs-scrub-full which runs btrfs scrub in the foreground, waits for
it to finish, and handles errors. It's much more like fsck tools in
other filesystems.
Thanks: Eric Sandeen
Fixes: https://issues.redhat.com/browse/RHEL-91936
This pulls in the commits below, requiring us to replace all uses of
String.is_prefix and String.is_suffix.
Mostly done with Perl like this, and carefully checked by hand
afterwards since this doesn't get everything right:
$ perl -pi.bak -e 's/String.is_prefix ([^[:space:]\)]+) ([^[:space:]\)]+)/String.starts_with \2 \1/g' -- `git ls-files`
Richard W.M. Jones (3):
mlstdutils: Fix comment that still referred to the old function names
mldrivers: Link to gettext-stub if ocaml-gettext is enabled
mlstdutils: Rename String.is_prefix -> starts_with, is_suffix -> ends_with
Pull in these commits which require minor changes:
Richard W.M. Jones (3):
mlstdutils: Remove Std_utils.identity
mlstdutils: Remove Std_utils.protect
mlstdutils: Remove List.filter_map
Adds these commits:
Richard W.M. Jones (5):
mlstdutils: Modify List.take, List.drop to match OCaml 5.3
mlstdutils: Rename List.dropwhile -> drop_while, takewhile -> take_while
mlstdutils: Add List.last function
mlstdutils: Move List module first
mlstdutils: Add String.common_prefix, longest_common_prefix
zfs-fuse has been unmaintained for a very long time. In fact, the
website has gone. This means that whatever we need for zfs (probably
some kind of license-compatible kernel module) is not going to be
provided by zfs-fuse.
See: https://bugzilla.redhat.com/show_bug.cgi?id=2214965
A customer case was found where /etc/fstab contained multiple root
mountpoints, something like:
LABEL=System / xfs ...
LABEL=Boot /boot ext2 ...
LABEL=System / xfs ...
This causes libguestfs and virt-v2v to fail. Either (on RHEL 9) we
try to mount the second instance of / which gives an error. Or (on
upstream kernels) we are able to mount the second instance but then
libguestfs gets confused when trying to unmount them.
In this case as the mounted devices are the same we can just delete
the duplicate. It's also possible that there could be multiple
non-identical root mountpoints, in which case we have to pick one, and
this code arbitrarily picks the first[*] (but emits a warning).
We don't do anything for non-root mountpoints.
Update common submodule to add 'List.same' function from mlstdutils.
[*] Which one is "the first" depends on what version of ocaml-augeas
we are using. ocaml-augeas version 0.6 Augeas.matches function
returns entries in reverse order (compared to augeas itself). This is
fixed in version 0.7:
http://git.annexia.org/?p=ocaml-augeas.git;a=commitdiff;h=b703b92e3d26690aa6f7b822132049ce5435983e
Fixes: https://issues.redhat.com/browse/RHEL-90168
We could match on 1- and 2-digit RHEL issues, thus matching strings
like "RHEL-9" and "RHEL-10". No such Jira issues exist for us.
(cherry picked from commit 846ecb2b3840212c6f2cad6a15a6b33cd2e912bf)
We didn't correctly check for an error from the ioctl. Thus when
using a raw block device you would often see a bogus warning in
debugging output.
Related: https://issues.redhat.com/browse/MTV-2441
Commit e11583a03a ("ocaml: Don't embed -L../lib/.libs in final
mlguestfs.cma or .cmxa") was correct in itself, but as a side effect
caused mlguestfs.cma to be linked to the system libguestfs. If system
libguestfs was not installed, the build would fail entirely with:
ocamlmklib -o mlguestfs \
-ldopt '-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 ' \
libguestfsocaml_a-guestfs-c.o libguestfsocaml_a-guestfs-c-actions.o libguestfsocaml_a-guestfs-c-errnos.o ../common/utils/libguestfsocaml_a-cleanups.o ../common/utils/libguestfsocaml_a-stringlists-utils.o guestfs.cmo \
\
-lguestfs
/usr/bin/ld: cannot find -lguestfs: No such file or directory
Fix (from Mohamed Akram) is to add -L../lib/.libs to the -ldopt flag.
This does not embed the option in the final mlguestfs.cma file, but
seems to only use it while building the library.
Updates: commit e11583a03a
Reported-by: Mohamed Akram
Fixes: https://github.com/libguestfs/libguestfs/issues/191
$ ocamlobjinfo /usr/lib64/ocaml/guestfs/mlguestfs.cma
File /usr/lib64/ocaml/guestfs/mlguestfs.cma
Force custom: no
Extra C object files: -lmlguestfs -L../lib/.libs -lguestfs
...
(and the same in mlguestfs.cmxa). This was caused because we run
ocamlmklib with this -L parameter so that OCaml tests and examples
will link with the local libguestfs.so (instead of the
system-installed version), but for other reasons that doesn't actually
work either.
Comment from Eric Blake:
> We don't have any hard-links in our source. GNU cp -d is the same as
> cp -P --preserve=links, but --preserve=links only matters for hard
> links, so losing it doesn't hurt our usage. I'm inclined to just go
> with cp -P unconditionally rather than needing the configure.ac
> logic for $(CP_D).
Fixes: https://github.com/libguestfs/libguestfs/issues/183
Reported-by: Mohamed Akram
Thanks: Eric Blake
These libraries are only needed if building the daemon (so not if
using ./configure --disable-daemon). Move the tests so we only check
for these libraries if the daemon is enabled in the build.
The daemon also requires ocaml-augeas (it uses both the augeas C
library directly and the OCaml bindings for augeas, for reasons).
However that test is already disabled if the daemon is not being
built.
Reported-by: Mohamed Akram
Fixes: commit dfa9dee775
Fixes: commit 8a723ca62e
Fixes: commit 228d49bb84
Fixes: https://github.com/libguestfs/libguestfs/issues/184
libmagic (part of the "file" command) was required in earlier versions
of libguestfs, but this requirement was effectively removed in
commit b48da89dd6 ("daemon: Reimplement ‘file_architecture’ API in
OCaml") back in 2017. Or to be more precise, we now use the "file"
command inside the daemon, which may or may not use libmagic but we
no longer link to the library directly.
Reported-by: Mohamed Akram
Fixes: commit b48da89dd6
Related: https://github.com/libguestfs/libguestfs/issues/184
Linux + LVM supports device names like /dev/disk/by-id/dm-uuid-LVM-
followed by two concatenated UUIDs, firstly for the volume group and
secondly for the logical volume. We can reverse those to get the
device name (/dev/VG/LV).
fstab entries look like:
# / was on /dev/vg0/lv-0 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YOOycQ533186x288FdU6UubU3OlnWJz6D / ext4 defaults 0 1
# /usr was on /dev/vg0/lv-1 during curtin installation
/dev/disk/by-id/dm-uuid-LVM-OzFWT6NHkstr1hcmrWRRMDGPn9xdZj1YZu53m4ZssZ8Jeb3I14RAJwIj5YlHIb9P /usr ext4 defaults 0 1
The upshot of this fix is that we are now able to correctly inspect
and run virt-v2v on Ubuntu 22+ guests with split /usr. In particular,
we correctly map /etc/fstab entries like the above to LV device names,
which means that /usr merging now works correctly.
Reported-by: Jaroslav Spanko
Thanks: Daniel Berrange
Fixes: https://issues.redhat.com/browse/RHEL-87493
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
This was implemented wrongly. In the XDR protocol, UUIDs are fixed
buffers of length 32. We can just use memcpy to copy from the OCaml
string to the UUID, but we have to ensure the string length returned
by OCaml is correct (if not we just assert, it's an internal error).
(It didn't even compile before, so we know it was never used).
Ubuntu 22= uses /dev/disk/by-uuid/ followed by a filesystem UUID in
fstab entries. Resolve these to mountables.
A typical fstab entry looks like this:
# /boot was on /dev/vda2 during curtin installation
/dev/disk/by-uuid/b4e56462-5a64-4272-b76d-f5e58bd8f128 /boot ext4 defaults 0 1
The comment is generated by the installer and appears in the fstab.
This entry would be translated to /dev/sda2.
This acts just like FString except that we do reverse device name
translation on it. The only use is in the 'pvs-full' API where we
will use it (in a subsequent commit) to reverse translate the pv_name
field (a device name) before returning it from the daemon.
Compare this to the 'pvs' API which also returns a list of device
names, but using the generator's 'RStructList (RDevice,...)' return
type, where RDevice is similarly reverse translated.
Note in the library-side bindings, because the name has already been
translated in the daemon, we just treat it exactly the same as
FString. The vast majority of this patch is this mechanical change.
Cole Robinson (2):
mltools: decouple and simplify osinfo device support checks
mlcustomize: disable `--inject-virtio-win osinfo`
Richard W.M. Jones (3):
mltools: Fix de-oUnit-ized tests
mltools: Unreference various objects
Revert "mltools: Unreference various objects"
And update customize docs to match
Signed-off-by: Cole Robinson <crobinso@redhat.com>
These are not required by libguestfs. They were used by virt-builder,
but that tool has now been moved to a separate project
(guestfs-tools).
Fixes: commit 733d2182b6
Useful for debugging problems caused by the host kernel. In
particular we were looking at a problem with passt creating a user
namespace but didn't know what exact kernel was being used.
Internal error type did not implement the necessary traits to be
treated as a proper Rust error. This would cause ergonomic issues
with manual error handling and with error handling crates like
anyhow. Implement the Display and Error trait for internal Error
type. This also fixes the build warnings.
Signed-off-by: Jacob Reger <regerjacob@gmail.com>
Fixed break in Rust bindings where callback registering expected a
*const i8, but were provided with a *const c_char. Changed to be both
*const c_char in alignment with the libguestfs documentation.
Signed-off-by: Jacob Reger <regerjacob@gmail.com>
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
There were several, large *.o files left over after 'make clean':
$ ls ruby/ext/guestfs/
actions-0.c actions-2.c actions-4.c actions-6.c handle.c
actions-0.o actions-2.o actions-4.o actions-6.o handle.o
actions-1.c actions-3.c actions-5.c actions.h module.c
actions-1.o actions-3.o actions-5.o extconf.rb.in module.o
tests/Makefile.am assumes the test image is located in
tests/syslinux/, but we actually created it in tests/. Fix the
location, and also ensure the test will clean it up after running
successfully.
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Fixes virt-builder error:
virt-builder: error: libguestfs error: inspect_os: execvp: sfdisk: No
such file or directory
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
Debian has replaced the deprecated isc-dhcp-client (dhclient) with
dhcpcd-base (dhcpcd) as the default DHCP client [1]. This causes network
issues in virt-builder since isc-dhcp-client is no longer installed
by default.
Add support for dhcpcd (dhcpcd-base package) in Debian.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041066
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
As far as I can tell using po4a-gettextize was wrong here. The right
subtool for combining multiple documents into a .pot is po4a-updatepo,
so use that one instead.
OCaml 4.08.0 was released on 2019-06-14, over 5 years ago. By
requiring a slightly later OCaml version, we can drop more
compatibility code which was only used by older versions.
Consistent with qemu & libvirt, this drops support for compiling
upstream libguestfs on RHEL 8 (ocaml-4.07.0-4.el8.x86_64).
Qemu policy:
https://www.qemu.org/docs/master/about/build-platforms.html
Libvirt policy:
https://libvirt.org/platforms.html
Update the common submodule, pulling in:
Richard W.M. Jones (4):
qemuopts: Add ability to add raw, unquoted output to qemu scripts
qemuopts: Fix missing break statement
mlstdutils: Remove Option module
Remove test for caml_alloc_initialized_string
Stop generating these files. They are currently only used by virt-v2v
-o qemu mode, and there are better ways to locate the UEFI files
there.
Update the common submodule to bring in:
Richard W.M. Jones (5):
mlcustomize: Add heuristic support for Windows Server 2025
mlcustomize/customize_run.ml: Move 'in' to new line
mlstdutils/guestfs_config: Define host_os
mlcustomize, mltools: Check guest OS is compatible before allowing --run
Remove mlv2v/ subdirectory
dhcpcd fails to update /etc/resolv.conf if is a dangling symlink, so remove it
if it is.
This happens on Arch Linux when systemd-resolved is enabled by symlinking
/etc/resolv.conf to /run/systemd/resolve/stub-resolv.conf. The symlink is copied
into the supermin appliance, but the target file is not and this causes dhcpcd to
fail when it tries to update /etc/resolv.conf.
The handling of /etc/resolv.conf could be improved in dhcpcd but it's not their
job to decide when to remove the symlink. We do have the authority to decide
whether or not to use systemd-resolved or resolvconf in the appliance, so we can
remove it.
After previous changes, this library is no longer used. We have
switched to json-c, for better compatibility with libvirt.
(cherry picked from
guestfs-tools commit e6dcf7e3a7e9170978e57ce6df1b34f92fac5ae3)
We tested for QEMU >= 2.10 support for mandatory locking. I believe
this is for all practical purposes always enabled now (and qemu 2.10
is ancient history) so simply assume it's true always.
Although we don't use JSON_parser in libguestfs, update the submodule
just to get rid of another Jansson dependency:
Richard W.M. Jones (1):
mltools: Replace jansson with json-c
This will eventually replace Jansson for all JSON parsing. However
this commit simply introduces the new dependency in the configure
script and documents it.
I chose json-c 0.14 as the baseline since that is the version in RHEL 9.
Probably earlier versions would work.
All recent compilers support this (except MS compilers which we don't
care about). Assume it is supported. We test it in ./configure and
hard fail if it doesn't work.
We still define HAVE_ATTRIBUTE_CLEANUP but you can now assume it is
always defined and don't have to check it.
The guest was found to have these inspection fields:
type: windows
distro: windows
product_name: Windows Server 2025 Standard
product_variant: Server
version: 10.0
arch: x86_64
hostname: WIN-84V4KKQ30SM
build ID: 26080
windows_systemroot: /Windows
windows_software_hive: /Windows/System32/config/SOFTWARE
windows_system_hive: /Windows/System32/config/SYSTEM
windows_current_control_set: ControlSet001
Reported-by: Ming Xie
Fixes: https://issues.redhat.com/browse/RHEL-62935
Add these two files from the common/ submodule:
common/mlcustomize/v2v-customize-options.pod
common/mlcustomize/v2v-customize-synopsis.pod
Updates: commit fe1ce09242
openEuler is simliar to CentOS, but the ID is not lower-case string,
as below:
NAME="openEuler"
VERSION="24.03 (LTS)"
ID="openEuler"
VERSION_ID="24.03"
PRETTY_NAME="openEuler 24.03 (LTS)"
ANSI_COLOR="0;31"
Signed-off-by: Wang Guoquan <wangguoquan03@foxmail.com>
Virt-v2v already includes facilities for injecting QEMU guest agent
etc. We shouldn't add the virt-customize options for this.
Update common submodule to include:
Richard W.M. Jones (2):
mlcustomize: Move virt-customize modules to mlcustomize/
mlcustomize: Update generated options for virt-v2v
Pull in some fixes from the common submodule. None of these should be
applicable to libguestfs, they apply only to guestfs-tools and
virt-v2v.
Richard W.M. Jones (13):
mlcustomize: Add Inject_virtio_win.inject_blnsvr implementation
mlcustomize: firstboot: Use Linux path for Powershell script path
mlcustomize: firstboot: Use powershell.exe instead of path
mlcustomize: firstboot: Use Powershell -NoProfile flag
mlcustomize: Revert delay installation of qemu-ga MSI
mldrivers/linux_kernels.ml: Prefix general information with ^info:
mlcustomize: Use Start-Process -Wait to run qemu-ga installer
mlcustomize: Add Firstboot.firstboot_dir function
mlcustomize: Place powershell scripts into <firstboot_dir>\Temp
mlcustomize: Inject qemu-ga & blnsvr into <firstboot_dir>/Temp
mlcustomize: Write qemu-ga log file name to log.txt
mlcustomize: Add some comments to firstboot batch file
mlcustomize: Reboot Windows between each firstboot script
Since RHEL 7.4, the noop scheduler is no longer a thing. Trying to
set it results in the error:
+ echo noop
/init: line 108: echo: write error: Invalid argument
The current recommendation (https://access.redhat.com/solutions/5427)
is to use mq-deadline, but that's also the default so we don't have to
do anything.
A bigger reason to remove these lines is that kernel 6.11.0 has
introduced a hang where -- rarely -- the ext4 filesystem hangs if you
try to change the scheduler while handing a page fault, even if you're
setting a scheduler that doesn't exist. I couldn't get much detail
except for a couple of stack traces from different VMs:
crash> set 234
PID: 234
COMMAND: "modprobe"
TASK: ffff9f5ec3a22f40 [THREAD_INFO: ffff9f5ec3a22f40]
CPU: 0
STATE: TASK_UNINTERRUPTIBLE
crash> bt
PID: 234 TASK: ffff9f5ec3a22f40 CPU: 0 COMMAND: "modprobe"
#0 [ffffb21e002e7840] __schedule at ffffffffa718f6d0
#1 [ffffb21e002e78f8] schedule at ffffffffa7190a27
#2 [ffffb21e002e7908] __bio_queue_enter at ffffffffa67e121c
#3 [ffffb21e002e7968] blk_mq_submit_bio at ffffffffa67f358c
#4 [ffffb21e002e79f0] __submit_bio at ffffffffa67e1e3c
#5 [ffffb21e002e7a58] submit_bio_noacct_nocheck at ffffffffa67e2326
#6 [ffffb21e002e7ac0] ext4_mpage_readpages at ffffffffa65ceafc
#7 [ffffb21e002e7be0] read_pages at ffffffffa6381d17
#8 [ffffb21e002e7c40] page_cache_ra_unbounded at ffffffffa6381ff5
#9 [ffffb21e002e7ca8] filemap_fault at ffffffffa63761b5
#10 [ffffb21e002e7d48] __do_fault at ffffffffa63d1892
#11 [ffffb21e002e7d70] do_fault at ffffffffa63d2425
#12 [ffffb21e002e7da0] __handle_mm_fault at ffffffffa63d8c6b
#13 [ffffb21e002e7e88] handle_mm_fault at ffffffffa63d95c2
#14 [ffffb21e002e7ec8] do_user_addr_fault at ffffffffa60b34ea
#15 [ffffb21e002e7f28] exc_page_fault at ffffffffa7186e4e
#16 [ffffb21e002e7f50] asm_exc_page_fault at ffffffffa72012a6
RIP: 000055d16159f8d8 RSP: 00007ffdd4c1f340 RFLAGS: 00010206
RAX: 00000000000bec82 RBX: 00007ff2fd00dc82 RCX: 000055d1615b492a
RDX: 00007ffdd4c216b0 RSI: 00000000200bec82 RDI: 000055d185725960
RBP: 00007ffdd4c1f5a0 R8: 0000000000000000 R9: 0000000000000000
R10: 0000000000000000 R11: 0000000000000202 R12: 00000000200bec82
R13: 000055d185725960 R14: 00007ffdd4c216b0 R15: 000055d1615b9708
ORIG_RAX: ffffffffffffffff CS: 0033 SS: 002b
crash> set 230
PID: 230
COMMAND: "modprobe"
TASK: ffff98ce03ca3040 [THREAD_INFO: ffff98ce03ca3040]
CPU: 0
STATE: TASK_UNINTERRUPTIBLE
crash> bt
PID: 230 TASK: ffff98ce03ca3040 CPU: 0 COMMAND: "modprobe"
#0 [ffffaf9940307840] __schedule at ffffffff9618f6d0
#1 [ffffaf99403078f8] schedule at ffffffff96190a27
#2 [ffffaf9940307908] __bio_queue_enter at ffffffff957e121c
#3 [ffffaf9940307968] blk_mq_submit_bio at ffffffff957f358c
#4 [ffffaf99403079f0] __submit_bio at ffffffff957e1e3c
#5 [ffffaf9940307a58] submit_bio_noacct_nocheck at ffffffff957e2326
#6 [ffffaf9940307ac0] ext4_mpage_readpages at ffffffff955ceafc
#7 [ffffaf9940307be0] read_pages at ffffffff95381d1a
#8 [ffffaf9940307c40] page_cache_ra_unbounded at ffffffff95381ff5
#9 [ffffaf9940307ca8] filemap_fault at ffffffff953761b5
#10 [ffffaf9940307d48] __do_fault at ffffffff953d1895
#11 [ffffaf9940307d70] do_fault at ffffffff953d2425
#12 [ffffaf9940307da0] __handle_mm_fault at ffffffff953d8c6b
#13 [ffffaf9940307e88] handle_mm_fault at ffffffff953d95c2
#14 [ffffaf9940307ec8] do_user_addr_fault at ffffffff950b34ea
#15 [ffffaf9940307f28] exc_page_fault at ffffffff96186e4e
#16 [ffffaf9940307f50] asm_exc_page_fault at ffffffff962012a6
RIP: 0000556b7a7468d8 RSP: 00007ffde2ffb560 RFLAGS: 00000206
RAX: 00000000000bec82 RBX: 00007f5331a0dc82 RCX: 0000556b7a75b92a
RDX: 00007ffde2ffd8d0 RSI: 00000000200bec82 RDI: 0000556ba8edf960
RBP: 00007ffde2ffb7c0 R8: 0000000000000000 R9: 0000000000000000
R10: 0000000000000000 R11: 0000000000000202 R12: 00000000200bec82
R13: 0000556ba8edf960 R14: 00007ffde2ffd8d0 R15: 0000556b7a760708
ORIG_RAX: ffffffffffffffff CS: 0033 SS: 002b
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2303267
Add INSTALL_OCAMLLIB parameter for allowing ocaml install to a user
defined path. If not defined, fallback to `ocamlc -where`.
Signed-off-by: Antonio Caggiano <quic_acaggian@quicinc.com>
The documentation for Module::Build is completely opaque on how you're
supposed to pass @CFLAGS@ correctly. However I noticed that we were
not passing -g through when generating Guestfs.so:
gcc -lpthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now '-specs=/usr/lib/rpm/redhat/redhat-hardened-ld' '-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' '-Wl,--build-id=sha1' -L/usr/local/lib -fstack-protector-strong -lperl -o blib/arch/auto/Sys/Guestfs/Guestfs.so lib/Sys/Guestfs.o -L../lib/.libs -lguestfs
This debuginfo was not always being generated correctly. (For some
reason it still manages to be generated in Fedora and RHEL 9, but not
in RHEL 10, this part is still unclear to me.)
Anyway it seems we ought to pass at least -g to the linker, and we
might as well pass the full set of @CFLAGS@, hence this change.
The actual output of sfdisk --part-attrs is bizarre and doesn't match
the documentation. After looking at the source from util-linux, fix
the parsing to match what sfdisk produces.
Reported-by: Yongkui Guo
Fixes: commit c6c266a85d
Fixes: https://issues.redhat.com/browse/RHEL-35998
Commit d5b6f1df5f ("daemon: Allow parts of the daemon and APIs to be
written in OCaml.", 2017) contained a bug where in any OCaml function
that returns int64_t, the result was truncated to an int. This
particularly affected part_get_gpt_attributes as that returns large 64
bit numbers, but probably affects other functions too, undetected.
Fixes: commit d5b6f1df5f
This was only theoretically supported, via curl. It's unlikely that
it really worked as it was never tested.
If needed it's better to use nbdkit-curl-plugin instead (this applies
to http and ftp as well).
After many, many years, although libvirt does still often fail to
work, it's generally more secure to stick with libvirt than to try
running qemu directly. The main issue here is that people have
cargo-culted LIBGUESTFS_BACKEND=direct everywhere (even when it's not
necessary).
In case network was requested, but the host lacks both dhclient and
dhcpcd, skip the loop which waits for a resolv.conf update.
This reduces boot time by 10 seconds.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
This was failing with recent Linux:
libguestfs: error: btrfs_subvolume_snapshot: /dir/test3: /dir/test6: ERROR: cannot snapshot '/sysroot/dir/test3': Invalid argument
I tried to change the test to use 1/1000 instead, but that fails with
a different error which I don't understand at all.
As we're not meant to be testing btrfs here, only that libguestfs can
translate between the guestfs API and btrfs commands and we know it
can do that, I simply deleted the sub-test entirely.
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
This "new" parameter was added in 2014:
commit 8eab3194ce1737a167812d5e84d83b0dfc253fac
Author: Karel Zak <kzak@redhat.com>
Date: Mon Sep 15 12:37:52 2014 +0200
sfdisk: add --parttype
The patch also makes --{id,change-id,print-id} deprecated in favour
of --parttype. The original --id is too generic option name and the
--print-id and --change-id are unnecessary and inconsistent with
another sfdisk options (e.g. we don't have --change-bootable)
Also remove an extraneous / incorrect comment about parted. As
history has played out, sfdisk proves to be the better tool and parted
is a PITA.
These were regenerated by Dan Berrange from a known good Debian
container image. In addition they are stripped. The commands were:
$ podman run -it registry.gitlab.com/qemu-project/qemu/qemu/debian-loongarch-cross
# echo 'int main(){}' > bin.c
# loongarch64-unknown-linux-gnu-gcc bin.c -o bin-loongarch64
# loongarch64-unknown-linux-gnu-strip --strip-all bin-loongarch64
# echo '' > lib.c
# loongarch64-unknown-linux-gnu-gcc -shared lib.c -o lib-loongarch64.so
# loongarch64-unknown-linux-gnu-strip --strip-all lib-loongarch64.so
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
These binaries are not meant to be run, they are purely data files
used for testing. Remove the +x attribute to prevent accidentally
running them.
However to avoid breaking the phony guests, we need to chmod +x the
files when we upload them into those guests.
guestmount.1 depends on translated files blocksize-option.pod,
key-option.pod & keys-from-stdin-option.pod (via __INCLUDE__
directives). If these are not yet translated by the time we try to
generate guestmount.1 then it will fail with:
podwrapper.pl: key-option.pod: cannot find input file on path at /builddir/build/BUILD/libguestfs-1.50.1/podwrapper.pl line 672.
This happens especially in parallel builds. Fix this by writing the
guestmount.1 rule explicitly, with the correct dependencies.
I noticed that 1-byte translated POD files were being generated in the
output directory (po-docs/ja/). This seems to have happened because
po4a-translate was generating an error, but because we were
immediately pipeing the output into sed the error was suppressed.
By running them as two separate commands this cannot happen.
Fixes: commit bd896d68c0
The previous code split it on ',' which was completely wrong.
(It reveals the lack of testing however).
Fixes: commit c08032ebe2
Reported-by: Yongkui Guo
The list of patches is below. The one which matters for guestfish is
addition of --key all:... selector.
Andrey Drobyshev (1):
mldrivers: look for bootloader config in /boot/grub/grub.cfg in case of UEFI
Richard W.M. Jones (5):
mlxml: Include <libxml/parser.h> for xmlReadMemory
options/keys.c: Rewrite confusing match statement
options: Rewrite --key documentation fragment
options: Allow --key all:SELECTOR to be used to match any device
mltools/libosinfo-c.c: Fix off-by-one error
Fixes: https://issues.redhat.com/browse/RHEL-19367
Since OCaml 5.1.1, changes to custom blocks caused C finalizers that
call caml_enter_blocking_section to stop working (if they ever did
before). They are relatively inflexible compared to registering an
OCaml finalizer (Gc.finalise) to call Guestfs.close, so use that
instead.
Suggested-by: Guillaume Munch-Maccagnoni
See: https://github.com/ocaml/ocaml/issues/12820
See: db48794fa8
Using 'virt-customize --tar-in some.tar:/dir -a disk.img' will unpack
'some.tar' into '/dir' in the guest. Note that this will not work for
compressed tar files as written since the underlying guestfs_tar_in
function requires the compression type to be set explicitly and
defaults to no compression (it does not auto-detect or default to
compression).
OCaml 5.1 changes the names of these libraries for some reason.
Also in OCaml 5.1, if using those libraries you must link with -lzstd.
Since zstd was already described as "required" (although we only used
it in the appliance), there is no official change to the requirements,
but I have added a configure time check for the library.
Thanks: Jerry James <loganjerry@gmail.com>
Intermittent test failures in virt-filesystems showed that when using
the pvs_full API, the pv_name field in the returned list of structures
was not being reverse translated. As a result internal partition
names could appear in the output of virt-filesystems.
See: https://listman.redhat.com/archives/libguestfs/2023-July/032058.html
The just added --chown option previously used StringPair, splitting
the argument as ‘UID.GID:FILENAME’. However this will not work if we
ever extend this with the ability to use user or group names, since
they may contain dot (but not colon). Add a new StringTriplet type
and split the argument string three ways. The new option becomes:
virt-customize ... --chown UID:GID:FILENAME
Include the following commit from the common submodule:
commit e70d89a58dae068be2e19c7c21558707261af96a
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Sat Jul 15 16:42:06 2023 +0100
customize: Update generated files for --chown with StringTriplet
Updates: commit d8e48bff21
On QEMU 7.2.0+, if "passt" is available, ask QEMU for passt ("stream")
rather than SLIRP ("user") networking.
For this, we need to run passt ourselves. Given that passt daemonizes by
default, start it with our traditional function guestfs_int_cmd_run(). Ask
passt to save its PID file, because in case something goes wrong before
we're completely sure the appliance (i.e. QEMU) is up and running, we'll
need to kill passt, the *grandchild*, ourselves.
Pass "--one-off" to passt (same as libvirt). This way, once we have proof
that QEMU has connected to passt (because the appliance shows signs of
life), we need not clean up passt ourselves -- once QEMU exits, passt will
see an EOF on the unix domain socket, and exit as well.
Passt is way more flexible than SLIRP, and passt normally intends to
imitate the host environment in the guest as much as possible. This means
that, when switching from SLIRP to passt, the guest would see changes to
the following:
- guest IP address,
- guest subnet mask,
- host (= gateway) IP address,
- host (= gateway) MAC address.
Extract the SLIRP defaults into the new macros NETWORK_GW_IP and
NETWORK_GW_MAC, and pass them explicitly to passt. In particular,
"tests/rsync/test-rsync.sh" fails without setting the host address
(NETWORK_GW_IP) properly.
(These artifacts can be verified in the appliance with "virt-rescue
--network", by running "ip addr", "ip route", and "ip neighbor" at the
virt-rescue prompt. There are four scenarios: two libguest backends, times
passt being installed or not installed.)
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20230714132213.96616-8-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Introduce a small function for creating pathnames for PID files.
guestfs_int_make_pid_path() is something of an amalgamation of
guestfs_int_make_temp_path() [1] and guestfs_int_create_socketname() [2]:
- it creates a pathname under sockdir, like [2],
- it uses the handle's unique counter, like [1],
- it takes a name like both [1] and [2], but the name is not size-limited
like in [2], plus we hardcode the suffix from [1] as ".pid".
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20230714132213.96616-7-lersek@redhat.com>
Consider the following inverted call tree (effectively a dependency tree
-- callees are at the top and near the left margin):
lazy_make_tmpdir() [lib/tmpdirs.c]
guestfs_int_lazy_make_tmpdir() [lib/tmpdirs.c]
guestfs_int_make_temp_path() [lib/tmpdirs.c]
guestfs_int_lazy_make_sockdir() [lib/tmpdirs.c]
guestfs_int_create_socketname() [lib/launch.c]
lazy_make_tmpdir() is our common workhorse / helper function that
centralizes the mkdtemp() function call.
guestfs_int_lazy_make_tmpdir() and guestfs_int_lazy_make_sockdir() are the
next level functions, both calling lazy_make_tmpdir(), just feeding it
different dirname generator functions, and different "is_runtime_dir"
qualifications. These functions create temp dirs for various, more
specific, purposes (see the manual and "lib/guestfs-internal.h" for more
details).
On a yet higher level are guestfs_int_make_temp_path() and
guestfs_int_create_socketname() -- they serve for creating *entries* in
those specific temp directories.
The discrepancy here is that, although all the other functions live in
"lib/tmpdirs.c", guestfs_int_create_socketname() is defined in
"lib/launch.c". That makes for a confusing code reading; move the function
to "lib/tmpdirs.c", just below its sibling function
guestfs_int_make_temp_path().
While at it, correct the leading comment on
guestfs_int_create_socketname() -- the socket pathname is created in the
socket directory, not in the temporary directory.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20230714132213.96616-6-lersek@redhat.com>
We generate the <interface type="user"> element on libvirt 3.8.0+ already.
For selecting passt rather than SLIRP, we only need to insert the child
element <backend type='passt'>. Make that child element conditional on
libvirt 9.0.0+, plus "passt --help" being executable.
For the latter, place the new helper function guestfs_int_passt_runnable()
in "lib/launch.c" -- we're going to use the same function for the direct
backend as well.
This change exposes a number of (perceived) shortcomings in libvirt; I've
filed <https://bugzilla.redhat.com/show_bug.cgi?id=2222766> about those.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20230714132213.96616-3-lersek@redhat.com>
Currently we #define NETWORK_ADDRESS as "169.254.0.0". That's a bug in
libguestfs, but it's been invisible -- thus far it's been canceled out by
*independent* bugs in *both* QEMU and libvirt.
(1) With the direct backend, the current definition of NETWORK_ADDRESS
results in the following QEMU command line option:
-netdev user,id=usernet,net=169.254.0.0/16
According to the QEMU documentation, the "net" property here is supposed
to specify the *exact* IP address that the guest will receive. The guest
however receives 169.254.2.15 (I've checked that with virt-rescue).
In other words, libguestfs doesn't do what the QEMU documentation says,
and QEMU doesn't do what the QEMU documentation says either. The end
result has been good enough -- but only until now.
(2) With the libvirt backend, the current definition of NETWORK_ADDRESS
results in the following domain XML snippet:
<interface type="user">
<model type="virtio"/>
<ip family="ipv4" address="169.254.0.0" prefix="16"/>
</interface>
which libvirt translates, in turn, to
-netdev {"type":"user","net":"169.254.0.0/16","id":"hostnet0"}
According to the domain XML documentation, the @address attribute here is
again supposed to specify the *exact* IP address that the guest will
receive. However, the guest receives 169.254.2.15 (I've checked that with
virt-rescue).
In other words, libguestfs doesn't do what the libvirt documentation says,
and libvirt doesn't do what the libvirt documentation says either. The end
result has been good enough -- but only until now.
Where things break down though is the subsequent passt enablement, in the
rest of this series. For example, when using the libvirt backend together
with passt, libvirt translates the @address attribute to passt's
"--address" option, but passt takes the address *verbatim*, and not as a
subnet base address. That difference is visible in the appliance; for
example, when running virt-rescue on a Fedora 38 image, and issuing "ip
addr". Namely, after enabling passt for the libvirt backend, the
guest-visible IP address changes from 169.254.2.15 to 169.254.0.0, which
is an IP address that makes no sense for an endpoint.
Fix the latent bug by specifying the actual guest IP address we want, in
NETWORK_ADDRESS.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20230714132213.96616-2-lersek@redhat.com>
The last (only?) caller of guestfs_int_cmd_clear_close_files() disappeared
in commit e4c3968880 ("lib/info: Remove /dev/fd hacking and pass a true
filename to qemu-img info.", 2018-01-23), part of v1.37.36.
Simplify the code by removing guestfs_int_cmd_clear_close_files().
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20230711113906.107340-1-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
When finalizing the handle we call guestfs_close. This function could
be long-running (eg. it may have to shut down the qemu subprocess), so
release the runtime lock.
Since OCaml 4 the old and confusing caml_enter_blocking_section and
caml_leave_blocking_section calls have been replaced with
caml_release_runtime_system and caml_acquire_runtime_system (in that
order). Use the new names.
Various errors like this:
In function ‘test_fuse’,
inlined from ‘main’ at test-fuse.c:133:11:
test-fuse.c:274:5: error: argument 1 null where non-null expected [-Werror=nonnull]
274 | fclose (fp);
| ^~~~~~~~~~~
In file included from test-fuse.c:26:
/usr/include/stdio.h: In function ‘main’:
/usr/include/stdio.h:183:12: note: in a call to function ‘fclose’ declared ‘nonnull’
183 | extern int fclose (FILE *__stream) __nonnull ((1));
| ^~~~~~
Since OCaml 4.07 (released 2018-07-10) the always-loaded standard
library module has been called Stdlib. The old Pervasives module was
finally removed in OCaml 5.
$ perl -pi.bak -e 's/Pervasives\./Stdlib./g' -- `git ls-files`
OCaml >= 4.07 is now required.
Also update the common submodule with:
commit d61cd820b49e403848d15c5deaccbf8dd7045370
Author: Jürgen Hötzel
Date: Sat May 20 18:16:40 2023 +0200
Add support for OCaml 5.0
Since SELinux userspace v3.4 [1], setfiles command supports "-T nthreads"
option, which allows parallel execution. "-T 0" allows using as many
threads as there're available CPU cores. This might speed up the process
of filesystem relabeling in case the appliance is being run with multiple
vCPUs. The latter is true for at least v2v starting from d2b64ecc67
("v2v: Set the number of vCPUs to same as host number of pCPUs.").
For instance, when running virt-v2v-in-place on my 12-core Xeon host
with SSD, with appliance being run with 8 vCPUs (the upper limit specified
in d2b64ecc67), and on the ~150GiB disk VM (physical size on the host),
I get the following results:
./in-place/virt-v2v-in-place -i libvirt fedora37-vm -v -x
Without this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 17.94 secs
...
With this patch:
...
commandrvf: setfiles -F -e /sysroot/dev -e /sysroot/proc -e /sysroot/sys -m -C -T 0 -r /sysroot -v /sysroot/etc/selinux/targeted/contexts/files/file_contexts /sysroot/^M
libguestfs: trace: v2v: selinux_relabel = 0
libguestfs: trace: v2v: rm_f "/.autorelabel"
guestfsd: => selinux_relabel (0x1d3) took 5.88 secs
...
So in my scenario it's getting 3 times faster.
[1] https://github.com/SELinuxProject/selinux/releases/tag/3.4
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
'X' in the setiles' stderr doesn't necessarily mean that option 'X'
doesn't exist. For instance, when passing '-T' we get: "setfiles:
option requires an argument -- 'T'".
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Since RHBZ#726528, filesystem.rpm doesn't include /selinux. setfiles
then gives us the warning: "Can't stat exclude path "/sysroot/selinux",
No such file or directory - ignoring."
Though the warning is harmless, let's get rid of it by checking the
existence of /selinux directory.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Andrey Drobyshev (2):
inject_virtio_win: add Virtio_SCSI to block_type
inject_virtio_win: write the proper block controller PCI ID to Win registry
Richard W.M. Jones (2):
mlcustomize: Fix overlong comment
mlcustomize: Add accessors for block driver priority list
Roman Kagan (1):
inject_virtio_win: match only vendor/device/revision
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
This was not being set because of some impenetrable problem with
autoconf. The actual line which set the shell variable was simply
being deleted for some reason. Using an m4 definition works.
Updates: commit f68752462e
It seems as if this is just a variation on the previous libvirt
suppression.
==2042391== 61 (48 direct, 13 indirect) bytes in 1 blocks are definitely lost in
loss record 267 of 507
==2042391== at 0x4846464: calloc (vg_replace_malloc.c:1340)
==2042391== by 0x54EE318: g_malloc0 (gmem.c:163)
==2042391== by 0x5017D4C: virClassNew (virobject.c:189)
==2042391== by 0x523CCB9: virDataTypesOnce.lto_priv.0 (datatypes.c:111)
==2042391== by 0x4D8BE42: __pthread_once_slow (pthread_once.c:116)
==2042391== by 0x50345E1: virOnce (virthread.c:44)
==2042391== by 0x523CD6A: virGetConnect (datatypes.c:121)
==2042391== by 0x51FBC1F: virConnectOpenInternal (libvirt.c:893)
==2042391== by 0x51FCC11: virConnectOpenAuth (libvirt.c:1271)
==2042391== by 0x4A3ECAD: guestfs_int_open_libvirt_connection.constprop.0 (li
bvirt-auth.c:224)
==2042391== by 0x4A1D62A: launch_libvirt.lto_priv.0 (launch-libvirt.c:389)
==2042391== by 0x4993739: guestfs_launch (launch.c:114)
(cherry picked from guestfs-tools commit 8790af0266a808c8a04927bb5039be06cbc3da54)
OCaml PCRE.compile doesn't free up regexps if they are stored in
globals. However the valgrind suppression for this matched the old
function name, not the new PCRE2 name. Valgrind errors looked like:
==1799651== 145 bytes in 1 blocks are possibly lost in loss record 2,927 of 4,168
==1799651== at 0x484186F: malloc (vg_replace_malloc.c:381)
==1799651== by 0x4D5F2E5: pcre2_compile_8 (pcre2_compile.c:10250)
==1799651== by 0x29E077: guestfs_int_pcre_compile (pcre-c.c:196)
==1799651== by 0x2B725F: camlOVA__entry (OVA.ml:392)
==1799651== by 0x2859A8: caml_program (in /home/rjones/d/virt-v2v/v2v/virt-v2v)
==1799651== by 0x40F9D8: caml_start_program (in /home/rjones/d/virt-v2v/v2v/virt-v2v)
==1799651== by 0x40FD86: caml_startup_common (in /home/rjones/d/virt-v2v/v2v/virt-v2v)
==1799651== by 0x40FDDC: caml_startup (in /home/rjones/d/virt-v2v/v2v/virt-v2v)
==1799651== by 0x28523F: main (in /home/rjones/d/virt-v2v/v2v/virt-v2v)
(cherry picked from virt-v2v commit 2949109ff9f7a081b1a4b475b9f7bcbef5b45ee0)
OCaml 4.08 introduces a stdlib Option module which looks a bit like
ours but has a number of differences. In particular our functions
Option.may and Option.default have no corresponding functions in
stdlib, although there are close enough equivalents.
This change was automated using this command:
$ perl -pi.bak \
-e 's/Option.may/Option.iter/g; s/Option.default /Option.value ~default:/g' \
`git ls-files`
Update common module to include:
commit cffa077323fafcdfcf78e230c022afa891a6b3ff
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Mon Feb 20 12:11:51 2023 +0000
mlstdutils: Rework the Option module to be compatible with stdlib
This reverts one part of commit 85235aec83 related to reference
counts. Py_BuildValue always increments the reference count (when it
succeeds) and I could not reproduce the Python 3 problem that was
described there.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
The presence of this file complicates ./configure and you also have to
remember to update it on each release. Replace it with a simple
RELEASE_DATE set in ./configure when the version is updated.
I also had to make a minor change to the generator which was using
this file both to check it was run from the source directory and to
get an exclusive lock. We now use podwrapper.pl.in for this.
It fails when run twice with:
../run cargo clean
error: could not find `Cargo.toml` in `/home/rjones/d/libguestfs/rust` or any parent directory
As this is an optional step don't fail here.
We require libvirt >= 0.10.2, and we included code to check this at
configure-, compile- and run-time. Remove the checks at compile and
run time (keep the ./configure check). Libvirt 0.10.2 was released
over 10 years ago so it's safe to assume that everyone has it by now.
The event callback gets a buffer parameter which is usually something
like a log message. However as it comes from C it is not necessarily
well-formed (eg) UTF-8 but could contain any old sequence of bytes.
In the test case provided by the reporter, we failed to encode the
buffer as 'str' with this error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 137: unexpected end of data
Use 'bytes' instead. Strictly speaking this changes the type
signature of the callbacks, but our existing Python tests which just
print the buffer using '%s' don't fail and in any case we don't
guarantee the stability of non-C APIs.
Reported-by: Yonatan Shtarkman
See: https://listman.redhat.com/archives/libguestfs/2023-February/030653.html
In the case that building the parameters to the Python event callback
fails, args was returned as NULL. We immediately tried to call
Py_INCREF on this which crashed. Returning NULL means the Python
function threw an exception, so print the exception and return (there
is no way to return an error here - the event is lost).
Reported-by: Yonatan Shtarkman
See: https://listman.redhat.com/archives/libguestfs/2023-February/030653.html
The tool only supports an older version of the diskimage-builder
metadata, and we do not have the time or inclination to update it to a
newer version.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1910039
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
Update common module to 3253cd99d135d5eb788a0477459b43a269e8cca6
No significant changes that affect virt-v2v now, but it adds some
extra functions to Std_utils that might be used one day.
Cherry picked from virt-v2v commit c7c9dac4f84580190b0e4f7c2e68970192f4bad3
It might be left over from a previous failed run. Best to unlink the
old file before starting qemu-nbd, so there's no possibility of
getting confused later when we wait for the file to appear.
This test fails for reasons I have not fully understood yet. However
one thing I noticed is that the Unix domain socket and PID file used
the tests are placed in the tests/ directory, not the tests/nbd/
subdirectory, so let's fix that:
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -p 63668 ...
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -p 60684 ...
Starting qemu-nbd fedora-nbd.img -t --pid-file /home/rjones/d/libguestfs-rhel-9.2/tests/nbd.pid --format raw -k /home/rjones/d/libguestfs-rhel-9.2/tests/unix.sock ...
Fixes: commit 6d32773e81
Add an API to return the build ID of the guest. This to allow a
future change to be able to distinguish between Windows 10 and Windows 11
which can only be done using the build ID.
For Windows we can read the CurrentBuildNumber key from the registry.
For Linux there happens to be a BUILD_ID field in /etc/os-release.
I've never seen a Linux distro that actually uses this.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
The file(1) manual suggests using -S (disable seccomp) with -z since
the set of system calls provided by the seccomp policy does not allow
the subprocess to run. This is obvious when you use file -z on a
compressed file on a Linux distro that enables file's seccomp policy
(Arch does this, Fedora does not):
$ file -zbsL lib-i586.so.zst
Bad system call
I also fixed some incorrect text in the manual.
Thanks: Toolybird for pointing to this fix
Reported-by: David Runge
Fixes: https://github.com/libguestfs/libguestfs/issues/100
I cannot reproduce the originally reported error:
libguestfs: error: mkfs: xfs: /dev/VG/LV: Filesystem must be larger than 300MB.
Thanks: David Runge
Related: https://github.com/libguestfs/libguestfs/issues/100
$ ls -l `find -name c-ctype.h`
-rwxr-xr-x. 1 rjones rjones 9647 Dec 3 2021 ./gnulib/lib/c-ctype.h
$ chmod -x `find -name c-ctype.h`
$ ls -l `find -name c-ctype.h`
-rw-r--r--. 1 rjones rjones 9647 Dec 3 2021 ./gnulib/lib/c-ctype.h
RPM builds actually gave a warning about this which is how I noticed
the problem:
*** WARNING: ./usr/src/debug/guestfs-tools-1.48.2-2.fc36.x86_64/gnulib/lib/c-ctype.h is executable but has no shebang, removing executable bit
(cherry picked from
guestfs-tools commit 566267a3d447eb97b4a0637adbe3e45c09ba090f)
Commit 133a491677 ("Use guestfsd binary to auto-generate library
dependencies for appliance", October 2020) removed explicit
dependencies for various system packages that the daemon links
directly to, ie. all of these libraries:
$ objdump -p daemon/guestfsd | grep NEEDED | sort
NEEDED libacl.so.1
NEEDED libaugeas.so.0
NEEDED libcap.so.2
NEEDED libc.so.6
NEEDED libgcc_s.so.1
NEEDED libhivex.so.0
NEEDED libjansson.so.4
NEEDED libm.so.6
NEEDED libpcre2-8.so.0
NEEDED librpm.so.9
NEEDED libselinux.so.1
NEEDED libsystemd.so.0
NEEDED libtirpc.so.3
(plus libyara which I don't have installed.)
This avoids having to update these dependencies if they change, eg.
when we switched from PCRE to PCRE2 we did not need to update this
file.
However the same commit also incorrectly removed two apparent library
packages (libldm, libxml2) which the daemon does not link to, but
which we'd like to pull in because of tools they provide, in
particular ldmtool.
Re-add those two explicit dependencies.
Fixes: commit 133a491677
Many warnings such as:
src/session.c: In function 'guestfs_session_internal_test_rstruct':
src/session.c:14755:7: warning: the comparison will always evaluate as 'true' for the address of 'pv_uuid' will never be NULL [-Waddress]
14755 | if (ret->pv_uuid) memcpy (s->pv_uuid, ret->pv_uuid, sizeof (s->pv_uuid));
| ^~~
In file included from src/session.c:40:
../include/guestfs.h:551:8: note: 'pv_uuid' declared here
551 | char pv_uuid[32]; /* this is NOT nul-terminated, be careful when printing */
| ^~~~~~~
Deprecation warnings include:
src/optargs-xfs_growfs.c: In function 'guestfs_xfs_growfs_init':
src/optargs-xfs_growfs.c:311:13: warning: Deprecated pre-processor symbol: replace with "G_ADD_PRIVATE"
311 | o->priv = GUESTFS_XFS_GROWFS_GET_PRIVATE (o);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is based on the same change made here:
https://github.com/nzjrs/osm-gps-map/pull/78/files
setenv can call malloc and is not safe to call here. Glibc is usually
tolerant of this and we haven't had problems before, but if you use
GLIBC_TUNABLES glibc.malloc.check=1 (or any alternate malloc / libc
which serializes) then you would see hangs if starting multiple
libguestfs handles from different threads at the same time.
This commit also updates the common submodule to pick up:
commit 3c64bcdeaf684f05f46f3928b55aadafdfe72720
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Fri Oct 14 11:07:21 2022 +0100
utils: Add function for copying the environment and adding new entries
libguestfs is currently calling setenv at an unsafe location between
fork and exec. To fix this we need a way to copy and modify the
environment before fork and then we can pass the modified environ to
execve-like functions. nbdkit already does the same so use that code.
This function is copied and adapted from here under a compatible license:
https://gitlab.com/nbdkit/nbdkit/-/blob/master/common/utils/environ.c
Thanks: Siddhesh Poyarekar
Downstream patched openssl in Fedora 37+ broke unless
/etc/crypto-policies/back-ends/opensslcnf.conf is present. Files in
this directory are generated by %post rules that use scripting
languages so cannot easily be created by supermin.
Add a symlink to the DEFAULT policy file if the configuration file
doesn't exist.
A symptom of this problem is the error:
Requested hash sha256 is not supported.
Failed to set pbkdf parameters.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2133884
Updates: commit d6ba398825
Downstream patched openssl in Fedora 37+ broke unless
/etc/crypto-policies/back-ends/opensslcnf.conf is present. Files in
this directory are generated by %post rules that use scripting
languages so cannot easily be created by supermin.
Force a copy of the host files into the appliance. This is not ideal
and is hopefully a temporary fix until Fedora's openssl is fixed.
A symptom of this problem is the error:
Requested hash sha256 is not supported.
Failed to set pbkdf parameters.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2133884
If the LVM ("lvm2") feature is not available, these calls would fail.
Really they ought to be part of the "lvm2" optgroup which would cause
the generator to call reply_with_unavailable_feature and generate the
correct ENOTSUP error. When vgscan was originally added in 2010 it
was not added to the optgroup, and when lvm_scan was later added in
2018 and deprecating vgscan, the same mistake was copied.
Before this commit they will try to run the lvm pvscan command which
will fail returning some other error (instead of ENOTSUP).
Fix this by turning the calls into no-ops if the LVM feature is not
available, since scanning for LVM objects when there is no LVM can be
safely turned into a no-op.
See also
https://listman.redhat.com/archives/libguestfs/2022-September/thread.html#29908
Also this updates the common module to pick up a related fix:
commit 4b4a5b84647b1496d034bcdff910930ca5f5c486
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Fri Sep 23 15:18:43 2022 +0100
options: Don't attempt to scan LVs if "lvm2" feature is not available
Reported-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Fixes: 55dfcb2211 ("New API: lvm_scan, deprecate vgscan")
Fixes: 9752039e52 ("New API: vgscan")
gnulib itself has a replacement for <unistd.h> which redefines pipe2
as rpl_pipe2 (etc), which is why the apparently recursive call in the
implementation of pipe2 isn't actually recursive. Since I didn't copy
that file, none of that worked and instead on platforms which have
pipe2 it recursed.
Reported-by: Laszlo Ersek
Fixes: commit 908e41e556
commit 9d40590852e0755d4719adf97122758fa98e90f9
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Tue Aug 16 16:19:29 2022 +0100
options/decrypt.c: Ignore #pragma GCC with clang
Added in 2021 as a workaround for GCC 11 and since fixed upstream.
On macOS (clang):
tilde.c:43:32: error: unknown warning group '-Wanalyzer-null-argument', ignored [-Werror,-Wunknown-warning-option]
^
tilde.c:86:32: error: unknown warning group '-Wanalyzer-null-argument', ignored [-Werror,-Wunknown-warning-option]
^
2 errors generated.
These were added for GCC 11. The problem has been fixed in GCC 12.
On macOS (clang) these produced errors like this:
tsk.c:75:32: error: unknown warning group '-Wanalyzer-file-leak', ignored [-Werror,-Wunknown-warning-option]
^
On macOS, several pages of errors like:
In file included from readdir.c:26:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/rpc/xdr.h:126:3: error: type name requires a specifier or qualifier
bool_t (*x_getlong)(struct __rpc_xdr *, int *);
^
launch.c:191:3: error: implicit declaration of function 'sigemptyset' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
sigemptyset (&sigset);
^
launch.c:192:3: error: implicit declaration of function 'sigaddset' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
sigaddset (&sigset, SIGTERM);
^
launch.c:193:3: error: implicit declaration of function 'sigprocmask' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
sigprocmask (SIG_UNBLOCK, &sigset, NULL);
^
3 errors generated.
commit 1bf092f3c22e93c152de9dea3f5c13df23dc571c
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Tue Aug 16 15:08:40 2022 +0100
utils: Include <stdio.h> for FILE*
These were added in libguestfs 1.14, but never really used. Only a
handful of probes were available. When I was benchmarking libguestfs
in 2016 I didn't even use these probes because better/simpler
techniques were available.
The macOS rpcgen actually generates calls to xdr_uint64_t but doesn't
define them. Despite this we can just use xdr_int64_t instead since
it's just byte swapping.
We have traditionally used custom printf formatters %Q and %R, where
%Q replaces the argument with a shell-quoted string, and %R replaces
the argument with a sysroot-prefixed shell-quoted string. They are
actually pretty useful, but unfortunately only supported by glibc.
We only used them in about a dozen places in the daemon (much code
having been replaced by OCaml which does not need them).
In every remaining case we were constructing a command using code like
this:
asprintf_nowarn (&cmd,
"cd %Q && find -print0 | %s -0 -o -H %s --quiet", ...);
We can replace this with:
char *cmd;
size_t cmd_size;
fp = open_memstream (&cmd, &cmd_size);
fprintf (fp, "cd ");
shell_quote (dir, fp);
fprintf (fp, " && find -print0 | %s -0 -o -H %s --quiet", ...);
fclose (fp);
This code is attempting to construct a grub-install command like:
grub-install --root-directory=/sysroot/boot /dev/sda
In fact it was adding quoting to the --root-directory parameter where
it was not needed (because our "command" function uses exec).
Remove use of %R here (to avoid the extra quoting) and just use the
sysroot prefix directly.
This is required so we can determine the file architecture of
zstd-compressed Linux kernel modules as used by OpenSUSE and maybe
other distros in future.
Note that zstd becomes a required package, but it is widely available
in current Linux distros.
The package names come from https://pkgs.org/download/zstd and my own
research.
OCaml is required to compile libguestfs, however we should still be
able to disable the OCaml bindings. This didn't work because using
--disable-ocaml caused various configure tests to be skipped which are
required to compile the daemon. In particular the check for
caml_alloc_initialized_string, resulting in this error:
pcre-c.c:47:1: error: static declaration of ‘caml_alloc_initialized_string’ follows non-static declaration
caml_alloc_initialized_string (mlsize_t len, const char *p)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Also OCaml gettext is not required by libguestfs. There are no *.ml
files used by libguestfs which require translation.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2108425
Fixes: https://bugs.gentoo.org/820053
Fixes: commit 733d2182b6
qemu (7.0) does not support -cpu max for TCG.
Note this change is necessary but not sufficient for getting
libguestfs to run on RISC-V, because there is also currently no
working path to make -kernel work.
Previously we noted in a comment that stat("hello.txt") is cached (and
not called again), so the test of the link count failed. Something
has changed, possibly in the kernel, but it results in even more
aggressive caching so that an earlier, similar test also fails in the
same way. I checked by enabling debugging that the stat call doesn't
result in guestfs_lstatns being called, and the old value for the
statbuf was being returned.
Call the C-language helper key_store_requires_network() in guestfish and
guestmount.
(Short log for the "common" submodule, commit range
35467027f657..af6cb55bc58a:
Laszlo Ersek (12):
options: fix UUID comparison logic bug in get_keys()
mltools/tools_utils: remove unused function "key_store_to_cli"
mltools/tools_utils: allow multiple "--key" options for OCaml tools too
options: replace NULL-termination with number-of-elements in get_keys()
options: wrap each passphrase from get_keys() into a struct
options: add back-end for LUKS decryption with Clevis+Tang
options: introduce selector type "key_clevis"
options: generalize "--key" selector parsing for C-language utilities
mltools/tools_utils-c: handle internal type error with abort()
mltools/tools_utils: generalize "--key" selector parsing for OCaml utils
options, mltools/tools_utils: parse "--key ID:clevis" options
options, mltools/tools_utils: add helper for network dependency
).
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1809453
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20220630122048.19335-4-lersek@redhat.com>
On riscv64:
readdir.c: In function ‘guestfs_impl_readdir’:
readdir.c:127:3: error: implicit declaration of function ‘unlink’ [-Werror=implicit-function-declaration]
127 | unlink (tmpfn);
| ^~~~~~
I also changed the #include lines to make them look a bit more
like use in other files.
On older GCC:
debug.c:116:32: error: unknown option after ‘#pragma GCC diagnostic’ kind [-Werror=pragmas]
116 | #pragma GCC diagnostic ignored "-Wanalyzer-mismatching-deallocation"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[3]: *** [Makefile:2039: guestfsd-debug.o] Error 1
The upstream bug (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99193)
has now been fixed so the workaround is not necessary with the latest
GCC, so just delete the workaround.
Starting with PHP8, arginfo is mandatory for PHP extensions. This patch
updates the generator for the PHP bindings to generate the arginfo
structures, using the Zend API macros. Only basic arginfo is added,
without full documentation of argument and return types, in order to
ensure compatibility with as many versions of PHP as possible.
In guestfs-tools commit 4fe8a03cd2d3 ('sysprep: remove lvm2's default
"system.devices" file', 2022-04-11), we disabled the use of LVM2's new
"devicesfile" feature, which could interfere with the cloning of virtual
machines.
We suspected in
https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c6
that the same lvm2 feature could affect the libguestfs appliance itself,
but decided in
https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c8https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c10
that this would not be the case, because "appliance/init" already
constructed a pristine LVM_SYSTEM_DIR.
Unfortunately, that's not enough: due to the "use_devicesfile=1" default
(on RHEL9 anyway), some "lvm" invocation, possibly inside the
lvm-set-filter API, *creates* "$LVM_SYSTEM_DIR/devices/system.devices".
And then we get (minimally) warnings such as
> Please remove the lvm.conf global_filter, it is ignored with the devices
> file.
> Please remove the lvm.conf filter, it is ignored with the devices file.
when using the lvm-set-filter API.
Explicitly disable the "devices file" in "appliance/init", and also
whenever we rewrite "lvm.conf" -- that is, in set_filter()
[daemon/lvm-filter.c]. In the former, check for the feature by locating
the devicesfile-related utilities "lvmdevices" and "vgimportdevices". In
the C code, invoke the utilities with the "--help" option instead. (In
"appliance/init", I thought it was best not to call any lvm2 utilities
even with "--help", with our lvm2.conf still under construction there.) If
either utility is available, set "use_devicesfile = 0".
Cc: David Teigland <teigland@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1965941
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220530141027.16167-1-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: style fix: break "devicesfile_feature" in the function
definition to a new line]
CentOS Stream has:
ID_LIKE="rhel fedora"
which confused the existing script. If there are multiple "likes"
arbitrarily pick the first one in the list.
Fixes: commit 63b722b6c0
We previously didn't bother to check the return values from any librpm
calls. In some cases where possibly the RPM database is faulty, this
caused us to return a zero-length list of installed applications (but
no error indication).
One way to reproduce this is given below. Note this reproducer will
only work when run on a RHEL 8 host (or more specifically, with
rpm <= 4.16):
$ virt-builder fedora-28
$ guestfish -a fedora-28.img -i rm /var/lib/rpm/Packages
$ guestfish --ro -a fedora-28.img -i inspect-list-applications /dev/sda4 -vx
...
chroot: /sysroot: running 'librpm'
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
librpm returned 0 installed packages
...
With this commit we get an error instead:
...
chroot: /sysroot: running 'librpm'
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
ocaml_exn: 'internal_list_rpm_applications' raised 'Failure' exception
guestfsd: error: rpmtsInitIterator
guestfsd: => internal_list_rpm_applications (0x1fe) took 0.01 secs
libguestfs: trace: internal_list_rpm_applications = NULL (error)
libguestfs: error: internal_list_rpm_applications: rpmtsInitIterator
libguestfs: trace: inspect_list_applications2 = NULL (error)
libguestfs: trace: inspect_list_applications = NULL (error)
...
Not in this case, but in some cases of corrupt RPM databases it is
possible to recover them by running "rpmdb --rebuilddb" as a guest
command (ie. with guestfs_sh).
See-also: https://bugzilla.redhat.com/show_bug.cgi?id=2089623#c12
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2089623
Fixes: commit c9ee831aff
Reported-by: Xiaodai Wang
Reported-by: Ming Xie
Acked-by: Laszlo Ersek <lersek@redhat.com>
The current code for working out the distro uses the ID entry from
/etc/os-release, and then we map those strings into a smaller set of
values (basically, what package manager to use). However it was
suggested that we should try ID_LIKE first so that distros which act
like other distros would work. On an Arch Linux 32 system:
ID=arch32
ID_LIKE=arch
See-also: https://github.com/libguestfs/libguestfs/issues/81
Thanks: S D Rausty
Instead of continuing on regardless and failing with a weird error
later, error out early if we don't know about the distro and so cannot
set QUERY_FILES_CMD. This avoids situations like
https://github.com/libguestfs/libguestfs/issues/81
The `git-publish`[1] tool is a wrapper around `git-format-patch` and
`git-send-email`. It's a handy tool that automates some of the tedious
aspects of manual patch submission:
- Submitting a patch to the list (with a small config in place) is as
simple as `git publish`
- On next revisions, it automatically increments version numbers
- It auto-copies the list of To: and Cc: from your previous iteration
- It lets you preview/edit emails before submission
- You can also use standard `git-format-patch` and `git-send-email`
options with `git publish`
- You can send pull requests with `git publish --pull-request`
- It also provides custom hooks ... and more[2]
[1] https://github.com/stefanha/git-publish
[2] https://github.com/stefanha/git-publish/blob/master/git-publish.pod
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
In https://bugzilla.redhat.com/show_bug.cgi?id=2082806 we've been
tracking an insidious qemu bug which intermittently prevents the
libguestfs appliance from starting. The symptoms are that SeaBIOS
starts and displays its messages, but the kernel isn't reached. We
found that the kernel does in fact start, but when it tries to set up
page tables and jump to protected mode it gets a triple fault which
causes the emulated CPU in qemu to reset (qemu exits).
This seems to only affect TCG (not KVM).
Yesterday I found that this is caused by using -cpu max which enables
the "la57" feature (5-level page tables[0]), and that we can make the
problem go away using -cpu max,la57=off. Note that I still don't
fully understand the qemu bug, so this is only a workaround.
I chose to disable 5-level page tables for both TCG and KVM, partly to
make the patch simpler, and partly because I guess it's not a feature
(ie. 57 bit linear addresses) that is useful for the libguestfs
appliance case, where we have limited physical memory and no need to
run any programs with huge address spaces.
I tested this by running both the direct & libvirt paths overnight. I
expect that this patch will fail with old qemu/libvirt which doesn't
understand the "la57" feature, but this is only intended as a
temporary workaround.
[0] Article about 5-level page tables as background:
https://lwn.net/Articles/717293/
Thanks: Laszlo Ersek
Fixes: https://answers.launchpad.net/ubuntu/+source/libguestfs/+question/701625
Acked-by: Laszlo Ersek <lersek@redhat.com>
Option "-C" of setfiles(8) causes setfiles(8) to exit with status 1 rather
than status 255 if it encounters relabeling errors, but no other (fatal)
error. Pass "-C" to setfiles(8) in "selinux-relabel", because we don't
want the "selinux-relabel" API to fail if setfiles(8) only encounters
relabeling errors.
(NB even without "-C", setfiles(8) continues traversing the directory
tree(s) and relabeling files across relabeling errors, so this change is
specifically about the exit status.)
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1794518
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220511122345.14208-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
The documentation currently says that the user should avoid passing
"--selinux-relabel" on the command line if the guest does not support
SELinux. However, the "is_selinux_guest" helper function in
"common/mlcustomize/SELinux_relabel.ml" already turns "--selinux-relabel"
into a no-op if some key SELinux files are absent from the guest, so there
is no need to caution the user.
This change is relevant because the subsequent patches will turn on
"--selinux-relabel" by default, and therefore "is_selinux_guest" will grow
in importance.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220510102757.14466-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Representing "iface" in the "drive_create_data" and "drive" structures is
now useless; the direct backend ignores "iface", while the libvirt one
rejects it unless it is empty. Unify both backends -- make them both
ignore "iface". (Which only relaxes the libvirt backend, so it cannot
cause compatibility problems.) This lets us remove the fields. Update the
documentation as well.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1844341
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220504134155.11832-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
In guestfs_readdir(), the daemon currently sends each XDR-encoded
"guestfs_int_dirent" to the library with a separate send_file_write()
call.
Determine the largest encoded size (from the longest filename that a
"guestfs_int_dirent" could carry, from readdir()'s "struct dirent"), and
batch up the XDR encodings until the next encoding might not fit in
GUESTFS_MAX_CHUNK_SIZE. Call send_file_write() only then.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1674392
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220502085601.15012-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Currently the guestfs_readdir() API can not list long directories, due to
it sending back the whole directory listing in a single guestfs protocol
response, which is limited to GUESTFS_MESSAGE_MAX (approx. 4MB) in size.
Introduce the "internal_readdir" action, for transferring the directory
listing from the daemon to the library through a FileOut parameter.
Rewrite guestfs_readdir() on top of this new internal function:
- The new "internal_readdir" action is a daemon action. Do not repurpose
the "readdir" proc_nr (138) for "internal_readdir", as some distros ship
the binary appliance to their users, and reusing the proc_nr could
create a mismatch between library & appliance with obscure symptoms.
Replace the old proc_nr (138) with a new proc_nr (511) instead; a
mismatch would then produce a clear error message. Assume the new action
will first be released in libguestfs-1.48.2.
- Turn "readdir" from a daemon action into a non-daemon one. Call the
daemon action guestfs_internal_readdir() manually, receive the FileOut
parameter into a temp file, then deserialize the dirents array from the
temp file.
This patch sneakily fixes an independent bug, too. In the pre-patch
do_readdir() function [daemon/readdir.c], when readdir() returns NULL, we
don't distinguish "end of directory stream" from "readdir() failed". This
rewrite fixes this problem -- I didn't see much value separating out the
fix for the original do_readdir().
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1674392
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220502085601.15012-2-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Before commit 3a00c4d179 ("Remove inspection from the C library and
switch to daemon/OCaml implementation") in 2017 the name parameter
passed to add_drive was used by inspection to override the device name
that is determined from fstab. None of our tools ever actually used
this parameter, and when the inspection code was moved inside the
daemon we stopped using this hint field at all.
So it's no longer used, and likely hasn't been used ever. Therefore
document that the field does nothing.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
These were added to ocamldep in Jan 2012, over 10 years ago. They
were not present in RHEL 6, but we don't care about that now.
(cherry picked from virt-v2v commit f6108bbd661d3e922d07b47f00daa901ab846e59)
cdrtools writes "CDROM" into the Volume Identifier field in the PVD,
whereas genisoimage and xorriso write "ISOIMAGE". Recognise either
string as valid in the test.
Fixes: https://github.com/libguestfs/libguestfs/issues/79
Reported-by: David Runge
If the appliance is a QCOW2 image, function get_root_uuid_with_file()
fails to read ext filesystem signature (0x53EF at offset 0x438) from it.
This results in the following error:
libguestfs: error: /usr/lib64/guestfs/appliance/root: appliance is not
an extfs filesystem
The error itself is harmless, but misleading. So let's skip retrieving
the signature and UUID in case the image contains QCOW2 header. It's
safe because in this case we'll retrieve it later from RAW image dumped
from that QCOW2 by "qemu-img dd".
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
On RHEL 7 (rpm-devel-4.11.3-45.el7.x86_64):
rpm-c.c: In function ‘guestfs_int_daemon_rpm_start_iterator’:
rpm-c.c:97:44: error: ‘RPMVSF_MASK_NOSIGNATURES’ undeclared (first use in this function)
rpmtsSetVSFlags (ts, rpmtsVSFlags (ts) | RPMVSF_MASK_NOSIGNATURES);
^
rpm-c.c:97:44: note: each undeclared identifier is reported only once for each function it appears in
Fixes: commit aa6f8038f8
Bytes.get_uint8 was added in OCaml 4.08. To support OCaml >= 4.04 (in
particular, RHEL 8 has OCaml 4.07) we have to replace this function
with the equivalent native call. We can remove this commit once the
baseline OCaml moves up.
Updates: commit edfebee404
Test fails as it cannot find .xml file. Turns out $pwd=./tests and
and not ./tests/regressions as supposed in script.
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@openvz.org>
Test fails as it cannot find .in file. Turns out $srcdir=. (which is ./tests) and
and not ./tests/regressions as supposed in script. Same apply to
$abs_srcdir.
Also put .out file in ./tests/regressions too.
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@openvz.org>
Older distros (eg CentOS 6) used SHA-1 RPM package signatures which
some newer distros (eg RHEL 9.0) prevent us from verifying.
This resulted in packages with SHA-1 signatures being skipped by
librpm (there is a warning in debug output, but if you're not looking
at that then the package is silently ignored). In some cases
essential packages like the kernel were skipped, which would be
visible as a failure of virt-v2v. In other cases (eg virt-inspector)
you'd just see fewer installed packages in the <applications> list.
Since verifying package signatures is not essential for inspection,
disable this feature in librpm.
Reported-by: Xiaodai Wang
Thanks: Panu Matilainen
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2064182
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
This script is used to prepare the list of fixed bugs for release
notes. This commit contains miscellaneous fixes:
- Recognise other bug formats such as direct links.
- Ensure user is logged in so that BZ API doesn't silently truncate
results.
(cherry picked from virt-v2v commit fac1e4fce0d7114bd044b274788de01569089e18)
This was a feature that allowed you to add drives to the appliance
after launching it. It was complicated to implement, and only worked
for the libvirt backend (not "direct", which is the default backend).
It also turned out to be a bad idea. The original concept was that
appliance creation was slow, so to examine multiple guests you should
launch the handle once then hot-add the disks from each guest in turn
to manipulate them. However this is terrible from a security point of
view, especially for multi-tenant, because the drives from one guest
might compromise the appliance and thus the filesystems/drives from
subsequent guests.
It also turns out that hotplugging is very slow. Nowadays appliance
creation should be faster than hotplugging.
The main use case for this was virt-df, but virt-df no longer uses it
after we discovered the problems outlined above.
These APIs were an experimental feature for passing through 9p
filesystems from the host to the libguestfs appliance. It was never
possible to use this without hacking the qemu command line of the
appliance to add such drives by hand. It also didn't fit the
libguestfs model very well. And 9p is generally deprecated in
upstream qemu.
Note that for ABI reasons these APIs are not actually removed, they
have been changed so that they always return an error. These APIs
were actually hard-removed from all versions of RHEL.
See-also: https://bugzilla.redhat.com/921710
User-Mode Linux was an alternative hypervisor that could run the
appliance, instead of using qemu. It had many limitations including
lack of network, and UML support in Linux has been semi-broken for a
long time. It was also slower than KVM on baremeal in general and had
various corner cases which were much slower including the emulated
serial port which made bulk uploads and downloads painful. Also of
course it lacked qemu-specific features like qcow2 or any
network-backed disk, so many disk images could not be opened this way.
This was never supported in RHEL.
See-also: https://bugzilla.redhat.com/1144197
This experimental feature allowed you (in theory) to connect to an
existing instance of the libguestfs daemon. (Again, in theory) it
allowed you to attach to running guests. This didn't work well in
practice. If you want to do this, install qemu-guest-agent inside
your guest instead.
This also disables the --live options in guestfish and guestmount.
(The option now prints an error).
This was never supported in RHEL.
The daemon tests relied on this connection method to perform tests on
a bare daemon, so this removes those tests. They were not especially
valuable.
See-also: https://bugzilla.redhat.com/798980
Commit 6d32773e81 ("tests: Run the tests in parallel.") makes all of
the tests run in the same directory. Previously tests expected to be
run in their own subdirectories and so were freer about using generic
filenames. When run in parallel these filenames now clash. Fix
another case.
Fixes: commit 6d32773e81
Commit 6d32773e81 ("tests: Run the tests in parallel.") combined
multiple Makefiles into one. During this conversion I accidentally
made several mistakes - duplicating the EXTRA_DIST entry for
mountable/test-mountable-inspect.sh and omitting the backslash
continuation of the TESTS line.
Fixes: commit 6d32773e81
Commit e9eaf4d889 ("docs: Split release notes by release.") split the
old guestfs-release-notes(1) page by release, but left now-broken
links to guestfs-release-notes(1) in various places in the
documentation.
The easiest way to fix this is to recreate this page by renaming
guestfs-release-notes-historical(1) as guestfs-release-notes(1) and
adding links to the other release notes pages.
Fixes: commit e9eaf4d889
When running the OCaml bytecode tests, small but apparently
unimportant differences in the stack frames caused the existing
suppressions not to match. Adding an extra "..." will ignore these
differences allowing the tests to pass.
Valgrind output before this commit (note "UnknownInlinedFun" frame):
==1733400== 79 (48 direct, 31 indirect) bytes in 1 blocks are definitely lost in loss record 320 of 518
==1733400== at 0x484A464: calloc (vg_replace_malloc.c:1328)
==1733400== by 0x59685D0: g_malloc0 (gmem.c:136)
==1733400== by 0x531BB59: virClassNew (virobject.c:191)
==1733400== by 0x553A3BF: UnknownInlinedFun (datatypes.c:110)
==1733400== by 0x553A3BF: virDataTypesOnce (datatypes.c:121)
==1733400== by 0x49EA0C8: __pthread_once_slow (pthread_once.c:116)
==1733400== by 0x53312E9: virOnce (virthread.c:44)
==1733400== by 0x553A74A: UnknownInlinedFun (datatypes.c:121)
==1733400== by 0x553A74A: virGetConnect (datatypes.c:133)
==1733400== by 0x54FA94F: virConnectOpenInternal (libvirt.c:895)
==1733400== by 0x54FB883: virConnectOpenAuth (libvirt.c:1277)
==1733400== by 0x50E842A: guestfs_int_open_libvirt_connection.constprop.0 (libvirt-auth.c:224)
==1733400== by 0x50C6120: launch_libvirt.lto_priv.0 (launch-libvirt.c:390)
==1733400== by 0x5040E30: UnknownInlinedFun (launch.c:114)
==1733400== by 0x5040E30: guestfs_launch (actions-3.c:513)
==1733400==
==1733400== 256 bytes in 1 blocks are definitely lost in loss record 481 of 518
==1733400== at 0x484586F: malloc (vg_replace_malloc.c:381)
==1733400== by 0x137A0E: UnknownInlinedFun (memory.c:824)
==1733400== by 0x137A0E: caml_executable_name (unix.c:367)
==1733400== by 0x14C224: UnknownInlinedFun (startup_byt.c:502)
==1733400== by 0x14C224: caml_main (startup_byt.c:457)
==1733400== by 0x11CEE1: main (main.c:41)
LUKS support used to work best if the LUKS device resided on a partition,
and contained a Physical Volume for an LVM Volume Group. This scheme, also
called LVM-on-LUKS, is commonly created by installers of various Linux
distributions. (See RHBZ#1451665.)
Libguestfs now also supports the scheme wherein the LUKS device resides on
an LVM Logical Volume, and contains a filesystem. This is called
LUKS-on-LVM, it is the inverse of the above scheme, and is created by
installers of other Linux distributions. (See RHBZ#1658126.)
Both schemes are now decrypted by libguestfs-based utilities when
inspection is enabled (such as in "guestfish -i", virt-inspector,
virt-v2v), through the inspect_mount() function in utilities written in C,
and through the "inspect_decrypt" function in ones written in OCaml.
We don't seem to need an API like "list-luks-devices", as
"list-dm-devices" returns decrypted (i.e., opened) LUKS devices too; for
example, in the LUKS-on-LVM case:
> ><fs> list-dm-devices
> /dev/mapper/luks-0d619854-ccd5-43b1-8883-991fec5ef713
> /dev/mapper/luks-4e9e7a6f-a68c-42fd-92b4-8f4f2579a389
Thus, the subject TODO section is now out of date, and it's unclear what
remains "to do" there; let's just remove the section.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1658126
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220223162120.16729-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Create a new (fake) Fedora disk image with two partitions. /dev/sda1 is
the boot partition as usual, /dev/sda2 is used as an LVM PV. The VG has
four LVs, Root and LV1 through LV3.
Each LV holds a LUKS device (with a different key). Each decrypted LUKS
device holds an ext2 filesystem, with "/dev/mapper/Root-luks" holding the
root filesystem.
Each filesystem has a dedicated label (ROOT, LV1, LV2, LV3).
In the test case, run guestfish in inspector mode, twice.
In the first invocation, provide the LUKS passphrases by LV name. Also
specific to the first invocation, fetch the LUKS UUIDs by LV name.
In the second invocation, provide the LUKS passphrases by UUID.
In both invocations, after decryption, check the filesystem labels, the
/dev/mapper/* names generated for the decrypted LUKS block devices, and
the existence of "/etc/fedora-release" on the root filesystem.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1658126
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220223162120.16729-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
$ git shortlog 5b5fac3e0b10..41126802097f
Laszlo Ersek (3):
Demote "Std_utils.wrap" to an internal function in Tools_utils
Tools_utils.wrap: only wrap text for TTYs
add common ("standard") option --wrap
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
GCC 12 gives a warning about our previous attempt to check the length
of the socket path. In the ensuing discussion it was pointed out that
it is easier to get snprintf to do the hard work. snprintf will
return an int >= UNIX_PATH_MAX if the path is too long, or < 0 if
there are other errors such as locale/encoding problems. So we should
just check for those two cases instead.
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NPKWMTSJ2A2ABNJJEH6WTZIAEFTX6CQY/
Thanks: Martin Sebor and Laszlo Ersek
The 169.254.0.0/16 network specification (for the appliance) is currently
duplicated between the direct backend and the libvirt backend. In a
subsequent patch, we're going to need the network specification in yet
another spot; extract it now to the NETWORK_ADDRESS and NETWORK_PREFIX
macros (simply as strings).
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2034160
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211223103701.12702-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
The <qemu:commandline> trick we use for adding our virtio-net-pci device
in the libvirt backend can conflict with libvirtd's and QEMU's PCI address
assignment. Try to mitigate that by placing our device in slot 0x1e on the
root bus. In practice this could only conflict with a "dmi-to-pci-bridge"
device model, which libvirtd itself places in slot 0x1e. However, given
the XMLs we generate, and modern QEMU versions, libvirtd has no reason to
auto-add "dmi-to-pci-bridge". Refer to
<https://libvirt.org/formatdomain.html#controllers>.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2034160
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211223103701.12702-2-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Warning 6 "labels-omitted" is not useful. It's fine to omit labels on
positional arguments.
Example:
File "perl_edit.ml", line 30, characters 2-13:
30 | c_edit_file (verbose ()) g (Guestfs.c_pointer g) file expr
^^^^^^^^^^^
Warning 6 [labels-omitted]: label verbose was omitted in the application of this function.
The function is specified as:
external c_edit_file : verbose:bool -> Guestfs.t -> int64 -> string -> string -> unit
The complaint is that the verbose: label has been omitted from the
first argument when the function is called, but IMO this is a
stylistic thing, not a bug.
(cherry picked from
guestfs-tools commit 577f7aee4b1c720f4c4826115b49a0c3870b149e)
In OCaml 4.13:
File "perl_edit.ml", line 30, characters 2-13:
30 | c_edit_file (verbose ()) g (Guestfs.c_pointer g) file expr
^^^^^^^^^^^
Error (warning 6 [labels-omitted]): label verbose was omitted in the application of this function.
(cherry picked from
guestfs-tools commit a4930f5fad82e5358d565b8cf3610970e9646259)
In some places in the generator we were still generating "noalloc".
It was hidden from the previous regexp I used to replace these because
of string escaping.
Updates: commit a69cde79ca
Picks up this commit:
mlstdutils/std_utils.mli: Remove export of deprecated String.copy
Since OCaml strings are at long last immutable, the String.copy
function is deprecated. Stop exporting it from our Std_utils.String.
Any places we use it are wrong and should be fixed.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2030709
Thanks: label@rockylinux.org
---
RWMJ notes: I fixed the original patch so it compiled. This patch
sets osinfo to "rocky8", which doesn't exist in the osinfo db yet.
Arguably we might want to set this to "centos8", but we can see what
libosinfo decides to do. Here is partial virt-inspector output on a
Rocky Linux disk image:
$ ./run virt-inspector -a disk.img
<?xml version="1.0"?>
<operatingsystems>
<operatingsystem>
<root>/dev/rl/root</root>
<name>linux</name>
<arch>x86_64</arch>
<distro>rocky</distro>
<product_name>Rocky Linux 8.5 (Green Obsidian)</product_name>
<major_version>8</major_version>
<minor_version>5</minor_version>
<package_format>rpm</package_format>
<package_management>dnf</package_management>
<hostname>localhost.localdomain</hostname>
<osinfo>rocky8</osinfo>
<mountpoints>
<mountpoint dev="/dev/rl/root">/</mountpoint>
<mountpoint dev="/dev/sda1">/boot</mountpoint>
</mountpoints>
<filesystems>
<filesystem dev="/dev/rl/root">
<type>xfs</type>
<uuid>fed8331f-9f25-40cd-883e-090cd640559d</uuid>
</filesystem>
<filesystem dev="/dev/rl/swap">
<type>swap</type>
<uuid>6da2c121-ea7d-49ce-98a3-14a37fceaadd</uuid>
</filesystem>
<filesystem dev="/dev/sda1">
<type>xfs</type>
<uuid>4efafe61-2d20-4d93-8055-537e09bfd033</uuid>
</filesystem>
</filesystems>
The "is_partition_can_hold_filesystem" function calls
"Parted.part_get_gpt_type" on the partition if:
- the partition table type is GPT,
- or the partition table type is MBR, and the partition is primary or
logical.
The one entry in the fake MBR partition table described in the previous
patch passes the second branch of this check, therefore
"Parted.part_get_gpt_type" is reached, and it invokes "sgdisk -i 1" on the
disk.
Surprisingly (not), while "sgdisk -i" copes fine with valid MBR partition
tables, it chokes on the fake one. The output does not contain the
"Partition GUID code" line, and so "sgdisk_info_extract_field" throws an
exception.
Prevent calling "Parted.part_get_gpt_type" on a bogus MBR partition table,
similarly to the "extended entry in MBR partition table" case; the
difference is that the bogus primary entry, unlike a valid extended entry,
*can* hold a filesystem.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-6-lersek@redhat.com>
"parted" incorrectly reports "loop" rather than "msdos" for the partition
table type, when the (fake) partition table comes from the "--mbr" option
of "mkfs.fat" (in dosfstools-4.2+), and the FAT variant in question is
FAT16 or FAT32. (See RHBZ#2026224.) Work this around by
- parsing the partition table ourselves, and
- overriding "loop" with "msdos" when appropriate.
Note that when the FAT variant is FAT12, "parted" fails to parse the fake
MBR partition table completely (see RHBZ#2026220), which we cannot work
around. However, FAT12 should be a rare corner case in libguestfs usage --
"mkfs.fat" auto-chooses FAT12 only below 9MB disk size, and even "-F 12"
can only be forced up to and including 255MB disk size.
Add the helper function "has_bogus_mbr" to the Utils module; we'll use it
elsewhere too.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-5-lersek@redhat.com>
[lersek@redhat.com: drop "fun" keyword, and use partial application, in
the definition of "sec0at" [Rich]]
Since commit 994ca1f8eb ("daemon: Reimplement 'part_get_mbr_part_type'
API in OCaml.", 2018-05-02), we've not had any calls to
print_partition_table() that would pass a "false" argument for the
"add_m_option" parameter.
Remove the parameter, and inside part_get_mbr_part_type(), remove the dead
branch.
Relatedly, update the comment on the
"print_partition_table_machine_readable" OCaml function, originally from
commit 32e661f421 ("daemon: Reimplement ‘part_list’ API in OCaml.",
2017-07-27). Because print_partition_table() now passes "-m" to "parted"
unconditionally, and there are no use cases left that would *forbid* "-m",
"print_partition_table_machine_readable" is almost equivalent to
print_partition_table() -- modulo the enforcement of the "BYT;" header.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-4-lersek@redhat.com>
The directory that readdir() and closedir() work on is BUS_PATH
("/sys/bus/virtio/drivers/9pnet_virtio"), not "/sys/block". Fix the error
messages that are sent when readdir() or closedir() fails.
(The invalid "sys/block" pathname could be a leftover from when the
directory reading logic was (perhaps) copied from "daemon/sync.c".)
Fixes: 5f10c33503
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-3-lersek@redhat.com>
Search the usage output of "mkfs.fat" for "--mbr[="; cache the result for
further invocations. If the option is supported, pass "--mbr=n" to
"mkfs.fat". This will prevent the creation of a bogus partition table
whose first (and only) entry describes a partition that contains the
partition table.
(Such a bogus partition table breaks "parted", which is a tool used by
libguestfs extensively, both internally and in public libguestfs APIs.)
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-2-lersek@redhat.com>
Another test was removing all disk_*.img files, which removed the file
used by this test. This was ultimately caused by us using parallel
tests.
Put the disk image files back into the tests/gdisk/ subdirectory to
avoid this race.
Fixes: commit 6d32773e81
"my" variable $output masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 73.
"my" variable $end_sectors masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 78.
Actually cargo caches downloaded libraries. The previous change
caused cargo to download and rebuild these after make clean which is
overly aggressive. Use make distclean instead.
Updates: commit 1834f19d20
Commit e597fc5317 ("daemon/yara: fix undefined behavior due to Yara 4.0
API changes", 2021-10-12) prevents the daemon from using such a Yara
version that precedes 4.0.0.
If only yara < 4 is found, treat the library as absent, rather than
attempting and failing to compile the yara module of the daemon. Note the
version requirement in the documentation too.
Suggested-by: Eric Blake <eblake@redhat.com>
Suggested-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-4-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Eliminate the AC_CHECK_LIB / AC_CHECK_HEADER tests for Yara, for the
following reasons:
- Upstream Yara has provided a pkg-config file since 2015, so the
(now-fixed) pkg-config check should always find it, without the
AC_CHECK_LIB / AC_CHECK_HEADER fallback branch.
- In a subsequent patch, we'll want to test for the incompatible Yara API
changes described at
<https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API>.
That's easy to do with pkg-config, but impossible with AC_CHECK_*,
without a custom test. Namely, both AC_CHECK_DECLS and AC_CHECK_TYPES
appear unable to check the parameter list of a function pointer typedef
(namely YR_CALLBACK_FUNC and YR_COMPILER_CALLBACK_FUNC). And writing a
dedicated test for this is overkill.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-3-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
The upstream Yara project has always provided its "libyara/yara.pc.in"
file with "Name: yara" -- ever since its introduction in commit
334bd1a671ca ("Add support for pkg-config", 2015-01-08).
In spite of this, when the (optional) Yara dependency was added to
libguestfs in commit 2e24129da3 ("appliance: add yara dependency",
2017-05-02), PKG_CHECK_MODULES was invoked with [libyara] as
list-of-modules.
(That was clearly a bug. I'm unsure what Debian calls their Yara
pkg-config module, but calling it anything else than "yara" would be a
distribution bug: upstream projects provide pkg-config files specifically
so that dependent projects can find their dependencies *regardless of
distribution*.)
As a consequence, on Fedora today, the PKG_CHECK_MODULES macro always
fails, and only the AC_CHECK_LIB / AC_CHECK_HEADER branch finds Yara.
In a subsequent patch, we'll want to add a version requirement to the
PKG_CHECK_MODULES macro invocation, so at first, fix the pkg-config
identifier of Yara.
Fixes: 2e24129da3
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-2-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Currently, the Yara test case ("yara/test-yara-scan.sh") fails, with the
following obscure error message:
> ><fs> yara-scan /text.txt
> libguestfs: error: deserialise_yara_detection_list:
Namely, the Yara rule match list serialization / de-serialization, between
the daemon and the library, is broken. It is caused by the following
incompatible pointer passing (i.e., undefined behavior), in function
do_internal_yara_scan(), file "daemon/yara.c":
> r = yr_rules_scan_fd (rules, fd, 0, yara_rules_callback, (void *) path, 0);
^^^^^^^^^^^^^^^^^^^
The prototype of yara_rules_callback() is:
> static int
> yara_rules_callback (int code, void *message, void *data)
however, in Yara commit 2b121b166d25 ("Track string matches using
YR_SCAN_CONTEXT.", 2020-02-27), which was included in the upstream v4.0.0
release, the rules callback prototype was changed as follows:
> diff --git a/libyara/include/yara/types.h b/libyara/include/yara/types.h
> index cad095cd70c2..f415033c4aa6 100644
> --- a/libyara/include/yara/types.h
> +++ b/libyara/include/yara/types.h
> @@ -661,6 +644,7 @@ struct YR_MEMORY_BLOCK_ITERATOR
>
>
> typedef int (*YR_CALLBACK_FUNC)(
> + YR_SCAN_CONTEXT* context,
> int message,
> void* message_data,
> void* user_data);
Therefore, the yara_rules_callback() function is entered with a mismatched
parameter list in the daemon, and so it passes garbage to
send_detection_info(), for serializing the match list.
This incompatible change was in fact documented by the Yara project:
https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API#scanning-callback
Gcc too warns about the incompatible pointer type, under
"-Wincompatible-pointer-types". However, libguestfs is built without
"-Werror" by default, so the warning is easy to miss, and the bug only
manifests at runtime.
(The same problem exists for yr_compiler_set_callback() /
compile_error_callback():
https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API#compiler-callback
except that this instance of the problem is not triggered by the test
case, as the rule list always compiles.)
Rather than simply fixing the parameter lists, consider the following
approach.
If Yara's YR_CALLBACK_FUNC and YR_COMPILER_CALLBACK_FUNC typedefs were not
for *pointer* types but actual *function* prototypes, then we could use
them directly in the declarations of our callback functions. Then any
future changes in the param lists would force a "conflicting types"
*compilation error* (not a warning). Illustration:
/* this is *not* a pointer type */
typedef int HELLO_FUNC (void);
/* function declarations */
static HELLO_FUNC my_hello_good;
static HELLO_FUNC my_hello_bad;
/* function definition, with explicit parameter list */
static int my_hello_good (void) { return 1; }
/* function definition with wrong param list -> compilation error */
static int my_hello_bad (int i) { return i; }
Unfortunately, given that the Yara-provided typedefs are already pointers,
we can't do this, in standard C anyway. Type derivation only allows for
array, structure, union, function, and pointer type derivation; it does
not allow "undoing" previous derivations.
However, using gcc's "typeof" keyword, the idea is possible. Given
YR_CALLBACK_FUNC, the expression
(YR_CALLBACK_FUNC)NULL
is a well-defined null pointer, and the function designator expression
*(YR_CALLBACK_FUNC)NULL
has the desired function type.
Of course, evaluating this expression would be undefined behavior, but in
the GCC extension expression
typeof (*(YR_CALLBACK_FUNC)NULL)
the operand of the "typeof" operator is never evaluated, as it does not
have a variably modified type. We can therefore use this "typeof" in the
same role as HELLO_FUNC had in the above example.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211011223627.20856-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: clean up whitespace in "YR_RULE *rule"]
gcc emits the following warning:
> proto.c: In function ‘send_file_complete’:
> proto.c:437:10: error: ‘buf’ may be used uninitialized
> [-Werror=maybe-uninitialized]
> 437 | return send_file_chunk (g, 0, buf, 0);
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In theory, passing the 1-byte array "buf", with indeterminate contents, to
xdr_bytes() ultimately, could be fine -- assuming xdr_bytes() never reads
the contents of the buffer, due to the buffer size being zero. However,
the xdr_bytes() manual does not seem to guarantee this (it also does not
explicitly permit passing a NULL buffer alongside size=0, which would be
even simpler for the caller).
In order to shut up the compiler, just zero-initialize the buffer --
that's simpler than adding diagnostics pragmas. The "maybe-uninitialized"
warning is otherwise very useful, so keep it globally enabled (per
WARN_CFLAGS / WERROR_CFLAGS).
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211011223627.20856-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
In OCaml 4.13:
Alert ocaml_deprecated_cli: Setting a warning with a sequence of lowercase or uppercase letters,
like 'CDEFLMPSUVYZX', is deprecated.
Use the equivalent signed form: +C+D+E+F+L+M+P+S+U+V+Y+Z+X+52-3.
(cherry picked from
guestfs-tools commit fa4f59e1d99c08d7e0bae2a7cb54f254a6506d67)
The current "arg_string_list" and "free_string_list" implementations go
back to commit b6f01f3260 ("Add Go (language) bindings.", 2013-07-03).
There are two problems with them:
- "free_string_list" doesn't actually free anything,
- at the *first* such g.Internal_test() call site that passes an
Ostringlist member inside the Optargs argument, namely:
> g.Internal_test ("abc",
> string_addr ("def"),
> []string{},
> false,
> 0,
> 0,
> "123",
> "456",
> []byte{'a', 'b', 'c', '\000', 'a', 'b', 'c'},
> &guestfs.OptargsInternal_test{Ostringlist_is_set: true,
> Ostringlist: []string{}
> }
> )
the "golang/run-bindtests" case crashes:
> panic: runtime error: cgo argument has Go pointer to Go pointer
>
> goroutine 1 [running]:
> libguestfs.org/guestfs.(*Guestfs).Internal_test.func7(0xc000018180,
> 0xadfb60, 0xadfb80, 0xc000010048, 0x0, 0x0, 0x0, 0xae3e10, 0xae3e30,
> 0xade3a0, ...)
> golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0xa9
> libguestfs.org/guestfs.(*Guestfs).Internal_test(0xc000018180, 0x4ee3a5,
> 0x3, 0xc000061be8, 0xc000061af8, 0x0, 0x0, 0xc000061a00, 0x0, 0x0, ...)
> golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0x3c9
> main.main()
> golang/bindtests/bindtests.go:77 +0x151e
> exit status 2
> FAIL run-bindtests (exit status: 1)
In Daniel P. Berrangé's words [1],
> You're allowed to pass a Go pointer to C via CGo, but the memory that
> points to is not allowed to contained further Go pointers. So the struct
> fields must strictly use a C pointer.
One pattern to solve the problem has been shown on stackoverflow [2].
Thus, rewrite the "arg_string_list" and "free_string_list" functions
almost entirely in C, following that example.
While this approach is not the most idiomatic Go, as a solution exists
without C helper functions [3], it should still be acceptable, at least as
an incremental improvement, for letting "golang/run-bindtests" pass.
[1] https://listman.redhat.com/archives/libguestfs/2021-September/msg00118.html
[2] https://stackoverflow.com/questions/35924545/golang-cgo-panic-runtime-error-cgo-argument-has-go-pointer-to-go-pointer
[3] https://listman.redhat.com/archives/libguestfs/2021-September/msg00106.html
Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: "Richard W.M. Jones" <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210921192939.32468-1-lersek@redhat.com>
Tested-by: "Richard W.M. Jones" <rjones@redhat.com>
Acked-by: "Richard W.M. Jones" <rjones@redhat.com>
According to xfs_admin(8):
> -c 0|1 Enable (1) or disable (0) lazy-counters in the filesys‐
> tem.
>
> Lazy-counters may not be disabled on Version 5 su‐
> perblock filesystems (i.e. those with metadata CRCs en‐
> abled).
>
> [...]
According to mkfs.xfs(1):
> -m global_metadata_options
> Section Name: [metadata]
> These options specify metadata format options that ei‐
> ther apply to the entire filesystem or aren't easily
> characterised by a specific functionality group. The
> valid global_metadata_options are:
>
> [...]
>
> crc=value
> This is used to create a filesystem which
> maintains and checks CRC information in all
> metadata objects on disk. The value is ei‐
> ther 0 to disable the feature, or 1 to en‐
> able the use of CRCs.
>
> [...]
>
> By default, mkfs.xfs will enable metadata
> CRCs.
Consistently with the above, the first "xfs_admin" test case in
"generator/actions_core.ml", which attempts to disable lazy counters,
always fails:
> 534/550 test_xfs_admin_0
> libguestfs: error: xfs_admin: /dev/sda1: Cannot disable lazy-counters on V5 fs
We can resolve this test failure in three ways:
(1) Extend do_mkfs() [daemon/mkfs.c], possibly even introduce
do_mkfs_xfs(), and permit the caller to specify "-m crc=0" for
mkfs.xfs. Then use this option when the temporary filesystem is
created in the XFS test that disables lazy counters.
(2) Extend the "guestfs_int_xfsinfo" structure in the libguestfs-common
project, with an "xfs_crc" field. Extend parse_xfs_info()
[daemon/xfs.c] to populate the field from "meta-data=...crc=[01]".
Modify the test case to check the following post-condition:
xfs_crc || xfs_lazycount == 0
instead of the current
xfs_lazycount == 0
effectively ignoring "xfs_lazycount" when "xfs_crc" is set.
(3) Remove the test altogether that attempts to disable lazy counters
after filesystem creation.
Given that new XFS filesystems are created with metadata CRCs enabled by
default, and several XFS features depend on metadata CRCs being enabled,
this patch implements option (3).
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
The "test-md-and-lvm-devices" test case creates, among other things, a
RAID0 array (md127) that spans two *differently sized* block devices
(sda1: 20MB, lv0: 16MB).
In Linux v3.14, the layout of such arrays was changed incompatibly and
undetectably. If an array were created with a pre-v3.14 kernel and
assembled on a v3.14+ kernel, or vice versa, data could be corrupted.
In Linux v5.4, a mitigation was added, requiring the user to specify the
layout version of such RAID0 arrays explicitly, as a module parameter. If
the user fails to specify a layout version, the v5.4+ kernel refuses to
assemble such arrays. This is why "test-md-and-lvm-devices" currently
fails, with any v5.4+ appliance kernel.
Until we implement a more general solution (see the bugzilla link below),
work around the issue by sizing sda1 and lv0 identically. For this,
increase the size of sdb1 to 24MB: when one 4MB extent is spent on LVM
metadata, the resultant lv0 size (20MB) will precisely match the size of
sda1.
This workaround only affects sizes, and does not interfere with the
original purpose of this test case, which is to test various *stackings*
between disk partitions, software RAID (md), and LVM logical volumes.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=2005485
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: remove stray empty line]
In commit 6d32773e81 ("tests: Run the tests in parallel.", 2021-03-18),
the "abs_srcdir" macro value that the 9p test would see changed from
".../tests/9p" to just ".../tests" -- the last component got dropped.
(Said commit updated some "abs_srcdir"-based references accordingly, for
example under "tests/disks", but "tests/9p/test-9p.sh" was missed.)
Therefore, the guest-visible location of the "/test-9p.sh" file changed to
"/9p/test-9p.sh", and a non-recursive listing of the guest-visible root
directory would not return the file. Thus, the test fails now.
Restore the host-side base directory to ".../tests/9p".
Fixes: 6d32773e81
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
bash can read input from a spawned process, and even provide input to
such process. This feature relies on /dev/fd/ being present. In the
past udev silently created this symlink, so this bash feature worked
more or less by accident. With recent systemd versions, such as 246
which is included in Leap 15.3, the symlink is not created anymore. As
a result scripts, such as /sbin/dhclient-script, fail to work
properly.
This symlink should have been created in version 1 of this variant of /init.
https://bugzilla.opensuse.org/show_bug.cgi?id=1190501
Signed-off-by: Olaf Hering <olaf@aepfle.de>
systemd-sysvinit contains the reboot command, which is used to
properly stop the VM. This was required by other packages, and as a
result always available. Since Leap 15.3 it will not be installed, and
as a result the VM will just panic because /init died.
If the appliance is started with --network, dhclient will run
/usr/sbin/dhclient-script, which in turn may call /sbin/netconfig to
update /etc/resolv.conf. Install sysconfig-netconfig to make sure DNS
resolving actually works.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Pass $(HIVEX_LIBS) with -cclib under the "daemon_utils_tests_LINK" target;
otherwise the OCaml compiler does not tell the linker where "-lhivex" can
be found, and the linking step fails if "-lhivex" is not on a system
library path.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
"ocamlc -where" is supposed to "print the location of the standard library
and exit". While this directory contains core OCaml C header files, it
does not contain hivex-related C header files. Trim "guestfsd_CPPFLAGS"
accordingly.
Furthermore, the hivex module for OCaml may exist elsewhere than under the
OCaml standard library directory. Invoke "ocamlfind query hivex" to find
this module. This is what AC_CHECK_OCAML_PKG(hivex) does too, in
"m4/guestfs-ocaml.m4" and "m4/ocaml.m4".
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
This warning is bogus, caused by the analyzer cannot track that len ==
0 if roots == NULL. I just changed the code to make it easier to
analyze, this doesn't fix any real bug.
guestfs-c.c: In function 'guestfs_finalize':
guestfs-c.c:85:9: error: dereference of NULL '0B' [CWE-476] [-Werror=analyzer-null-dereference]
85 | caml_remove_generational_global_root (roots[i]);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Found by GCC -fanalyzer:
xattr.c:478:32: error: '%zu' directive output may be truncated writing between 1 and 19 bytes into a region of size 16 [-Werror=format-truncation=]
478 | snprintf (num, sizeof num, "%zu", nr_attrs);
| ^
xattr.c:478:32: note: directive argument in the range [0, 2305843009213693950]
/usr/include/bits/stdio2.h:71:10: note: '__builtin___snprintf_chk' output between 2 and 20 bytes into a destination of size 16
71 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
| ^
Short log:
Laszlo Ersek (2):
Makefile.am: supply missing $(LIBGUESTFS_CFLAGS)
Makefile.am: use $(LIBGUESTFS_LIBS) for linking OCaml test programs
Richard W.M. Jones (4):
tools: Refactor create_standard_options
tools: Add optional --program-name flag.
mltools/test-getopt.sh: Allow test to work for tools version 2
mltools: Add JSON_parser.object_get_bool
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
In commit 6d32773e81 ("tests: Run the tests in parallel.", 2021-03-18),
the working directory relative to which "test-parallel-mount-local" would
be launched (by the test machinery) changed from "tests/mount-local" to
just "tests".
While the relative pathname of the "guestunmount" executable was updated
inside "test-parallel-mount-local" accordingly, the relative pathname of
the FUSE client ("test-parallel-mount-local" itself, just invoked with
"--test") was not. This issue guarantees that the exec call fails in the
child, and so the test case always hangs.
Because we had removed "mount-local" from the end of the working
directory, prepend it now to the relative pathname of the FUSE client
executable.
Fixes: 6d32773e81
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210902135124.15191-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Each worker thread of "test-parallel-mount-local" performs the following
steps (among others):
(1) it starts an appliance dedicated to that thread, using a private
scratch disk image,
(2) exports a dedicated FUSE mount point on the host, exposing the file
system on the appliance's disk,
(3) launches a child process for manipulating the particular FUSE mount
point on the host,
(4) enters a FUSE request processing loop, translating requests between
the host kernel (coming in via the FUSE mount point) and the
appliance.
Items to note:
- The child process from step (3) consists of a single thread of execution
(see fork() in POSIX): a duplicate of the parent process's respective
worker thread.
- The child process from step (3) blocks on any FUSE mount point access on
the host until the worker thread in the parent process starts processing
FUSE requests, in step (4).
- The FUSE request processing in step (4), in the worker thread living in
the parent process, terminates if and only if the child process unmounts
the FUSE mount point originating from (2).
Should the exec call in step (3) fail for any reason, the child currently
jumps to the "error" label. This is wrong: under the error label, we call
guestfs_close() on the appliance -- but the appliance is owned by the
parent process's worker thread, not the child. What happens is that the
child kills off the appliance while the parent's worker thread is in the
FUSE request processing loop (4).
The "error" label was never meant to be reached by the child process -- if
exec fails for any reason, exit the child immediately. The parent will
remain in the FUSE request processing loop (4) forever, but no state will
be corrupted. For example, using another (interactive) session on the
host, the FUSE mount points can be interacted with, and if all of them are
manually unmounted, the FUSE request processing (4) completes in every
worker thread.
This patch does not fix the primary issue with
"test-parallel-mount-local", but removes "chaos" from the symptoms. The
next patch will fix the actual regression in this test case.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210902135124.15191-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
If /lib is a symlink to usr/lib, paths to shared libraries as
determined by ld.so may differ from dpkg's file lists.
We turn the filename search pattern into a glob expression by
prefixing it with a '*', so the required packages are found again:
$ dpkg -S /lib/x86_64-linux-gnu/libpcre2-8.so.0
dpkg-query: no path found matching pattern /lib/x86_64-linux-gnu/libpcre2-8.so.0
$ dpkg -S */lib/x86_64-linux-gnu/libpcre2-8.so.0
libpcre2-8-0:amd64: /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0
qemu 6.1 has decided to change qemu-img create so that a backing
format (-F) is required if a backing file (-b) is specified. Since we
don't want to change the libguestfs API to force callers to specify
this because that would be an API break, autodetect it.
This is similar to commit c8c181e8d9 ("launch: libvirt: Autodetect
backing format for readonly drive overlays").
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1998820
When using option `--with-distro`, `HAVE_RPM`, `HAVE_DPKG` and
`HAVE_PACMAN` are not defined and make the configure phase fail.
This makes sure that these conditionals are always defined.
Windows Server 2022 preview is identified as <osinfo>win2k16</osinfo>.
Although current osinfo-db does not have an entry "win2k22", return
this instead.
osinfo-db issue to add win2k22:
https://gitlab.com/libosinfo/osinfo-db/-/issues/82
Inspection information for the guest:
type: windows
distro: windows
product_name: Windows Server 2022 Datacenter
product_variant: Server
version: 10.0
arch: x86_64
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1997446
Reported-by: Yongkui Guo
On RISC-V there is no default machine type. Invoking QEMU requires to
specify a board model with the -M option. So let's define MACHINE_TYPE
as virt.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
$ guestfish -N fs:vfat:2G syslinux /dev/sda1
libguestfs: error: syslinux: Error converting to codepage 850 Invalid argument
...
This happens because of the default codepage requested by syslinux
(code page 850) combined with the appliance missing the iconv
converter for this codepage.
Reported-by: Yongkui Guo
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1990720
When creating and returning a Python struct we were adding fields from
the C struct, but did not reduce the ref count on the temporary value
after it had been moved to the struct, resulting in a memory leak.
Reported-by: 朱丹 <zhudan24@huawei.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1985912
The `test` builtin/binary usually accepts `==` for string comparison, it is
mostly accepted for typos and people being used to double equals, but is not
documented and not always accepted either. Since autoconf uses the default
shell, it might just fail in some cases with:
./configure: 29986: test: xrustc: unexpected operator
./configure: 29990: test: xcargo: unexpected operator
Just change it to single equals as it is done everywhere else.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Commit 0f54df53d2 ("build: Remove gnulib") introduced a bug when I
rewrote existing code that used gnulib areadlink().
A missing "continue" statement on the path where fstatat(2) failed
caused fall-through to the case where it tries to use malloc(3) on the
value from the uninitialized stat buf. This caused a huge amount of
memory to be allocated, invoking the oom-killer inside the appliance.
Reported-by: Yongkui Guo
Fixes: commit 0f54df53d2
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1960217
Commit 2f587bbaec ("daemon: Read ISO9660 Primary Volume Descriptor
directly.") changed daemon/isoinfo.ml to read the PVD directly. This
was fine for guestfs_isoinfo_device which opens a device name, but did
not work for ISOs embedded within filesystems opened using
guestfs_isoinfo because we did not chroot into the filesystem first.
Example reproducer (run from the libguestfs source directory):
$ guestfish -N fs -m /dev/sda1 upload ./test-data/test.iso /test.iso
$ guestfish --ro -a test1.img -m /dev/sda1 isoinfo /test.iso
libguestfs: error: isoinfo: open: /test.iso: No such file or directory
After this fix:
$ guestfish --ro -a test1.img -m /dev/sda1 isoinfo /test.iso
iso_system_id:
iso_volume_id: ISOIMAGE
iso_volume_space_size: 2490
[etc.]
Reported-by: Yongkui Guo
Fixes: commit 2f587bbaec
Fixes: https://bugzilla.redhat.com/show_bug.cgi
warning: panic message is not a string literal
--> src/bin/event_leak.rs:9:30
|
9 | Err(e) => panic!(format!(" could not create handle {:?}", e)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(non_fmt_panic)]` on by default
= note: this is no longer accepted in Rust 2021
= note: this warning originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
QEMU has deprecated this option:
commit 166310299a1e7824bbff17e1f016659d18b4a559
Author: Daniel P. Berrangé
Date: Tue Oct 20 17:08:27 2020 +0100
os: deprecate the -enable-fips option and QEMU's FIPS enforcement
The -enable-fips option was added a long time ago to prevent the use of
single DES when VNC when FIPS mode is enabled. It should never have been
added, because apps are supposed to unconditionally honour FIPS mode
based on the '/proc/sys/crypto/fips_enabled' file contents.
In addition there is more to achieving FIPS compliance than merely
blocking use of certain algorithms. Those algorithms which are used
need to perform self-tests at runtime.
QEMU's built-in cryptography provider has no support for self-tests,
and neither does the nettle library.
If QEMU is required to be used in a FIPS enabled host, then it must be
built with the libgcrypt library enabled, which will unconditionally
enforce FIPS compliance in any algorithm usage.
Thus there is no need to keep either the -enable-fips option in QEMU, or
QEMU's internal FIPS checking methods.
In RHEL 8+, /usr/etc no longer exists. Since we were looking for this
directory in order to detect a separate /usr partition, those were no
longer detected, so the merging of /usr data into the root was not
being done. The result was incomplete inspection data and failure of
virt-v2v.
All Linux systems since forever have had /usr/src but not /src, so
detect this instead.
Furthermore the merging code didn't work, because we expected that the
root filesystem had a distro assigned, but in this configuration we
may need to look for that information in /usr/lib/os-release (not on
the root filesystem). This change makes the merging work even if we
have incomplete information about the root filesystem, so long as we
have an /etc/fstab entry pointing to the /usr mountpoint.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1949683
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1930133
Fixes: commit 394d11be49
./guestfs.go:4945:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
./guestfs.go:7320:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
./guestfs.go:7335:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
As part of our efforts to clean up and simplify libguestfs, removing
gnulib deletes a large dependency that we mostly no longer use and
causes problems for new users trying to build the library from source.
A few modules from gnulib are still used (under a compatible license)
and these are copied into gnulib/lib/
This gnulib feature abstracts away threads, locks and TLS, and also
allowed libguestfs to be linked with or without pthread. However
since pthread these days is part of glibc and so every program is
using pthread, and we want to get rid of gnulib as a dependency, just
use pthread directly.
It turns out we can read the information we need for the isoinfo API
directly from the ISO9660 PVD. We don't need to use either isoinfo or
xorriso. This also has the advantages of reducing by 1 the number of
dependencies in the appliance, and reducing potential vulnerability to
a crafted ISO file.
This also fixes timezone calculation for the datetime fields.
Thanks: Thomas Schmitt
Updates: commit efb8a766ca
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
This Debian page explains the upstream situation:
https://wiki.debian.org/genisoimage
On Fedora, xorriso provides a compatibility program called "mkisofs".
However this is not present in Debian. Hence the choice to look for
the program called "xorrisofs".
libguestfs 1.45.3 now reads the RPM database using librpm, which means
our old phony database created by db_dump can no longer work. Instead
provide a real (but very minimal) sqlite database.
This commit also fixes the virt-inspector test since the RPM database
contents are now different.
The child (chrooted) process wrote its answer on the pipe and then
exited. Meanwhile the parent waiting for the child to exit before
reading from the pipe. Thus if the output was larger than a Linux
pipebuffer then the whole thing would deadlock.
Latest btrfs seems to reject 512 byte sector size. It may be because
of the specific hardware that I'm running the test on. Anyway using a
4K sector size works.
libguestfs: error: mkfs_btrfs: /dev/sda1: ERROR: invalid sectorsize 512, expected range is [4K, 64K]
When shipping the libguestfs tarball we do not necessarily have
common/mlcustomize. If we run the generator in this situation then
don't try to generate files in the non-existent directory.
See-also: commit 7ced2b9354
This was returning "readdir: Invalid argument" which is actually
impossible (readdir(3) cannot fail with EINVAL). It turns out that
the problem is just errno from some other place leaking out.
This was deprecated in btrfs 4.14.1 and recently removed (see
btrfs-progs commit 4bd94dba8a "btrfs-progs: mkfs: remove alloc start
options and docs"). If the option is set simply ignore it.
Use qemu-nbd --pid-file option so we don't have to use an arbitrary
sleep.
Enable all parts of the test, since everything should work now with
various upstream bugs having been fixed in the 8 years since the test
was originally written.
libcap commit 177cd41803 ("A more compact form for the text
representation of capabilities.") changed the format used by
cap_to_text(3), breaking our test. Change the test to cope. This
will break with older libcap now, but there's not a lot we can do
about it.
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
This test was correctly copied into the new guestfs-tools repository
when virt-resize was moved there along with the other tools. However
it was never removed from libguestfs. We were effectively running the
test on the installed virt-resize.
Fixes: commit 733d2182b6
Note this requires libvirt >= 7.1.0 which was only released in March 2021.
With an older libvirt you will see this error:
Original error from libvirt: unsupported configuration: Invalid mode attribute 'maximum' [code=67 int1=-1]
In theory we could check if this is supported by looking at the
libvirt capabilities and fall back, but this commit does not do that,
in the expectation that most people will be using the default backend
(direct) and on Fedora/RHEL we will add an explicit minimum version
dependency to the package.
qemu support has been around quite a bit longer (at least since 2017).
Fixes: commit 30f74f38bd
This is used in virt-win-reg, but that tool have moved out to
guestfs-tools so the dependency is no longer needed by libguestfs
itself.
Fixes: commit 733d2182b6
This was only used for a single rule (check-valgrind-local-guests)
which ran "make check-valgrind" on local guests. This was never
really used by me and was fairly inadvisable anyway, so we can easily
remove it and thus remove the dependency on perl Sys::Virt.
CC virt_builder-setlocale-c.o
setlocale-c.c:38: error: "Val_none" redefined [-Werror]
38 | #define Val_none (Val_int (0))
|
In file included from /usr/lib64/ocaml/caml/alloc.h:24,
from setlocale-c.c:23:
/usr/lib64/ocaml/caml/mlvalues.h:395: note: this is the location of the previous definition
395 | #define Val_none Val_int(0)
|
QEMU has a newish feature (from about 2017 / qemu 2.9) called -cpu max
which is supposed to select the best CPU, ideal for libguestfs.
After this change, on x86-64:
KVM TCG
Direct -cpu max -cpu max
(non-libvirt)
Libvirt <cpu mode="host-passthrough"> <cpu mode="host-model">
<model fallback="allow"/> <model fallback="allow"/>
</cpu> </cpu>
Thanks: Daniel Berrangé
Previously this was in common/utils. However it is not used anywhere
else, and guestfs-tools wants to remove gnulib dependencies, so move
this to libguestfs.
Normally packagers should try to use ./configure --with-guestfs-path=...
However if they do not, then a regression in 1.44 caused the path to
constain the literal string "${exec_prefix}". This was because of
autoconf stupidity.
Try to resolve this by expanding ${exec_prefix} to something sensible.
The expansion will often be wrong (but at least it will now be a valid
path), hence the recommendation to always set --libdir and/or
--with-guestfs-path when configuring.
Fixes: commit 94ff24b880
Fixes: https://bugzilla.redhat.com/1915646
In commit 4bbbf03b8b we started to
ignore bogus GCC 11 warnings. Unfortunately earlier versions of GCC
don't know about those pragmas so give warnings [hence errors in
developer builds] like:
tsk.c:75:32: error: unknown option after '#pragma GCC diagnostic' kind [-Werror=pragmas]
Turn off these warnings.
Updates: commit 4bbbf03b8b
Current GNU tar does not restore all extended attributes. In
particular only user.* capabilities are restored (although all
are saved in the tarball).
To restore capabilities, SELinux security attributes, and other things
we need to use --xattrs-include=*
For further information on the tar bug, see:
https://bugzilla.redhat.com/show_bug.cgi?id=771927
This fixes:
"-supermin: ext2fs_namei: parent directory not found:
/var/lib/rpm: File not found by ext2_lookup"
This because chkconfig contains /var/lib/rpm/alternatives on Mageia
This package in Mageia enables optional support for Windows 10
"CompactOS" (file-level compression), read-only, which is sufficient
for inspecting Windows guests and doing certain types of modifications
to them. Virt-v2v appears to work, but it may be that anything that
involves modifying a compressed file might not work.
See commit e6764a5415
The ELF NEEDED are used to determine guestfsd's library dependencies
with help from the dynamic linker and the package manager.
This was prompted by Debian bug #972241 which was caused by a
libtirpc package renaming in Debian/unstable because the SONAME had
been changed.
Bug originally reported here by trysis:
https://stackoverflow.com/questions/64273334/test-x-in-mounted-filesystem
If the user is root then we override normally access controls in FUSE,
see https://bugzilla.redhat.com/show_bug.cgi?id=1106548.
However this causes test -x to mark all files as executable. We
shouldn't let root execute any file, only ones which have the 'x' bit
set. Therefore this narrows the fix in bug 1106548 so it only applies
to read and write bits.
To test this I created a disk with guestfish which had an executable
and a non-executable file:
$ guestfish -N fs -m /dev/sda1
><fs> touch /file1
><fs> touch /file2
><fs> chmod 0755 /file1
><fs> ll /
total 24
drwxr-xr-x 3 root root 4096 Oct 12 14:04 .
drwxr-xr-x 19 root root 4096 Oct 12 14:04 ..
-rwxr-xr-x 1 root root 0 Oct 12 14:04 file1
-rw-r--r-- 1 root root 0 Oct 12 14:04 file2
drwx------ 2 root root 16384 Oct 12 14:04 lost+found
I then mounted and tested it as non-root:
$ guestmount -a test1.img -m /dev/sda1 /tmp/mnt -v -x
$ ls -l /tmp/mnt
total 16
-rwxr-xr-x. 1 root root 0 Oct 12 15:04 file1
-rw-r--r--. 1 root root 0 Oct 12 15:04 file2
drwx------. 2 root root 16384 Oct 12 15:04 lost+found
$ test -x /tmp/mnt/file1; echo $?
0
$ test -x /tmp/mnt/file2; echo $?
1
and as root:
$ sudo guestmount -a test1.img -m /dev/sda1 /tmp/mnt -v -x
$ test -x /tmp/mnt/file1; echo $?
0
$ test -x /tmp/mnt/file2; echo $?
0
In the debug output for non-root we can see the difference:
libguestfs: /file1: testing access mask X_OK: caller UID:GID = 1000:1000, file UID:GID = 0:0, file mode = 100755, result = OK
libguestfs: /file2: testing access mask X_OK: caller UID:GID = 1000:1000, file UID:GID = 0:0, file mode = 100644, result = EACCESS
and for root:
libguestfs: /file1: testing access mask X_OK: caller UID:GID = 0:0, file UID:GID = 0:0, file mode = 100755, result = OK
libguestfs: /file2: testing access mask X_OK: caller UID:GID = 0:0, file UID:GID = 0:0, file mode = 100644, result = OK
After this commit the root output changes to this (ie. same decision
as non-root):
libguestfs: /file1: testing access mask X_OK: caller UID:GID = 0:0, file UID:GID = 0:0, file mode = 100755, result = OK
libguestfs: /file2: testing access mask X_OK: caller UID:GID = 0:0, file UID:GID = 0:0, file mode = 100644, result = EACCESS
When guestfs_lvm_canonical_lv_name was called with a /dev/dm* or
/dev/mapper* name which was not an LV then a noisy error would be
printed. This would typically have happened with encrypted disks, and
now happens very noticably when inspecting Windows BitLocker-
encrypted guests.
This commit hides this error in all cases, although it is still logged
to debug. See comment and the thread below for detailed rationale.
https://www.redhat.com/archives/libguestfs/2020-October/thread.html#00055
Previously callers were unable to distinguish a regular error (like an
I/O error) from the case where you call this API on something which is
valid but not a logical volume. Set errno to a known value in this
case.
In case any bare filesystems were decrypted using cryptsetup-open,
they would appear as /dev/mapper/name devices. Since list-filesystems
did not consider those when searching for filesystems, the unencrypted
filesystems would not be returned.
Note that previously this worked for LUKS because the common case
(eg. for Fedora) was that whole devices were encrypted and thoes
devices contained LVs, so luks-open + vgactivate would activate the
LVs which would then be found by list-filesystems. For Windows
BitLocker, the common case seems to be that each separate NTFS
filesystem is contained in a separate BitLocker wrapper.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Use a LINGUAS file with the list of available translations instead of
defining them in a make variable. This way Weblate will be able to
update the list using an available addon, and we do not need to list
those not built.
Accordingly, rename the variable with built languages to
'linguas_translated'.
Start the message extraction from the toplevel source directory, so the
file references are relative to that, instead of relative to this
po-docs subdirectory.
Also update/regenerate podfiles accordingly.
Start the message extraction from the toplevel source directory, so the
file references are relative to that, instead of relative to this po
subdirectory.
With the Weblate adoption, we let it update the po files from the
catalog template. The po4a behaviour of extracting the template,
merging the existing translations, and creating the translated PODs at
once is problematic. Hence, split the extraction and the translated POD
generation in two.
Use po4a-gettextize to extract the catalog template only, not doing it
anymore automatically at each build. There is no more need for a
po4a.conf file.
Use po4a-translate to create translated PODs from the po files, keeping
the fixup of the generated files (to avoid spurious =encoding, etc).
Add a silent rule to hide the po4a-translate command lines by default.
These changes also allow us to get rid of the POD existance checks with
associated error message pointing to the update-po rule. Now each
translated POD file is generated because of make dependency, and it
depends only on its po file.
Signed-off-by: Pino Toscano <ptoscano@redhat.com>
The guestfish man page uses also additional POD snippets, so list them
as dependencies to make sure they are up-to-date.
This does not change the behaviour at the moment, however it will matter
when each traslated POD file will be generated on its own.
Weblate will handle the update of the po files from the translation
catalog, so avoid stomping on its feet by doing the same.
The translation catalog will be regenerated manually periodically.
Use a LINGUAS file with the list of available translations instead of
defining them in a make variable. This way Weblate will be able to
update the list using an available addon.
Signed-off-by: Pino Toscano <ptoscano@redhat.com>
For the appliance of the QCOW2 format, the function get_root_uuid()
fails to get the UUID of the disk image.
In this case, let us read the first 256k bytes of the disk image with
the 'qemu-img dd' command. Then pass the read block to the 'file'
command.
Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Even though it seems clear from the code that roots cannot be used
uninitialized, GCC 10.1 cannot seem to work it out (possibly an LTO
bug). Easiest way out here is to just initialize it.
rescue.c:396:37: error: 'roots' may be used uninitialized in this function [-Werror=maybe-uninitialized]
396 | CLEANUP_FREE_STRING_LIST char **roots;
| ^
I couldn't get GCC 10.1 to ignore this warning any longer, possibly
because I am using LTO. In any case dereferencing a pointer is
undefined behaviour, so let's use GCC's __builtin_trap() function
instead (also supported by clang).
debug.c: In function 'debug_segv':
debug.c:1002:8: error: null pointer dereference [-Werror=null-dereference]
1002 | *ptr = 1;
| ^
Generated code in virt-builder causes the error below. Since we
cannot control what is generated by bison, increase the limit.
CCLD virt-index-validate
index-parse.c: In function 'yyparse':
index-parse.c:1857:1: error: the frame size of 5152 bytes is larger than 5000 bytes [-Werror=frame-larger-than=]
1857 | }
| ^
lto1: all warnings being treated as errors
lto-wrapper: fatal error: gcc returned 1 exit status
Argon2 is the default LUKS Password-Based Key Derivation Function
(PBKDF) for some new guests such as RHEL 8.2 and Fedora. It is
designed to be "memory hard", meaning that by design it requires large
amounts of memory, making it expensive to brute-force. Unfortunately
the default for guests which had more than a few GB of RAM at install
time is to require about 1 GB of RAM to decrypt the block device,
which is considerably larger than the default available in the
libguestfs appliance.
To make it possible to open these encrypted disks we need to make the
appliance larger. This could be done as a one-off, and the current
workaround is simply to set LIBGUESTFS_MEMSIZE=2048 or a similar
amount. However since we don't know in advance whether we could be
dealing with an encrypted disk, partition, etc. or what PBKDF it uses,
the only way to deal with this in all circumstances is to increase the
default memsize. This commit increases it quite a lot (768 -> 1280)
which is unfortunate.
Note as there is some confusion on this point: Since libguestfs does
not attempt to decrypt disks in parallel, you only need ~ 1GB in
total, not per encrypted disk.
For a reproducer, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1837765#c14
Python 2 reached end of life on 2020-01-01:
https://python3statement.org/https://pythonclock.org/
The minimum version required is now Python 3.4 (since that is the
version in Debian oldoldstable), but 3.6 is the minimum version that
I actually test.
Ubuntu 20.04 could not be built because their installer has changed in
a way which is not compatible with the current build script. This
needs some work to fix.
Replace the use of liberl_interface, which is removed in Erlang 23,
by libei. The implementation uses the ei_decode_iodata() function
which has been introduces only for Erlang 23, so it doesnt work with
earlier Erlang versions.
This new operation removes the Kerberos /etc/krb5.keytab file from the
guest.
Thanks to Christian Heimes and François Cami for the hints.
Related to RHBZ#1789592.
This new operation unenrolls the guest from a IPA server offline, by
removing the configuration files and certificates.
Thanks to Christian Heimes and François Cami for the hints.
Add a simple side effect to make operation flag that a regeneration of
the system CA store is needed. In case it is flagged, regenerate the
system CA store directly, or using a firstboot script in case of
incompatible architectures.
This change is almost a no-op, since no operation requires the
regeneration of the system CA store yet.
For unclear reasons this broke recently, although the breakage is only
reproducible in Fedora Koji. It appears to be caused by the
calculation of the internal autoconf variable $acl_libdirstem
changing. In the reproducer system:
checking for the common suffixes of directories in the library search path... lib,lib,lib64
On my local system:
checking for the common suffixes of directories in the library search path... lib64,lib64
My local system would be the correct one. The actual code that
calculates this does some crazy stuff with ‘gcc -print-search-dirs’
which would hint that this output has changed in some way that subtly
breaks the generated configure script. However even with match GCC
versions on my local system I could still not reproduce the issue.
None of this matters, as the easiest fix here is simply to stop using
internal autoconf variables at all. I also added some AC_MSG_*
statements so we can more easily see what's going on in the configure
output.
Fixes commit 0f79400c7f.
The kernel returns xattr names in a slightly peculiar format. We
parsed this format several times in the code. Refactor this parsing
so we only do it in one place.
At the moment it is empty, so probably it does not exist. Remove it to
avoid adding spurious content to the pkg-config file in case that
variable will get a value in the future.
You have to use:
../libguestfs/run ./configure
../libguestfs/run make
Use of the second ../libguestfs/run against make is unfortunate but I
believe it's unavoidable due to the way that ocamlfind works.
We use a similar trick to libvirt to allow external C programs that
use libguestfs to be compiled against the built (but not installed)
libguestfs with:
../libguestfs/run ./configure
make
What actually happens is we have a second pkg-config file
(lib/local/libguestfs.pc) which points to the locally built
libguestfs. The ./run script sets up PKG_CONFIG_PATH to point to this
directory. Assuming that ./configure is using pkg-config/pkgconf and
not some other half-baked solution it will pick up the libguestfs.pc
file from here which will set CFLAGS and LIBS appropriately.
Causes this error if you compile libguestfs 1.42.0 from the tarball
without invoking the generator:
make[4]: *** No rule to make target '../common/mlv2v/uefi.ml', needed by 'libguestfs.pot'. Stop.
Appliance device names are not reliable since the kernel no longer
enumerates virtio-scsi devices serially. Instead get the UUID of the
appliance and pass this as the parameter.
Note this requires supermin >= 5.1.18 (from around July 2017).
Linux from around 5.6 now enumerates individual disks in any order
(whereas previously it enumerated only drivers in parallel). This
means that /dev/sdX ordering is no longer stable - in particular we
cannot be sure that /dev/sda inside the guest is the first disk that
was attached to the appliance, /dev/sdb the second disk and so on.
However we can still use SCSI PCI device numbering as found in
/dev/disk/by-path. Use this to translate device names in and out of
the appliance.
Thanks: Vitaly Kuznetsov, Paolo Bonzini, Dan Berrangé.
In the guestfs_disk_create API we have traditionally allowed you to
set backingfile without setting backingformat. The meaning of this is
to let qemu autodetect the backing format when opening the overlay
disk.
However libvirt >= 6.0 refuses to even pass such disks to qemu (see
https://bugzilla.redhat.com/show_bug.cgi?id=1798148).
For this reason, move the autodetection earlier and make it explicit.
We now autodetect the format of the backing disk at the time of
creation of the overlay, and set that as the backing format in the
overlay disk itself, allowing libvirt to open the disk later.
This symbol is not present in Python 2.7 or 3.6. It's not really
necessary to call this, it just avoids a crash in certain corner cases
when the interpreter is shutting down. So make the call conditional
on the function existing.
Fixes commit e6f9e0b0f6.
This function doesn't work reliably with the proposed change to device
name translation. The reason is that strings returned by
Devsparts.list_devices contained translated names, so their indexes
did not correspond to the untranslated names used outside the
appliance..
We can avoid this and make the function much simpler and faster by
implementing it on the library side instead.
Although it's highly unlikely in normal use, while testing device name
translation patches it did happen and caused the test to segfault
instead of exiting with an error.
Already outdated, but rounded ;)
I literally just opened the 5yrs logo, changed the text and then done:
inkscape -z -o logo/fish-10yrs.{png,svg}
cp {logo,website}/fish-10yrs.svg
and then updated the rest of the files.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
The current way to get the size of a filesystem is to query the size in
bytes of the device. However, this gives the whole size of the device
where a filesystem is stored, and it does not consider the actual size
for which the filesystem is configured (e.g. in case it was shrunk).
A simple reproducer for this is:
$ guestfish -N test.img=fs:ext4:2G resize2fs-size /dev/sda1 1073741824
As result, try to mount the filesystem, and get its actual statistics to
determine its full size. In case mounting fails, fall back to the
previous method, which is still a good value in the majority of the
cases.
Thanks to: Erik Skultety.
This patch adds '--blocksize' command line option for virt-make-fs
tool. This option allows specifying disk sector size as described in
'guestfs_add_drive_opts' libguestfs API.
On current Fedora releases the ocaml modules will fail to
link unless CFLAGS contains -fPIC.
The autogen.sh script only updates the 'gnulib' submodule,
and so the build will fail due to the missing 'common'
submodule. This needs to be manually initialized at checkout.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The first split does not care about the whole string, it is just trying to get
the command name in front, so triml is just right.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Seeing `g.add_drive_opt :readonly => 1` allows one to imply
that ensuring writable access to drive should happen via
`g.add_drive_opt :readonly => 0`. However, the passed option
value gets passed down to C according to Ruby Boolean semantics,
that is, any value apart from `false` and `nil` will be true
(see RTEST in Ruby C API).
So its more idiomatic and provides a better hint if we use
`g.add_drive_opt :readonly => true` in Ruby samples.
Go API functions returned (<val>, *GuestfsError) that made
code like this fail to build:
n, err := os.Stdin.Read(buf)
if err != nil {
log.Fatal(err)
}
n, err = g.Pwrite_device(dev, buf[:n], off)
...
As err should be of error (interface) type as of the stdlib call,
and should be of *GuestfsError type as of the libguestfs call.
The concrete error value that libguestfs functions return can be
a *GuestfsError, but the function signature should have (<val>, error)
as return value.
This patch adds '--blocksize' command line option for virt-get-kernel
tool. This option allows specifying disk sector size as described in
'guestfs_add_drive_opts' libguestfs API.
This package in Fedora enables optional support for Windows 10
"CompactOS" (file-level compression), read-only, which is sufficient
for inspecting Windows guests and doing certain types of modifications
to them. Virt-v2v appears to work, but it may be that anything that
involves modifying a compressed file might not work.
I couldn't find the equivalent package in Debian or SUSE. It's
available in Arch AUR although I didn't verify that part of the change
actually works there (but should be safe because supermin ignores
packages that are not known about on the target system).
Nowadays there are hard drives and operating systems which support
"4K native" sector size. In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.
GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes. libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.
In order to fix the issue we need a way to provide correct logical block
size for attached disks. Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.
After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important. Thus both physical and logical
block size will be set to the same value.
In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method. Valid values are 512 and 4096.
add_drive_scratch has the same optional parameter for a consistency and
testing purpose.
add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
Libvirt 6.0 now requires that every disk in the backing chain has an
explicit backing format. For example this will be rejected by
libvirt:
qemu-img create -f qcow2 -b backing-disk disk.qcow2
with the error:
Original error from libvirt: Requested operation is not valid:
format of backing image 'backing-disk' of image 'disk.qcow2' was not
specified in the image metadata (See
https://libvirt.org/kbase/backing_chains.html for troubleshooting)
[code=55 int1=-1]
Instead you have to use the -F option to specify the format, eg:
qemu-img create -f qcow2 -b backing-disk -F raw disk.qcow2
I believe this warning is bogus, but simply initializing the local
variable is enough to avoid it.
log.c: In function 'do_log':
log.c:390:7: error: 'comm_len' may be used uninitialized in this function [-Werror=maybe-uninitialized]
390 | printf (" %.*s", (int) comm_len, comm);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The current code was broken, as the field 1 of the exception value is
the error code (int), not an error string, and thus it would have
crashed. This did not happen in practice, as all the usage of
ocaml-augeas were only in the inspection code with ad-hoc exception
catching blocks.
Other than fixing the aforementioned issue, enhance the error reporting
to be as close as possible to what the current AUGEAS_ERROR() macro
does: error message, error minor message (if available), error details
(if available).
Enhance the UEFI firmware lookup function with the information on the
libvirt firmware autoselection, allowing it to return a value to use for
the appliance.
At the moment no firmware is selected this way, so there is no behaviour
change.
This fails with Fedora 32, and possibly earlier versions:
checking if OCaml ‘-runtime-variant _pic’ works... no
The reason is:
$ ocamlc -runtime-variant _pic test.ml -o test
File "test.ml", line 1:
Error: Cannot find file camlheader_pic
which may even be a packaging error in the Fedora package. However it
makes no sense to test the bytecode compiler since we don't use it on
any architecture we care about and bytecode doesn't even contain a
linked runtime. Changing the test to use ocamlopt instead of ocamlc
fixes the problem.
Properly mark the long line with the license classifier in setup.py:
the classifier is long on its own, and changing the code to stay within
79 characters would result in worse code.
Fix continuation indentation, and whitespaces around operators.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
Add or remove empty lines to match the needed ones around
blocks/functions/etc.
Adapt the generation of guestfs.py to emit the separating empty line
before adding a new function/alias, rather than after it.
This is just formatting, no behaviour changes.
In Python 2 code branches use PyString_FromStringAndSize directly
instead of the guestfs_int_py_fromstringsize wrapper (which calls
PyString_FromStringAndSize anyway on Python < 3).
The PyString API is specific to Python < 3, replaced by PyBytes and
PyUnicode in Python 3+; hence, drop the compile time check, and use the
right API depending on the Python version.
If it is not possible to detect the distribution of a Linux OS, do not
propose "unknownX.Y" (where X is the major version number, and Y the
minor) as short osinfo ID. Just return "unknown" instead.
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list. That source
contained also functions using gnulib functions, however without
linking to gnulib. When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.
As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
needed
Move the interal static libraries as the last items in the list of
libraries of guestfsd, to make sure their symbols are used for all the
other libraries. This is because GCC resolves the symbols looking at
the arguments from the beginning to the end of the command line.
This currently does not cause failures, however it "just works" because
of the tricky situation set up.
The situation is the following:
1) common/utils contains few utility sources: one of them is utils.c,
which contains various functions -- for example
guestfs_int_free_string_list and guestfs_int_drive_name --, it is built
as utils.o, and bundled in the static library libutils.a
2) common/mlutils builds a OCaml library with bindings for some utility
functions in libutils.a, in particular guestfs_int_drive_name (but not
guestfs_int_free_string_list); there are two versions of this library,
one OCaml library (dllmlcutils.so) that links with libutils.a, and one
static library (libmlcutils.a), which cannot specify the libraries it
links to (as it is static)
3) when the daemon is linked, the command line was the following
(simplified):
$ gcc [...] -o guestfsd guestfsd-9p.o other_daemon_object.o [...] \
../common/utils/.libs/libutils.a [...] -lmlcutils [...]
Some of the objects of the daemon itself use
guestfs_int_free_string_list, and thus the compiler opens libutils.a
(it is after the objects in the command line) and picks utils.o, which
contains also guestfs_int_drive_name (not used directly in the daemon);
when linking later on with libmlcutils.a, the symbols for this static
library (like guestfs_int_drive_name) are already resolved, and thus
all the symbols are resolved, and the linking succeeds
This fragile situation can be easily broken by moving e.g.
guestfs_int_drive_name out of common/utils/utils.c to a new source (say
utils2.c) still built as part of libutils.a: since nothing before
-lmlcutils actually needs to pick utils2.o from libutils.a for symbols,
then GCC will not be able to resolve all the symbols in libmlcutils.a.
As solution, move libutils.a (and other internal static libraries) as
last libraries to link guestfsd to: this way, GCC knows where to find
all the symbols needed by all the objects and libraries specified in
the command line.
Prune from the list of sources where to extract messages various sources
with no messages:
- .pl and .pm files, as they do not contain messages: almost all the
.pl files are tests, and the only .pm file is the Perl Sys::Guestfs
module, which wraps the XS extension
- dummy.c sources; they are empty sources used to build OCaml-only
targets using automake
- gperf generated sources
- C/OCaml tests
As proposed in the Debian bug #946594 [1], the implicit .cmi/.cmo/.cmx
dependencies are tied to files in the source directory only, and thus
they break when either the .ml or the .mli files are in the build
directory. Since there is already VPATH set up by automake, rely on it
to locate the right source.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946594#36
Solution proposed by Vincent Danjean, thanks!
This reverts commit 802c5d2055 in this
file.
Add a copy of the libvirt-ocaml library, currently available at:
http://git.annexia.org/?p=ocaml-augeas.git;a=summary
This is a snapshot at commit 1cf5aef99b26a46529ca797547c0b49627fffe78,
which has all the features we need (and that builds fine).
It is expected to stay synchronized with upstream, until there is a new
upstream release, and it will be widespread enough.
This helper script will contain variables with results of configure
checks, so other scripts can source it.
Source it automatically in test-functions.sh, so every test can already
make use of it.
Invoke make-internal-documentation.pl by its path in the source
directory.
Also, depend on it to regenerate the documentation in case
make-internal-documentation.pl changes.
The current method of adding multiple --insert or --verbatim
parameters to the podwrapper command is not very easy to use because
it involves modifying the Makefile.am in every place where this is
used, plus under po-docs/$language/Makefile.am. It's better if the
POD file itself can do the inclusion.
This enhances support so that the special sequences
__INCLUDE:file.pod__
or
__VERBATIM:file.txt__
are treated as file inclusion.
The filename must be a plain file and is searched along a path
(supplied by the optional podwrapper --path parameter). The purpose
of the path is to allow translations to happen more easily. For
example we can include a particular POD fragment from common/options/
for the English version, but copy the translated file to
po-docs/$language/ for every translated version.
Remove gnulib modules that provide stuff clearly not used within
libguestfs (library, daemon, and C tools). Among directly and
indirectly modules used previous (and now no more), they are:
cycle-check
d-ino
dev-ino
dup3
dup3-tests
fcntl-safer
fcntl-safer-tests
fdopendir
fdopendir-tests
filevercmp
filevercmp-tests
ftell
ftell-tests
ftello
ftello-tests
fts
getaddrinfo
getaddrinfo-tests
getcwd
getcwd-tests
gnu-make
hostent
i-ring
i-ring-tests
iconv
iconv-tests
inet_ntop
inet_ntop-tests
isatty
isatty-tests
openat-safer
openat-safer-tests
opendirat
ptsname_r
ptsname_r-tests
read-file
read-file-tests
rewinddir
servent
ttyname_r
ttyname_r-tests
xgetcwd
Some of the removed modules are still used pulled indirectly as
dependency of other modules. There should be no behaviour change on
recent Linux distros, although older distros were not tested (adding
a module back is easy, anyway).
Remove accordingly unused automake variables, and ignored files.
Move the read_whole_file function to the common utilities of the daemon,
so other parts can use it. For this purpose, add an out parameter to
get the amount of bytes read.
Except from the parameter addition, this should be just refactoring.
xgetcwd is used only in a test, so there is no need to pull a gnulib
module just for it.
Switch to use getcwd directly with a fixed buffer: the tests would have
failed with paths longer than 992 characters, as the libvirt_uri would
have been truncated. Since there were no reports of issues, we can
assume that the current working directory will fit in 1024 characters;
adapt the size of libvirt_uri accordingly.
If you've registered a callback in Python and the handle is implicitly
closed when the Python interpreter exits, then it can be that the
following happens:
- Python interpreter is finalized.
- guestfs_close is called
- callback is invoked (eg. close event or to display a debug message)
- Python code runs from the callback wrapper
This cause a segfault on shutdown. Catch this case and stop the
callback from running (we lose the event but given the above sequence
of events there's not much we can do about it).
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1773520#c7
All Py_* functions should be protected by the GIL.
Otherwise internal python data structures can get corrupted.
Move PyGILState_Ensure to the beginning of the block and
PyGILState_Release to the bottom.
Signed-off-by: Sam Eiderman <sameid@google.com>
The generator creates these files with 0444 mode, so they cannot be
overwritten by a simple ‘cp’ command. We could use ‘cp -f’ or ‘rm -f’.
Fixes commit 15394cb4dd.
With the proposed split we will only run the generator from the
libguestfs repo. When compiling virt-v2v or the guestfs-tools we will
need certain generated files to be present already in the
libguestfs-common repo, and therefore these files must be added to
git. Hopefully they won't change very often.
After the proposed split of the libguestfs repo, we will end up with
the following layout:
libguestfs.git
common -> git submodule libguestfs-common.git
generator
virt-v2v.git
common -> git submodule libguestfs-common.git
guestfs-tools.git
common -> git submodule libguestfs-common.git
The generator will only be able to write to libguestfs directories and
the common directory/submodule. This is mostly the case already with
only 6 exceptions:
customize/customize-options.pod
customize/customize-synopsis.pod
customize/customize_cmdline.ml
customize/customize_cmdline.mli
v2v/uefi.ml
v2v/uefi.mli
This commit moves these files around so they appear under common/ml*
It is somewhat unsatisfactory because it involves copying files
around, but there are some mitigating factors:
(1) Any changes now give us more freedom to develop faster and thus
clean things up in future.
(2) The v2v/uefi files ought to go away in future anyway.
This is simple code motion and should have no effect on the built
programs or tests.
These two modules are a dependency of virt-v2v. Since we intend to
split virt-v2v from the other OCaml virt-* programs, we cannot have a
dependency between virt-v2v and virt-customize. Instead we must move
the modules to a common directory (common/mlcustomize) and have both
tools depending on the modules from there.
This is simple refactoring and should not affect how the programs work
or are tested.
The existing output objects only used the s_name field so pass that
instead. (There was one minor exception: -o libvirt adjusted an error
message based on the hypervisor field but that was simply to fix.)
Existing conversions do not use anything except the source disks, so
only pass this.
This also renames src_disks -> source_disks in a few places for
consistency.
Pure refactoring with no change in function, designed to reduce
coupling between stages.
Use firstboot script to install MSI with QEMU-GA from virtio-win ISO or
oVirt/RHV guest tools ISO.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
For Linux the guest itself remembers the IP address associated with
each MAC address. Thus it doesn't matter if the interface type
changes (ie. to virtio-net), because as long as we preserve the MAC
address the guest will use the same IP address or the same DHCP
configuration.
However on Windows this association is not maintained by MAC address.
In fact the MAC address isn't saved anywhere in the guest registry.
(It seems instead this is likely done through PCI device type and
address which we don't record at the moment and is almost impossible
to preserve.) When a guest which doesn't use DHCP is migrated, the
guest sees the brand new virtio-net devices and doesn't know what to
do with them, and meanwhile the right static IPs are still associated
with the old and now-defunct interfaces in the registry.
We cannot collect the required information from within the guest.
However we can collect it outside the tool by some other means
(eg. using VMware Tools APIs) and present this information to virt-v2v
which then writes it into the Windows guest at firstboot time.
This commit adds the --mac ..:ip:.. sub-option which creates a
Powershell script to set network adapters at firstboot. An option
such as:
--mac 00:0c:29:e6:3d:9d:ip:192.168.0.89,192.168.0.1,24,192.168.0.254
approximately turns into this script:
# Wait for the netkvm (virtio-net) driver to become active.
$adapters = @()
While (-Not $adapters) {
Start-Sleep -Seconds 5
$adapters = Get-NetAdapter -Physical |
Where DriverFileName -eq "netkvm.sys"
}
$mac_address = '00-0c-29-e6-3d-9d'
$ifindex = (Get-NetAdapter -Physical |
Where MacAddress -eq $mac_address).ifIndex
if ($ifindex) {
New-NetIPAddress -InterfaceIndex $ifindex
-IPAddress '192.168.0.89'
-DefaultGateway '192.168.0.1'
-PrefixLength 24
Set-DnsClientServerAddress -InterfaceIndex $ifindex
-ServerAddresses ('192.168.0.254')
}
Thanks: Brett Thurber for diagnosing the problem and suggesting paths
towards a fix.
This experimental filter can be used to work around brief
interruptions in service, such as the network going down, firewalls
timing out connections etc., without requiring virt-v2v to be rerun.
For input methods which use nbdkit, we can cheaply add
nbdkit-rate-filter to control input-side network bandwidth. These
options control that filter. We can choose to set the bandwidth
statically and optionally change it dynamically:
--bandwidth 10M
# static bandwidth of 10 Mbps, no dynamic adjustment possible
--bandwidth 5M --bandwidth-file /tmp/bw
# initial static bandwidth of 5 Mbps, adjustable by writing to /tmp/bw
--bandwidth-file /tmp/bw
# no initial bandwidth cap, can be added later by writing to /tmp/bw
It only makes sense to control the input side since virt-v2v writes a
lot less data than it reads.
‘virt-v2v -i libvirtxml’ has a little-known feature where it can read
network disks over HTTP or HTTPS. This can be used to test VMware
conversions without needing VMware, although as far as I can tell the
current test suite does not use the feature. This commit changes this
functionality to use nbdkit-curl-plugin instead of the qemu curl
driver.
This change shouldn't affect functionality. The readahead size is
changed from a fixed 1M buffer to using the readahead filter which is
self-configuring.
See also commit 38bf2a0f97 which
originally introduced this functionality in 2017.
The readahead filter is a self-configuring filter that makes
sequential reads faster when the plugin is slow (and all of the
plugins we use here are always slow).
I observed the behaviour of the readahead filter with our qcow2
overlay when converting a guest from a vCenter source. Even when
doing random reads, qemu issues 64K reads which happen to also be the
minimum request size of the readahead filter, so there is no extra
overhead. When doing the sequential copy the readahead filter
performed better than qemu curl’s readahead because it scaled the
prefetched data appropriately on long contiguous stretches and then
shrunk it back to 64K around fragmented parts of the base image.
This uses the technique described in the nbdkit-probing(1) man page.
It should work with a wide range of versions of nbdkit, and is the one
which the nbdkit developers currently recommend.
Initially this is a like-for-like replacement, but in future commits
this will allow us to implement:
- password authentication (instead of SSH agent)
- bandwidth throttling
- readahead
Note this requires nbdkit >= 1.12.
In forthcoming commits we will be adding support for ssh, curl and
other features that require nbdkit >= 1.12.
As a prelude to that work, add generic code for querying ‘nbdkit
--dump-config’ and ‘nbdkit plugin --dump-plugin’ and checking the
minimum version number.
This changes the minimum version from 1.1.16 to 1.2, although that was
released about a year ago and is widely available, and in any case
we're going to require 1.12 in the next commit.
This refactoring takes the nbdkit-specific code from the ‘-it vddk’
mode and puts it into a separate module. The idea is to reuse this
module (in future commits) to support ssh, curl and rate limiting.
This refactoring is not quite neutral because it moves some of the
prechecking into the Nbdkit.create function. This means it will
happen a little bit later in the cycle (in input#source instead of
input#precheck - refer to the diagram in v2v/types.mli). However it's
still almost in the same place and importantly still happens before we
start copying.
cgo does not allow arbitrary CFLAGS to be used. Instead it contains a
list of flags (safelist) that are allowed to be passed to the compiler.
Sadly -U option (introduced in commit d8d8c856a1) is not among them.
See: https://github.com/golang/go/issues/23672
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
It makes little sense to require the oVirt certificate, especially when
the verification of the connection (-oo rhv-verifypeer) is disabled by
default. The only work done with the certificate in that case is
checking that it is a valid certificate file.
Hence, make -oo rhv-cafile optional, requiring it only when
-oo rhv-verifypeer is enabled.
By default the installer will set the hostname to what the DHCP returns,
exposing details of the machine where make-template.ml runs.
Instead, force "unassigned-hostname.unassigned-domain" as hostname, so
plays nicely with the hostname setting code in virt-customize.
According to the Erlang website:
The old legacy erl_interface library (functions with prefix erl_) is
deprecated as of OTP 22, and will be removed in OTP 23. This does
not apply to the ei library. Reasonably new gcc compilers will issue
deprecation warnings. In order to disable these warnings, define the
macro EI_NO_DEPR_WARN.
That's a shame and probably means we will have to drop the Erlang
bindings soon unless someone ports them to this new API (stable APIs
FTW people!). In the meantime add the flag to prevent warn-errors
about deprecation.
This way it is possible to override the UUIDs of the uploaded disks,
instead of letting RHV generate them.
This can be useful to force certain UUIDs, and to specify the disks in
--no-copy mode (which now can be used).
Check for the INITIALIZING state of the image transfer right away,
without waiting 5 seconds even before the first time: this way, if the
transfer is already in the right state then there is no need to wait.
Otherwise it complains about missing files that it has no rules for, for example
`builder/index-parser.c`.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Instead of waiting for the completion of the nbdkit transfers to get the
UUIDs of the disks, use the new #disk_copied hook to do that after each
disk is copied.
This has almost no behaviour on rhv-upload, except for the --no-copy
mode:
- previously it used to hit the 5 minute timeout while waiting for the
finalization of the first disk
- now it asserts on the different number of collected UUIDs vs the
actual targets; at the moment there is nothing else that can be done,
as this assumption is needed e.g. when creating the OVF file
Improve the way the precheck script checks for the specified resources:
- look directly for a data center with the specified storage domain
- get the storage domain object from the storage domains attached to the
data center found
- similarly, look for the specified cluster among the ones attached to
the data center found
When everything is found, return the UUID of the storage domain, and of
the cluster back to virt-v2v, which will store them.
Similarly, rework the createvm script to directly get the requested
cluster, instead of looking for it once again. Also, since the UUID of
the storage domain is available in virt-v2v already, use it directly
instead of using a placeholder.
This should fix a number of issues:
- unexisting/unattached storage domains are rejected outright
- the cluster is rejected if not part of the same data center of the
selected storage domain
- renaming the specified storage domain during the data copying will not
cause the conversion to fail (which will still use the specified
storage domain, no matter the new name)
Based on the hints by Daniel Erez in RHBZ#1612653.
This way it is possible to communicate data from the precheck script
back to virt-v2v.
For now there are no results, so the resulting JSON is discarded.
Split the VM existance check out of the precheck script to a new vmcheck
script, and invoke that in #prepare_targets. Invoke the precheck script
in #precheck, as now it can be run with only values of command line
options.
This does not change which checks are performed; however, an invalid
cluster name will make virt-v2v fail way earlier (even before connecting
to the source).
This removes only the tool itself, and all the bits strictly needed to
not break the build.
This is now available as separate tool in its own repository:
https://github.com/libguestfs/virt-p2v
Added in virt-p2v commit:
commit e83b6f50af34ce650063ecc520bfabab400e8e73
Author: Matthew Booth <mbooth@redhat.com>
Date: Fri Mar 26 09:40:20 2010 +0000
Add export to RHEV
Allow guests to be written to a RHEV NFS export storage domain.
Add 'rhev' output method and -osd command-line option.
Example command line:
virt-v2v -f virt-v2v.conf -ic 'esx://yellow.rhev.marston/' \
-o rhev -osd blue:/export/export RHEL3-32
This will connect to an ESX server and write the guest 'RHEL3-32' to
blue:/export/export, which is a pre-initialised RHEV export storage domain.
make[2]: *** No rule to make target '../p2v/about-authors.c', needed by 'internal-documentation.pod'. Stop.
Since we're going to remove the whole p2v subdirectory shortly anyway,
it was simplest to ignore the whole directory.
Avoids this error when building from git:
make[2]: *** No rule to make target '../common/mllibvirt/libvirt_c.c', needed by 'internal-documentation.pod'. Stop.
This file doesn't have any generated docs or gettext annotations in it
so adding it to these files is useless anyway.
Disable this warning/error in the Python 3.8 header files:
In file included from /usr/include/python3.8/abstract.h:837,
from /usr/include/python3.8/Python.h:147,
from actions.h:31,
from actions-6.c:34:
/usr/include/python3.8/cpython/abstract.h: In function '_PyVectorcall_Function':
/usr/include/python3.8/cpython/abstract.h:91:11: error: cast increases required alignment of target type [-Werror=cast-align]
91 | ptr = (vectorcallfunc*)(((char *)callable) + offset);
| ^
Previously to work around some problems in Python 2 header files we
had to include <Python.h> before any other config file.
For Python 3 which is all we really care about now this is no longer
needed. We can move the include from three files into the local
"actions.h" file, bringing all the Python definitions and workarounds
into a single place.
GtkAttachOptions is part of GtkTable, which is used only with GTK < 3.4;
however, these enum values are used also in the GtkGrid version of the
code, so they are needed also when disabling deprecated stuff.
As easy solution to make the current code working without deprecated
stuff of GTK, copy the GtkAttachOptions enum when using GtkGrid (i.e.
with GTK >= 3.4).
- feed the content directly to stdin, avoid the need of read (and write)
a temporary file
- read all the output at once, without a tail-recursive function
- apply trimming and first line discarding after closing the process
Isolate the logic for the memoized disk cache in a small module, so it
can be reused for other tools.
Other than refactoring, there should be no behaviour changes.
- add a no-op Connection.close(), as it called explicitly in the
close() callback of the nbdkit plugin (rhv-upload-plugin.py)
- fix the types of the 'id' variables, which are strings
- fix names of arguments & optional arguments in C<..> markers
- use https for URLs where possible
- fix links to other guestfs APIs
- use more C<..> markers for special tests, shell commands, values of
arguments, and names of fields
- link to command man pages where an explicit command is mentioned
- fix few incorrect documentation bits
This patch includes:
- Event callback handlers
- Tests related to events(410-430)
src/bin/event.rs and src/bin/event_leak.rs
are the PoCs that Boxes related to callbacks are
not leaked.
Without clarifying handle's lifetime, it is unable
to see how long the callbacks which the handle
owns will live. Then, Rust compiler will infer
that the callbacks have 'static lifetime. It is
not convenient for users.
BUGS
docs/C_SOURCE_FILES
po/POTFILES
po/POTFILES-ml
These files are normally updated by ‘make dist’ but as we've not had a
release in a while they had not been updated for some time.
Since there is no more need to build nbdkit from sources, then there is
no need to set $PATH with a custom build of nbdkit.
Followup of commit 0704d8eb0b.
Point to files in the source directory using the right variables, so
they are found also when the build directory is different than the
source directory.
While it is generated at configure time for dependency reasons, consider
it a generated source nevertheless: after the first run, there will be
dependency rules available, so automake will trigger the proper rule to
generate it again using generate-p2v-config.pl.
Soon only the virt-p2v authors will be available, so remove all the
other roles. This leaves only in the virt-p2v about dialog, which is a
mild regression compared to the current situation, although it is just
for user information.
Move (or copy the general ones) all the p2v entries in the top-level
.gitignore file to a new file specific for p2v. This will make it
easier to keep them when splitting p2v.
Copy from generator/p2v_config.ml also the bits for POD documentation,
adding them to generate-p2v-config.pl; this way,
virt-p2v-kernel-config.pod can be generated at build time directly,
instead of statically shipped as generator output.
Instead of generating the p2v kernel config using the OCaml generator,
create a Perl script to do this job, mostly at build time. This is done
to rely less on the generator for p2v, and because the generation of
these sources is quick enough that it can be done at build time (instead
of shipping the generated sources in dist tarballs).
The generate-p2v-config.pl mimics what generator/p2v_config.ml --
namings, and general structure are kept close to that for comparison.
The two C sources are created at build time by the script; however, the
p2v-config.h header is generated at configure time: this is done because
p2v-config.h is included by p2v.h (another header), which in turn is
included by all the p2v C sources -- automake is not able to properly
resolve the dependency, and thus it would not be generated properly.
Python 3.8 no longer links C extensions to -lpython, instead relying
on the fact that the python binary itself already contains those
symbols. This means $PYTHON_LIBS is empty and so the Python bindings
are not built.
Use a different test to see if the python module is available.
This way no non-namespaced OCaml C symbols are used, reducing the risk
of clashes with other code.
The only exception is ocaml-augeas, which does not build with
CAML_NAME_SPACE; it will be fixed upstream, and it affects only
ocaml-augeas itself.
While svirt_t can be used for sockets it does not always guarantee that it will
be accessible from a virtual machine. The VM might be running under svirt_tcg_t
context which will need a svirt_tcg_t label on the socket in order to access it.
There is, however, another label, svirt_socket_t, which is accessible from
virt_domain:
# sesearch -A -s svirt_t -c unix_stream_socket -p connectto
...
allow virt_domain svirt_socket_t:unix_stream_socket { ... connectto ... };
...
And virt_domain is a type attribute of both svirt_t and svirt_tcg_t:
# seinfo -x -a virt_domain
Type Attributes: 1
attribute virt_domain;
svirt_t
svirt_tcg_t
Resolves: https://bugzilla.redhat.com/1698437
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
It seems that on older Python 3 versions importlib.util is not imported
automatically when importlib is imported; as solution, import
importlib.util directly (as it is what is used, anyway).
Fixes commit f79129b8dc.
Previously, is_custom_hv() used to compare the QEMU executable found
during configure to the hypervisor set to check whether it is a custom
one; however, the QEMU found at configure time can be different than
what libvirt was configured with.
This fixes the libvirt backend when libguestfs is configured with a
different QEMU, that now will be specified as emulator overriding the
libvirt one.
We ship our own copy of it, so we do not need the external version.
(Also, the latest upstream version of ocaml-libvirt was already not
usable to build the test harness of v2v.)
Now that we have a proper libvirt connection object, use it directly to
refresh the storage pool, and define the final guest. This avoids
spawning a new virsh process twice, with no possibility to even share a
possible authentication required.
Store the Libvirt.Connect.t object as instance variable, so it can be
used also outside of prepare_targets. Use a private method to access
it, so there is no need to directly use the Lazy object.
Currently virt-v2v has few custom C-based functions for libvirt
operations, which are limited in what they do, and there is a lot of
duplicated code.
Instead, switch to ocaml-libvirt for all the libvirt interaction
currently done by the Libvirt_utils module. This has few advantages:
- each input & output module now opens a libvirt connection only once,
only when needed
- no need to pass URIs and passwords around, if not needed
- a wider range of libvirt APIs can now be used, with no need to create
bindings manually
The hierarchy of input_libvirt* classes is changed to take a Lazy object
with the libvirt connection, accessing it through a "proctected" method:
this way, the connection is opened only at the first access.
Also, the Libvirt_utils module now is just helpers around the Libvirt
module, to centralize error handling, and few common operations.
Add a copy of the libvirt-ocaml library, currently available at:
https://libvirt.org/git/?p=libvirt-ocaml.git;a=summary
This is a snapshot at commit d3ed8dcf1b0a6a8a855ceecbe0bb97f21e6665e3,
which has all the features we need (and that builds fine).
It is expected to stay synchronized with upstream, until there is a new
upstream release, and it will be widespread enough.
While there are input modes that do not use libvirt, making libvirt
mandatory for virt-v2v slightly simplifies the code now, and allow for
further improvements/integration with libvirt later on.
Adapt make-fedora-img.pl to not use deprecated APIs anymore:
- set_e2uuid -> set_uuid
- txz_in -> tar_in + compress:xz
This causes no changes in the generated test images.
Since we already assume Python >= 2.7, modernize this example to make it
work also on Python 3:
- use print() as function
- sort the mount points using a key for sorted(), instead of a
comparison function
- remove extra newline escape
- reident two lines according to the PEP 8 style
Add a simple way to do not even provide prototypes of deprecated
functions in the C library: this way, users (like our tools) can build
against the library making sure to not use any deprecated function, not
even when compiler deprecation warnings are disabled.
Add it to the majority of our tools/internal libraries, and make sure
that it is not defined when building the API bridges of our bindings.
Right now, deprecated functions of the library do not trigger any
compiler deprecation warning by default; they do that only if
GUESTFS_WARN_DEPRECATED=1 is defined. However, this is not something
that seems to be done often -- at least none of the projects using the
libguestfs C API does that.
Hence, do a small behaviour change to change this on the other way
round: now deprecated functions trigger compiler deprecation warnings by
default, using GUESTFS_NO_WARN_DEPRECATED to disable this (and revert
to the previous behaviour). Even though deprecated functions will not
be removed, we really want users to migrate away from them, as they were
deprecated for good reasons.
Define GUESTFS_NO_WARN_DEPRECATED where needed:
- in all the bindings, as they bind all the functions including the
deprecated ones
- in the guestfish actions, as it exposes almost all the APIs
- in the C API test, as it runs the automated tests of all the APIs that
have them
- for two tests that explicitly test deprecated functions
part-expand-gpt takes extreme cautions and doesn't proceed to writing
to the disk if the preliminary dry run of sgdisk has generated any
warnings on stdout.
This blocks the use of part-expand-gpt on disk shrink (with disk
resize being the main usecase for part-expand-gpt), because sgdisk dry
run produces a warning in that case.
So remove the excessive safety check, and leave it up to the caller.
Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
This is not quite a neutral refactoring, because it means we now run
the getenforce command every time virt-v2v starts up. However it's a
trivial command that reads a single /sys file and it can't fail even
if the command is missing or on platforms that know nothing about
SELinux.
virt-v2v obviously cannot convert this kind of devices, since they are
specific to the host of the hypervisor. Thus, emit a warning about the
presence of direct network interfaces, so at least this can be noticed
when converting a guest.
Starting with 5.2.0, libvirt has a way to select automatically the
firmware for a guest using an attribute of the <os> tag. Hence, use
this information (when available, of course) to flag the firmware used
by the guest.
Return the right osinfo short IDs for some rolling Linux distributions,
such as Arch Linux, Gentoo, and Void Linux. Their IDs were recently
added to osinfo-db.
Consider Arch Linux as rolling distribution, so it is recognized using
/etc/os-release.
The end result does not change, although this makes Arch Linux inspected
using os-release only, instead of getting inspection details mixed from
both os-release and lsb-release.
Add "gentoo" as recognized ID in /etc/os-release, and consider it as
rolling distribution (so without VERSION_ID in os-release).
This avoids using a not-useful version read from /etc/gentoo-release,
e.g. "Gentoo Base System release 2.6".
Use a static list of rolling distros, so it is easier to handle them
differently; use this list for handling the lack of VERSION_ID in
os-release files.
This is just refactoring, no behaviour changes.
Add a new output mode to virt-v2v: similar to -o local, the written
metadata is a JSON file with the majority of the data that virt-v2v
knowns about (or collects) during the conversion.
This is meant to be used only when no existing output mode is usable,
and a guest needs to be converted to run on KVM anyway. The user of
this mode is supposed to use all the data in the JSON, as they contain
important details on how even run the guest (e.g. w.r.t. firmware,
drivers of disks/NICs, etc).
When the machine readable mode is enabled, print all the messages
(progress, info, warning, and errors) also as JSON in the machine
readable stream: this way, users can easily parse the status of the
OCaml tool, and report that back.
The formatting of the current date time into the RFC 3999 format is done
in C, because of the lack of OCaml APIs for this.
Allow to specify a file descriptor for the machine readable output.
Use the same assumption as done in v2v, i.e. that Unix.file_descr is
simply the int file descriptor.
Enhance the helper printf function for machine readable output to always
flush after each string: this way, readers of the machine readable
stream can get the output as soon as it is outputted.
Store log from MSI installer. Log file will be located in firstboot
scripts-done directory with name rhev-apt.log. The path has to be
double-quoted to handle spaces in path name properly.
Hopefully this can help resolve RHBZ#1584678 someday.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The debug message was slightly changed too to better match the similar
message for ISO case. It refers to the root directory instead of the
specific subdirectory inside guest tools.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The RHV Tools ISO is provided as Red Hat only product, and thus not
available for all the virt-v2v users. Hence, change the way we report
the status of the installation of the qemu guest agent from the RHV
Tools ISO:
- do not warn if virt-v2v does not know how to install the package for
the current guest
- do not warn if the ISO does not contain packages for the current guest
- on successful installations, show an info message
Related: RHBZ#1691659
Right now the code copies, and then tries to install, all the files
found for the directory of a distro. This does not take into account
differences in the architectures of the packages available, so a x86_64
package must be installed only on x86_64 guests.
As solution, filter the packages using the typical suffix used by
packages (so architecture + file extension), to pick only packages
compatible with the current guest.
Schema parsing was failing with errors such as:
libguestfs: QMP parse error: '[' or '{' expected near end of file (ignored)
This happened because the QMP command was actually completely failing
and never printing a result at all. This happens because the qemu
audio driver can't be set up without a console. We can suppress this
by setting the environment variable QEMU_AUDIO_DRV=none, which is the
same thing that libvirt does, and also the same thing that we are
already doing when launching the real appliance subprocess.
See also: https://bugzilla.redhat.com/show_bug.cgi?id=1692047
Make sure the create guest is always up-to-date: this way it is possible
to test the conversion even taking into account the latest upgrades
available.
ubuntu-server depends on open-vm-tools on Ubuntu, so if v2v tries to
uninstall open-vm-tools then dpkg will (rightfully) fail with a
dependency issue.
Since open-vm-tools is harmless after the conversion anyway (it will
not even run), then do not attempt to uninstall it if ubuntu-server is
installed too.
Followup of commit 2bebacf8bf.
Thanks to: Ming Xie.
When writing the libosinfo metadata in the libvirt XML, use the newly
added (in osinfo-db) ID for Windows Server 2019; sadly, this version of
Windows has the same version as Windows Server 2016, so distinguish it
by looking at its product name.
Use a better icon for RHEL guests, still provided by redhat-logos (or
equivalent in downstream distributions), and which fits a better
definition of logo for the distribution.
Thanks to Ray Strode for the hints.
Try to look for a well known kernel module (so far only virtio, or kvm)
to use for detecting the architecture of a kernel. This way, we can
avoid picking 3rd party modules that cause troubles.
Kernel modules can be also symlinks to files available outside the
"canonical" module directory; the "file" API, used by the
"file-architecture" API, return the actual type of a file (so symlinks,
block devices, etc), and thus "file-architecture" fails on symlinks.
To prevent this situation, canonicalize the path of the module picked
for architecture detection: this way, "file-architecture" will act on a
real file.
test-relabel.pl performs a full SELinux relabelling of the phony guest,
and the selinux_relabel API does not work if the "selinuxrelabel"
feature is not available.
The icon is installed by x11-themes/gentoo-artwork, which is not
installed by default (and most probably only on "desktop"
installations), although it's the best hit so far...
Newer versions do not have the 24px distributor.png icon; OTOH they have
the 48px version, so look for that one before the 24px one.
Also, bump the size limit to 10K, as newer versions of the icon are
bigger than 3K.
Isolate the code for looking for a PNG file among a specified list from
the icon_ubuntu implementation, and switch that function to it.
This is just code refactoring, with no behaviour changes.
Use NEVR when querying RPM for the list of files of a package, instead
of ENVR. Also, use the epoch only when non-zero, and version of RPM
supports it.
The approach is basically copied from what supermin does in its RPM
package handler.
So far RBufferOut return values, and FBuffer struct fields are 'str' on
all the versions of Python. Python 3 distinguishes between 'str'
(unicode strings), and 'bytes', with 'str' no more able to hold
arbitrary data.
For this reason, switch the return value of RBufferOut functions, and
FBuffer struct fields to bytes on Python 3: while this is a potentially
incompatibile change, this is the only way to handle safely sequences
of arbitrary bytes.
Make sure to reference the arguments, to make sure they are kept alive
during the function call; this is visible when setting an event handler
for the CLOSE event, and testing it with Python 3.
This does not seem to create a memory leak e.g. with Python 2.
Also, switch away from the quasi-internal PyEval_CallObject to the
public PyObject_CallObject, which takes care of doing safety checks.
error(f_"no GNU Privacy Guard (GnuPG, gpg) binary was found.\n\nEither gpg v1 or v2 can be installed to check signatures. Virt-builder looks for a binary called either ‘gpg2’ or ‘gpg‘ on the $PATH. You can also specify a binary using the ‘--gpg’ option. If you don't want to check signatures, use ’--no-check-signature’ but note that this may make you vulnerable to Man-In-The-Middle attacks.")
(* --- If we get here, we want to create a guest. --- *)
(* Warn if the user might be writing to a partition on a USB key. *)
(matchcmdline.outputwith
|Somedevicewhenis_partitiondevice->
ifcmdline.warn_if_partitionthen
warning(f_"output device (%s) is a partition. If you are writing to a USB key or external drive then you probably need to write to the whole device, not to a partition. If this warning is wrong then you can disable it with --no-warn-if-partition")
device;
|Some_|None->()
);
(* Download the template, or it may be in the cache. *)
error(f_"%s checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the ‘-v’ option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
error(f_"no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.")in
error(f_"no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.")
in
Customize_run.rungrootcmdline.customize_ops;
(* Collect some stats about the final output file.
* Notes:
* - These are virtual disk stats.
* - Never fail here.
*)
letstats=
ifnot(quiet())then(
try
(* Calculate the free space (in bytes) across all mounted
elseifi=len1||j=len2thenfalse(* no match - different lengths *)
else(
letx1=getxdigitfp1.[i]andx2=getxdigitfp2.[j]in
matchx1,x2with
|Somex1,Somex2whenx1=x2->loop(i+1)(j+1)
|Somex1,Somex2->false(* no match - different content *)
|Some_,None->loopi(j+1)
|None,Some_->loop(i+1)j
|None,None->loop(i+1)(j+1)
)
in
loop00
andgetxdigit=function
|'0'..'9'asc->Some(Char.codec-Char.code'0')
|'a'..'f'asc->Some(Char.codec-Char.code'a')
|'A'..'F'asc->Some(Char.codec-Char.code'A')
|_->None
letverifying_signaturest=
t.check_signature
letrecverifytfilename=
ift.check_signaturethen(
letargs=quotefilenamein
do_verifytargs
)
andverify_detachedtfilenamesigfile=
ift.check_signaturethen(
matchsigfilewith
|None->
error(f_"there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means you are susceptible to man-in-the-middle attacks.")
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the ‘-v’ option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
