Reimplement xfs_info by returning a hash table of values (rather than
a limited struct), and by writing it in OCaml with PCRE which makes
string parsing a lot simpler. This will now flexibly return all the
fields from the underlying xfs_info command, even (hopefully) future
fields.
Note the field values are returned as strings, because the actual
fields in xfs_info output are fairly random and free-form. There is a
trade off here between returning as much information as we can, and
requiring the user to do a bit of (simple) field parsing.
Fixes: https://issues.redhat.com/browse/RHEL-143673
Add a new optional boolean argument 'keepdirlink' to tar_in that passes
--keep-directory-symlink to tar. This preserves existing symlinks to
directories when extracting, which is important for usrmerge systems
where /lib is a symlink to /usr/lib.
Without this option, extracting a tarball containing lib/modules/...
to / would replace the /lib symlink with a real directory, breaking
the system.
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Python is broken and requires that we include <Python.h> before all
other headers so it can make inadvisable definitions of
_POSIX_C_SOURCE and other things. This wasn't a problem before, but a
recent change to glibc makes this necessary now.
See also a similar commit in nbdkit:
f924c3c34b
I also removed the -Wcast-align suppression as that is no longer
needed.
Windows group policy objects (GPOs) are restrictions that can be added
by an administrator to Windows to lock down various operations. From
our point of view the ones that matter involve restricting the ability
to inject device drivers.
Previously virt-v2v detected group policy here:
9bb2e7d470/convert/convert_windows.ml (L69)
We would like to report group policy through the libguestfs API and
tools such as virt-inspector, so move the code that is used to detect
group policy to libguestfs. A new API is introduced that returns
whether group policy was found (only for Windows guests) during
inspection of the software registry.
Fixes: https://issues.redhat.com/browse/RHEL-125846
Existing virt-v2v code uses some simple heuristics for detecting
Windows anti-virus software:
7520185504/convert/windows.ml
Replicate exactly this code as a new field in the struct returned by
guestfs_inspect_get_applications2. Because of limitations with the
API, we must use one of the existing spare fields in the struct, and
it must have the same type (a string), so we are limited in the design
of this new API. I chose to return a primary classification for the
application, with the only classification possible so far being
"antivirus" (or "" if not). This allows the possibility of future
expansion of use of this field if we need to in future.
Fixes: https://issues.redhat.com/browse/RHEL-125846
Convert to TypedData_Get_Struct which has been in ruby since 2009
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
virt-customize and friends have the following options:
--sm-register
--sm-unregister
--sm-attach
--sm-remove
--sm-credentials
These run `subscription-manager` commands inside the VM, which
is often necessary to get working package install for RHEL VMs.
These were added in 2015ish. Conceptually I understand why we
might want native support in the tools: virt-customize and
virt-builder have package management knowledge, and
for some distros subscription-manager is necessary.
But this support doesn't add much convenience over calling the commands
directly with `--run-command`. And on RHEL10, subscription-manager was
reworked and complete drops the backing commands for `--sm-attach` and
`--sm-remove`.
We _could_ make this code smarter, try to detect that situation,
and not error when subscription-manager is new enough. But if
subscription-manager itself doesn't care about maintaining that kind
of compat, I'm not sure why we should care either.
Instead we decided to just rip it all out and document the
alternatives.
common module is updated to sync related changes:
Cole Robinson (1):
mlcustomize: deprecate and remove --sm-* options
Richard W.M. Jones (3):
mlstdutils: Export List.find_opt
daemon, generator: Use power of 2 for initial size of Hashtbl.create
mlcustomize/inject_virtio_win.ml: Use viostor.inf instead of guestor
https://issues.redhat.com/browse/RHEL-113397https://issues.redhat.com/browse/RHEL-113398
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Before 2011 it was recommended to use a prime number for the initial
size. In 2011 the OCaml hash table was reimplemented using a hash
function based on Murmur 3. Hashtbl.create now adjusts the initial
size to the next power of 2 (minimum 16). So replace obsolete
'Hashtbl.create 13' with 'Hashtbl.create 16'.
Add an API to do the equivalent of `chmod [-r] MODE PATH` for
NTFS filesystems.
Files created on a linux ntfs-3g mount can not change permissions
directly. New files and directories are created with rough windows
equivalent of `chmod 777`. These wide open permissions can generate
security warnings on windows after virt-v2v installs bits into
`Program Files\Guestfs`.
Behind the scenes we use `ntfssecaudit(8)` from `ntfsprogs`
which is already part of the appliance. We only expose the chmod-style
feature; the rest of `ntfssecaudit` is concerned reporting and
managing fine grained windows security info which is way more than
we need.
Also note, `ntfssecaudit` needs to run on an unmounted partition
so using this is more complicated than a traditional `chmod` call.
Related: https://issues.redhat.com/browse/RHEL-104352
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This change was done almost entirely automatically using the script
below. This uses the OCaml lexer to read the source files and extract
the strings and locations. Strings which are "candidates" (in this
case, longer than 3 lines) are replaced in the output with quoted
string literals.
Since the OCaml lexer is used, it already substitutes all escape
sequences correctly. I diffed the output of the generator and it is
identical after this change, except for UUIDs, which change because of
how Utils.stable_uuid is implemented.
Thanks: Nicolas Ojeda Bar
$ ocamlfind opt -package unix,compiler-libs.common find_strings.ml \
-o find_strings.opt -linkpkg
$ for f in $( git ls-files -- \*.ml ) ; do ./find_strings.opt $f ; done
open Printf
let read_whole_file path =
let buf = Buffer.create 16384 in
let chan = open_in path in
let maxlen = 16384 in
let b = Bytes.create maxlen in
let rec loop () =
let r = input chan b 0 maxlen in
if r > 0 then (
Buffer.add_substring buf (Bytes.to_string b) 0 r;
loop ()
)
in
loop ();
close_in chan;
Buffer.contents buf
let count_chars c str =
let count = ref 0 in
for i = 0 to String.length str - 1 do
if c = String.unsafe_get str i then incr count
done;
!count
let subs = ref []
let consider_string str loc =
let nr_lines = count_chars '\n' str in
if nr_lines > 3 then
subs := (str, loc) :: !subs
let () =
Lexer.init ();
let filename = Sys.argv.(1) in
let content = read_whole_file filename in
let lexbuf = Lexing.from_string content in
let rec loop () =
let token = Lexer.token lexbuf in
(match token with
| Parser.EOF -> ();
| STRING (s, loc, sopt) ->
consider_string s loc; (* sopt? *)
loop ();
| token ->
loop ();
)
in
loop ();
(* The list of subs is already reversed, which is convenient
* because we must the file substitutions in reverse order.
*)
let subs = !subs in
let new_content = ref content in
List.iter (
fun (str, loc) ->
let { Location.loc_start = { pos_cnum = p1 };
loc_end = { pos_cnum = p2 } } = loc in
let len = String.length !new_content in
let before = String.sub !new_content 0 (p1-1) in
let after = String.sub !new_content (p2+1) (len - p2 - 1) in
new_content := before ^ "{|" ^ str ^ "|}" ^ after
) subs;
let new_content = !new_content in
if content <> new_content then (
(* Update the file in place. *)
let new_filename = filename ^ ".new"
and backup_filename = filename ^ ".bak" in
let chan = open_out new_filename in
fprintf chan "%s" new_content;
close_out chan;
Unix.rename filename backup_filename;
Unix.rename new_filename filename
)
The guestfs_selinux_relabel function was very hard to use. In
particular it didn't just do an SELinux relabel as you might expect.
Instead you have to write a whole bunch of code around it (example[1])
to make it useful.
Another problem is that it doesn't let you pass multiple paths to the
setfiles command, but the command itself does permit that (and, as it
turns out, will require it). There is no backwards compatible way to
extend the existing definition to allow a list parameter without
breaking API.
So deprecate guestfs_selinux_relabel. Reimplement it as
guestfs_setfiles. The new function is basically the same as the old
one, but allows you to pass a list of paths. The old function calls
the new function with a single path parameter.
[1] https://github.com/libguestfs/libguestfs-common/blob/master/mlcustomize/SELinux_relabel.ml
No existing OCaml functions have a StringList parameter, but we would
like to add one.
The original plan seems to have been to map these to 'string array'
types, but 'string list' is more natural, albeit marginally less
efficient. The implementation here just has to convert the 'char **'
into the OCaml linked list of values.
The old btrfs-fsck API used "btrfs check" which appears to be broken
or deprecated. The real tool you should use is "btrfs scrub". We
have already implemented that API, but it is very awkward to use from
libguestfs. In particular there's no existing way to run the scrub
and wait for it to finish.
Fix this by deprecating btrfs-fsck. Implement a new API
btrfs-scrub-full which runs btrfs scrub in the foreground, waits for
it to finish, and handles errors. It's much more like fsck tools in
other filesystems.
Thanks: Eric Sandeen
Fixes: https://issues.redhat.com/browse/RHEL-91936
This pulls in the commits below, requiring us to replace all uses of
String.is_prefix and String.is_suffix.
Mostly done with Perl like this, and carefully checked by hand
afterwards since this doesn't get everything right:
$ perl -pi.bak -e 's/String.is_prefix ([^[:space:]\)]+) ([^[:space:]\)]+)/String.starts_with \2 \1/g' -- `git ls-files`
Richard W.M. Jones (3):
mlstdutils: Fix comment that still referred to the old function names
mldrivers: Link to gettext-stub if ocaml-gettext is enabled
mlstdutils: Rename String.is_prefix -> starts_with, is_suffix -> ends_with
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
This was implemented wrongly. In the XDR protocol, UUIDs are fixed
buffers of length 32. We can just use memcpy to copy from the OCaml
string to the UUID, but we have to ensure the string length returned
by OCaml is correct (if not we just assert, it's an internal error).
(It didn't even compile before, so we know it was never used).
This acts just like FString except that we do reverse device name
translation on it. The only use is in the 'pvs-full' API where we
will use it (in a subsequent commit) to reverse translate the pv_name
field (a device name) before returning it from the daemon.
Compare this to the 'pvs' API which also returns a list of device
names, but using the generator's 'RStructList (RDevice,...)' return
type, where RDevice is similarly reverse translated.
Note in the library-side bindings, because the name has already been
translated in the daemon, we just treat it exactly the same as
FString. The vast majority of this patch is this mechanical change.
Cole Robinson (2):
mltools: decouple and simplify osinfo device support checks
mlcustomize: disable `--inject-virtio-win osinfo`
Richard W.M. Jones (3):
mltools: Fix de-oUnit-ized tests
mltools: Unreference various objects
Revert "mltools: Unreference various objects"
And update customize docs to match
Signed-off-by: Cole Robinson <crobinso@redhat.com>
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Stop generating these files. They are currently only used by virt-v2v
-o qemu mode, and there are better ways to locate the UEFI files
there.
Update the common submodule to bring in:
Richard W.M. Jones (5):
mlcustomize: Add heuristic support for Windows Server 2025
mlcustomize/customize_run.ml: Move 'in' to new line
mlstdutils/guestfs_config: Define host_os
mlcustomize, mltools: Check guest OS is compatible before allowing --run
Remove mlv2v/ subdirectory
All recent compilers support this (except MS compilers which we don't
care about). Assume it is supported. We test it in ./configure and
hard fail if it doesn't work.
We still define HAVE_ATTRIBUTE_CLEANUP but you can now assume it is
always defined and don't have to check it.
openEuler is simliar to CentOS, but the ID is not lower-case string,
as below:
NAME="openEuler"
VERSION="24.03 (LTS)"
ID="openEuler"
VERSION_ID="24.03"
PRETTY_NAME="openEuler 24.03 (LTS)"
ANSI_COLOR="0;31"
Signed-off-by: Wang Guoquan <wangguoquan03@foxmail.com>
Virt-v2v already includes facilities for injecting QEMU guest agent
etc. We shouldn't add the virt-customize options for this.
Update common submodule to include:
Richard W.M. Jones (2):
mlcustomize: Move virt-customize modules to mlcustomize/
mlcustomize: Update generated options for virt-v2v
The actual output of sfdisk --part-attrs is bizarre and doesn't match
the documentation. After looking at the source from util-linux, fix
the parsing to match what sfdisk produces.
Reported-by: Yongkui Guo
Fixes: commit c6c266a85d
Fixes: https://issues.redhat.com/browse/RHEL-35998
Commit d5b6f1df5f ("daemon: Allow parts of the daemon and APIs to be
written in OCaml.", 2017) contained a bug where in any OCaml function
that returns int64_t, the result was truncated to an int. This
particularly affected part_get_gpt_attributes as that returns large 64
bit numbers, but probably affects other functions too, undetected.
Fixes: commit d5b6f1df5f
This was only theoretically supported, via curl. It's unlikely that
it really worked as it was never tested.
If needed it's better to use nbdkit-curl-plugin instead (this applies
to http and ftp as well).
This was failing with recent Linux:
libguestfs: error: btrfs_subvolume_snapshot: /dir/test3: /dir/test6: ERROR: cannot snapshot '/sysroot/dir/test3': Invalid argument
I tried to change the test to use 1/1000 instead, but that fails with
a different error which I don't understand at all.
As we're not meant to be testing btrfs here, only that libguestfs can
translate between the guestfs API and btrfs commands and we know it
can do that, I simply deleted the sub-test entirely.
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
