Before 2011 it was recommended to use a prime number for the initial
size. In 2011 the OCaml hash table was reimplemented using a hash
function based on Murmur 3. Hashtbl.create now adjusts the initial
size to the next power of 2 (minimum 16). So replace obsolete
'Hashtbl.create 13' with 'Hashtbl.create 16'.
Add an API to do the equivalent of `chmod [-r] MODE PATH` for
NTFS filesystems.
Files created on a linux ntfs-3g mount can not change permissions
directly. New files and directories are created with rough windows
equivalent of `chmod 777`. These wide open permissions can generate
security warnings on windows after virt-v2v installs bits into
`Program Files\Guestfs`.
Behind the scenes we use `ntfssecaudit(8)` from `ntfsprogs`
which is already part of the appliance. We only expose the chmod-style
feature; the rest of `ntfssecaudit` is concerned reporting and
managing fine grained windows security info which is way more than
we need.
Also note, `ntfssecaudit` needs to run on an unmounted partition
so using this is more complicated than a traditional `chmod` call.
Related: https://issues.redhat.com/browse/RHEL-104352
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This change was done almost entirely automatically using the script
below. This uses the OCaml lexer to read the source files and extract
the strings and locations. Strings which are "candidates" (in this
case, longer than 3 lines) are replaced in the output with quoted
string literals.
Since the OCaml lexer is used, it already substitutes all escape
sequences correctly. I diffed the output of the generator and it is
identical after this change, except for UUIDs, which change because of
how Utils.stable_uuid is implemented.
Thanks: Nicolas Ojeda Bar
$ ocamlfind opt -package unix,compiler-libs.common find_strings.ml \
-o find_strings.opt -linkpkg
$ for f in $( git ls-files -- \*.ml ) ; do ./find_strings.opt $f ; done
open Printf
let read_whole_file path =
let buf = Buffer.create 16384 in
let chan = open_in path in
let maxlen = 16384 in
let b = Bytes.create maxlen in
let rec loop () =
let r = input chan b 0 maxlen in
if r > 0 then (
Buffer.add_substring buf (Bytes.to_string b) 0 r;
loop ()
)
in
loop ();
close_in chan;
Buffer.contents buf
let count_chars c str =
let count = ref 0 in
for i = 0 to String.length str - 1 do
if c = String.unsafe_get str i then incr count
done;
!count
let subs = ref []
let consider_string str loc =
let nr_lines = count_chars '\n' str in
if nr_lines > 3 then
subs := (str, loc) :: !subs
let () =
Lexer.init ();
let filename = Sys.argv.(1) in
let content = read_whole_file filename in
let lexbuf = Lexing.from_string content in
let rec loop () =
let token = Lexer.token lexbuf in
(match token with
| Parser.EOF -> ();
| STRING (s, loc, sopt) ->
consider_string s loc; (* sopt? *)
loop ();
| token ->
loop ();
)
in
loop ();
(* The list of subs is already reversed, which is convenient
* because we must the file substitutions in reverse order.
*)
let subs = !subs in
let new_content = ref content in
List.iter (
fun (str, loc) ->
let { Location.loc_start = { pos_cnum = p1 };
loc_end = { pos_cnum = p2 } } = loc in
let len = String.length !new_content in
let before = String.sub !new_content 0 (p1-1) in
let after = String.sub !new_content (p2+1) (len - p2 - 1) in
new_content := before ^ "{|" ^ str ^ "|}" ^ after
) subs;
let new_content = !new_content in
if content <> new_content then (
(* Update the file in place. *)
let new_filename = filename ^ ".new"
and backup_filename = filename ^ ".bak" in
let chan = open_out new_filename in
fprintf chan "%s" new_content;
close_out chan;
Unix.rename filename backup_filename;
Unix.rename new_filename filename
)
The guestfs_selinux_relabel function was very hard to use. In
particular it didn't just do an SELinux relabel as you might expect.
Instead you have to write a whole bunch of code around it (example[1])
to make it useful.
Another problem is that it doesn't let you pass multiple paths to the
setfiles command, but the command itself does permit that (and, as it
turns out, will require it). There is no backwards compatible way to
extend the existing definition to allow a list parameter without
breaking API.
So deprecate guestfs_selinux_relabel. Reimplement it as
guestfs_setfiles. The new function is basically the same as the old
one, but allows you to pass a list of paths. The old function calls
the new function with a single path parameter.
[1] https://github.com/libguestfs/libguestfs-common/blob/master/mlcustomize/SELinux_relabel.ml
No existing OCaml functions have a StringList parameter, but we would
like to add one.
The original plan seems to have been to map these to 'string array'
types, but 'string list' is more natural, albeit marginally less
efficient. The implementation here just has to convert the 'char **'
into the OCaml linked list of values.
The old btrfs-fsck API used "btrfs check" which appears to be broken
or deprecated. The real tool you should use is "btrfs scrub". We
have already implemented that API, but it is very awkward to use from
libguestfs. In particular there's no existing way to run the scrub
and wait for it to finish.
Fix this by deprecating btrfs-fsck. Implement a new API
btrfs-scrub-full which runs btrfs scrub in the foreground, waits for
it to finish, and handles errors. It's much more like fsck tools in
other filesystems.
Thanks: Eric Sandeen
Fixes: https://issues.redhat.com/browse/RHEL-91936
This pulls in the commits below, requiring us to replace all uses of
String.is_prefix and String.is_suffix.
Mostly done with Perl like this, and carefully checked by hand
afterwards since this doesn't get everything right:
$ perl -pi.bak -e 's/String.is_prefix ([^[:space:]\)]+) ([^[:space:]\)]+)/String.starts_with \2 \1/g' -- `git ls-files`
Richard W.M. Jones (3):
mlstdutils: Fix comment that still referred to the old function names
mldrivers: Link to gettext-stub if ocaml-gettext is enabled
mlstdutils: Rename String.is_prefix -> starts_with, is_suffix -> ends_with
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
This was implemented wrongly. In the XDR protocol, UUIDs are fixed
buffers of length 32. We can just use memcpy to copy from the OCaml
string to the UUID, but we have to ensure the string length returned
by OCaml is correct (if not we just assert, it's an internal error).
(It didn't even compile before, so we know it was never used).
This acts just like FString except that we do reverse device name
translation on it. The only use is in the 'pvs-full' API where we
will use it (in a subsequent commit) to reverse translate the pv_name
field (a device name) before returning it from the daemon.
Compare this to the 'pvs' API which also returns a list of device
names, but using the generator's 'RStructList (RDevice,...)' return
type, where RDevice is similarly reverse translated.
Note in the library-side bindings, because the name has already been
translated in the daemon, we just treat it exactly the same as
FString. The vast majority of this patch is this mechanical change.
Cole Robinson (2):
mltools: decouple and simplify osinfo device support checks
mlcustomize: disable `--inject-virtio-win osinfo`
Richard W.M. Jones (3):
mltools: Fix de-oUnit-ized tests
mltools: Unreference various objects
Revert "mltools: Unreference various objects"
And update customize docs to match
Signed-off-by: Cole Robinson <crobinso@redhat.com>
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
Stop generating these files. They are currently only used by virt-v2v
-o qemu mode, and there are better ways to locate the UEFI files
there.
Update the common submodule to bring in:
Richard W.M. Jones (5):
mlcustomize: Add heuristic support for Windows Server 2025
mlcustomize/customize_run.ml: Move 'in' to new line
mlstdutils/guestfs_config: Define host_os
mlcustomize, mltools: Check guest OS is compatible before allowing --run
Remove mlv2v/ subdirectory
All recent compilers support this (except MS compilers which we don't
care about). Assume it is supported. We test it in ./configure and
hard fail if it doesn't work.
We still define HAVE_ATTRIBUTE_CLEANUP but you can now assume it is
always defined and don't have to check it.
openEuler is simliar to CentOS, but the ID is not lower-case string,
as below:
NAME="openEuler"
VERSION="24.03 (LTS)"
ID="openEuler"
VERSION_ID="24.03"
PRETTY_NAME="openEuler 24.03 (LTS)"
ANSI_COLOR="0;31"
Signed-off-by: Wang Guoquan <wangguoquan03@foxmail.com>
Virt-v2v already includes facilities for injecting QEMU guest agent
etc. We shouldn't add the virt-customize options for this.
Update common submodule to include:
Richard W.M. Jones (2):
mlcustomize: Move virt-customize modules to mlcustomize/
mlcustomize: Update generated options for virt-v2v
The actual output of sfdisk --part-attrs is bizarre and doesn't match
the documentation. After looking at the source from util-linux, fix
the parsing to match what sfdisk produces.
Reported-by: Yongkui Guo
Fixes: commit c6c266a85d
Fixes: https://issues.redhat.com/browse/RHEL-35998
Commit d5b6f1df5f ("daemon: Allow parts of the daemon and APIs to be
written in OCaml.", 2017) contained a bug where in any OCaml function
that returns int64_t, the result was truncated to an int. This
particularly affected part_get_gpt_attributes as that returns large 64
bit numbers, but probably affects other functions too, undetected.
Fixes: commit d5b6f1df5f
This was only theoretically supported, via curl. It's unlikely that
it really worked as it was never tested.
If needed it's better to use nbdkit-curl-plugin instead (this applies
to http and ftp as well).
This was failing with recent Linux:
libguestfs: error: btrfs_subvolume_snapshot: /dir/test3: /dir/test6: ERROR: cannot snapshot '/sysroot/dir/test3': Invalid argument
I tried to change the test to use 1/1000 instead, but that fails with
a different error which I don't understand at all.
As we're not meant to be testing btrfs here, only that libguestfs can
translate between the guestfs API and btrfs commands and we know it
can do that, I simply deleted the sub-test entirely.
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
The previous code split it on ',' which was completely wrong.
(It reveals the lack of testing however).
Fixes: commit c08032ebe2
Reported-by: Yongkui Guo
Since OCaml 5.1.1, changes to custom blocks caused C finalizers that
call caml_enter_blocking_section to stop working (if they ever did
before). They are relatively inflexible compared to registering an
OCaml finalizer (Gc.finalise) to call Guestfs.close, so use that
instead.
Suggested-by: Guillaume Munch-Maccagnoni
See: https://github.com/ocaml/ocaml/issues/12820
See: db48794fa8
Using 'virt-customize --tar-in some.tar:/dir -a disk.img' will unpack
'some.tar' into '/dir' in the guest. Note that this will not work for
compressed tar files as written since the underlying guestfs_tar_in
function requires the compression type to be set explicitly and
defaults to no compression (it does not auto-detect or default to
compression).
The just added --chown option previously used StringPair, splitting
the argument as ‘UID.GID:FILENAME’. However this will not work if we
ever extend this with the ability to use user or group names, since
they may contain dot (but not colon). Add a new StringTriplet type
and split the argument string three ways. The new option becomes:
virt-customize ... --chown UID:GID:FILENAME
Include the following commit from the common submodule:
commit e70d89a58dae068be2e19c7c21558707261af96a
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Sat Jul 15 16:42:06 2023 +0100
customize: Update generated files for --chown with StringTriplet
Updates: commit d8e48bff21
