Reimplement xfs_info by returning a hash table of values (rather than
a limited struct), and by writing it in OCaml with PCRE which makes
string parsing a lot simpler. This will now flexibly return all the
fields from the underlying xfs_info command, even (hopefully) future
fields.
Note the field values are returned as strings, because the actual
fields in xfs_info output are fairly random and free-form. There is a
trade off here between returning as much information as we can, and
requiring the user to do a bit of (simple) field parsing.
Fixes: https://issues.redhat.com/browse/RHEL-143673
Add a new optional boolean argument 'keepdirlink' to tar_in that passes
--keep-directory-symlink to tar. This preserves existing symlinks to
directories when extracting, which is important for usrmerge systems
where /lib is a symlink to /usr/lib.
Without this option, extracting a tarball containing lib/modules/...
to / would replace the /lib symlink with a real directory, breaking
the system.
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add an API to do the equivalent of `chmod [-r] MODE PATH` for
NTFS filesystems.
Files created on a linux ntfs-3g mount can not change permissions
directly. New files and directories are created with rough windows
equivalent of `chmod 777`. These wide open permissions can generate
security warnings on windows after virt-v2v installs bits into
`Program Files\Guestfs`.
Behind the scenes we use `ntfssecaudit(8)` from `ntfsprogs`
which is already part of the appliance. We only expose the chmod-style
feature; the rest of `ntfssecaudit` is concerned reporting and
managing fine grained windows security info which is way more than
we need.
Also note, `ntfssecaudit` needs to run on an unmounted partition
so using this is more complicated than a traditional `chmod` call.
Related: https://issues.redhat.com/browse/RHEL-104352
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This change was done almost entirely automatically using the script
below. This uses the OCaml lexer to read the source files and extract
the strings and locations. Strings which are "candidates" (in this
case, longer than 3 lines) are replaced in the output with quoted
string literals.
Since the OCaml lexer is used, it already substitutes all escape
sequences correctly. I diffed the output of the generator and it is
identical after this change, except for UUIDs, which change because of
how Utils.stable_uuid is implemented.
Thanks: Nicolas Ojeda Bar
$ ocamlfind opt -package unix,compiler-libs.common find_strings.ml \
-o find_strings.opt -linkpkg
$ for f in $( git ls-files -- \*.ml ) ; do ./find_strings.opt $f ; done
open Printf
let read_whole_file path =
let buf = Buffer.create 16384 in
let chan = open_in path in
let maxlen = 16384 in
let b = Bytes.create maxlen in
let rec loop () =
let r = input chan b 0 maxlen in
if r > 0 then (
Buffer.add_substring buf (Bytes.to_string b) 0 r;
loop ()
)
in
loop ();
close_in chan;
Buffer.contents buf
let count_chars c str =
let count = ref 0 in
for i = 0 to String.length str - 1 do
if c = String.unsafe_get str i then incr count
done;
!count
let subs = ref []
let consider_string str loc =
let nr_lines = count_chars '\n' str in
if nr_lines > 3 then
subs := (str, loc) :: !subs
let () =
Lexer.init ();
let filename = Sys.argv.(1) in
let content = read_whole_file filename in
let lexbuf = Lexing.from_string content in
let rec loop () =
let token = Lexer.token lexbuf in
(match token with
| Parser.EOF -> ();
| STRING (s, loc, sopt) ->
consider_string s loc; (* sopt? *)
loop ();
| token ->
loop ();
)
in
loop ();
(* The list of subs is already reversed, which is convenient
* because we must the file substitutions in reverse order.
*)
let subs = !subs in
let new_content = ref content in
List.iter (
fun (str, loc) ->
let { Location.loc_start = { pos_cnum = p1 };
loc_end = { pos_cnum = p2 } } = loc in
let len = String.length !new_content in
let before = String.sub !new_content 0 (p1-1) in
let after = String.sub !new_content (p2+1) (len - p2 - 1) in
new_content := before ^ "{|" ^ str ^ "|}" ^ after
) subs;
let new_content = !new_content in
if content <> new_content then (
(* Update the file in place. *)
let new_filename = filename ^ ".new"
and backup_filename = filename ^ ".bak" in
let chan = open_out new_filename in
fprintf chan "%s" new_content;
close_out chan;
Unix.rename filename backup_filename;
Unix.rename new_filename filename
)
The guestfs_selinux_relabel function was very hard to use. In
particular it didn't just do an SELinux relabel as you might expect.
Instead you have to write a whole bunch of code around it (example[1])
to make it useful.
Another problem is that it doesn't let you pass multiple paths to the
setfiles command, but the command itself does permit that (and, as it
turns out, will require it). There is no backwards compatible way to
extend the existing definition to allow a list parameter without
breaking API.
So deprecate guestfs_selinux_relabel. Reimplement it as
guestfs_setfiles. The new function is basically the same as the old
one, but allows you to pass a list of paths. The old function calls
the new function with a single path parameter.
[1] https://github.com/libguestfs/libguestfs-common/blob/master/mlcustomize/SELinux_relabel.ml
The old btrfs-fsck API used "btrfs check" which appears to be broken
or deprecated. The real tool you should use is "btrfs scrub". We
have already implemented that API, but it is very awkward to use from
libguestfs. In particular there's no existing way to run the scrub
and wait for it to finish.
Fix this by deprecating btrfs-fsck. Implement a new API
btrfs-scrub-full which runs btrfs scrub in the foreground, waits for
it to finish, and handles errors. It's much more like fsck tools in
other filesystems.
Thanks: Eric Sandeen
Fixes: https://issues.redhat.com/browse/RHEL-91936
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
The actual output of sfdisk --part-attrs is bizarre and doesn't match
the documentation. After looking at the source from util-linux, fix
the parsing to match what sfdisk produces.
Reported-by: Yongkui Guo
Fixes: commit c6c266a85d
Fixes: https://issues.redhat.com/browse/RHEL-35998
This was only theoretically supported, via curl. It's unlikely that
it really worked as it was never tested.
If needed it's better to use nbdkit-curl-plugin instead (this applies
to http and ftp as well).
This was failing with recent Linux:
libguestfs: error: btrfs_subvolume_snapshot: /dir/test3: /dir/test6: ERROR: cannot snapshot '/sysroot/dir/test3': Invalid argument
I tried to change the test to use 1/1000 instead, but that fails with
a different error which I don't understand at all.
As we're not meant to be testing btrfs here, only that libguestfs can
translate between the guestfs API and btrfs commands and we know it
can do that, I simply deleted the sub-test entirely.
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
The file(1) manual suggests using -S (disable seccomp) with -z since
the set of system calls provided by the seccomp policy does not allow
the subprocess to run. This is obvious when you use file -z on a
compressed file on a Linux distro that enables file's seccomp policy
(Arch does this, Fedora does not):
$ file -zbsL lib-i586.so.zst
Bad system call
I also fixed some incorrect text in the manual.
Thanks: Toolybird for pointing to this fix
Reported-by: David Runge
Fixes: https://github.com/libguestfs/libguestfs/issues/100
I cannot reproduce the originally reported error:
libguestfs: error: mkfs: xfs: /dev/VG/LV: Filesystem must be larger than 300MB.
Thanks: David Runge
Related: https://github.com/libguestfs/libguestfs/issues/100
This is required so we can determine the file architecture of
zstd-compressed Linux kernel modules as used by OpenSUSE and maybe
other distros in future.
Note that zstd becomes a required package, but it is widely available
in current Linux distros.
The package names come from https://pkgs.org/download/zstd and my own
research.
Currently the guestfs_readdir() API can not list long directories, due to
it sending back the whole directory listing in a single guestfs protocol
response, which is limited to GUESTFS_MESSAGE_MAX (approx. 4MB) in size.
Introduce the "internal_readdir" action, for transferring the directory
listing from the daemon to the library through a FileOut parameter.
Rewrite guestfs_readdir() on top of this new internal function:
- The new "internal_readdir" action is a daemon action. Do not repurpose
the "readdir" proc_nr (138) for "internal_readdir", as some distros ship
the binary appliance to their users, and reusing the proc_nr could
create a mismatch between library & appliance with obscure symptoms.
Replace the old proc_nr (138) with a new proc_nr (511) instead; a
mismatch would then produce a clear error message. Assume the new action
will first be released in libguestfs-1.48.2.
- Turn "readdir" from a daemon action into a non-daemon one. Call the
daemon action guestfs_internal_readdir() manually, receive the FileOut
parameter into a temp file, then deserialize the dirents array from the
temp file.
This patch sneakily fixes an independent bug, too. In the pre-patch
do_readdir() function [daemon/readdir.c], when readdir() returns NULL, we
don't distinguish "end of directory stream" from "readdir() failed". This
rewrite fixes this problem -- I didn't see much value separating out the
fix for the original do_readdir().
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1674392
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220502085601.15012-2-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Before commit 3a00c4d179 ("Remove inspection from the C library and
switch to daemon/OCaml implementation") in 2017 the name parameter
passed to add_drive was used by inspection to override the device name
that is determined from fstab. None of our tools ever actually used
this parameter, and when the inspection code was moved inside the
daemon we stopped using this hint field at all.
So it's no longer used, and likely hasn't been used ever. Therefore
document that the field does nothing.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
cdrtools writes "CDROM" into the Volume Identifier field in the PVD,
whereas genisoimage and xorriso write "ISOIMAGE". Recognise either
string as valid in the test.
Fixes: https://github.com/libguestfs/libguestfs/issues/79
Reported-by: David Runge
This was a feature that allowed you to add drives to the appliance
after launching it. It was complicated to implement, and only worked
for the libvirt backend (not "direct", which is the default backend).
It also turned out to be a bad idea. The original concept was that
appliance creation was slow, so to examine multiple guests you should
launch the handle once then hot-add the disks from each guest in turn
to manipulate them. However this is terrible from a security point of
view, especially for multi-tenant, because the drives from one guest
might compromise the appliance and thus the filesystems/drives from
subsequent guests.
It also turns out that hotplugging is very slow. Nowadays appliance
creation should be faster than hotplugging.
The main use case for this was virt-df, but virt-df no longer uses it
after we discovered the problems outlined above.
These APIs were an experimental feature for passing through 9p
filesystems from the host to the libguestfs appliance. It was never
possible to use this without hacking the qemu command line of the
appliance to add such drives by hand. It also didn't fit the
libguestfs model very well. And 9p is generally deprecated in
upstream qemu.
Note that for ABI reasons these APIs are not actually removed, they
have been changed so that they always return an error. These APIs
were actually hard-removed from all versions of RHEL.
See-also: https://bugzilla.redhat.com/921710
This experimental feature allowed you (in theory) to connect to an
existing instance of the libguestfs daemon. (Again, in theory) it
allowed you to attach to running guests. This didn't work well in
practice. If you want to do this, install qemu-guest-agent inside
your guest instead.
This also disables the --live options in guestfish and guestmount.
(The option now prints an error).
This was never supported in RHEL.
The daemon tests relied on this connection method to perform tests on
a bare daemon, so this removes those tests. They were not especially
valuable.
See-also: https://bugzilla.redhat.com/798980
According to xfs_admin(8):
> -c 0|1 Enable (1) or disable (0) lazy-counters in the filesys‐
> tem.
>
> Lazy-counters may not be disabled on Version 5 su‐
> perblock filesystems (i.e. those with metadata CRCs en‐
> abled).
>
> [...]
According to mkfs.xfs(1):
> -m global_metadata_options
> Section Name: [metadata]
> These options specify metadata format options that ei‐
> ther apply to the entire filesystem or aren't easily
> characterised by a specific functionality group. The
> valid global_metadata_options are:
>
> [...]
>
> crc=value
> This is used to create a filesystem which
> maintains and checks CRC information in all
> metadata objects on disk. The value is ei‐
> ther 0 to disable the feature, or 1 to en‐
> able the use of CRCs.
>
> [...]
>
> By default, mkfs.xfs will enable metadata
> CRCs.
Consistently with the above, the first "xfs_admin" test case in
"generator/actions_core.ml", which attempts to disable lazy counters,
always fails:
> 534/550 test_xfs_admin_0
> libguestfs: error: xfs_admin: /dev/sda1: Cannot disable lazy-counters on V5 fs
We can resolve this test failure in three ways:
(1) Extend do_mkfs() [daemon/mkfs.c], possibly even introduce
do_mkfs_xfs(), and permit the caller to specify "-m crc=0" for
mkfs.xfs. Then use this option when the temporary filesystem is
created in the XFS test that disables lazy counters.
(2) Extend the "guestfs_int_xfsinfo" structure in the libguestfs-common
project, with an "xfs_crc" field. Extend parse_xfs_info()
[daemon/xfs.c] to populate the field from "meta-data=...crc=[01]".
Modify the test case to check the following post-condition:
xfs_crc || xfs_lazycount == 0
instead of the current
xfs_lazycount == 0
effectively ignoring "xfs_lazycount" when "xfs_crc" is set.
(3) Remove the test altogether that attempts to disable lazy counters
after filesystem creation.
Given that new XFS filesystems are created with metadata CRCs enabled by
default, and several XFS features depend on metadata CRCs being enabled,
this patch implements option (3).
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
Latest btrfs seems to reject 512 byte sector size. It may be because
of the specific hardware that I'm running the test on. Anyway using a
4K sector size works.
libguestfs: error: mkfs_btrfs: /dev/sda1: ERROR: invalid sectorsize 512, expected range is [4K, 64K]
