These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
OCaml 5.1 changes the names of these libraries for some reason.
Also in OCaml 5.1, if using those libraries you must link with -lzstd.
Since zstd was already described as "required" (although we only used
it in the appliance), there is no official change to the requirements,
but I have added a configure time check for the library.
Thanks: Jerry James <loganjerry@gmail.com>
We have traditionally used custom printf formatters %Q and %R, where
%Q replaces the argument with a shell-quoted string, and %R replaces
the argument with a sysroot-prefixed shell-quoted string. They are
actually pretty useful, but unfortunately only supported by glibc.
We only used them in about a dozen places in the daemon (much code
having been replaced by OCaml which does not need them).
In every remaining case we were constructing a command using code like
this:
asprintf_nowarn (&cmd,
"cd %Q && find -print0 | %s -0 -o -H %s --quiet", ...);
We can replace this with:
char *cmd;
size_t cmd_size;
fp = open_memstream (&cmd, &cmd_size);
fprintf (fp, "cd ");
shell_quote (dir, fp);
fprintf (fp, " && find -print0 | %s -0 -o -H %s --quiet", ...);
fclose (fp);
This was a feature that allowed you to add drives to the appliance
after launching it. It was complicated to implement, and only worked
for the libvirt backend (not "direct", which is the default backend).
It also turned out to be a bad idea. The original concept was that
appliance creation was slow, so to examine multiple guests you should
launch the handle once then hot-add the disks from each guest in turn
to manipulate them. However this is terrible from a security point of
view, especially for multi-tenant, because the drives from one guest
might compromise the appliance and thus the filesystems/drives from
subsequent guests.
It also turns out that hotplugging is very slow. Nowadays appliance
creation should be faster than hotplugging.
The main use case for this was virt-df, but virt-df no longer uses it
after we discovered the problems outlined above.
Pass $(HIVEX_LIBS) with -cclib under the "daemon_utils_tests_LINK" target;
otherwise the OCaml compiler does not tell the linker where "-lhivex" can
be found, and the linking step fails if "-lhivex" is not on a system
library path.
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
"ocamlc -where" is supposed to "print the location of the standard library
and exit". While this directory contains core OCaml C header files, it
does not contain hivex-related C header files. Trim "guestfsd_CPPFLAGS"
accordingly.
Furthermore, the hivex module for OCaml may exist elsewhere than under the
OCaml standard library directory. Invoke "ocamlfind query hivex" to find
this module. This is what AC_CHECK_OCAML_PKG(hivex) does too, in
"m4/guestfs-ocaml.m4" and "m4/ocaml.m4".
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
Move the interal static libraries as the last items in the list of
libraries of guestfsd, to make sure their symbols are used for all the
other libraries. This is because GCC resolves the symbols looking at
the arguments from the beginning to the end of the command line.
This currently does not cause failures, however it "just works" because
of the tricky situation set up.
The situation is the following:
1) common/utils contains few utility sources: one of them is utils.c,
which contains various functions -- for example
guestfs_int_free_string_list and guestfs_int_drive_name --, it is built
as utils.o, and bundled in the static library libutils.a
2) common/mlutils builds a OCaml library with bindings for some utility
functions in libutils.a, in particular guestfs_int_drive_name (but not
guestfs_int_free_string_list); there are two versions of this library,
one OCaml library (dllmlcutils.so) that links with libutils.a, and one
static library (libmlcutils.a), which cannot specify the libraries it
links to (as it is static)
3) when the daemon is linked, the command line was the following
(simplified):
$ gcc [...] -o guestfsd guestfsd-9p.o other_daemon_object.o [...] \
../common/utils/.libs/libutils.a [...] -lmlcutils [...]
Some of the objects of the daemon itself use
guestfs_int_free_string_list, and thus the compiler opens libutils.a
(it is after the objects in the command line) and picks utils.o, which
contains also guestfs_int_drive_name (not used directly in the daemon);
when linking later on with libmlcutils.a, the symbols for this static
library (like guestfs_int_drive_name) are already resolved, and thus
all the symbols are resolved, and the linking succeeds
This fragile situation can be easily broken by moving e.g.
guestfs_int_drive_name out of common/utils/utils.c to a new source (say
utils2.c) still built as part of libutils.a: since nothing before
-lmlcutils actually needs to pick utils2.o from libutils.a for symbols,
then GCC will not be able to resolve all the symbols in libmlcutils.a.
As solution, move libutils.a (and other internal static libraries) as
last libraries to link guestfsd to: this way, GCC knows where to find
all the symbols needed by all the objects and libraries specified in
the command line.
Remove gnulib modules that provide stuff clearly not used within
libguestfs (library, daemon, and C tools). Among directly and
indirectly modules used previous (and now no more), they are:
cycle-check
d-ino
dev-ino
dup3
dup3-tests
fcntl-safer
fcntl-safer-tests
fdopendir
fdopendir-tests
filevercmp
filevercmp-tests
ftell
ftell-tests
ftello
ftello-tests
fts
getaddrinfo
getaddrinfo-tests
getcwd
getcwd-tests
gnu-make
hostent
i-ring
i-ring-tests
iconv
iconv-tests
inet_ntop
inet_ntop-tests
isatty
isatty-tests
openat-safer
openat-safer-tests
opendirat
ptsname_r
ptsname_r-tests
read-file
read-file-tests
rewinddir
servent
ttyname_r
ttyname_r-tests
xgetcwd
Some of the removed modules are still used pulled indirectly as
dependency of other modules. There should be no behaviour change on
recent Linux distros, although older distros were not tested (adding
a module back is easy, anyway).
Remove accordingly unused automake variables, and ignored files.
This way no non-namespaced OCaml C symbols are used, reducing the risk
of clashes with other code.
The only exception is ocaml-augeas, which does not build with
CAML_NAME_SPACE; it will be fixed upstream, and it affects only
ocaml-augeas itself.
After this commit, all annocheck errors are fixed except for:
Hardened: virt-get-kernel: MAYB: Gaps were detected in the annobin coverage. Run with -v to list.
After discussion with the annocheck maintainers this gap in coverage
(which corresponds to the OCaml runtime) seems to be caused either by
the runtime not being linked with the right flags, or might be a bug
in annocheck itself. In any case it's not something that can be
resolved within the scope of libguestfs.
Add a way to generate OCaml interfaces for all the modules in the
daemon that implement APIs: this makes sure that for them the
interface of each function matches the actual API specified in the
generator.
This way the Mount module contains only the OCaml implementations of
mount-related daemon APIs.
This is simple refactoring, with no functional changes.
While YAJL mostly works fine, it did not see any active development in
the last 3 years. OTOH, Jansson is another JSON C implementation, with
a very liberal license, and a much nicer API.
Hence, switch all of libguestfs from YAJL to Jansson:
- configure checks, and buildsystem in general
- packages pulled in the appliance
- actual implementations
- contrib scripts
- documentation
This also makes use of the better APIs available (e.g. json_object_get,
json_array_foreach, and json_object_foreach). This does not change the
API of our OCaml Yajl module.
Do a configure check for the OPEN_UNSAFE flag in the OCaml binding of
Hivex, using it only when available. This makes it possible to use
hivex < 1.3.14 to build libguestfs (the daemon, actually).
Amend the building documentation accordingly, bringing the minimum
version of hivex back as it was before
commit 64f49df747.
In OCaml 4.06 we need to link to daemon to libasmrun_pic.a (the OCaml
runtime linked with -fPIC). Otherwise you will see many errors like
this:
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup_aux.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup.o): relocation R_X86_64_32S against symbol `caml_data_segments' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(fail.o): relocation R_X86_64_32 against symbol `caml_exn_Failure' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(roots.o): relocation R_X86_64_32 against symbol `caml_frametable' can not be used when making a shared object; recompile with -fPIC
This commit use a configure-time test to find the best OCaml runtime.
common/mlutils: Unix_utils.StatVFS.statvfs: This commit implements a
full-featured binding for the statvfs(3) function.
We then use this to reimplement the daemon statvfs API in OCaml.
Note that the Gnulib fallback is dropped in this commit. It
previously referenced non-existent field names in the fs_usage struct
so it didn't work. Also it's not necessary as POSIX has supported
statvfs(3) since 2001, it's supported in *BSD, macOS > 10.4, and there
is already a Windows fallback.
Define the types which will be used to communicate between the
different parts of the inspection code. The main types are:
fs corresponds to ‘struct inspect_fs’ in C code
root no direct correspondence with the C code, but in the C
code, ‘inspect_fs’ was overloaded to store roots
inspection_data
the inspection data which is incrementally collected about
each filesystem as we perform inspection steps
Other types have simple and obvious correspondences with the
equivalent C code.
Add some utility function which will be used by inspection.
Note that this commit has no effect on its own, it just links extra
dead code into the daemon.
This commit bundles the ocaml-augeas library (upstream here:
http://git.annexia.org/?p=ocaml-augeas.git;a=summary). It's identical
to the upstream version and should remain so.
We can work towards using system ocaml-augeas, when it's more widely
available.
These are generated in many different ways in the various
subdirectories, and sometimes not generated correctly. Introduce a
script to do this in one place, and hopefully correctly.
This is mostly simple refactoring, but I got rid of a couple of
things:
(1) The ‘make depend’ rule doesn't appear to be needed. automake (or
make?) seems to rebuild the ‘.depend’ file automatically just because
it is included.
(2) I got rid of the hairy path rewriting sed expression. Possibly
that is needed for srcdir != builddir.
The following functions were previously reimplemented in OCaml. This
commit replaces them with calls to the C functions:
- is_root_device
- prog_exists
- udev_settle
plus the internal get_verbose_flag function.
However note that we cannot do this for every utility function. In
particular the C function must not call any reply* functions.
When parts of the daemon were previously converted to OCaml, the
previous PCRE regexps were converted to Str regexps. Restore the
original PCRE regexps.
There was also one case where an original call to glob(3) was replaced
by a Str regexp, and this is replaced by a PCRE regexp (although it is
in fact identical in this instance).
This updates commit b48da89dd6
and commit eeda6edca1
and commit 2ca0fa778d.
This also reimplements the lv_canonical function in OCaml. We cannot
call the original C function because it calls reply_with_perror which
would break the OCaml bindings.
