Reimplement xfs_info by returning a hash table of values (rather than
a limited struct), and by writing it in OCaml with PCRE which makes
string parsing a lot simpler. This will now flexibly return all the
fields from the underlying xfs_info command, even (hopefully) future
fields.
Note the field values are returned as strings, because the actual
fields in xfs_info output are fairly random and free-form. There is a
trade off here between returning as much information as we can, and
requiring the user to do a bit of (simple) field parsing.
Fixes: https://issues.redhat.com/browse/RHEL-143673
These were previously written in very convoluted C which had to deal
with parsing the crazy output of the "lvm" command. In fact the
parsing was so complex that it was generated by the generator. It's
easier to do this in OCaml.
These are basically legacy APIs. They cannot be expanded and LVM
already supports many more fields. We should replace these with APIs
for getting single named fields from LVM.
These APIs allow you to capture output from guest commands that
generate more output than the protocol limit allows.
Thanks: Nijin Ashok
Fixes: https://issues.redhat.com/browse/RHEL-80159
Replace strange $TEST_FUNCTIONS variable/expansion thing with
something more like what we use in nbdkit, a simple tests/functions.sh
script that gets sourced into each test script.
Update the common submodule to get:
commit 8137d47d0e654065391151eb275e3b64f230f6f5
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Feb 13 11:13:55 2025 +0000
mlcustomize, mltools: Replace $TEST_FUNCTIONS
TEST_FUNCTIONS is being removed from libguestfs and guestfs-tools (it
was removed from virt-v2v a while back). Make the same adjustment in
the common submodule.
(and some other commits which are not relevant to libguestfs)
sfdisk can now do everything with GPT that sgdisk was needed for
before. In particular we are able to reimplement the following
functions using sfdisk:
- part_set_disk_guid (replace with sfdisk --disk-id)
- part_get_disk_guid
- part_set_disk_guid_random
- part_set_gpt_attributes (sfdisk --part-attrs)
- part_get_gpt_attributes
- part_set_gpt_guid (sfdisk --part-uuid)
- part_get_gpt_guid
- part_set_gpt_type (sfdisk --part-type)
- part_get_gpt_type
This allows us to drop the requirement for gdisk in many cases.
There is only one API remaining which requires gdisk, part_expand_gpt,
which we do not use in our tools. In a prior commit I already moved
this solitary function to a new source file (daemon/gdisk.c).
Fixes: https://issues.redhat.com/browse/RHEL-35998
This is required so we can determine the file architecture of
zstd-compressed Linux kernel modules as used by OpenSUSE and maybe
other distros in future.
Note that zstd becomes a required package, but it is widely available
in current Linux distros.
The package names come from https://pkgs.org/download/zstd and my own
research.
Starting with PHP8, arginfo is mandatory for PHP extensions. This patch
updates the generator for the PHP bindings to generate the arginfo
structures, using the Zend API macros. Only basic arginfo is added,
without full documentation of argument and return types, in order to
ensure compatibility with as many versions of PHP as possible.
Commit e9eaf4d889 ("docs: Split release notes by release.") split the
old guestfs-release-notes(1) page by release, but left now-broken
links to guestfs-release-notes(1) in various places in the
documentation.
The easiest way to fix this is to recreate this page by renaming
guestfs-release-notes-historical(1) as guestfs-release-notes(1) and
adding links to the other release notes pages.
Fixes: commit e9eaf4d889
Create a new (fake) Fedora disk image with two partitions. /dev/sda1 is
the boot partition as usual, /dev/sda2 is used as an LVM PV. The VG has
four LVs, Root and LV1 through LV3.
Each LV holds a LUKS device (with a different key). Each decrypted LUKS
device holds an ext2 filesystem, with "/dev/mapper/Root-luks" holding the
root filesystem.
Each filesystem has a dedicated label (ROOT, LV1, LV2, LV3).
In the test case, run guestfish in inspector mode, twice.
In the first invocation, provide the LUKS passphrases by LV name. Also
specific to the first invocation, fetch the LUKS UUIDs by LV name.
In the second invocation, provide the LUKS passphrases by UUID.
In both invocations, after decryption, check the filesystem labels, the
/dev/mapper/* names generated for the decrypted LUKS block devices, and
the existence of "/etc/fedora-release" on the root filesystem.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1658126
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220223162120.16729-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
As part of our efforts to clean up and simplify libguestfs, removing
gnulib deletes a large dependency that we mostly no longer use and
causes problems for new users trying to build the library from source.
A few modules from gnulib are still used (under a compatible license)
and these are copied into gnulib/lib/
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
libguestfs 1.45.3 now reads the RPM database using librpm, which means
our old phony database created by db_dump can no longer work. Instead
provide a real (but very minimal) sqlite database.
This commit also fixes the virt-inspector test since the RPM database
contents are now different.
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
This was only used for a single rule (check-valgrind-local-guests)
which ran "make check-valgrind" on local guests. This was never
really used by me and was fairly inadvisable anyway, so we can easily
remove it and thus remove the dependency on perl Sys::Virt.
The ELF NEEDED are used to determine guestfsd's library dependencies
with help from the dynamic linker and the package manager.
This was prompted by Debian bug #972241 which was caused by a
libtirpc package renaming in Debian/unstable because the SONAME had
been changed.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
With the Weblate adoption, we let it update the po files from the
catalog template. The po4a behaviour of extracting the template,
merging the existing translations, and creating the translated PODs at
once is problematic. Hence, split the extraction and the translated POD
generation in two.
Use po4a-gettextize to extract the catalog template only, not doing it
anymore automatically at each build. There is no more need for a
po4a.conf file.
Use po4a-translate to create translated PODs from the po files, keeping
the fixup of the generated files (to avoid spurious =encoding, etc).
Add a silent rule to hide the po4a-translate command lines by default.
These changes also allow us to get rid of the POD existance checks with
associated error message pointing to the update-po rule. Now each
translated POD file is generated because of make dependency, and it
depends only on its po file.
Signed-off-by: Pino Toscano <ptoscano@redhat.com>
You have to use:
../libguestfs/run ./configure
../libguestfs/run make
Use of the second ../libguestfs/run against make is unfortunate but I
believe it's unavoidable due to the way that ocamlfind works.
We use a similar trick to libvirt to allow external C programs that
use libguestfs to be compiled against the built (but not installed)
libguestfs with:
../libguestfs/run ./configure
make
What actually happens is we have a second pkg-config file
(lib/local/libguestfs.pc) which points to the locally built
libguestfs. The ./run script sets up PKG_CONFIG_PATH to point to this
directory. Assuming that ./configure is using pkg-config/pkgconf and
not some other half-baked solution it will pick up the libguestfs.pc
file from here which will set CFLAGS and LIBS appropriately.
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list. That source
contained also functions using gnulib functions, however without
linking to gnulib. When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.
As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
needed
This helper script will contain variables with results of configure
checks, so other scripts can source it.
Source it automatically in test-functions.sh, so every test can already
make use of it.
With the proposed split we will only run the generator from the
libguestfs repo. When compiling virt-v2v or the guestfs-tools we will
need certain generated files to be present already in the
libguestfs-common repo, and therefore these files must be added to
git. Hopefully they won't change very often.
After the proposed split of the libguestfs repo, we will end up with
the following layout:
libguestfs.git
common -> git submodule libguestfs-common.git
generator
virt-v2v.git
common -> git submodule libguestfs-common.git
guestfs-tools.git
common -> git submodule libguestfs-common.git
The generator will only be able to write to libguestfs directories and
the common directory/submodule. This is mostly the case already with
only 6 exceptions:
customize/customize-options.pod
customize/customize-synopsis.pod
customize/customize_cmdline.ml
customize/customize_cmdline.mli
v2v/uefi.ml
v2v/uefi.mli
This commit moves these files around so they appear under common/ml*
It is somewhat unsatisfactory because it involves copying files
around, but there are some mitigating factors:
(1) Any changes now give us more freedom to develop faster and thus
clean things up in future.
(2) The v2v/uefi files ought to go away in future anyway.
This is simple code motion and should have no effect on the built
programs or tests.
These two modules are a dependency of virt-v2v. Since we intend to
split virt-v2v from the other OCaml virt-* programs, we cannot have a
dependency between virt-v2v and virt-customize. Instead we must move
the modules to a common directory (common/mlcustomize) and have both
tools depending on the modules from there.
This is simple refactoring and should not affect how the programs work
or are tested.
Move (or copy the general ones) all the p2v entries in the top-level
.gitignore file to a new file specific for p2v. This will make it
easier to keep them when splitting p2v.
Add a copy of the libvirt-ocaml library, currently available at:
https://libvirt.org/git/?p=libvirt-ocaml.git;a=summary
This is a snapshot at commit d3ed8dcf1b0a6a8a855ceecbe0bb97f21e6665e3,
which has all the features we need (and that builds fine).
It is expected to stay synchronized with upstream, until there is a new
upstream release, and it will be widespread enough.
When the machine readable mode is enabled, print all the messages
(progress, info, warning, and errors) also as JSON in the machine
readable stream: this way, users can easily parse the status of the
OCaml tool, and report that back.
The formatting of the current date time into the RFC 3999 format is done
in C, because of the lack of OCaml APIs for this.
