Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
Latest btrfs seems to reject 512 byte sector size. It may be because
of the specific hardware that I'm running the test on. Anyway using a
4K sector size works.
libguestfs: error: mkfs_btrfs: /dev/sda1: ERROR: invalid sectorsize 512, expected range is [4K, 64K]
When shipping the libguestfs tarball we do not necessarily have
common/mlcustomize. If we run the generator in this situation then
don't try to generate files in the non-existent directory.
See-also: commit 7ced2b9354
libcap commit 177cd41803 ("A more compact form for the text
representation of capabilities.") changed the format used by
cap_to_text(3), breaking our test. Change the test to cope. This
will break with older libcap now, but there's not a lot we can do
about it.
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
Previously callers were unable to distinguish a regular error (like an
I/O error) from the case where you call this API on something which is
valid but not a logical volume. Set errno to a known value in this
case.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Python 2 reached end of life on 2020-01-01:
https://python3statement.org/https://pythonclock.org/
The minimum version required is now Python 3.4 (since that is the
version in Debian oldoldstable), but 3.6 is the minimum version that
I actually test.
Replace the use of liberl_interface, which is removed in Erlang 23,
by libei. The implementation uses the ei_decode_iodata() function
which has been introduces only for Erlang 23, so it doesnt work with
earlier Erlang versions.
This function doesn't work reliably with the proposed change to device
name translation. The reason is that strings returned by
Devsparts.list_devices contained translated names, so their indexes
did not correspond to the untranslated names used outside the
appliance..
We can avoid this and make the function much simpler and faster by
implementing it on the library side instead.
The first split does not care about the whole string, it is just trying to get
the command name in front, so triml is just right.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Go API functions returned (<val>, *GuestfsError) that made
code like this fail to build:
n, err := os.Stdin.Read(buf)
if err != nil {
log.Fatal(err)
}
n, err = g.Pwrite_device(dev, buf[:n], off)
...
As err should be of error (interface) type as of the stdlib call,
and should be of *GuestfsError type as of the libguestfs call.
The concrete error value that libguestfs functions return can be
a *GuestfsError, but the function signature should have (<val>, error)
as return value.
Nowadays there are hard drives and operating systems which support
"4K native" sector size. In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.
GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes. libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.
In order to fix the issue we need a way to provide correct logical block
size for attached disks. Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.
After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important. Thus both physical and logical
block size will be set to the same value.
In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method. Valid values are 512 and 4096.
add_drive_scratch has the same optional parameter for a consistency and
testing purpose.
add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
Add or remove empty lines to match the needed ones around
blocks/functions/etc.
Adapt the generation of guestfs.py to emit the separating empty line
before adding a new function/alias, rather than after it.
This is just formatting, no behaviour changes.
In Python 2 code branches use PyString_FromStringAndSize directly
instead of the guestfs_int_py_fromstringsize wrapper (which calls
PyString_FromStringAndSize anyway on Python < 3).
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list. That source
contained also functions using gnulib functions, however without
linking to gnulib. When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.
As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
needed
After the proposed split of the libguestfs repo, we will end up with
the following layout:
libguestfs.git
common -> git submodule libguestfs-common.git
generator
virt-v2v.git
common -> git submodule libguestfs-common.git
guestfs-tools.git
common -> git submodule libguestfs-common.git
The generator will only be able to write to libguestfs directories and
the common directory/submodule. This is mostly the case already with
only 6 exceptions:
customize/customize-options.pod
customize/customize-synopsis.pod
customize/customize_cmdline.ml
customize/customize_cmdline.mli
v2v/uefi.ml
v2v/uefi.mli
This commit moves these files around so they appear under common/ml*
It is somewhat unsatisfactory because it involves copying files
around, but there are some mitigating factors:
(1) Any changes now give us more freedom to develop faster and thus
clean things up in future.
(2) The v2v/uefi files ought to go away in future anyway.
This is simple code motion and should have no effect on the built
programs or tests.
This removes only the tool itself, and all the bits strictly needed to
not break the build.
This is now available as separate tool in its own repository:
https://github.com/libguestfs/virt-p2v
Disable this warning/error in the Python 3.8 header files:
In file included from /usr/include/python3.8/abstract.h:837,
from /usr/include/python3.8/Python.h:147,
from actions.h:31,
from actions-6.c:34:
/usr/include/python3.8/cpython/abstract.h: In function '_PyVectorcall_Function':
/usr/include/python3.8/cpython/abstract.h:91:11: error: cast increases required alignment of target type [-Werror=cast-align]
91 | ptr = (vectorcallfunc*)(((char *)callable) + offset);
| ^
Previously to work around some problems in Python 2 header files we
had to include <Python.h> before any other config file.
For Python 3 which is all we really care about now this is no longer
needed. We can move the include from three files into the local
"actions.h" file, bringing all the Python definitions and workarounds
into a single place.
- feed the content directly to stdin, avoid the need of read (and write)
a temporary file
- read all the output at once, without a tail-recursive function
- apply trimming and first line discarding after closing the process
Isolate the logic for the memoized disk cache in a small module, so it
can be reused for other tools.
Other than refactoring, there should be no behaviour changes.
- fix names of arguments & optional arguments in C<..> markers
- use https for URLs where possible
- fix links to other guestfs APIs
- use more C<..> markers for special tests, shell commands, values of
arguments, and names of fields
- link to command man pages where an explicit command is mentioned
- fix few incorrect documentation bits
Without clarifying handle's lifetime, it is unable
to see how long the callbacks which the handle
owns will live. Then, Rust compiler will infer
that the callbacks have 'static lifetime. It is
not convenient for users.
