The current "arg_string_list" and "free_string_list" implementations go
back to commit b6f01f3260 ("Add Go (language) bindings.", 2013-07-03).
There are two problems with them:
- "free_string_list" doesn't actually free anything,
- at the *first* such g.Internal_test() call site that passes an
Ostringlist member inside the Optargs argument, namely:
> g.Internal_test ("abc",
> string_addr ("def"),
> []string{},
> false,
> 0,
> 0,
> "123",
> "456",
> []byte{'a', 'b', 'c', '\000', 'a', 'b', 'c'},
> &guestfs.OptargsInternal_test{Ostringlist_is_set: true,
> Ostringlist: []string{}
> }
> )
the "golang/run-bindtests" case crashes:
> panic: runtime error: cgo argument has Go pointer to Go pointer
>
> goroutine 1 [running]:
> libguestfs.org/guestfs.(*Guestfs).Internal_test.func7(0xc000018180,
> 0xadfb60, 0xadfb80, 0xc000010048, 0x0, 0x0, 0x0, 0xae3e10, 0xae3e30,
> 0xade3a0, ...)
> golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0xa9
> libguestfs.org/guestfs.(*Guestfs).Internal_test(0xc000018180, 0x4ee3a5,
> 0x3, 0xc000061be8, 0xc000061af8, 0x0, 0x0, 0xc000061a00, 0x0, 0x0, ...)
> golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0x3c9
> main.main()
> golang/bindtests/bindtests.go:77 +0x151e
> exit status 2
> FAIL run-bindtests (exit status: 1)
In Daniel P. Berrangé's words [1],
> You're allowed to pass a Go pointer to C via CGo, but the memory that
> points to is not allowed to contained further Go pointers. So the struct
> fields must strictly use a C pointer.
One pattern to solve the problem has been shown on stackoverflow [2].
Thus, rewrite the "arg_string_list" and "free_string_list" functions
almost entirely in C, following that example.
While this approach is not the most idiomatic Go, as a solution exists
without C helper functions [3], it should still be acceptable, at least as
an incremental improvement, for letting "golang/run-bindtests" pass.
[1] https://listman.redhat.com/archives/libguestfs/2021-September/msg00118.html
[2] https://stackoverflow.com/questions/35924545/golang-cgo-panic-runtime-error-cgo-argument-has-go-pointer-to-go-pointer
[3] https://listman.redhat.com/archives/libguestfs/2021-September/msg00106.html
Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: "Richard W.M. Jones" <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210921192939.32468-1-lersek@redhat.com>
Tested-by: "Richard W.M. Jones" <rjones@redhat.com>
Acked-by: "Richard W.M. Jones" <rjones@redhat.com>
According to xfs_admin(8):
> -c 0|1 Enable (1) or disable (0) lazy-counters in the filesys‐
> tem.
>
> Lazy-counters may not be disabled on Version 5 su‐
> perblock filesystems (i.e. those with metadata CRCs en‐
> abled).
>
> [...]
According to mkfs.xfs(1):
> -m global_metadata_options
> Section Name: [metadata]
> These options specify metadata format options that ei‐
> ther apply to the entire filesystem or aren't easily
> characterised by a specific functionality group. The
> valid global_metadata_options are:
>
> [...]
>
> crc=value
> This is used to create a filesystem which
> maintains and checks CRC information in all
> metadata objects on disk. The value is ei‐
> ther 0 to disable the feature, or 1 to en‐
> able the use of CRCs.
>
> [...]
>
> By default, mkfs.xfs will enable metadata
> CRCs.
Consistently with the above, the first "xfs_admin" test case in
"generator/actions_core.ml", which attempts to disable lazy counters,
always fails:
> 534/550 test_xfs_admin_0
> libguestfs: error: xfs_admin: /dev/sda1: Cannot disable lazy-counters on V5 fs
We can resolve this test failure in three ways:
(1) Extend do_mkfs() [daemon/mkfs.c], possibly even introduce
do_mkfs_xfs(), and permit the caller to specify "-m crc=0" for
mkfs.xfs. Then use this option when the temporary filesystem is
created in the XFS test that disables lazy counters.
(2) Extend the "guestfs_int_xfsinfo" structure in the libguestfs-common
project, with an "xfs_crc" field. Extend parse_xfs_info()
[daemon/xfs.c] to populate the field from "meta-data=...crc=[01]".
Modify the test case to check the following post-condition:
xfs_crc || xfs_lazycount == 0
instead of the current
xfs_lazycount == 0
effectively ignoring "xfs_lazycount" when "xfs_crc" is set.
(3) Remove the test altogether that attempts to disable lazy counters
after filesystem creation.
Given that new XFS filesystems are created with metadata CRCs enabled by
default, and several XFS features depend on metadata CRCs being enabled,
this patch implements option (3).
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
When creating and returning a Python struct we were adding fields from
the C struct, but did not reduce the ref count on the temporary value
after it had been moved to the struct, resulting in a memory leak.
Reported-by: 朱丹 <zhudan24@huawei.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1985912
warning: panic message is not a string literal
--> src/bin/event_leak.rs:9:30
|
9 | Err(e) => panic!(format!(" could not create handle {:?}", e)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(non_fmt_panic)]` on by default
= note: this is no longer accepted in Rust 2021
= note: this warning originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
./guestfs.go:4945:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
./guestfs.go:7320:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
./guestfs.go:7335:16: conversion from _Ctype_char (int8) to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
This gnulib feature abstracts away threads, locks and TLS, and also
allowed libguestfs to be linked with or without pthread. However
since pthread these days is part of glibc and so every program is
using pthread, and we want to get rid of gnulib as a dependency, just
use pthread directly.
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.
isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing. (For a summary of the situation
see: https://wiki.debian.org/genisoimage).
This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.
Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
Latest btrfs seems to reject 512 byte sector size. It may be because
of the specific hardware that I'm running the test on. Anyway using a
4K sector size works.
libguestfs: error: mkfs_btrfs: /dev/sda1: ERROR: invalid sectorsize 512, expected range is [4K, 64K]
When shipping the libguestfs tarball we do not necessarily have
common/mlcustomize. If we run the generator in this situation then
don't try to generate files in the non-existent directory.
See-also: commit 7ced2b9354
libcap commit 177cd41803 ("A more compact form for the text
representation of capabilities.") changed the format used by
cap_to_text(3), breaking our test. Change the test to cope. This
will break with older libcap now, but there's not a lot we can do
about it.
Before this change the tests ran in about 12m34 and afterwards in
about 6m20, although the real change is more dramatic if you only run
tests from the tests/ subdirectory (as language tests still run serially).
This breaks valgrinding for now, which I intend to fix properly later.
Previously callers were unable to distinguish a regular error (like an
I/O error) from the case where you call this API on something which is
valid but not a logical volume. Set errno to a known value in this
case.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Python 2 reached end of life on 2020-01-01:
https://python3statement.org/https://pythonclock.org/
The minimum version required is now Python 3.4 (since that is the
version in Debian oldoldstable), but 3.6 is the minimum version that
I actually test.
Replace the use of liberl_interface, which is removed in Erlang 23,
by libei. The implementation uses the ei_decode_iodata() function
which has been introduces only for Erlang 23, so it doesnt work with
earlier Erlang versions.
This function doesn't work reliably with the proposed change to device
name translation. The reason is that strings returned by
Devsparts.list_devices contained translated names, so their indexes
did not correspond to the untranslated names used outside the
appliance..
We can avoid this and make the function much simpler and faster by
implementing it on the library side instead.
The first split does not care about the whole string, it is just trying to get
the command name in front, so triml is just right.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Go API functions returned (<val>, *GuestfsError) that made
code like this fail to build:
n, err := os.Stdin.Read(buf)
if err != nil {
log.Fatal(err)
}
n, err = g.Pwrite_device(dev, buf[:n], off)
...
As err should be of error (interface) type as of the stdlib call,
and should be of *GuestfsError type as of the libguestfs call.
The concrete error value that libguestfs functions return can be
a *GuestfsError, but the function signature should have (<val>, error)
as return value.
Nowadays there are hard drives and operating systems which support
"4K native" sector size. In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.
GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes. libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.
In order to fix the issue we need a way to provide correct logical block
size for attached disks. Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.
After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important. Thus both physical and logical
block size will be set to the same value.
In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method. Valid values are 512 and 4096.
add_drive_scratch has the same optional parameter for a consistency and
testing purpose.
add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
Add or remove empty lines to match the needed ones around
blocks/functions/etc.
Adapt the generation of guestfs.py to emit the separating empty line
before adding a new function/alias, rather than after it.
This is just formatting, no behaviour changes.
In Python 2 code branches use PyString_FromStringAndSize directly
instead of the guestfs_int_py_fromstringsize wrapper (which calls
PyString_FromStringAndSize anyway on Python < 3).
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list. That source
contained also functions using gnulib functions, however without
linking to gnulib. When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.
As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
needed
After the proposed split of the libguestfs repo, we will end up with
the following layout:
libguestfs.git
common -> git submodule libguestfs-common.git
generator
virt-v2v.git
common -> git submodule libguestfs-common.git
guestfs-tools.git
common -> git submodule libguestfs-common.git
The generator will only be able to write to libguestfs directories and
the common directory/submodule. This is mostly the case already with
only 6 exceptions:
customize/customize-options.pod
customize/customize-synopsis.pod
customize/customize_cmdline.ml
customize/customize_cmdline.mli
v2v/uefi.ml
v2v/uefi.mli
This commit moves these files around so they appear under common/ml*
It is somewhat unsatisfactory because it involves copying files
around, but there are some mitigating factors:
(1) Any changes now give us more freedom to develop faster and thus
clean things up in future.
(2) The v2v/uefi files ought to go away in future anyway.
This is simple code motion and should have no effect on the built
programs or tests.
This removes only the tool itself, and all the bits strictly needed to
not break the build.
This is now available as separate tool in its own repository:
https://github.com/libguestfs/virt-p2v
