11637 Commits

Author SHA1 Message Date
Richard W.M. Jones
056b03f4a1 Version 1.46.2. v1.46.2 2021-12-24 10:13:01 +00:00
Laszlo Ersek
54b13344c4 launch-libvirt: add virtio-net via the standard <interface> element
Starting with version 3.8.0, libvirt allows us to specify the network
address and network mask (as prefix) for SLIRP directly via the
<interface> element in the domain XML:
<https://libvirt.org/formatdomain.html#userspace-slirp-stack>. This means
we don't need the <qemu:commandline> hack for virtio-net on such versions.

Restrict the hack in construct_libvirt_xml_qemu_cmdline() to
libvirt<3.8.0, and generate the proper <interface> element in
construct_libvirt_xml_devices() on libvirt>=3.8.0.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2034160
Suggested-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211223103701.12702-4-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 5858c2cf6c)
2021-12-24 09:45:42 +00:00
Laszlo Ersek
715afa4e9e lib: extract NETWORK_ADDRESS and NETWORK_PREFIX as macros
The 169.254.0.0/16 network specification (for the appliance) is currently
duplicated between the direct backend and the libvirt backend. In a
subsequent patch, we're going to need the network specification in yet
another spot; extract it now to the NETWORK_ADDRESS and NETWORK_PREFIX
macros (simply as strings).

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2034160
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211223103701.12702-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 216de164e0)
2021-12-24 09:45:37 +00:00
Laszlo Ersek
405cc10666 launch-libvirt: place our virtio-net-pci device in slot 0x1e
The <qemu:commandline> trick we use for adding our virtio-net-pci device
in the libvirt backend can conflict with libvirtd's and QEMU's PCI address
assignment. Try to mitigate that by placing our device in slot 0x1e on the
root bus. In practice this could only conflict with a "dmi-to-pci-bridge"
device model, which libvirtd itself places in slot 0x1e. However, given
the XMLs we generate, and modern QEMU versions, libvirtd has no reason to
auto-add "dmi-to-pci-bridge". Refer to
<https://libvirt.org/formatdomain.html#controllers>.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2034160
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211223103701.12702-2-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 5ce5ef6a97)
2021-12-24 09:45:31 +00:00
Richard W.M. Jones
5b0882e559 fish: Avoid valgrind test from creating fish/.cache
Work around for https://bugzilla.redhat.com/show_bug.cgi?id=2031135

(cherry picked from commit 4af6d68e2d)
2021-12-24 09:45:27 +00:00
Richard W.M. Jones
969b1249e1 valgrind: Update suppressions list
For OCaml 4.13, libvirt 7.7.

(cherry picked from commit c6b25262ee)
2021-12-24 09:45:21 +00:00
Richard W.M. Jones
d3275cc507 Disable OCaml warning 6 completely
Warning 6 "labels-omitted" is not useful.  It's fine to omit labels on
positional arguments.

Example:

  File "perl_edit.ml", line 30, characters 2-13:
  30 |   c_edit_file (verbose ()) g (Guestfs.c_pointer g) file expr
         ^^^^^^^^^^^
  Warning 6 [labels-omitted]: label verbose was omitted in the application of this function.

The function is specified as:

  external c_edit_file : verbose:bool -> Guestfs.t -> int64 -> string -> string -> unit

The complaint is that the verbose: label has been omitted from the
first argument when the function is called, but IMO this is a
stylistic thing, not a bug.

(cherry picked from
guestfs-tools commit 577f7aee4b1c720f4c4826115b49a0c3870b149e)

(cherry picked from commit 1941593585)
2021-12-24 09:45:17 +00:00
Richard W.M. Jones
f438640541 customize: Suppress OCaml warning
In OCaml 4.13:

File "perl_edit.ml", line 30, characters 2-13:
30 |   c_edit_file (verbose ()) g (Guestfs.c_pointer g) file expr
       ^^^^^^^^^^^
Error (warning 6 [labels-omitted]): label verbose was omitted in the application of this function.

(cherry picked from
guestfs-tools commit a4930f5fad82e5358d565b8cf3610970e9646259)

(cherry picked from commit 0d05a229f3)
2021-12-24 09:45:11 +00:00
Richard W.M. Jones
4ff4e123dc generator: Replace more "noalloc" with [@@noalloc]
In some places in the generator we were still generating "noalloc".
It was hidden from the previous regexp I used to replace these because
of string escaping.

Updates: commit a69cde79ca
(cherry picked from commit b64e9bffc1)
2021-12-24 09:45:06 +00:00
Richard W.M. Jones
d306db36c8 Update common submodule
Picks up this commit:

    mlstdutils/std_utils.mli: Remove export of deprecated String.copy

    Since OCaml strings are at long last immutable, the String.copy
    function is deprecated.  Stop exporting it from our Std_utils.String.
    Any places we use it are wrong and should be fixed.

(cherry picked from commit 1e60550c2a)
2021-12-24 09:44:42 +00:00
Richard W.M. Jones
e8c7b5f4ce Version 1.46.1. v1.46.1 2021-12-09 17:26:02 +00:00
Richard W.M. Jones
6c05d666b8 lib, lua: Fix usage of strerror_r
$ ./run guestfish -vx add-drive foo "readonly:true"
  libguestfs: trace: set_pgroup true
  libguestfs: trace: set_pgroup = 0
  libguestfs: trace: add_drive "foo" "readonly:true"
  libguestfs: error: foo:
  libguestfs: trace: add_drive = -1 (error)
  libguestfs: trace: close
  libguestfs: closing guestfs handle 0x55c0bacf12a0 (state 0)

Fix the usage of strerror_r by using the new wrapper defined in
libguestfs-common.  A similar fix is made in the Lua bindings.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2030396
Reported-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Tested-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
7b5e00f036 daemon/listfs: don't call "sgdisk -i" on bogus MBR partition table entry
The "is_partition_can_hold_filesystem" function calls
"Parted.part_get_gpt_type" on the partition if:
- the partition table type is GPT,
- or the partition table type is MBR, and the partition is primary or
  logical.

The one entry in the fake MBR partition table described in the previous
patch passes the second branch of this check, therefore
"Parted.part_get_gpt_type" is reached, and it invokes "sgdisk -i 1" on the
disk.

Surprisingly (not), while "sgdisk -i" copes fine with valid MBR partition
tables, it chokes on the fake one. The output does not contain the
"Partition GUID code" line, and so "sgdisk_info_extract_field" throws an
exception.

Prevent calling "Parted.part_get_gpt_type" on a bogus MBR partition table,
similarly to the "extended entry in MBR partition table" case; the
difference is that the bogus primary entry, unlike a valid extended entry,
*can* hold a filesystem.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-6-lersek@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
0efccea844 daemon/parted: work around part table type misreporting by "parted"
"parted" incorrectly reports "loop" rather than "msdos" for the partition
table type, when the (fake) partition table comes from the "--mbr" option
of "mkfs.fat" (in dosfstools-4.2+), and the FAT variant in question is
FAT16 or FAT32. (See RHBZ#2026224.) Work this around by
- parsing the partition table ourselves, and
- overriding "loop" with "msdos" when appropriate.

Note that when the FAT variant is FAT12, "parted" fails to parse the fake
MBR partition table completely (see RHBZ#2026220), which we cannot work
around. However, FAT12 should be a rare corner case in libguestfs usage --
"mkfs.fat" auto-chooses FAT12 only below 9MB disk size, and even "-F 12"
can only be forced up to and including 255MB disk size.

Add the helper function "has_bogus_mbr" to the Utils module; we'll use it
elsewhere too.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-5-lersek@redhat.com>
[lersek@redhat.com: drop "fun" keyword, and use partial application, in
 the definition of "sec0at" [Rich]]
2021-12-09 13:54:15 +00:00
Laszlo Ersek
08856ccda2 daemon/parted: simplify print_partition_table() prototype
Since commit 994ca1f8eb ("daemon: Reimplement 'part_get_mbr_part_type'
API in OCaml.", 2018-05-02), we've not had any calls to
print_partition_table() that would pass a "false" argument for the
"add_m_option" parameter.

Remove the parameter, and inside part_get_mbr_part_type(), remove the dead
branch.

Relatedly, update the comment on the
"print_partition_table_machine_readable" OCaml function, originally from
commit 32e661f421 ("daemon: Reimplement ‘part_list’ API in OCaml.",
2017-07-27). Because print_partition_table() now passes "-m" to "parted"
unconditionally, and there are no use cases left that would *forbid* "-m",
"print_partition_table_machine_readable" is almost equivalent to
print_partition_table() -- modulo the enforcement of the "BYT;" header.

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-4-lersek@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
e589faf92a daemon/9p: fix wrong pathname in error message
The directory that readdir() and closedir() work on is BUS_PATH
("/sys/bus/virtio/drivers/9pnet_virtio"), not "/sys/block". Fix the error
messages that are sent when readdir() or closedir() fails.

(The invalid "sys/block" pathname could be a leftover from when the
directory reading logic was (perhaps) copied from "daemon/sync.c".)

Fixes: 5f10c33503
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-3-lersek@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
e8e2f417c9 daemon/mkfs: disable creation of fake MBR partition table with "mkfs.fat"
Search the usage output of "mkfs.fat" for "--mbr[="; cache the result for
further invocations. If the option is supported, pass "--mbr=n" to
"mkfs.fat". This will prevent the creation of a bogus partition table
whose first (and only) entry describes a partition that contains the
partition table.

(Such a bogus partition table breaks "parted", which is a tool used by
libguestfs extensively, both internally and in public libguestfs APIs.)

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1931821
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20211125094954.9713-2-lersek@redhat.com>
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
07049b2a1f xfs: Document lazy-counters setting cannot be changed in XFS version 5
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2024022
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
9f7695e539 Update common submodule
Removes old compat Bytes module.

Fixes: commit b536c61a6d
Thanks: Laszlo Ersek
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
c6082b97ee tests/gdisk/test-expand-gpt.pl: Don't race with other tests
Another test was removing all disk_*.img files, which removed the file
used by this test.  This was ultimately caused by us using parallel
tests.

Put the disk image files back into the tests/gdisk/ subdirectory to
avoid this race.

Fixes: commit 6d32773e81
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
b0c69b6ebb tests/gdisk/test-expand-gpt.pl: Don't hide error message from qemu-img resize
In a test I ran qemu-img resize failed.  However because we redirected
stderr to /dev/null the error message was hidden.
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
7686580338 tests/gdisk/test-expand-gpt.pl: Fix some warnings
"my" variable $output masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 73.
"my" variable $end_sectors masks earlier declaration in same scope at ./gdisk/test-expand-gpt.pl line 78.
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
3c41324ef5 Move minimum OCaml version to 4.04.
Synchronize with common module which also requires 4.04.

Small adjustment to use of List.sort_uniq because the signature
changed slightly.
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
2ecccfce38 daemon: Replace "noalloc" with [@@noalloc] 2021-12-09 13:54:15 +00:00
Richard W.M. Jones
9a5c5ac5bd m4: Remove test for OCaml Bytes module 2021-12-09 13:54:15 +00:00
Richard W.M. Jones
c2c7dfd66b rust: Use distclean to clean cache rather than make clean
Actually cargo caches downloaded libraries.  The previous change
caused cargo to download and rebuild these after make clean which is
overly aggressive.  Use make distclean instead.

Updates: commit 1834f19d20
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
637f193189 rust: Wire up make clean so it runs cargo clean
Before this commit, after make clean:

$ du -sh rust
641M	rust

After:

776K	rust
2021-12-09 13:54:15 +00:00
Laszlo Ersek
ecf4449762 build, docs: spell out minimum version (4.0.0) for the (optional) Yara lib
Commit e597fc5317 ("daemon/yara: fix undefined behavior due to Yara 4.0
API changes", 2021-10-12) prevents the daemon from using such a Yara
version that precedes 4.0.0.

If only yara < 4 is found, treat the library as absent, rather than
attempting and failing to compile the yara module of the daemon. Note the
version requirement in the documentation too.

Suggested-by: Eric Blake <eblake@redhat.com>
Suggested-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-4-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
56172a193c build: eliminate the AC_CHECK_LIB / AC_CHECK_HEADER tests for Yara
Eliminate the AC_CHECK_LIB / AC_CHECK_HEADER tests for Yara, for the
following reasons:

- Upstream Yara has provided a pkg-config file since 2015, so the
  (now-fixed) pkg-config check should always find it, without the
  AC_CHECK_LIB / AC_CHECK_HEADER fallback branch.

- In a subsequent patch, we'll want to test for the incompatible Yara API
  changes described at
  <https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API>.

  That's easy to do with pkg-config, but impossible with AC_CHECK_*,
  without a custom test. Namely, both AC_CHECK_DECLS and AC_CHECK_TYPES
  appear unable to check the parameter list of a function pointer typedef
  (namely YR_CALLBACK_FUNC and YR_COMPILER_CALLBACK_FUNC). And writing a
  dedicated test for this is overkill.

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-3-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
12414ef705 build: fix the pkg-config identifier of the (optional) Yara library
The upstream Yara project has always provided its "libyara/yara.pc.in"
file with "Name: yara" -- ever since its introduction in commit
334bd1a671ca ("Add support for pkg-config", 2015-01-08).

In spite of this, when the (optional) Yara dependency was added to
libguestfs in commit 2e24129da3 ("appliance: add yara dependency",
2017-05-02), PKG_CHECK_MODULES was invoked with [libyara] as
list-of-modules.

(That was clearly a bug. I'm unsure what Debian calls their Yara
pkg-config module, but calling it anything else than "yara" would be a
distribution bug: upstream projects provide pkg-config files specifically
so that dependent projects can find their dependencies *regardless of
distribution*.)

As a consequence, on Fedora today, the PKG_CHECK_MODULES macro always
fails, and only the AC_CHECK_LIB / AC_CHECK_HEADER branch finds Yara.

In a subsequent patch, we'll want to add a version requirement to the
PKG_CHECK_MODULES macro invocation, so at first, fix the pkg-config
identifier of Yara.

Fixes: 2e24129da3
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211013133611.21599-2-lersek@redhat.com>
Acked-by: Eric Blake <eblake@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
4113eca69c daemon/yara: fix undefined behavior due to Yara 4.0 API changes
Currently, the Yara test case ("yara/test-yara-scan.sh") fails, with the
following obscure error message:

> ><fs> yara-scan /text.txt
> libguestfs: error: deserialise_yara_detection_list:

Namely, the Yara rule match list serialization / de-serialization, between
the daemon and the library, is broken. It is caused by the following
incompatible pointer passing (i.e., undefined behavior), in function
do_internal_yara_scan(), file "daemon/yara.c":

>   r = yr_rules_scan_fd (rules, fd, 0, yara_rules_callback, (void *) path, 0);
                                        ^^^^^^^^^^^^^^^^^^^

The prototype of yara_rules_callback() is:

> static int
> yara_rules_callback (int code, void *message, void *data)

however, in Yara commit 2b121b166d25 ("Track string matches using
YR_SCAN_CONTEXT.", 2020-02-27), which was included in the upstream v4.0.0
release, the rules callback prototype was changed as follows:

> diff --git a/libyara/include/yara/types.h b/libyara/include/yara/types.h
> index cad095cd70c2..f415033c4aa6 100644
> --- a/libyara/include/yara/types.h
> +++ b/libyara/include/yara/types.h
> @@ -661,6 +644,7 @@ struct YR_MEMORY_BLOCK_ITERATOR
>
>
>  typedef int (*YR_CALLBACK_FUNC)(
> +    YR_SCAN_CONTEXT* context,
>      int message,
>      void* message_data,
>      void* user_data);

Therefore, the yara_rules_callback() function is entered with a mismatched
parameter list in the daemon, and so it passes garbage to
send_detection_info(), for serializing the match list.

This incompatible change was in fact documented by the Yara project:

  https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API#scanning-callback

Gcc too warns about the incompatible pointer type, under
"-Wincompatible-pointer-types". However, libguestfs is built without
"-Werror" by default, so the warning is easy to miss, and the bug only
manifests at runtime.

(The same problem exists for yr_compiler_set_callback() /
compile_error_callback():

  https://github.com/VirusTotal/yara/wiki/Backward-incompatible-changes-in-YARA-4.0-API#compiler-callback

except that this instance of the problem is not triggered by the test
case, as the rule list always compiles.)

Rather than simply fixing the parameter lists, consider the following
approach.

If Yara's YR_CALLBACK_FUNC and YR_COMPILER_CALLBACK_FUNC typedefs were not
for *pointer* types but actual *function* prototypes, then we could use
them directly in the declarations of our callback functions. Then any
future changes in the param lists would force a "conflicting types"
*compilation error* (not a warning). Illustration:

  /* this is *not* a pointer type */
  typedef int HELLO_FUNC (void);

  /* function declarations */
  static HELLO_FUNC my_hello_good;
  static HELLO_FUNC my_hello_bad;

  /* function definition, with explicit parameter list */
  static int my_hello_good (void) { return 1; }

  /* function definition with wrong param list -> compilation error */
  static int my_hello_bad (int i) { return i; }

Unfortunately, given that the Yara-provided typedefs are already pointers,
we can't do this, in standard C anyway. Type derivation only allows for
array, structure, union, function, and pointer type derivation; it does
not allow "undoing" previous derivations.

However, using gcc's "typeof" keyword, the idea is possible. Given
YR_CALLBACK_FUNC, the expression

  (YR_CALLBACK_FUNC)NULL

is a well-defined null pointer, and the function designator expression

  *(YR_CALLBACK_FUNC)NULL

has the desired function type.

Of course, evaluating this expression would be undefined behavior, but in
the GCC extension expression

  typeof (*(YR_CALLBACK_FUNC)NULL)

the operand of the "typeof" operator is never evaluated, as it does not
have a variably modified type. We can therefore use this "typeof" in the
same role as HELLO_FUNC had in the above example.

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211011223627.20856-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: clean up whitespace in "YR_RULE *rule"]
2021-12-09 13:54:15 +00:00
Laszlo Ersek
421ab66c8c lib/proto: suppress "may be used uninitialized" in send_file_complete()
gcc emits the following warning:

> proto.c: In function ‘send_file_complete’:
> proto.c:437:10: error: ‘buf’ may be used uninitialized
> [-Werror=maybe-uninitialized]
>   437 |   return send_file_chunk (g, 0, buf, 0);
>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In theory, passing the 1-byte array "buf", with indeterminate contents, to
xdr_bytes() ultimately, could be fine -- assuming xdr_bytes() never reads
the contents of the buffer, due to the buffer size being zero. However,
the xdr_bytes() manual does not seem to guarantee this (it also does not
explicitly permit passing a NULL buffer alongside size=0, which would be
even simpler for the caller).

In order to shut up the compiler, just zero-initialize the buffer --
that's simpler than adding diagnostics pragmas. The "maybe-uninitialized"
warning is otherwise very useful, so keep it globally enabled (per
WARN_CFLAGS / WERROR_CFLAGS).

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211011223627.20856-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-12-09 13:54:15 +00:00
Laszlo Ersek
fc921c6515 build: fix typo in "--enable-werror" help string
While <https://libguestfs.org/guestfs-building.1.html> correctly documents
the "--enable-werror" option, the "./configure" help text itself doesn't.
Replace "--enable-error" with "--enable-werror" now.

Fixes: 0f54df53d2
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20211011223627.20856-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-12-09 13:54:15 +00:00
Richard W.M. Jones
aa78dc8ece m4/guestfs-ocaml.m4: Fix deprecated warning format
In OCaml 4.13:

Alert ocaml_deprecated_cli: Setting a warning with a sequence of lowercase or uppercase letters,
like 'CDEFLMPSUVYZX', is deprecated.
Use the equivalent signed form: +C+D+E+F+L+M+P+S+U+V+Y+Z+X+52-3.

(cherry picked from
guestfs-tools commit fa4f59e1d99c08d7e0bae2a7cb54f254a6506d67)
2021-12-09 13:54:15 +00:00
Laszlo Ersek
e2f8db27d0 Go bindings: fix "C array of strings" -- char** -- allocation
The current "arg_string_list" and "free_string_list" implementations go
back to commit b6f01f3260 ("Add Go (language) bindings.", 2013-07-03).
There are two problems with them:

- "free_string_list" doesn't actually free anything,

- at the *first* such g.Internal_test() call site that passes an
  Ostringlist member inside the Optargs argument, namely:

> g.Internal_test ("abc",
>                  string_addr ("def"),
>                  []string{},
>                  false,
>                  0,
>                  0,
>                  "123",
>                  "456",
>                  []byte{'a', 'b', 'c', '\000', 'a', 'b', 'c'},
>                  &guestfs.OptargsInternal_test{Ostringlist_is_set: true,
>                                                Ostringlist: []string{}
>                                               }
>                 )

  the "golang/run-bindtests" case crashes:

> panic: runtime error: cgo argument has Go pointer to Go pointer
>
> goroutine 1 [running]:
> libguestfs.org/guestfs.(*Guestfs).Internal_test.func7(0xc000018180,
> 0xadfb60, 0xadfb80, 0xc000010048, 0x0, 0x0, 0x0, 0xae3e10, 0xae3e30,
> 0xade3a0, ...)
>         golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0xa9
> libguestfs.org/guestfs.(*Guestfs).Internal_test(0xc000018180, 0x4ee3a5,
> 0x3, 0xc000061be8, 0xc000061af8, 0x0, 0x0, 0xc000061a00, 0x0, 0x0, ...)
>         golang/src/libguestfs.org/guestfs/guestfs.go:6729 +0x3c9
> main.main()
>         golang/bindtests/bindtests.go:77 +0x151e
> exit status 2
> FAIL run-bindtests (exit status: 1)

In Daniel P. Berrangé's words [1],

> You're allowed to pass a Go pointer to C via CGo, but the memory that
> points to is not allowed to contained further Go pointers. So the struct
> fields must strictly use a C pointer.

One pattern to solve the problem has been shown on stackoverflow [2].
Thus, rewrite the "arg_string_list" and "free_string_list" functions
almost entirely in C, following that example.

While this approach is not the most idiomatic Go, as a solution exists
without C helper functions [3], it should still be acceptable, at least as
an incremental improvement, for letting "golang/run-bindtests" pass.

[1] https://listman.redhat.com/archives/libguestfs/2021-September/msg00118.html
[2] https://stackoverflow.com/questions/35924545/golang-cgo-panic-runtime-error-cgo-argument-has-go-pointer-to-go-pointer
[3] https://listman.redhat.com/archives/libguestfs/2021-September/msg00106.html

Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: "Richard W.M. Jones" <rjones@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210921192939.32468-1-lersek@redhat.com>
Tested-by: "Richard W.M. Jones" <rjones@redhat.com>
Acked-by: "Richard W.M. Jones" <rjones@redhat.com>
2021-09-27 09:57:34 +02:00
Richard W.M. Jones
5c68392175 Version 1.46.0. v1.46.0 2021-09-23 14:52:46 +01:00
Richard W.M. Jones
5b432781e2 build: Remove BUGS file
Recent changes to Red Hat Bugzilla broke our ability to generate this
file.  In any case it's not very useful since a simple Bugzilla query
can be used instead:

https://bugzilla.redhat.com/buglist.cgi?component=libguestfs

Remove the file.  The generator used this file to ensure it was being
run from the correct directory and to take a lock.  Use RELEASES file
instead.

See also: https://github.com/python-bugzilla/python-bugzilla/issues/149
2021-09-23 14:52:46 +01:00
Richard W.M. Jones
ab2d624f46 docs: Finalize release notes for 1.46 release today 2021-09-23 09:44:08 +01:00
Laszlo Ersek
627f808e4b tests: xfs: remove lazy-counter disablement test
According to xfs_admin(8):

>        -c 0|1 Enable (1) or disable (0) lazy-counters in the  filesys‐
>               tem.
>
>               Lazy-counters  may  not  be  disabled  on  Version 5 su‐
>               perblock filesystems (i.e. those with metadata CRCs  en‐
>               abled).
>
>               [...]

According to mkfs.xfs(1):

>        -m global_metadata_options
>        Section Name: [metadata]
>               These  options  specify metadata format options that ei‐
>               ther apply to the entire  filesystem  or  aren't  easily
>               characterised  by  a  specific  functionality group. The
>               valid global_metadata_options are:
>
>                   [...]
>
>                    crc=value
>                           This  is  used  to create a filesystem which
>                           maintains and checks CRC information in  all
>                           metadata  objects  on disk. The value is ei‐
>                           ther 0 to disable the feature, or 1  to  en‐
>                           able the use of CRCs.
>
>                           [...]
>
>                           By  default,  mkfs.xfs  will enable metadata
>                           CRCs.

Consistently with the above, the first "xfs_admin" test case in
"generator/actions_core.ml", which attempts to disable lazy counters,
always fails:

> 534/550 test_xfs_admin_0
> libguestfs: error: xfs_admin: /dev/sda1: Cannot disable lazy-counters on V5 fs

We can resolve this test failure in three ways:

(1) Extend do_mkfs() [daemon/mkfs.c], possibly even introduce
    do_mkfs_xfs(), and permit the caller to specify "-m crc=0" for
    mkfs.xfs. Then use this option when the temporary filesystem is
    created in the XFS test that disables lazy counters.

(2) Extend the "guestfs_int_xfsinfo" structure in the libguestfs-common
    project, with an "xfs_crc" field. Extend parse_xfs_info()
    [daemon/xfs.c] to populate the field from "meta-data=...crc=[01]".
    Modify the test case to check the following post-condition:

      xfs_crc || xfs_lazycount == 0

    instead of the current

      xfs_lazycount == 0

    effectively ignoring "xfs_lazycount" when "xfs_crc" is set.

(3) Remove the test altogether that attempts to disable lazy counters
    after filesystem creation.

Given that new XFS filesystems are created with metadata CRCs enabled by
default, and several XFS features depend on metadata CRCs being enabled,
this patch implements option (3).

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-4-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-09-21 19:01:18 +02:00
Laszlo Ersek
24f98939ae test-md-and-lvm-devices: work around RAID0 regression in Linux v3.14/v5.4
The "test-md-and-lvm-devices" test case creates, among other things, a
RAID0 array (md127) that spans two *differently sized* block devices
(sda1: 20MB, lv0: 16MB).

In Linux v3.14, the layout of such arrays was changed incompatibly and
undetectably. If an array were created with a pre-v3.14 kernel and
assembled on a v3.14+ kernel, or vice versa, data could be corrupted.

In Linux v5.4, a mitigation was added, requiring the user to specify the
layout version of such RAID0 arrays explicitly, as a module parameter. If
the user fails to specify a layout version, the v5.4+ kernel refuses to
assemble such arrays. This is why "test-md-and-lvm-devices" currently
fails, with any v5.4+ appliance kernel.

Until we implement a more general solution (see the bugzilla link below),
work around the issue by sizing sda1 and lv0 identically. For this,
increase the size of sdb1 to 24MB: when one 4MB extent is spent on LVM
metadata, the resultant lv0 size (20MB) will precisely match the size of
sda1.

This workaround only affects sizes, and does not interfere with the
original purpose of this test case, which is to test various *stackings*
between disk partitions, software RAID (md), and LVM logical volumes.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=2005485
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: remove stray empty line]
2021-09-21 19:00:59 +02:00
Laszlo Ersek
8156c16520 test-9p: fix the base directory that's exported to the guest
In commit 6d32773e81 ("tests: Run the tests in parallel.", 2021-03-18),
the "abs_srcdir" macro value that the 9p test would see changed from
".../tests/9p" to just ".../tests" -- the last component got dropped.

(Said commit updated some "abs_srcdir"-based references accordingly, for
example under "tests/disks", but "tests/9p/test-9p.sh" was missed.)

Therefore, the guest-visible location of the "/test-9p.sh" file changed to
"/9p/test-9p.sh", and a non-recursive listing of the guest-visible root
directory would not return the file. Thus, the test fails now.

Restore the host-side base directory to ".../tests/9p".

Fixes: 6d32773e81
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210920052335.3358-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-09-21 19:00:11 +02:00
Olaf Hering
f47e0bb672 appliance: reorder mounting of special filesystems in init
Make sure proc and dev are available early.
No change in behavior intended.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
2021-09-15 12:37:08 +01:00
Olaf Hering
9db0c98c99 appliance: enable bash's Process Substitution feature
bash can read input from a spawned process, and even provide input to
such process. This feature relies on /dev/fd/ being present. In the
past udev silently created this symlink, so this bash feature worked
more or less by accident. With recent systemd versions, such as 246
which is included in Leap 15.3, the symlink is not created anymore. As
a result scripts, such as /sbin/dhclient-script, fail to work
properly.

This symlink should have been created in version 1 of this variant of /init.

https://bugzilla.opensuse.org/show_bug.cgi?id=1190501

Signed-off-by: Olaf Hering <olaf@aepfle.de>
2021-09-15 12:37:08 +01:00
Olaf Hering
c0de4de902 appliance: add reboot and netconfig for SUSE
systemd-sysvinit contains the reboot command, which is used to
properly stop the VM. This was required by other packages, and as a
result always available. Since Leap 15.3 it will not be installed, and
as a result the VM will just panic because /init died.

If the appliance is started with --network, dhclient will run
/usr/sbin/dhclient-script, which in turn may call /sbin/netconfig to
update /etc/resolv.conf. Install sysconfig-netconfig to make sure DNS
resolving actually works.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
2021-09-14 20:49:02 +01:00
Richard W.M. Jones
46ab3dbbc0 docs: Prepare draft release notes for 1.46 2021-09-13 19:29:17 +01:00
Richard W.M. Jones
ae8ff236ab bugs-in-changelog.sh: Various fixes to cope with new changelog format 2021-09-13 19:29:17 +01:00
Richard W.M. Jones
e14ff93742 lib: direct: Remove use of sga
sga (or "sgabios" or "Serial Graphics Adapter") is an option ROM for
seabios which directs output to the serial adapter.  This is very
useful for debugging BIOS problems during boot.

RHEL wants to deprecate this feature (in fact, they just deprecated it
without telling us).  However there is an equivalent feature in
seabios (seabios >= 1.11 / qemu >= 2.11.0) which can be enabled using
either -nographic or -machine graphics=off

This commit removes sga and enables -machine graphics=off in the
direct backend.

References (for RHEL 9 qemu change):
https://bugzilla.redhat.com/show_bug.cgi?id=2002325
https://bugzilla.redhat.com/show_bug.cgi?id=2000845
https://lists.nongnu.org/archive/html/qemu-devel/2021-09/msg02417.html
https://listman.redhat.com/archives/libvir-list/2021-September/msg00205.html

For the libvirt backend we will continue to use <bios useserial=yes>.
This currently breaks when sga is not available, but I talked to Dan
and the plan there is to adapt libvirt so the same XML will enable
-machine graphics=off.  IOW libguestfs does not need to make any
change.

References (for libvirt change):
https://bugzilla.redhat.com/show_bug.cgi?id=2003092
https://listman.redhat.com/archives/libvir-list/2021-September/msg00193.html

Thanks: Gerd Hoffman, Daniel Berrangé
2021-09-13 10:40:50 +01:00
Laszlo Ersek
523b0180d8 daemon_utils_tests: generalize ocaml-hivex[-devel] lookup
Pass $(HIVEX_LIBS) with -cclib under the "daemon_utils_tests_LINK" target;
otherwise the OCaml compiler does not tell the linker where "-lhivex" can
be found, and the linking step fails if "-lhivex" is not on a system
library path.

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-3-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-09-09 14:52:03 +02:00
Laszlo Ersek
53e03eefae daemon: generalize ocaml-hivex[-devel] lookup
"ocamlc -where" is supposed to "print the location of the standard library
and exit". While this directory contains core OCaml C header files, it
does not contain hivex-related C header files. Trim "guestfsd_CPPFLAGS"
accordingly.

Furthermore, the hivex module for OCaml may exist elsewhere than under the
OCaml standard library directory. Invoke "ocamlfind query hivex" to find
this module. This is what AC_CHECK_OCAML_PKG(hivex) does too, in
"m4/guestfs-ocaml.m4" and "m4/ocaml.m4".

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210908133542.19002-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
2021-09-09 14:52:02 +02:00
Richard W.M. Jones
ceb034c92c python, java: Avoid bogus -fanalyzer warnings
This is essentially the same as the previous OCaml commit.  It does
not fix a real bug.
2021-09-07 16:50:38 +01:00