Commit Graph

11449 Commits

Author SHA1 Message Date
Richard W.M. Jones
c8e0b45389 fish: Update documentation to refer to cryptsetup-open/close and BitLocker. 2020-10-12 10:44:29 +01:00
Richard W.M. Jones
c456ea0332 New APIs: cryptsetup-open and cryptsetup-close.
This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.

The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.

The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort.  As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.

The deprecated functions and the new functions are all (re-)written in
OCaml.

There is no new test here, unfortunately.  It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.

Also synchronise with common module.
2020-10-12 10:44:08 +01:00
Richard W.M. Jones
bfd0886765 golang: Fix path to include/guestfs.h.
Thanks: Martin Kletzander
Fixes: commit 75abec1f70
2020-10-06 15:26:14 +01:00
Pino Toscano
dbfab7d3b2 build: fix includedir in uninstalled libguestfs.pc
Update includedir with the new directory that contains guestfs.h.

Updates commit 75abec1f70.
2020-09-22 18:12:05 +02:00
Richard W.M. Jones
b89f377540 Version 1.43.2. v1.43.2 2020-09-21 18:56:18 +01:00
Richard W.M. Jones
75abec1f70 include: Move lib/guestfs.h to include/guestfs.h
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.

It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory.  Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up.  This is
certainly the case for examples, and some language bindings and some
tests.

In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
2020-09-21 18:38:28 +01:00
Richard W.M. Jones
3743cc89f6 golang: Capitalize fields in structs so they are publicly accessible.
See: https://github.com/libguestfs/libguestfs/issues/53
2020-09-21 10:56:29 +01:00
Yuri Chornoivan
fce82fe55a Fix minor typos 2020-08-24 16:24:38 +01:00
Pino Toscano
14ff95c511 po-docs: turn language list into LINGUAS file
Use a LINGUAS file with the list of available translations instead of
defining them in a make variable. This way Weblate will be able to
update the list using an available addon, and we do not need to list
those not built.

Accordingly, rename the variable with built languages to
'linguas_translated'.
2020-08-17 07:08:11 +02:00
Richard W.M. Jones
bd3fb65351 fish: Fix parsing of exportnames in NBD URIs.
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md#nbd-uri-export-name
2020-08-16 11:02:42 +01:00
Weblate
b96c8f5146 Update translation files
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: libguestfs/libguestfs-docs-master
Translate-URL: https://translate.fedoraproject.org/projects/libguestfs/libguestfs-docs-master/
2020-08-13 11:59:02 +02:00
Weblate
48b0799297 Update translation files
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: libguestfs/libguestfs-master
Translate-URL: https://translate.fedoraproject.org/projects/libguestfs/libguestfs-master/
2020-08-13 11:56:56 +02:00
Pino Toscano
9a986ed60a Remove references to Zanata
We migrated to Weblate, and Zanata is being decommissioned.
2020-08-13 11:27:44 +02:00
Pino Toscano
049d83f6e4 po/po-docs: update catalog templates
Regenerate the catalog templates according to the updated extraction
rules (mostly fixed paths).
2020-08-13 11:27:44 +02:00
Pino Toscano
d3b43bb47a po-docs: fix references to sources
Start the message extraction from the toplevel source directory, so the
file references are relative to that, instead of relative to this
po-docs subdirectory.

Also update/regenerate podfiles accordingly.
2020-08-13 11:27:44 +02:00
Pino Toscano
a16f67e404 po: fix references to OCaml sources
Start the message extraction from the toplevel source directory, so the
file references are relative to that, instead of relative to this po
subdirectory.
2020-08-13 11:27:44 +02:00
Pino Toscano
bd896d68c0 po-docs: split pot and po handling
With the Weblate adoption, we let it update the po files from the
catalog template. The po4a behaviour of extracting the template,
merging the existing translations, and creating the translated PODs at
once is problematic. Hence, split the extraction and the translated POD
generation in two.

Use po4a-gettextize to extract the catalog template only, not doing it
anymore automatically at each build. There is no more need for a
po4a.conf file.

Use po4a-translate to create translated PODs from the po files, keeping
the fixup of the generated files (to avoid spurious =encoding, etc).
Add a silent rule to hide the po4a-translate command lines by default.

These changes also allow us to get rid of the POD existance checks with
associated error message pointing to the update-po rule. Now each
translated POD file is generated because of make dependency, and it
depends only on its po file.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
2020-08-13 11:27:44 +02:00
Pino Toscano
3095825350 podfiles: add missing documentation 2020-08-13 11:27:44 +02:00
Pino Toscano
5be4970b1f po-docs: add missing dependencies for guestfish.1
The guestfish man page uses also additional POD snippets, so list them
as dependencies to make sure they are up-to-date.

This does not change the behaviour at the moment, however it will matter
when each traslated POD file will be generated on its own.
2020-08-13 11:27:44 +02:00
Pino Toscano
0dc0f87f20 po: remove rules for pot/po update
Weblate will handle the update of the po files from the translation
catalog, so avoid stomping on its feet by doing the same.

The translation catalog will be regenerated manually periodically.
2020-08-13 11:27:44 +02:00
Pino Toscano
eb7ba4b1d2 po: turn language list into LINGUAS file
Use a LINGUAS file with the list of available translations instead of
defining them in a make variable. This way Weblate will be able to
update the list using an available addon.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
2020-08-13 11:27:44 +02:00
Andrey Shinkevich
3cad943a85 appliance: extract UUID from QCOW2 disk image
For the appliance of the QCOW2 format, the function get_root_uuid()
fails to get the UUID of the disk image.
In this case, let us read the first 256k bytes of the disk image  with
the 'qemu-img dd' command. Then pass the read block to the 'file'
command.

Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
2020-08-13 10:11:09 +01:00
Richard W.M. Jones
2469b4b790 rescue: Initialize roots.
Even though it seems clear from the code that roots cannot be used
uninitialized, GCC 10.1 cannot seem to work it out (possibly an LTO
bug).  Easiest way out here is to just initialize it.

rescue.c:396:37: error: 'roots' may be used uninitialized in this function [-Werror=maybe-uninitialized]
  396 |     CLEANUP_FREE_STRING_LIST char **roots;
      |                                     ^
2020-08-01 07:33:45 +01:00
Richard W.M. Jones
0b8ef5a98d daemon/debug.c: Use __builtin_trap to cause segfault.
I couldn't get GCC 10.1 to ignore this warning any longer, possibly
because I am using LTO.  In any case dereferencing a pointer is
undefined behaviour, so let's use GCC's __builtin_trap() function
instead (also supported by clang).

debug.c: In function 'debug_segv':
debug.c:1002:8: error: null pointer dereference [-Werror=null-dereference]
 1002 |   *ptr = 1;
      |        ^
2020-08-01 07:27:17 +01:00
Richard W.M. Jones
599f2f7bd9 m4: Increase warning limit for frame-larger-than.
Generated code in virt-builder causes the error below.  Since we
cannot control what is generated by bison, increase the limit.

  CCLD     virt-index-validate
index-parse.c: In function 'yyparse':
index-parse.c:1857:1: error: the frame size of 5152 bytes is larger than 5000 bytes [-Werror=frame-larger-than=]
 1857 | }
      | ^
lto1: all warnings being treated as errors
lto-wrapper: fatal error: gcc returned 1 exit status
2020-08-01 07:09:52 +01:00
Richard W.M. Jones
5b29fba5b2 docs: Move shared POD for --keys-from-stdin to common code.
No functional change.
2020-07-30 14:06:45 +01:00
Richard W.M. Jones
eb78e990ac daemon, lib: Replace deprecated security_context_t with char *.
This gives deprecation warnings.  It always was simply a char *, and
the recommendation upstream is to replace uses with char *:

9eb9c93275
2020-07-30 13:58:35 +01:00
Richard W.M. Jones
224f373043 lib: Increase default memsize to 1280 (RHBZ#1837765).
Argon2 is the default LUKS Password-Based Key Derivation Function
(PBKDF) for some new guests such as RHEL 8.2 and Fedora.  It is
designed to be "memory hard", meaning that by design it requires large
amounts of memory, making it expensive to brute-force.  Unfortunately
the default for guests which had more than a few GB of RAM at install
time is to require about 1 GB of RAM to decrypt the block device,
which is considerably larger than the default available in the
libguestfs appliance.

To make it possible to open these encrypted disks we need to make the
appliance larger.  This could be done as a one-off, and the current
workaround is simply to set LIBGUESTFS_MEMSIZE=2048 or a similar
amount.  However since we don't know in advance whether we could be
dealing with an encrypted disk, partition, etc. or what PBKDF it uses,
the only way to deal with this in all circumstances is to increase the
default memsize.  This commit increases it quite a lot (768 -> 1280)
which is unfortunate.

Note as there is some confusion on this point: Since libguestfs does
not attempt to decrypt disks in parallel, you only need ~ 1GB in
total, not per encrypted disk.

For a reproducer, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1837765#c14
2020-07-17 10:47:18 +01:00
Richard W.M. Jones
c5eec6c17b docs: Increase minimum version of Python to 3.6.
3.5 is needed to support type hints.

Debian oldstable has 3.5 (oldoldstable needed 3.4).
2020-07-06 20:37:51 +01:00
Sam Eiderman
66a5913462 python: Add type hints
Since support for python2 is dropped we can use the new python3 syntax
for type hints.

Signed-off-by: Sam Eiderman <sameid@google.com>
2020-07-06 20:35:58 +01:00
Richard W.M. Jones
dfe9fe8de8 Version 1.43.1. v1.43.1 2020-07-06 16:25:36 +01:00
Richard W.M. Jones
75def0ef18 python: Remove deprecated use of PyEval_ThreadsInitialized.
The correct/modern way to do this is:

https://docs.python.org/3/c-api/init.html#releasing-the-gil-from-extension-code

PyEval_ThreadsInitialized was deprecated in 3.9 and will be removed
completely in 3.11, so we shouldn't use that function.
2020-07-06 12:40:33 +01:00
Richard W.M. Jones
403d17a876 python: Drop support for Python 2.
Python 2 reached end of life on 2020-01-01:
https://python3statement.org/
https://pythonclock.org/

The minimum version required is now Python 3.4 (since that is the
version in Debian oldoldstable), but 3.6 is the minimum version that
I actually test.
2020-07-06 12:40:33 +01:00
Richard W.M. Jones
38e2ad1fbc builder: Update index file.
Fixes: commit 4837698d83.
2020-07-06 12:40:33 +01:00
Richard W.M. Jones
4837698d83 builder: templates: Add CentOS 8.2 and RHEL 8.2. 2020-06-16 12:51:09 +01:00
Richard W.M. Jones
6670dc0fbf builder: templates: Add centos-7.8, fedora-32, rhel-7.8.
Ubuntu 20.04 could not be built because their installer has changed in
a way which is not compatible with the current build script.  This
needs some work to fix.
2020-06-04 15:22:38 +01:00
Sergei Golovan
987734fcca erlang: Port to libei for Erlang 23
Replace the use of liberl_interface, which is removed in Erlang 23,
by libei. The implementation uses the ei_decode_iodata() function
which has been introduces only for Erlang 23, so it doesnt work with
earlier Erlang versions.
2020-06-04 13:49:35 +01:00
Pino Toscano
faa5d8507f sysprep: add Kerberos keytab file removal
This new operation removes the Kerberos /etc/krb5.keytab file from the
guest.

Thanks to Christian Heimes and François Cami for the hints.

Related to RHBZ#1789592.
2020-05-26 11:02:46 +02:00
Pino Toscano
0a53e2c7fc sysprep: add IPA offline unenrollment (RHBZ#1789592)
This new operation unenrolls the guest from a IPA server offline, by
removing the configuration files and certificates.

Thanks to Christian Heimes and François Cami for the hints.
2020-05-26 11:02:37 +02:00
Pino Toscano
b9065fa7ad sysprep: ca-certificates: request system CA store update
In case any certificate is removed from the guest, regenerate the system
CA store.
2020-05-07 14:05:09 +02:00
Pino Toscano
bb7fc6d0a1 sysprep: add a update_system_ca_store side effect
Add a simple side effect to make operation flag that a regeneration of
the system CA store is needed. In case it is flagged, regenerate the
system CA store directly, or using a firstboot script in case of
incompatible architectures.

This change is almost a no-op, since no operation requires the
regeneration of the system CA store yet.
2020-05-07 14:05:09 +02:00
Pino Toscano
b25e3495f5 customize: port do_run to run_in_guest_command
Make use of the new helper function in Tools_utils to run commands in
the guest.
2020-05-07 14:04:56 +02:00
Richard W.M. Jones
94ff24b880 appliance: Set default guestfs appliance path to $libdir/guestfs.
For unclear reasons this broke recently, although the breakage is only
reproducible in Fedora Koji.  It appears to be caused by the
calculation of the internal autoconf variable $acl_libdirstem
changing.  In the reproducer system:

checking for the common suffixes of directories in the library search path... lib,lib,lib64

On my local system:

checking for the common suffixes of directories in the library search path... lib64,lib64

My local system would be the correct one.  The actual code that
calculates this does some crazy stuff with ‘gcc -print-search-dirs’
which would hint that this output has changed in some way that subtly
breaks the generated configure script.  However even with match GCC
versions on my local system I could still not reproduce the issue.

None of this matters, as the easiest fix here is simply to stop using
internal autoconf variables at all.  I also added some AC_MSG_*
statements so we can more easily see what's going on in the configure
output.

Fixes commit 0f79400c7f.
2020-05-06 12:39:41 +01:00
Richard W.M. Jones
26324d5445 builder: templates: Add Ubuntu 20.04 code name (focal). 2020-05-05 22:23:49 +01:00
Richard W.M. Jones
a6ed62b0a6 builder: Fix all download.devel links.
All broken in early 2020 for no discernible reason.
2020-04-30 12:33:11 +01:00
Geoff Toole
ef2761eaa0 build: Fix linking of OCaml components when srcdir != builddir
ocaml-link.sh is a generated file, so run it from the build directory.
Remove it from EXTRA_DIST and also adjust docs/guestfs-hacking.pod
accordingly.
2020-04-20 09:31:35 +01:00
Richard W.M. Jones
c2c11382bb daemon: xattr: Filter out user.WofCompressedData from xattrs (RHBZ#1811539).
See comment in code for justification.

Thanks: Yongkui Guo for finding the bug.
2020-03-30 10:40:13 +01:00
Richard W.M. Jones
af8ed266a2 daemon: Add filter_list utility function.
For filtering lists of strings based on a predicate.
2020-03-30 10:40:13 +01:00
Richard W.M. Jones
5c175fe732 daemon: xattr: Refactor code which splits attr names from the kernel.
The kernel returns xattr names in a slightly peculiar format.  We
parsed this format several times in the code.  Refactor this parsing
so we only do it in one place.
2020-03-30 10:40:13 +01:00
Pino Toscano
088b6d1c80 get-kernel: fallback to uncompressed vmlinux kernels
In case the Linux guest has no compressed linux kernel (vmlinuz),
fallback to uncompressed ones (vmlinux).

Reported by: Laurent Vivier.
2020-03-12 13:52:17 +01:00