Commit Graph

10298 Commits

Author SHA1 Message Date
Richard W.M. Jones
032148d416 daemon: Reimplement ‘file’ API in OCaml.
‘file’ is a small, self-contained API which runs a single command, so
it's a good test case for reimplementing APIs.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
d5b6f1df5f daemon: Allow parts of the daemon and APIs to be written in OCaml.
This change allows parts of the daemon to be written in the OCaml
programming language.  I am using the ‘Main Program in C’ method along
with ‘-output-obj’ to create an object file from the OCaml code /
runtime, as described here:
https://caml.inria.fr/pub/docs/manual-ocaml/intfc.html

Furthermore, change the generator to allow individual APIs to be
implemented in OCaml.  This is picked by setting:

  impl = OCaml <ocaml_function>;

The generator creates ‘do_function’ (the same one you would have to
write by hand in C), with the function calling the named
‘ocaml_function’ and dealing with marshalling/unmarshalling the OCaml
parameters.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
10cf01419a daemon: Reimplement handling of lvm.conf and filters.
LVM is fine with a completely empty configuration file (meaning "all
defaults"), so start with one instead of copying the system
configuration file.

Also this means we can very easily implement lvm_set_filter
functionality without using Augeas, since we no longer have to worry
about existing filters being present.

Thanks: Alasdair Kergon, Zdenek Kabelac.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
fe8c11ae4d v2v: Add slow tests of opensuse 13.1, 13.2 and 42.1 conversions. 2017-07-27 22:27:08 +01:00
Richard W.M. Jones
ba41cf1a51 v2v: bootloaders: Handle no Bootloader::Tools default section (RHBZ#1472208).
In SUSE guests, handle the case where
Bootloader::Tools::GetDefaultSection () returns undef.

Previously this would return an empty string and cause a bogus error
in subsequent code:

virt-v2v: error: libguestfs error: statns: statns_stub: path must start
with a / character
2017-07-27 22:27:08 +01:00
Richard W.M. Jones
024efc0f2f common/mlstdutils: Fix parallel builds of bytes.ml.
With OCaml < 4.02 when using the alternate Bytes module, this module
would be compiled twice during parallel builds, resulting in
occasional corruption.  The reason for this is that the ocamldep file
mentions ‘bytes.cmo’ whereas the ‘$(OCAML_BYTES_COMPAT_ML)’ macro
expands to ‘../../common/mlstdutils/bytes.ml’.  Make doesn't recognize
these as the same file.

Use an alternate way to specify this file to fix this.
2017-07-27 22:27:08 +01:00
Richard W.M. Jones
381c8b68c4 daemon: Remove GUESTFSD_EXT_CMD.
GUESTFSD_EXT_CMD was used by OpenSUSE to track which external commands
are run by the daemon and package those commands into the appliance.

It is no longer used by recent SUSE builds, so remove it.

Thanks: Pino Toscano, Olaf Hering.
2017-07-27 17:31:41 +01:00
Richard W.M. Jones
8d09d716aa common/mlstdutils: Build a bytecode version of this library.
Even if ocamlopt is available, always build a bytecode version of
‘common/mlstdutils’.

Furthermore, because this library is pure OCaml, we should not be
using ‘ocamlmklib’.  We should use ‘ocaml{c,opt} -a’ instead.  This
doesn't make any difference for native code, but for bytecode it was
building a broken library.

The original reason for making this change is because the generator is
always built as bytecode, and it depended on
‘../common/mlstdutils/guestfs_config.cmo’ and
‘../common/mlstdutils/std_utils.cmo’.  On native code platforms these
were not built before the generator and so the generator races to
build the .cmi and .cmo files.  Since the generator doesn't have
correct dependencies covering the ‘common/mlstdutils’ directory you
can get a broken link on fast machines:

  File "../common/mlstdutils/std_utils.ml", line 1:
  Error: Corrupted compiled interface
  ../common/mlstdutils/guestfs_config.cmi
  make[2]: *** [Makefile:1993: ../common/mlstdutils/std_utils.cmo] Error 2
  make[2]: *** Waiting for unfinished jobs....
  make[2]: Leaving directory '/builddir/build/BUILD/libguestfs-1.37.17/generator'
2017-07-27 14:39:46 +01:00
Richard W.M. Jones
a20859f563 tests: lvm: Make the lvm_set_filter test easier to understand.
No functional change.
2017-07-27 14:39:46 +01:00
Richard W.M. Jones
d098d314e6 daemon: Tidy up the <daemon.h> header file.
No change, just tidy up the header file.
2017-07-27 13:16:46 +01:00
Richard W.M. Jones
b981ac0a6e v2v: uefi: Add function to error out unless UEFI firmware is installed.
No change, just refactoring.
2017-07-27 13:11:43 +01:00
Richard W.M. Jones
7cd2753115 common/mlstdutils: Don't export StringSet.map (for older OCaml).
Fixes commit 8548729cf2.

Thanks: Pino Toscano.
2017-07-24 14:15:56 +01:00
Richard W.M. Jones
60408cecd6 common/mlutils: Remove bogus suffix parameter from Mkdtemp.temp_dir.
The C function mkdtemp(3) requires that the string ends with 6 'X'
characters, so appending a non-empty suffix causes the function to
raise EINVAL.

Luckily we only ever called this function with the last parameter "".
2017-07-24 11:48:24 +01:00
Richard W.M. Jones
8548729cf2 common/mlstdutils: Implement StringSet. 2017-07-24 11:46:22 +01:00
Richard W.M. Jones
ac3db3d664 common/mlstdutils: Add chomp function to remove \n from end of strings.
This is like the Perl chomp function, it removes a single \n from the
end of a string if present, else leaves the string alone.

I believe I found the only (two) places where such a function is used,
but there may be a few more lurking.
2017-07-21 15:56:59 +01:00
Richard W.M. Jones
95dfa054c0 builder: templates: Rebuild all Fedora 26 templates with LVM disabled.
This also fixes virt-p2v-make-disk which could not handle the base
disk images using LVM.

Fixes commit 48874066b9
and commit 0a73032120.
2017-07-19 20:56:11 +01:00
Richard W.M. Jones
b62419c05b builder: templates: Calculate revision correctly when existing file has no revision. 2017-07-19 15:24:05 +01:00
Richard W.M. Jones
11a329b1b0 builder: templates: Generate plain partition layouts (without LVM).
This was a mistake when the original shell scripts were transcribed to
make-template.ml (in commit 65a0570385).
2017-07-19 14:50:59 +01:00
Richard W.M. Jones
a2aabffd81 p2v: make-disk: Add verbose option to dracut in post-install script.
The verbose output goes to /tmp/builder.log so it wouldn't normally be
seen by the end user, but it's useful for debugging.
2017-07-19 14:47:15 +01:00
Richard W.M. Jones
5c86c64c79 fuse: Make it clearer that -o allow_other must be enabled in /etc/fuse.conf.
It wasn't very clear in the manual before.
2017-07-19 11:50:16 +01:00
Richard W.M. Jones
ffdcb7cfeb build: Require <caml/unixsupport.h>
Require <caml/unixsupport.h> (an OCaml header file) and remove
alternate defintions of ‘Nothing’ and ‘unix_error’ which are defined
in this header file.

We require OCaml >= 3.11 which has this header file, so there is no
need to test for it or provide alternative definitions.

Thanks: Pino Toscano.
2017-07-18 18:42:23 +01:00
Richard W.M. Jones
318ca68408 build: Make OCaml compiler required for all builds.
Previously the OCaml compiler was only required if building from git
but was at least theoretically optional if building from tarballs
(although this was never tested).  Since we want to write parts of the
daemon in OCaml, this makes OCaml required for all builds.

Note that the ‘--disable-ocaml’ option remains, but it now only
disables OCaml bindings and OCaml virt tools.  Using this option does
not disable the OCaml compiler requirement.

Also note that ‘HAVE_OCAML’ changes meaning slightly, so it now means
"build OCaml bindings and tools" (analogous to ‘HAVE_PERL’ and
others).  The generator, daemon [in a future commit], and some utility
libraries needed by the generator or daemon do not test for this macro
because we can assume OCaml compiler availability.
2017-07-18 18:37:35 +01:00
Richard W.M. Jones
8ecfa6c638 Version 1.37.18. v1.37.18 2017-07-18 15:15:47 +01:00
Richard W.M. Jones
93a2f9d213 v2v: Set correct root device when rebuilding SUSE kdump initrd.
The ‘vmware-uninstall-tools.pl’ script tries to rebuild the initrd.
On SUSE, if kdump initrd has been enabled, this would use the wrong
root device because ‘mkdumprd’ doesn't know what root device to use.
Fix that by setting the ‘rootdev’ environment variable.

See also https://bugzilla.redhat.com/1465849 (this is not a fix).
2017-07-18 14:13:36 +01:00
Richard W.M. Jones
98641e26c8 lib: Move guestfs_int_parse_unsigned_int* functions.
Just code motion, no functional change.
2017-07-18 13:22:00 +01:00
Richard W.M. Jones
0c98971040 p2v: make-disk: Install p2v service in multi-user.target instead of default.target.
Tested by doing ‘make -C p2v run-virt-p2v-in-a-vm’.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
db5e6bd221 customize: firstboot: Remove default.target.wants/guestfs-firstboot.service too.
This file could be left over from earlier use of the ‘--firstboot’
option.

Thanks: Pino Toscano.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
00c4c5c8f8 customize: firstboot: Install firstboot scripts in multi-user.target (RHBZ#1469655).
When the guest uses systemd, install firstboot scripts under
‘multi-user.target’.

See this link for details:
https://lists.freedesktop.org/archives/systemd-devel/2017-July/039325.html

This fixes firstboot on Fedora 26 (and in general with systemd >= 233).

Thanks: Lennart Poettering
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
1286129fac customize: firstboot: Factor out the name of the systemd target.
Currently we install the firstboot service under systemd target
‘default.target’.  This change simply factors out this name.

Note that the name is not factored out in the code which deletes the
old ‘/etc/systemd/system/default.target.wants/firstboot.service’ file,
since that would have always been installed in the same location.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
981d3c9630 docs: Add info about how the stable tarballs are signed. 2017-07-16 21:36:19 +01:00
Richard W.M. Jones
f60f8c47cc lib: Move implementation of ‘hivex_value_utf8’ to new file ‘lib/hivex.c’.
Just a code movement, no change.
2017-07-14 14:35:37 +01:00
Richard W.M. Jones
2afe62eb60 builder: templates: Prevent script from being run as root.
Or with LIBVIRT_DEFAULT_URI=qemu:///system which is the same
thing.

In either case the images are created as user qemu.qemu and then
aren't readable or modifiable by later parts of the script.
2017-07-13 12:07:35 +01:00
Richard W.M. Jones
9fd2231cb0 customize: Test firstboot in Fedora 26.
It currently fails because of this bug apparently in systemd:

https://bugzilla.redhat.com/show_bug.cgi?id=1469655
https://github.com/systemd/systemd/issues/6334
2017-07-13 12:00:38 +01:00
Pino Toscano
a1c559a0f4 ocaml: fix build with Bytes fallback
Place the Bytes fallback module in the right place (mlstdutils), with no
need to make it available directly also for generation, since it uses
mlstdutils now.

Fixes commit 61d4891ef4.
2017-07-12 09:52:46 +02:00
Pino Toscano
178a20f6c3 builder: fix paths to mlstdutils & mlutils
Followup/fix of commit 61d4891ef4.
2017-07-12 09:52:46 +02:00
Richard W.M. Jones
0a73032120 builder: Add Fedora 26 for aarch64, armv7l, i686, ppc64, ppc64le.
All the secondary architectures except s390x, which we cannot handle
because qemu doesn't emulate that architecture sufficiently yet.
2017-07-11 15:17:58 +01:00
Richard W.M. Jones
5aaa91b524 builder: templates: Use full path for nvram file.
Works around a virt-install or libvirt bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1469491
2017-07-11 13:02:55 +01:00
Richard W.M. Jones
904578e8b9 builder: templates: Use dl.fedoraproject.org for secondary architectures.
Avoid redirecting to incomplete mirrors.
2017-07-11 12:00:58 +01:00
Richard W.M. Jones
48874066b9 builder: Add fedora-26 (x86_64) image. 2017-07-11 10:29:57 +01:00
Richard W.M. Jones
09b14a6076 Version 1.37.17. v1.37.17 2017-07-10 22:06:45 +01:00
Richard W.M. Jones
538ef8107c common/mlutils: Add test .ml file to EXTRA_DIST.
Fixes commit 8be72935dc.
2017-07-10 22:04:41 +01:00
Richard W.M. Jones
f30b2065a2 gobject: Add outline guestfs-gobject(3) manual page.
Since we removed gtk doc, we might as well replace it with a
manual page explaining the basics of how to run gjs.
2017-07-10 17:03:19 +01:00
Richard W.M. Jones
f5f7f00a46 gobject: Remove gtk-doc (RHBZ#1465665).
The gobject bindings are adequately covered in the usual manual pages:
guestfs(3).  There is no need for separate generation of gtk-doc.
Also generating gtk documentation is the slowest part of the build,
and the tooling around gtk-doc is broken
(https://bugzilla.redhat.com/show_bug.cgi?id=1465665).

Note this removes the configure ‘--enable-gtk-doc’ option.  Using this
option now gives a warning, but is otherwise ignored:

  configure: WARNING: unrecognized options: --enable-gtk-doc
2017-07-10 17:03:19 +01:00
Richard W.M. Jones
2cb0413049 gobject: Use TEST_FUNCTIONS framework in tests. 2017-07-10 17:03:19 +01:00
Richard W.M. Jones
707f5bcfe0 daemon: Link guestfsd with libutils.
After the previous refactoring, we are able to link the daemon to
common/utils, and also remove some of the "duplicate" functions that
the daemon carried ("duplicate" in quotes because they were often not
exact duplicates).

Also this removes the duplicate reimplementation of (most) cleanup
functions in the daemon, since those are provided by libutils now.

It also allows us in future (but not in this commit) to move utility
functions from the daemon into libutils.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
e6c89f9631 utils: Rename ‘guestfs-internal-frontend.h’ to ‘guestfs-utils.h’.
The reason it's not just ‘utils.h’ is because Pino is worried that we
might pick up /usr/include/utils.h from a rogue library.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
8be72935dc mllib, v2v: Split out OCaml utils bindings ‘common/mlutils’.
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.

The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
5efebd8c7e utils: Split out structs cleanups and printing into common/structs.
These won't be used by the daemon, so interferes with us using
common/utils in the daemon, so they are moved to a different library.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
78587cc3c6 common/utils: cleanups: Add CLEANUP_CLOSE function.
This is present in the daemon, but the function could be used
throughout the code.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
b2469a6d96 common/utils: Refactor stdlib, gnulib and libxml2 cleanup functions.
This refactoring change just moves the cleanup functions around in the
common/utils directory.

libxml2 cleanups are moved to a separate object file, so that we can
still link to libutils even if the main program is not using libxml2
anywhere.  Similarly gnulib cleanups.

cleanup.c is renamed to cleanups.c.

A new header file cleanups.h is introduced which will replace
guestfs-internal-frontend-cleanups.h (fully replaced in a later commit).
2017-07-10 17:01:59 +01:00