Commit Graph

10283 Commits

Author SHA1 Message Date
Richard W.M. Jones
95dfa054c0 builder: templates: Rebuild all Fedora 26 templates with LVM disabled.
This also fixes virt-p2v-make-disk which could not handle the base
disk images using LVM.

Fixes commit 48874066b9
and commit 0a73032120.
2017-07-19 20:56:11 +01:00
Richard W.M. Jones
b62419c05b builder: templates: Calculate revision correctly when existing file has no revision. 2017-07-19 15:24:05 +01:00
Richard W.M. Jones
11a329b1b0 builder: templates: Generate plain partition layouts (without LVM).
This was a mistake when the original shell scripts were transcribed to
make-template.ml (in commit 65a0570385).
2017-07-19 14:50:59 +01:00
Richard W.M. Jones
a2aabffd81 p2v: make-disk: Add verbose option to dracut in post-install script.
The verbose output goes to /tmp/builder.log so it wouldn't normally be
seen by the end user, but it's useful for debugging.
2017-07-19 14:47:15 +01:00
Richard W.M. Jones
5c86c64c79 fuse: Make it clearer that -o allow_other must be enabled in /etc/fuse.conf.
It wasn't very clear in the manual before.
2017-07-19 11:50:16 +01:00
Richard W.M. Jones
ffdcb7cfeb build: Require <caml/unixsupport.h>
Require <caml/unixsupport.h> (an OCaml header file) and remove
alternate defintions of ‘Nothing’ and ‘unix_error’ which are defined
in this header file.

We require OCaml >= 3.11 which has this header file, so there is no
need to test for it or provide alternative definitions.

Thanks: Pino Toscano.
2017-07-18 18:42:23 +01:00
Richard W.M. Jones
318ca68408 build: Make OCaml compiler required for all builds.
Previously the OCaml compiler was only required if building from git
but was at least theoretically optional if building from tarballs
(although this was never tested).  Since we want to write parts of the
daemon in OCaml, this makes OCaml required for all builds.

Note that the ‘--disable-ocaml’ option remains, but it now only
disables OCaml bindings and OCaml virt tools.  Using this option does
not disable the OCaml compiler requirement.

Also note that ‘HAVE_OCAML’ changes meaning slightly, so it now means
"build OCaml bindings and tools" (analogous to ‘HAVE_PERL’ and
others).  The generator, daemon [in a future commit], and some utility
libraries needed by the generator or daemon do not test for this macro
because we can assume OCaml compiler availability.
2017-07-18 18:37:35 +01:00
Richard W.M. Jones
8ecfa6c638 Version 1.37.18. v1.37.18 2017-07-18 15:15:47 +01:00
Richard W.M. Jones
93a2f9d213 v2v: Set correct root device when rebuilding SUSE kdump initrd.
The ‘vmware-uninstall-tools.pl’ script tries to rebuild the initrd.
On SUSE, if kdump initrd has been enabled, this would use the wrong
root device because ‘mkdumprd’ doesn't know what root device to use.
Fix that by setting the ‘rootdev’ environment variable.

See also https://bugzilla.redhat.com/1465849 (this is not a fix).
2017-07-18 14:13:36 +01:00
Richard W.M. Jones
98641e26c8 lib: Move guestfs_int_parse_unsigned_int* functions.
Just code motion, no functional change.
2017-07-18 13:22:00 +01:00
Richard W.M. Jones
0c98971040 p2v: make-disk: Install p2v service in multi-user.target instead of default.target.
Tested by doing ‘make -C p2v run-virt-p2v-in-a-vm’.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
db5e6bd221 customize: firstboot: Remove default.target.wants/guestfs-firstboot.service too.
This file could be left over from earlier use of the ‘--firstboot’
option.

Thanks: Pino Toscano.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
00c4c5c8f8 customize: firstboot: Install firstboot scripts in multi-user.target (RHBZ#1469655).
When the guest uses systemd, install firstboot scripts under
‘multi-user.target’.

See this link for details:
https://lists.freedesktop.org/archives/systemd-devel/2017-July/039325.html

This fixes firstboot on Fedora 26 (and in general with systemd >= 233).

Thanks: Lennart Poettering
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
1286129fac customize: firstboot: Factor out the name of the systemd target.
Currently we install the firstboot service under systemd target
‘default.target’.  This change simply factors out this name.

Note that the name is not factored out in the code which deletes the
old ‘/etc/systemd/system/default.target.wants/firstboot.service’ file,
since that would have always been installed in the same location.
2017-07-17 16:04:07 +01:00
Richard W.M. Jones
981d3c9630 docs: Add info about how the stable tarballs are signed. 2017-07-16 21:36:19 +01:00
Richard W.M. Jones
f60f8c47cc lib: Move implementation of ‘hivex_value_utf8’ to new file ‘lib/hivex.c’.
Just a code movement, no change.
2017-07-14 14:35:37 +01:00
Richard W.M. Jones
2afe62eb60 builder: templates: Prevent script from being run as root.
Or with LIBVIRT_DEFAULT_URI=qemu:///system which is the same
thing.

In either case the images are created as user qemu.qemu and then
aren't readable or modifiable by later parts of the script.
2017-07-13 12:07:35 +01:00
Richard W.M. Jones
9fd2231cb0 customize: Test firstboot in Fedora 26.
It currently fails because of this bug apparently in systemd:

https://bugzilla.redhat.com/show_bug.cgi?id=1469655
https://github.com/systemd/systemd/issues/6334
2017-07-13 12:00:38 +01:00
Pino Toscano
a1c559a0f4 ocaml: fix build with Bytes fallback
Place the Bytes fallback module in the right place (mlstdutils), with no
need to make it available directly also for generation, since it uses
mlstdutils now.

Fixes commit 61d4891ef4.
2017-07-12 09:52:46 +02:00
Pino Toscano
178a20f6c3 builder: fix paths to mlstdutils & mlutils
Followup/fix of commit 61d4891ef4.
2017-07-12 09:52:46 +02:00
Richard W.M. Jones
0a73032120 builder: Add Fedora 26 for aarch64, armv7l, i686, ppc64, ppc64le.
All the secondary architectures except s390x, which we cannot handle
because qemu doesn't emulate that architecture sufficiently yet.
2017-07-11 15:17:58 +01:00
Richard W.M. Jones
5aaa91b524 builder: templates: Use full path for nvram file.
Works around a virt-install or libvirt bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1469491
2017-07-11 13:02:55 +01:00
Richard W.M. Jones
904578e8b9 builder: templates: Use dl.fedoraproject.org for secondary architectures.
Avoid redirecting to incomplete mirrors.
2017-07-11 12:00:58 +01:00
Richard W.M. Jones
48874066b9 builder: Add fedora-26 (x86_64) image. 2017-07-11 10:29:57 +01:00
Richard W.M. Jones
09b14a6076 Version 1.37.17. v1.37.17 2017-07-10 22:06:45 +01:00
Richard W.M. Jones
538ef8107c common/mlutils: Add test .ml file to EXTRA_DIST.
Fixes commit 8be72935dc.
2017-07-10 22:04:41 +01:00
Richard W.M. Jones
f30b2065a2 gobject: Add outline guestfs-gobject(3) manual page.
Since we removed gtk doc, we might as well replace it with a
manual page explaining the basics of how to run gjs.
2017-07-10 17:03:19 +01:00
Richard W.M. Jones
f5f7f00a46 gobject: Remove gtk-doc (RHBZ#1465665).
The gobject bindings are adequately covered in the usual manual pages:
guestfs(3).  There is no need for separate generation of gtk-doc.
Also generating gtk documentation is the slowest part of the build,
and the tooling around gtk-doc is broken
(https://bugzilla.redhat.com/show_bug.cgi?id=1465665).

Note this removes the configure ‘--enable-gtk-doc’ option.  Using this
option now gives a warning, but is otherwise ignored:

  configure: WARNING: unrecognized options: --enable-gtk-doc
2017-07-10 17:03:19 +01:00
Richard W.M. Jones
2cb0413049 gobject: Use TEST_FUNCTIONS framework in tests. 2017-07-10 17:03:19 +01:00
Richard W.M. Jones
707f5bcfe0 daemon: Link guestfsd with libutils.
After the previous refactoring, we are able to link the daemon to
common/utils, and also remove some of the "duplicate" functions that
the daemon carried ("duplicate" in quotes because they were often not
exact duplicates).

Also this removes the duplicate reimplementation of (most) cleanup
functions in the daemon, since those are provided by libutils now.

It also allows us in future (but not in this commit) to move utility
functions from the daemon into libutils.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
e6c89f9631 utils: Rename ‘guestfs-internal-frontend.h’ to ‘guestfs-utils.h’.
The reason it's not just ‘utils.h’ is because Pino is worried that we
might pick up /usr/include/utils.h from a rogue library.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
8be72935dc mllib, v2v: Split out OCaml utils bindings ‘common/mlutils’.
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.

The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
5efebd8c7e utils: Split out structs cleanups and printing into common/structs.
These won't be used by the daemon, so interferes with us using
common/utils in the daemon, so they are moved to a different library.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
78587cc3c6 common/utils: cleanups: Add CLEANUP_CLOSE function.
This is present in the daemon, but the function could be used
throughout the code.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
b2469a6d96 common/utils: Refactor stdlib, gnulib and libxml2 cleanup functions.
This refactoring change just moves the cleanup functions around in the
common/utils directory.

libxml2 cleanups are moved to a separate object file, so that we can
still link to libutils even if the main program is not using libxml2
anywhere.  Similarly gnulib cleanups.

cleanup.c is renamed to cleanups.c.

A new header file cleanups.h is introduced which will replace
guestfs-internal-frontend-cleanups.h (fully replaced in a later commit).
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
7489d22843 common/utils: Move ‘uefi.c’ to ‘lib/’.
This was only used inside the library, so move it there.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
add53125a4 common/mlstdutils: Implement ‘Char.mem’, ‘String.span’ and ‘String.cspan’.
Char.mem tells you if a byte is a member of a string.

String.span and String.cspan are like the C functions strspn and
strcspn.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
b31a076717 common/mlstdutils: Implement complete set of byte swapping functions.
This implements all of:

 val int_of_le16 : string -> int64
 val le16_of_int : int64 -> string
 val int_of_be16 : string -> int64
 val be16_of_int : int64 -> string
 val int_of_le32 : string -> int64
 val le32_of_int : int64 -> string
 val int_of_be32 : string -> int64
 val be32_of_int : int64 -> string
 val int_of_le64 : string -> int64
 val le64_of_int : int64 -> string
 val int_of_be64 : string -> int64
 val be64_of_int : int64 -> string

and tests.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
61d4891ef4 mllib: Split ‘Common_utils’ into ‘Std_utils’ + ‘Common_utils’.
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib.  Therefore these functions
may be used by the generator.

The new module is moved to ‘common/mlstdutils’.

This also removes the "<stdlib>" hack, and the code which copied the
library around.

Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.

The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
989292c146 mllib: Move Xml (libxml2) OCaml bindings to common/mlxml.
The ‘Xml’ module is a self-contained library of bindings for libxml2,
with no other dependencies.

Move it to a separate ‘common/mlxml’ directory.

This is not pure refactoring.  For unclear reasons, the previous
version of ‘Xml.parse_file’ read the whole file into memory and then
called ‘xmlReadMemory’.  This was quite inefficient, and unnecessary
because we could use ‘xmlReadFile’ to read and parse the file
efficiently.  Changing the code to use ‘xmlReadFile’ also removes the
unnecessary dependency on ‘Common_utils.read_whole_file’.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
9254c8f152 mllib: Move Progress OCaml bindings to common/mlprogress.
The ‘Progress’ module is a self-contained library with the only
dependencies being:

 - the C ‘progress’ implementation

Move it to a separate ‘common/mlprogress’ directory.

This change is pure code refactoring.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
746a87cbf8 mllib: Move Visit OCaml bindings to common/mlvisit.
The ‘Visit’ module is a self-contained library with the only
dependencies being:

 - the C ‘visit’ implementation

 - the guestfs OCaml bindings

Move it to a separate ‘common/mlvisit’ directory.

This change is not entirely refactoring.  Two other fixes are made:

 - remove unsafe use of CLEANUP_FREE from a function which could
   raise an OCaml exception (cleanup handlers would not be called
   correctly if the exception is thrown)

 - don't link directly to common/visit/visit.c, but instead use
   the library (common/visit/libvisit.la)
2017-07-10 17:01:59 +01:00
Pino Toscano
28c375701d builder: templates: pass empty proxy to d-i when not set
When there is no proxy set, pass an empty string to the Debian
Installer.  This way, the installer will not ask for a proxy.
2017-07-10 17:56:13 +02:00
Pino Toscano
941545bde6 builder: templates: save NVRAM before removing guest
When installing guests that need NVRAM variables, the cleanup of the
guest with `virsh undefine` will remove that file too, which is not what
we want.  Instead, compress the NVRAM file right before the cleanup,
to ensure we have it.

Also, fix the filename for it, removing the double "-nvram" suffixes.
2017-07-10 17:56:13 +02:00
Pino Toscano
a67346b2fe builder: templates: improve debian.preseed
- set the mirror to deb.debian.org, which is the official redirector, so
  generated images will use the closest mirror depending on their
  location
- automake the task selection, using the choice we want
- do a full upgrade of the distro, so the template is already up-to-date
- install grub on the default device, which should also be the only one
  in the automated installation
2017-07-10 17:56:13 +02:00
Pino Toscano
eb830e89dc builder: templates: generalize debian location
Add an helper function to translate our architecture to the Debian
naming, and use it to get the location for all the architectures.
2017-07-10 17:56:13 +02:00
Pino Toscano
2ff6938935 builder: templates: update Debian preseed from Stretch
Update our debian.preseed file from a Stretch example file, leaving all
the customization done already.
2017-07-10 17:56:13 +02:00
Richard W.M. Jones
4cea914d24 builder: make-template: Combine both os_variant_of_os functions into one.
Cosmetic only change, no functional change.
2017-07-10 16:49:32 +01:00
Richard W.M. Jones
77d25587b8 v2v: docs: VDSM location of virt-v2v log file.
See this bug for background information:
https://bugzilla.redhat.com/show_bug.cgi?id=1350465

Thanks: Tomáš Golembiovský, Pino Toscano
2017-07-07 17:17:54 +01:00
Richard W.M. Jones
d96209ef07 lib: libvirt: Pass copyonread flag through to the libvirt XML (RHBZ#1466563).
We were dropping the add_drive copyonread flag when using the libvirt
backend.  This resulted in significant performance degradation (2x-3x
slower) when running virt-v2v against VMware servers.

Thanks: Kun Wei.
2017-07-01 11:40:12 +01:00