Commit Graph

10602 Commits

Author SHA1 Message Date
Richard W.M. Jones
a93a06dc59 daemon: Rewrite inspection code using ‘return’ statements.
When rewriting this, I revisited the original C code and changed the
OCaml code so it's now reasonably similar to the original.
2017-12-08 16:22:14 +00:00
Richard W.M. Jones
506e90cf35 common/mlstdutils: Add ‘return’ statement for OCaml.
This adds a ‘return’ statement as found in other programming
languages.  You can use it like this:

  with_return (fun {return} ->
    some code ...
  )

where ‘some code’ may either return implicitly (as usual), or may call
‘return x’ to immediately return ‘x’.  All returned values must have
the same type.

The OCaml >= 4.04 implementation is by Petter A. Urkedal and octachron.
See this thread:
https://sympa.inria.fr/sympa/arc/caml-list/2017-11/msg00017.html

The version that works for any OCaml is by me.  (Note that my version
cannot be nested).
2017-12-08 16:22:14 +00:00
Richard W.M. Jones
0eb23230fa common/mlstdutils: Move list functions into extended List module.
However some existing functions had names which shadowed existing
functions in the List module, so I had to rename them:

  assoc -> List.assoc_lbl
  append -> List.push_back_list
  prepend -> List.push_front_list

This is an extension of the previous commit.
2017-12-08 16:22:11 +00:00
Richard W.M. Jones
b585fea976 common/mlstdutils: Extend the List module.
We defined a number of functions on lists which are not provided by
the standard library.  As with Char and String, let's extend List to
add these new functions to a List pseudo-module (really
Std_utils.List, but called List when you ‘open Std_utils’).

The initial exported functions are all List functions from OCaml 3.11
+ iteri + mapi.  We can add other functions as needed.
2017-12-08 16:21:10 +00:00
Richard W.M. Jones
4699c7b6e1 v2v: -o null: Use the qemu null device driver.
Instead of writing to a temporary file and deleting it, use the null
block driver (the "null-co://" URI) in qemu to throw it away.
2017-12-08 16:21:06 +00:00
Richard W.M. Jones
e29296cfa2 v2v: Abstract the target file so it could be a QEMU URI.
This complicated bit of refactoring abstracts the target file so it is
allowed to be either a filename or a QEMU URI.
2017-12-08 16:20:59 +00:00
Richard W.M. Jones
c6b2c0a008 v2v: vddk: Fix printing of nbdkit command.
Fixes commit 99a29125f6.
2017-12-08 11:48:43 +00:00
Pino Toscano
b34f457cf2 builder: use the template arch when caching all templates
When caching all the templates, use the architecture of each template,
instead of the architecture passed as --arch (or the host architecture,
as default).  This way, the right destination filename will be used.

Fixes commit b1cf6246f3.

Thanks to: Erik Skultety.
2017-12-07 18:16:17 +01:00
Richard W.M. Jones
99a29125f6 v2v: vddk: Print nbdkit command before running it.
Very useful for debugging.
2017-12-07 16:22:09 +00:00
Richard W.M. Jones
2b185d1e90 v2v: cmdline: Reformat array of arguments.
No change, just code formatting.
2017-12-07 16:22:09 +00:00
Richard W.M. Jones
eecf41956d v2v: Remove unused ‘scheme’ parameter.
It wasn't actually used for anything significant.  Even worse
virt-v2v-copy-to-local made up a scheme out of thin air.

Just code refactoring, no functional change.
2017-12-07 10:53:31 +00:00
Richard W.M. Jones
4b43697f9b v2v: docs: State that --vddk-thumbprint is only required if --vddk present (RHBZ#1513884).
Thanks: Ming Xie, Pino Toscano.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1513884#c2
2017-12-06 15:56:47 +00:00
Richard W.M. Jones
5ac24b490e v2v: docs: Remove min version of VDDK, recommend nbdkit >= 1.1.25 (RHBZ#1513884).
Requires -> recommends because older versions of nbdkit will likely
work also.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1513884#c3
2017-12-06 15:55:57 +00:00
Pino Toscano
614f680dc6 lib: libvirt: stop using <shareable/> for appliance disk (RHBZ#1518517)
Commit aa9e0057b1 made the libvirt backend
use <shareable/> for the disk of the appliance, since at that time all
the instances were using the disk directly.
OTOH, commit 3ad44c8660 switched to
overlays for read-only disks, including the appliance, so effectively
protecting the appliance.

Because of this, the libvirt backend does not need <shareable/> anymore.

Thanks to: Daniel Berrange, Richard W.M. Jones,  Peter Krempa.
2017-12-05 18:14:21 +01:00
Pino Toscano
1b2020ba04 v2v: bootloaders: handle no default grubby kernel (RHBZ#1519204)
When using grubby to get the default kernel of a guest, do not fail
with a bogus error like:

virt-v2v: error: libguestfs error: statns: statns_stub: path must start
with a / character

in case there is no default kernel that can be determined (e.g. because
of a bogus configuration).
2017-12-04 10:15:29 +01:00
Richard W.M. Jones
af4e3b3d3b bugs-in-changelog.sh: Python bugzilla client -t option changed to -s. 2017-11-25 21:07:29 +00:00
Richard W.M. Jones
5eabcdb9fd v2v: Add generator_built and BUILT_SOURCES listing generated files.
Fixes commit 0c396650b0.
2017-11-23 18:48:44 +00:00
Pino Toscano
06df910491 docs: add virt-builder-repository.pod to podfiles
Updates commit a442d2c321.
2017-11-23 14:49:44 +01:00
Lars Seipel
35320dd0ed launch: direct: Omit locking option for non-file disks (RHBZ#1516094)
QEMU does not accept options unrecognized by the block driver
in use. Disable locking only for read-only disks that are
file-backed, as that's the only block driver it is supported
with.

Signed-off-by: Lars Seipel <ls@slrz.net>
2017-11-23 08:42:46 +00:00
Yuri Chornoivan
1563ba5827 Fix minor typos 2017-11-22 20:33:23 +00:00
Richard W.M. Jones
67b0de7399 builder: Add missing builder/repository_main.mli file.
Fixes commit a442d2c321.
2017-11-21 23:11:34 +00:00
Cédric Bosdonnat
a442d2c321 New tool: virt-builder-repository
virt-builder-repository allows users to easily create or update
a virt-builder source repository out of disk images. The tool can
be run in either interactive or automated mode.
2017-11-21 16:36:32 +00:00
Cédric Bosdonnat
efaf570aaa builder: add a template parameter to get_index
get_index now gets a new template parameter. Setting it to true will
make the index parsing less picky about missing important data. This
can be used to parse a partial index file.
2017-11-21 16:16:19 +00:00
Cédric Bosdonnat
448a61fbd5 builder: change arch type to distinguish guesses
Change Index.arch to the type (Arch of string | GuessedArch of string).

In a future commit, the index parser will allow arch not to be set
for some cases. Thus arch value will be guessed by inspecting the
image. However we need to distinguish between a set value and a guessed
one. Using this new type will help it:

    match arch with
    | Arch s        -> (* This is a set value *)
    | GuessedArch s -> (* This is a guessed value *)
2017-11-21 16:16:19 +00:00
Martin Kletzander
946d5bf8d4 build: Reference RPC_CFLAGS where needed
Since glibc-2.26 removed the /usr/include/rpc/rpc.h, it caused problem even
though probing for libtirpc has taken place and worked properly.  The problem is
that RPC_CFLAGS (usually `-I/usr/include/tirpc` on Linux) was not used in places
where it was needed.  So the build failed with messages like the following:

In file included from guestfs_protocol.c:6:0:
guestfs_protocol.h:9:10: fatal error: rpc/rpc.h: No such file or directory
 #include <rpc/rpc.h>
          ^~~~~~~~~~~

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-11-19 19:59:01 +00:00
Richard W.M. Jones
31acbfef2f common/mlpcre: Compile bindings even with --disable-ocaml (RHBZ#1514756).
Thanks: Jean-Christophe Manciot
2017-11-18 12:33:05 +00:00
Richard W.M. Jones
50ca24b634 Version 1.37.34. v1.37.34 2017-11-16 23:05:17 +00:00
Richard W.M. Jones
6407f23e3d tests: Make filtering code in rhbz557655.sh test more liberal.
Because of vagaries in how the debug messages get printed they can get
split over lines:

  gulibguestfs: trace: truncate_size = 0
  estfsd: => truncate_size (0xc8) took 0.00 secs

and the code which filtered out those messages before comparison was
failing.  This makes the code more liberal on how it matches and
filters out these messages.
2017-11-16 22:43:53 +00:00
Richard W.M. Jones
068078ba95 Version 1.37.33. v1.37.33 2017-11-16 14:03:23 +00:00
Richard W.M. Jones
f42a6f056f po: Add osinfo_config.ml to POTFILES-ml.
Updates commit 02184f55f9.
2017-11-16 14:03:03 +00:00
Richard W.M. Jones
d156072c6b Update PO files from Zanata.
Adds po/cs.po and removes po/zh_TW.po.
2017-11-16 14:02:10 +00:00
Richard W.M. Jones
9dd81036ce builder: Add osinfo_config.mli file.
Every .ml file now needs a corresponding .mli file so that
dependencies are generated correctly.  The check-mli.sh script picks
up the problem:

  FAIL: check-mli.sh
  ==================
  ./builder/osinfo_config.ml: missing ./builder/osinfo_config.mli
  FAIL check-mli.sh (exit status: 1)

Fixes commit 46abca0ec3
and commit 02184f55f9.
2017-11-16 13:22:02 +00:00
Richard W.M. Jones
f2fe422909 lib: command: If command fails, exit with 126 or 127 as defined by POSIX.
If running the external command fails in "argv mode" (ie. when
not using the shell), then exit with either 126 or 127 as defined
by POSIX.

This is mostly the same as what bash does, see
execute_cmd.c:shell_execve in the bash sources.

Note: saving errno around perror(3) if necessary, otherwise you will
see different behaviour between verbose and non-verbose mode.  In
non-verbose mode, perror(3) tried to print to a closed file
descriptor, failing and overwriting errno.  That took some time to
debug.
2017-11-16 12:55:19 +00:00
Richard W.M. Jones
3ef6794b09 common/mlstdutils: Add with_openfile function.
This safe wrapper around Unix.openfile ensures that exceptions
escaping cannot leave unclosed files.

There are only a few places in the code where this wrapper can be used
currently.  There are other occurences of Unix.openfile but they are
not suitable for replacement.
2017-11-16 12:32:58 +00:00
Richard W.M. Jones
d17a70324c v2v: vddk: Atomicly chmod the password file.
Ensure we are chmodding the correct file, avoids possible races.

Thanks: Pino Toscano.
2017-11-16 12:32:58 +00:00
Richard W.M. Jones
2d1e2d12af common/mlstdutils: Add with_open_in and with_open_out functions.
These safe wrappers around Pervasives.open_in and Pervasives.open_out
ensure that exceptions escaping cannot leave unclosed files.
2017-11-16 12:32:58 +00:00
Richard W.M. Jones
6c24213d78 Version 1.37.32. v1.37.32 2017-11-16 10:36:19 +00:00
Richard W.M. Jones
46abca0ec3 builder: Add osinfo to virt-builder and hence to EXTRA_DIST.
Fixes commit 02184f55f9.
2017-11-16 10:33:42 +00:00
Richard W.M. Jones
dca92e4cf1 daemon: Use a configure-time test to find the best OCaml runtime.
In OCaml 4.06 we need to link to daemon to libasmrun_pic.a (the OCaml
runtime linked with -fPIC).  Otherwise you will see many errors like
this:

    /usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup_aux.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup.o): relocation R_X86_64_32S against symbol `caml_data_segments' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(fail.o): relocation R_X86_64_32 against symbol `caml_exn_Failure' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(roots.o): relocation R_X86_64_32 against symbol `caml_frametable' can not be used when making a shared object; recompile with -fPIC

This commit use a configure-time test to find the best OCaml runtime.
2017-11-14 19:54:43 +00:00
Richard W.M. Jones
994ca8d873 builder: Add Fedora 27 templates for aarch64, armv7l, i686, ppc64, ppc64le. 2017-11-14 19:54:43 +00:00
Richard W.M. Jones
52f8e2b830 builder: Fedora 27 (x86_64) image.
Note this uses GPT, see commit a06e50e263.
2017-11-14 14:58:53 +00:00
Richard W.M. Jones
a52b7bdccc builder: template: Save the final virt-install command to a file.
For documentation purposes only.
2017-11-14 14:08:34 +00:00
Richard W.M. Jones
143d61745b builder: templates: Rebuild Fedora 26 ppc64 & ppc64le templates with GPT. 2017-11-14 14:08:34 +00:00
Richard W.M. Jones
a06e50e263 builder: templates: For RHEL, CentOS >= 7 and all Fedora, default to GPT.
The immediate issue is with Fedora/ppc64 and /ppc64le which currently
use extended partitions, breaking the virt-builder ‘--size’ parameter,
eg:

  $ virt-builder --arch ppc64le fedora-26 --size 20G
  ...
  [  21.6] Resizing (using virt-resize) to expand the disk to 20.0G
  virt-resize: error: /dev/sda5: partition not found in the source disk image
  (this error came from '--expand' option on the command line).  Try running
  this command: virt-filesystems --partitions --long -a /var/tmp/vbf67b8c.img

However more generally MBR is broken and should die.  GPT is supported
by all modern virtual bootloaders, so just default to it.

Notes:

* This is different from mandating a UEFI bootloader.

* I am not planning to rebuild any existing images except the
  F26 ppc64 & ppc64le ones.
2017-11-14 14:07:56 +00:00
Richard W.M. Jones
a88385add6 fuse: Don't use errno if libfuse fuse_mount fails.
libfuse prints errors on stderr and there seems to be no way to change
that.  It doesn't make any attempt to preserve errno either, so
printing an error based on errno is wrong.

Thanks: Assaf Gordon.
2017-11-14 09:41:05 +00:00
Cédric Bosdonnat
7e3689bfe0 builder: add Index.write_entry function
Add a function to properly write virt-builder source index entries.
Note that this function is very similar to Index.print_entry that is
meant for debugging purposes.
2017-11-07 12:51:28 +00:00
Cédric Bosdonnat
0933d2d818 builder: add Utils.get_image_infos function
This helper function calls qemu-img info on an image file and
returns the output as a JSON Yajl tree.

This function will be used in future commits.
2017-11-07 12:41:31 +00:00
Pino Toscano
02184f55f9 builder: add simple OCaml osinfo-db reader
Add a simple OCaml-based implementation of reader of the osinfo-db:
the only interface is an iterator that invokes an user-supplied
function with each XML file found.

This implementation behaves like the current C implementation, and
still supports the old libosinfo db.

[RWMJ: Fixed trailing whitespace]
2017-11-07 12:24:42 +00:00
Cédric Bosdonnat
8a661b1c48 mllib: add XPath helper xpath_get_nodes
This function will allow more OCaml-ish processing of XPath queries
with multiple results.
2017-11-07 12:16:21 +00:00
Cédric Bosdonnat
628141f302 Ignore builder/*.out and *.img files
These ignores are covering test-console-ubuntu-12.04 test data.
2017-11-07 12:01:31 +00:00