Commit Graph

10958 Commits

Author SHA1 Message Date
Pino Toscano
4ff573c352 ocaml: make sure to pass LDFLAGS to ocamlmklibs linker (RHBZ#1624130)
Pass the LDFLAGS properly as arguments for the C linker when using
ocamlmklibs via the -ldopt option.

Followup of commit 34c23403c5.
2018-09-27 15:51:01 +02:00
Richard W.M. Jones
b54b58c44e v2v: -o rhv-upload: Add a test.
Previously this output method was almost completely untested.

This commit adds a fake ovirtsdk4 module so we can test the
-o rhv-upload method fairly completely without needing an actual
oVirt instance around.

Thanks: Pino Toscano.
2018-09-26 19:00:40 +01:00
Richard W.M. Jones
6694028f98 v2v: -o rhv-upload: Only set SSL context for https connections.
For real imageio servers the destination will always be https.  This
change has no effect there.

However when testing we want to use an http server for simplicity.  As
there is no certificate or cafile in this case the call to create the
context will fail.

This also simplifies creation of the context object and recognizes the
"insecure" flag for connecting to imageio.

Thanks: Nir Soffer.
2018-09-26 19:00:40 +01:00
Richard W.M. Jones
edd4980cef v2v: -o rhv-upload: Print commands before running them in verbose mode. 2018-09-26 19:00:40 +01:00
Richard W.M. Jones
e2bafffce2 Version 1.39.11. v1.39.11 2018-09-21 16:15:37 +01:00
Richard W.M. Jones
da48f4e5ff sparsify: Fix test-virt-sparsify-in-place-fstrim-unsupported.sh
Since Linux commit f663b5b38fff trimming vfat is now supported by
Linux.  This broke the test which assumed it was not supported.  Use
another filesystem (minix) which does not support trimming instead.

Thanks: Daniel P. Berrangé and Pino Toscano.
2018-09-21 13:32:58 +01:00
Richard W.M. Jones
165add060e Version 1.39.10. v1.39.10 2018-09-21 11:57:08 +01:00
Richard W.M. Jones
6f605a0804 lib: direct: Query qemu binary for availability of KVM (RHBZ#1605071).
When using the direct backend, you should see the result of testing
the qemu binary for the availability of KVM:

  libguestfs: qemu KVM: enabled

Thanks: Andrea Bolognani.
2018-09-21 10:56:25 +01:00
Richard W.M. Jones
34c23403c5 tools: Ensure CFLAGS and LDFLAGS are passed to all OCaml binaries (RHBZ#1624130).
After this commit, all annocheck errors are fixed except for:

  Hardened: virt-get-kernel: MAYB: Gaps were detected in the annobin coverage.  Run with -v to list.

After discussion with the annocheck maintainers this gap in coverage
(which corresponds to the OCaml runtime) seems to be caused either by
the runtime not being linked with the right flags, or might be a bug
in annocheck itself.  In any case it's not something that can be
resolved within the scope of libguestfs.
2018-09-21 10:16:34 +01:00
Richard W.M. Jones
09abb9c990 tools: Link OCaml programs with -runtime-variant _pic if available.
OCaml has a small runtime which is statically linked into the virt
tools (providing things like GC and primitives).  Since OCaml 4.03 it
has been possible to select variants of this runtime, one of which is
compiled with -fPIC, using ‘ocamlopt -runtime-variant _pic’.

This has performance implications on i686, but is relatively free on
other architectures.  Since it (in theory) adds to the security of the
final binary this commit enables it whenever it is available.
2018-09-21 10:16:34 +01:00
Pino Toscano
4b1e5b0c3f Introduce a --key option in tools that accept keys
The majority of the tools have already options (--echo-keys &
--keys-from-stdin) to deal with LUKS credentials, although there is no
way to automatically provide credentials.  --keys-from-stdin is
suboptimal, because it is a usable solution only when there is just one
device to open, and no other input passed via stdin to the tool (like
the commands for guestfish).

To overcome this limitation, introduce a new --key option in tools:
* --key /dev/device:file:/filename/with/key
* --key /dev/device:string:the-actual-key
this way it is possible to pass all the credentials needed for the
specific devices to open, with no risk of conflict with stdin, and also
in a secure way (when using the "file" way).

On the technical side: this adds a new "key_store" API for the C tools,
making sure it is used only when needed.  Partially mirror it also for
the OCaml tools, although there will be a conversion to the C API
because the decryption helpers used are in the common C parts.
2018-09-21 10:30:05 +02:00
Pino Toscano
ec438b7b85 mltools: create a cmdline_options struct
Instead of returning directly a Getopt.t handle, now
Tools_utils.create_standard_options returns a struct, which at the
moment contains only the Getopt.t handle.  This way, it will be easy to
add more data needed for handling standard command line options.

This is mostly refactoring, with no functional changes.
2018-09-20 17:52:53 +02:00
Richard W.M. Jones
51a9c874d3 v2v: Add a slow test of Windows conversions.
As with the previous commit this requires that Windows guests have
been created first using the procedure from:

https://rwmj.wordpress.com/2018/09/13/creating-windows-templates-for-virt-builder/

For me:

PASS: test-v2v-conversion-of-windows-6.3-server.sh
PASS: test-v2v-conversion-of-windows-6.2-server.sh
PASS: test-v2v-conversion-of-windows-10.0-server.sh

If you don't have these templates in virt-builder then the tests will
skip.
2018-09-14 14:59:35 +01:00
Richard W.M. Jones
4b6341072c customize: tests: Add slow test of Windows firstboot.
This requires that Windows guests have been created using the
procedure outlined here:

https://rwmj.wordpress.com/2018/09/13/creating-windows-templates-for-virt-builder/

For me:

PASS: test-firstboot-windows-6.2-server.sh
PASS: test-firstboot-windows-6.3-server.sh
PASS: test-firstboot-windows-10.0-server.sh

An incidental change is that we dump the firstboot log from the guest
(even on success).  If the firstboot fails this is very useful for
determining the real cause.
2018-09-14 14:39:46 +01:00
Richard W.M. Jones
d54be028a2 customize: Use set -x for test-firstboot.sh slow test.
Aids in debugging failed tests.
2018-09-14 13:49:54 +01:00
Richard W.M. Jones
bbefbc8a48 v2v: -it vddk: Fix check for incorrect LD_LIBRARY_PATH.
In nbdkit >= 1.7.2, nbdkit vddk --dump-plugin will print the shared
library name in normal output, which breaks this test.

The actual error when LD_LIBRARY_PATH is not set includes this line:

  nbdkit: error: libvixDiskLib.so.6: cannot open shared object file: No such file or directory

so search instead for the error message "cannot open shared object file".
2018-09-14 12:01:53 +01:00
Richard W.M. Jones
089f0345f4 builder: templates: Add Windows template support.
Provides support for building:

 - Windows 7
 - Windows Server 2008 R2
 - Windows Server 2012
 - Windows Server 2012 R2
 - Windows Server 2016

Note that these images cannot be released to the public because of
obvious licensing issues.  But this documents how we build them for
internal consumption so that others can also build them.

Thanks: Christophe Fergeau, Tomáš Golembiovský.
2018-09-13 09:51:58 +01:00
Richard W.M. Jones
755834c797 builder: templates: Use virt-sparsify --in-place (instead of copying).
virt-sparsify in copying mode takes a huge amount of temporary space
and takes a very long time.  In any case it's not really necessary for
modern guests since in-place sparsification is fully supported now.
2018-09-13 09:17:21 +01:00
Nikolay Ivanets
eaa9c8cbc6 daemon: consider /etc/mdadm/mdadm.conf while inspecting mountpoints.
Inspection code checks /etc/mdadm.conf to map MD device paths listed in
mdadm.conf to MD device paths in the guestfs appliance. However on some
operating systems (e.g. Ubuntu) mdadm.conf has alternative location:
/etc/mdadm/mdadm.conf.

This patch consider an alternative location of mdadm.conf as well.
2018-09-13 09:17:21 +01:00
Richard W.M. Jones
e6895bd9d2 builder: templates: Specify --arch explicitly, even on x86_64. 2018-09-11 16:52:10 +01:00
Richard W.M. Jones
cc01166a24 builder: templates: Make UEFI code arch-independent.
Previously it only worked for aarch64.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
613c1782ea builder: templates: Refactor code for using UEFI.
Simple refactoring.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
00fff03485 builder: templates: Refactor code for adding --initrd-inject.
Initrd is a Linux thing, so this code should only be used for
Linux-based OSes.  Refactor to make it a bit cleaner.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
d0ff42dfa2 builder: templates: Add a can_sysprep_os function.
Does virt-sysprep know how to sysprep this disk image?  Add a function
to find out.

Just refactoring.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
112880e61b builder: templates: Rename make_kickstart function.
This function will soon be used to generate Windows unattended.xml
files (as well as Debian preseed) so just rename it back to
‘make_kickstart’ rather than attempting to explain in the name every
way it could be used.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
e9bd010f6a builder: templates: Add generic method for deleting files on exit.
Simple refactoring.
2018-09-11 16:52:10 +01:00
Richard W.M. Jones
2c349a00d2 Revert "build: Maintain a central list of skipped tests."
This reverts commit 79a3ac5d0e.
2018-09-04 11:25:56 +01:00
Richard W.M. Jones
46bb1428ec Revert "tests: Disable 255 disk test until we can find a way to make it reliable."
This reverts commit 9b911a2a85.
2018-09-04 11:25:56 +01:00
Richard W.M. Jones
969cacead9 v2v: Add -o openstack target, writes to OpenStack & Cinder using APIs. 2018-09-03 18:29:33 +01:00
Richard W.M. Jones
e4f4ed6188 mltools: Add with_timeout utility function.
Higher-order function which performs an operation repeatedly until it
succeeds or fails, or a timeout is reached.
2018-09-03 18:29:33 +01:00
Richard W.M. Jones
16c9dad598 v2v: -o glance: Refactor OpenStack image properties into a new module.
Straightforward refactoring.  Apart from small modifications to clean
up the code and reorder the properties more logically there should be
no functional change.
2018-09-03 18:29:33 +01:00
Richard W.M. Jones
e12c56176a v2v: windows: Fix rhev-apt command line (RHBZ#1624902).
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1584678#c15

Fixes commit bcdbe6405c.  However this
bug was copied directly from old virt-v2v which did the same thing
(from lib/Sys/VirtConvert/Converter/Windows.pm):

  echo installing rhev-apt >>log.txt
  "rhev-apt.exe" /S /v /qn >>log.txt

Thanks: Lev Veyde
2018-09-03 18:29:33 +01:00
Richard W.M. Jones
741ef228cd v2v: docs: Describe support for SHA-2 certs for Windows 7 / 2008 R2 (RHBZ#1624878).
Thanks: Yan Vugenfirer.
2018-09-03 18:29:33 +01:00
Richard W.M. Jones
56f1e6cecd java: Link with gnulib to resolve missing hash_free symbol. 2018-08-28 22:24:46 +01:00
Richard W.M. Jones
9b911a2a85 tests: Disable 255 disk test until we can find a way to make it reliable. 2018-08-28 22:24:46 +01:00
Richard W.M. Jones
79a3ac5d0e build: Maintain a central list of skipped tests.
I was maintaining this list unofficially.  Let's maintain it in the
sources so it is versioned and branched along with the libguestfs
code.
2018-08-28 22:24:46 +01:00
Richard W.M. Jones
5b87b9f132 ppc64le: Remove bogus __powerpc64le__.
Not a thing - tested on P9.
2018-08-28 22:24:46 +01:00
Richard W.M. Jones
56318f0b5f ppc64le: Don't use -cpu parameter under any circumstances (RHBZ#1605071). 2018-08-28 22:24:46 +01:00
Richard W.M. Jones
598e879ad1 Version 1.39.9. v1.39.9 2018-08-28 22:17:02 +01:00
Richard W.M. Jones
af5b117ade mltools: Add test-machine-readable-script.sh to EXTRA_DIST.
Fixes commit afa8111b75.
2018-08-28 22:15:30 +01:00
Nir Soffer
8f250c00c8 v2v: rhv-upload-plugin: Use BrokenPipeError
With python 3, we have a nicer way to handle socket.error with errno set
to EPIPE (or ESHUTDOWN).

This is also more correct since in some cases (that I could not
reproduce yet with v2v), using e[0] with BrokenPipeError will fail with:

>>> OSError(errno.EPIPE, "Broken pipe")[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'BrokenPipeError' object is not subscriptable

For python 2 e[0] seems to work, but is leftover from historic python
version that used to raise a tuple instead of socket.error instance.
In python 2.7 library code e.args[0] is used. If we ever port this to
python 2 this is the best form.
2018-08-28 21:56:11 +01:00
Richard W.M. Jones
0d0b551130 v2v: Document how to adjust ESXi NFC memory limit (RHBZ#1614276). 2018-08-24 14:25:27 +01:00
Richard W.M. Jones
9f678c3bde v2v: Add --print-estimate option to print copy size estimate.
This option prints the estimated size of the data that will be copied
from the source disk.

Currently this overestimates by the size of the qcow2 header, but for
real disk images that doesn't matter much.

For example:

$ virt-builder fedora-27
$ virt-v2v -i disk fedora-27.img -o null --print-estimate
[...]
virt-v2v: This guest has virtio drivers installed.
[  44.0] Mapping filesystem data to avoid copying unused and blank areas
[  44.5] Closing the overlay
disk 1: 1047920640
total: 1047920640
2018-08-24 14:13:01 +01:00
Pino Toscano
10f9779c08 v2v: test-harness: use Xml.parse_file
Use Xml.parse_file instead of read_whole_file + Xml.parse_memory.
2018-08-24 13:11:09 +02:00
Pino Toscano
d7772fb78b builder: index_parser_tests: really use parameter
Really use the parameter of the "read_file" function, instead of
hardcoding "out".  This does not change the behaviour, since all the
callers already use "out" as the file name to read.

Fixes commit 7e3689bfe0.
2018-08-24 13:06:36 +02:00
Pino Toscano
ec345aca02 builder: use the new json_parser_tree_parse_file
No need to read the whole file manually.
2018-08-23 19:12:46 +02:00
Pino Toscano
bb00c482e8 mltools: JSON: add json_parser_tree_parse_file
Easy way to parse JSON from a file, without reading it all to string
first.
2018-08-23 19:06:22 +02:00
Sebastian Meyer
fd43730e2b Fix error with --uninstall option on SUSE
The `-l` option for some `zypper` subcommands is short for
`--auto-agree-with-licenses` and not available on the `zypper rm` command.
2018-08-23 18:05:04 +02:00
Pino Toscano
afa8111b75 OCaml tools: add output selection for --machine-readable
Add an optional argument for --machine-readable to select the output,
adding a new function to specifically write data to that output stream.
The possible choices are:
* --machine-readable: to stdout, like before
* --machine-readable=file:name-of-file: to the specified file
* --machine-readable=stream:stdout: explicitly to stdout
* --machine-readable=stream:stderr: explicitly to stderr

Adapt all the OCaml-based tools to use the new function, so the
--machine-readable choice is respected.
2018-08-23 18:03:39 +02:00
Pino Toscano
8863ccbd64 common/mltools: getopt: add Getopt.OptString
Introduce a new type of option with an optional string argument.
2018-08-23 16:04:27 +02:00