Commit Graph

10634 Commits

Author SHA1 Message Date
Richard W.M. Jones
c3dd3f51a6 lua, perl: Use thread-safe strerror_r instead of strerror (RHBZ#1536763). 2018-01-22 17:52:46 +00:00
Richard W.M. Jones
e04aa823d1 perl: Free array on error path to avoid leak (RHBZ#1536765). 2018-01-22 17:52:46 +00:00
Richard W.M. Jones
a30b51747f launch: direct: Use old-style file= and format= parameters when not disabling locking (RHBZ#1503497).
Ancient qemu 1.5 (in RHEL 7) does not understand the
file.file.filename= and file.driver= parameters.  Go back to using the
old-style file= and format= parameters when we're not trying to set
the file.backing.file.locking=off parameter.

Fixes commit 9fe592808c.

Thanks: Yongkui Guo, Václav Kadlčík.
2018-01-19 16:01:01 +00:00
Richard W.M. Jones
7e43ef4171 common/mlxml: Remove -cclib from ocamlmklib.
ocamlmklib will get confused if the -cclib parameter expands to
anything except a single -lxml2 argument.  In any case it's not
necessary to specify the linker command here since every use of this
library links in a program to libxml2.
2018-01-16 13:03:39 +00:00
Richard W.M. Jones
49263be47a configure: Don't define _FORTIFY_SOURCE.
We routinely test the upstream code by running everything under
valgrind, and in any case _FORTIFY_SOURCE is usually defined by
downstream Linux distros and we can leave the optimization vs safety
decision to them.

See this bug: https://bugs.gentoo.org/640494
2018-01-16 13:03:39 +00:00
Cédric Bosdonnat
7c986228d5 resize: copy GPT partition flags
In some cases, the first stage bootloader needs the 'Legacy BIOS
bootable' flag to be set on the partition. This change copies all
flags (including this one) for each partition of the old disk to the
new one to avoid ending up with non-bootable disks.
2018-01-16 13:03:39 +00:00
Cédric Bosdonnat
c5fdc4f764 New APIs: part_set_gpt_attributes and part_get_gpt_attributes
Allow reading and setting the GPT partition attribute flags.
2018-01-16 13:03:39 +00:00
Cédric Bosdonnat
5f26f70262 daemon: make sgdisk_info_extract_uuid_field more generic
Rename the sgdisk_info_extract_uuid_field to
sgdisk_info_extract_field in order to reuse it for other field types.
Just like its C ancestor, it now needs an extractor function to be
passed as parameter.
2018-01-16 12:51:46 +00:00
Mykola Ivanets
079681058e tests: md: test guestfish finds md and LV devices in different combinations
Test guestfish finds:
1. md device created from physical block device and LV,
2. md device created from LVs
3. LV created on md device

raid0 is used for md device because it is inoperable if one of its components is inaccessible so it is easy observable that md device is missing (raid1 in this case will be operable but in degraded state).
2018-01-16 09:49:21 +00:00
Mykola Ivanets
c71e9e9acd tests: md: test guestfish finds logical volume on md device
Test guestfish finds logical volume (LV) created on md device
2018-01-16 09:49:21 +00:00
Mykola Ivanets
b00379fbd6 appliance: init: Avoid running degraded md devices
The issue:
- raid1 will be in degraded state if one of its components is logical volume (LV)
- raid0 will be inoperable at all (inacessible from within appliance) if one of its component is LV
- raidN: you can expect the same issue for any raid level depends on how many components are inaccessible at the time mdadm is running and raid redundency.

It happens because mdadm is launched prior to lvm AND it is instructed to run found arrays immediately (--run flag) regardless of completeness of their components.
Later (when lvm activates found LVs) md signature on LV might be recognized BUT newly found raid components could't be inserted into already running (in degraded state)
or marked as inoperable raid arrays.

The patch fixes the issue in the following way:

1. Found arrays won't be run immediately unless ALL expected drives (components) are present. Here '--no-degraded' flag comes into a play. See mdadm(8).
2. Second mdadm call (after LVM is scanned) will scan UNUSED yet devices and make an attempt to run all found arrays (even they will be in degraded state).

There is no performance penalty because second pass scans UNUSED yet devices. Here is 'boot-benchmark' before and after patch:

             : libvirt backend : direct backend
------------------------------------------------
master       : 835.2ms ±1.1ms  : 670.4ms ±0.3ms
master+patch : 837.7ms ±2.4ms  : 671.8ms ±0.2ms
2018-01-16 09:49:21 +00:00
Mykola Ivanets
9bdd2cacfd cat: virt-tail: Fix spontaneously failing test
According to 'guestfs_sync' API method documentation: "You should always call this if you have modified a disk image, before closing the handle."
So, 'guestfish --remote sync' is required because changes made on the disk (guestfish --remote rm /tail) should be visible to 'virt-tail' which works in different process and also accessing the same disk.
A you can see, other changes done via 'guestfish --remote' in this test are flushed via a 'sync' command.

The test is failing spontaneously beasue without 'sync' it very depends on outside factors like qemu caching policy, underlying host filesystem etc.
2018-01-16 08:37:16 +00:00
Richard W.M. Jones
4ac3be3d04 daemon: tests: Compile tests with compiler warnings.
And fix the problems.
2018-01-09 12:17:53 +00:00
Richard W.M. Jones
04031269f6 tests: Further instances of $(RPC_CFLAGS). 2018-01-09 12:17:53 +00:00
Richard W.M. Jones
df7f6b9c9a common/protocol: Fix indentation of $(RPC_CFLAGS).
Fixes commit 946d5bf8d4.
2018-01-09 12:17:53 +00:00
Richard W.M. Jones
0a9a3d62d9 diff: Set different identifiers on the two handles.
This allows us to easily see from trace output (‘virt-diff -x’) which
handle is being used for each call.  Otherwise you get meaningless
output such as:

libguestfs: trace: download "/Windows/WindowsUpdate.log" "/tmp/virtdiffy9m5YW/a"
libguestfs: trace: download = 0
libguestfs: trace: download "/Windows/WindowsUpdate.log" "/tmp/virtdiffy9m5YW/b"
libguestfs: trace: download = 0

Thanks: Xiang Hua Chen
2018-01-09 11:12:19 +00:00
Richard W.M. Jones
ab0a3e0276 resize: Clarify example in manual page.
Someone following this example started by literally copying "olddisk"
to "newdisk", which doesn't work (the target must always be blank).
Clarify the wording.
2018-01-05 15:01:20 +00:00
Richard W.M. Jones
212762c593 Update copyright dates for 2018.
Run the following command over the source:

  perl -pi.bak -e 's/(20[01][0-9])-2017/$1-2018/g' `git ls-files`
2018-01-04 15:30:10 +00:00
Richard W.M. Jones
dab065a8ee v2v: docs: Match the two instances of /path/to/nbdkit in the documentation.
Fixes commit d81a2ee185.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1513884#c5
2017-12-18 10:30:25 +00:00
Richard W.M. Jones
d3afdb6e3e df: Handle block sizes smaller than 1024 bytes (RHBZ#1525241).
Thanks: Nikolay Ivanets
2017-12-14 22:42:06 +00:00
Richard W.M. Jones
2c359583b3 Version 1.37.35. v1.37.35 2017-12-10 09:34:50 +00:00
Richard W.M. Jones
1190b8ce56 generator: Update AUTHORS. 2017-12-10 09:34:29 +00:00
Richard W.M. Jones
66f5aceda8 builder: Fix name of repository_main.mli file.
Fixes commit 67b0de7399.
2017-12-10 09:23:19 +00:00
Richard W.M. Jones
442653d9cd php: Ignore php/extension/configure.ac
Latest php finally renames this (from configure.in).
2017-12-10 09:17:40 +00:00
Richard W.M. Jones
62da9d819f configure: If multiple entries in RELEASES file, pick first.
Multiple entries matching the grep would corrupt the output Makefile.
2017-12-10 09:16:45 +00:00
Richard W.M. Jones
1d38216d20 v2v: -i vmx: Enhance VMX support with ability to use ‘-it ssh’ transport.
This enhances the existing VMX input support allowing it to be
used over SSH to the ESXi server.

The original command (for local .vmx files) was:

 $ virt-v2v -i vmx guest.vmx -o local -os /var/tmp

Adding ‘-it ssh’ and using an SSH remote path gives the new syntax:

 $ virt-v2v \
     -i vmx -it ssh \
     "ssh://root@esxi.example.com/vmfs/volumes/datastore1/guest/guest.vmx" \
     -o local -os /var/tmp

I anticipate that this input method will be widely used enough that it
deserves its own example at the top of the man page.
2017-12-09 21:18:17 +00:00
Richard W.M. Jones
6f347b4f80 v2v: vddk: Switch to using ‘-it vddk’ to specify VDDK as input transport.
Previously the presence of the ‘--vddk <libdir>’ option magically
enabled VDDK mode.  However we want to introduce other transports for
VMware conversions so this wasn't a very clean choice.

With this commit you must use ‘-it vddk’ to specify that you want VDDK
as a disk transport.  The previous ‘--vddk <libdir>’ option has been
renamed to ‘--vddk-libdir <libdir>’ to be consistent with the other
passthrough options, and it is no longer required.

A new command line looks like:

  $ export PATH=/path/to/nbdkit:$PATH
  $ virt-v2v \
      -ic 'vpx://root@vcenter.example.com/Datacenter/esxi?no_verify=1' \
|     -it vddk \
|     --vddk-libdir /path/to/vmware-vix-disklib-distrib \
      --vddk-thumbprint xx:xx:xx:... \
      "Windows 2003" \
      -o local -os /var/tmp

where only the two lines marked with ‘|’ have changed.
2017-12-09 21:11:52 +00:00
Richard W.M. Jones
0bbedd47c0 mltools: Move urandom module to common/mltools.
So that we can use it in virt-v2v code.
2017-12-09 21:11:52 +00:00
Richard W.M. Jones
c2898c50fd urandom: Fail if /dev/urandom returns EOF.
This is an unexpected error, so fail hard instead of leaking
End_of_file exception.

Nothing that calls into the Urandom module expects or handles
End_of_file.
2017-12-09 21:11:52 +00:00
Richard W.M. Jones
fd0bfba7b2 urandom: Use with_openfile. 2017-12-09 21:11:52 +00:00
Richard W.M. Jones
7960eb9bc6 urandom: Use O_CLOEXEC. 2017-12-09 21:11:52 +00:00
Richard W.M. Jones
1f66344961 builder: planner: Don't add some impossible transitions.
Certain transitions where the input and output filename are the same
are impossible, eg copying a file to itself.  Don't add these.

Reported-by: David Kaylor.
2017-12-09 21:11:52 +00:00
Richard W.M. Jones
a93a06dc59 daemon: Rewrite inspection code using ‘return’ statements.
When rewriting this, I revisited the original C code and changed the
OCaml code so it's now reasonably similar to the original.
2017-12-08 16:22:14 +00:00
Richard W.M. Jones
506e90cf35 common/mlstdutils: Add ‘return’ statement for OCaml.
This adds a ‘return’ statement as found in other programming
languages.  You can use it like this:

  with_return (fun {return} ->
    some code ...
  )

where ‘some code’ may either return implicitly (as usual), or may call
‘return x’ to immediately return ‘x’.  All returned values must have
the same type.

The OCaml >= 4.04 implementation is by Petter A. Urkedal and octachron.
See this thread:
https://sympa.inria.fr/sympa/arc/caml-list/2017-11/msg00017.html

The version that works for any OCaml is by me.  (Note that my version
cannot be nested).
2017-12-08 16:22:14 +00:00
Richard W.M. Jones
0eb23230fa common/mlstdutils: Move list functions into extended List module.
However some existing functions had names which shadowed existing
functions in the List module, so I had to rename them:

  assoc -> List.assoc_lbl
  append -> List.push_back_list
  prepend -> List.push_front_list

This is an extension of the previous commit.
2017-12-08 16:22:11 +00:00
Richard W.M. Jones
b585fea976 common/mlstdutils: Extend the List module.
We defined a number of functions on lists which are not provided by
the standard library.  As with Char and String, let's extend List to
add these new functions to a List pseudo-module (really
Std_utils.List, but called List when you ‘open Std_utils’).

The initial exported functions are all List functions from OCaml 3.11
+ iteri + mapi.  We can add other functions as needed.
2017-12-08 16:21:10 +00:00
Richard W.M. Jones
4699c7b6e1 v2v: -o null: Use the qemu null device driver.
Instead of writing to a temporary file and deleting it, use the null
block driver (the "null-co://" URI) in qemu to throw it away.
2017-12-08 16:21:06 +00:00
Richard W.M. Jones
e29296cfa2 v2v: Abstract the target file so it could be a QEMU URI.
This complicated bit of refactoring abstracts the target file so it is
allowed to be either a filename or a QEMU URI.
2017-12-08 16:20:59 +00:00
Richard W.M. Jones
c6b2c0a008 v2v: vddk: Fix printing of nbdkit command.
Fixes commit 99a29125f6.
2017-12-08 11:48:43 +00:00
Pino Toscano
b34f457cf2 builder: use the template arch when caching all templates
When caching all the templates, use the architecture of each template,
instead of the architecture passed as --arch (or the host architecture,
as default).  This way, the right destination filename will be used.

Fixes commit b1cf6246f3.

Thanks to: Erik Skultety.
2017-12-07 18:16:17 +01:00
Richard W.M. Jones
99a29125f6 v2v: vddk: Print nbdkit command before running it.
Very useful for debugging.
2017-12-07 16:22:09 +00:00
Richard W.M. Jones
2b185d1e90 v2v: cmdline: Reformat array of arguments.
No change, just code formatting.
2017-12-07 16:22:09 +00:00
Richard W.M. Jones
eecf41956d v2v: Remove unused ‘scheme’ parameter.
It wasn't actually used for anything significant.  Even worse
virt-v2v-copy-to-local made up a scheme out of thin air.

Just code refactoring, no functional change.
2017-12-07 10:53:31 +00:00
Richard W.M. Jones
4b43697f9b v2v: docs: State that --vddk-thumbprint is only required if --vddk present (RHBZ#1513884).
Thanks: Ming Xie, Pino Toscano.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1513884#c2
2017-12-06 15:56:47 +00:00
Richard W.M. Jones
5ac24b490e v2v: docs: Remove min version of VDDK, recommend nbdkit >= 1.1.25 (RHBZ#1513884).
Requires -> recommends because older versions of nbdkit will likely
work also.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1513884#c3
2017-12-06 15:55:57 +00:00
Pino Toscano
614f680dc6 lib: libvirt: stop using <shareable/> for appliance disk (RHBZ#1518517)
Commit aa9e0057b1 made the libvirt backend
use <shareable/> for the disk of the appliance, since at that time all
the instances were using the disk directly.
OTOH, commit 3ad44c8660 switched to
overlays for read-only disks, including the appliance, so effectively
protecting the appliance.

Because of this, the libvirt backend does not need <shareable/> anymore.

Thanks to: Daniel Berrange, Richard W.M. Jones,  Peter Krempa.
2017-12-05 18:14:21 +01:00
Pino Toscano
1b2020ba04 v2v: bootloaders: handle no default grubby kernel (RHBZ#1519204)
When using grubby to get the default kernel of a guest, do not fail
with a bogus error like:

virt-v2v: error: libguestfs error: statns: statns_stub: path must start
with a / character

in case there is no default kernel that can be determined (e.g. because
of a bogus configuration).
2017-12-04 10:15:29 +01:00
Richard W.M. Jones
af4e3b3d3b bugs-in-changelog.sh: Python bugzilla client -t option changed to -s. 2017-11-25 21:07:29 +00:00
Richard W.M. Jones
5eabcdb9fd v2v: Add generator_built and BUILT_SOURCES listing generated files.
Fixes commit 0c396650b0.
2017-11-23 18:48:44 +00:00
Pino Toscano
06df910491 docs: add virt-builder-repository.pod to podfiles
Updates commit a442d2c321.
2017-11-23 14:49:44 +01:00