Commit Graph

10160 Commits

Author SHA1 Message Date
Richard W.M. Jones
8c9a4bc84f common/options: Change drv struct to store drive index instead of device name.
The device name is only used by guestfish (when using the -N option to
prepare drives).  We constructed the device name very naively,
basically ‘sprintf ("/dev/sd%c", next_drive)’.

This stores the device index instead, and only constructs the device
name in guestfish.  Also the device name is constructed properly using
guestfs_int_drive_name so it can cope with #drives > 26.

As a side effect of this change we can remove the extra parameter of
the add_drives macro.

Thanks: Pino Toscano
2017-05-03 19:32:37 +01:00
Richard W.M. Jones
b168147834 generator: Don't permit certain String/stringt combinations.
After the previous commit it become possible to construct various
"impossible" argument types, such as lists of FileIn strings.  This
commit prevents these from happening.
2017-05-03 19:26:18 +01:00
Richard W.M. Jones
30411ef623 generator: Simplify the handling of string parameters.
Previously we had lots of types like String, Device, StringList,
DeviceList, etc. where Device was just a String with magical
properties (but only inside the daemon), and DeviceList was just a
list of Device strings.

Replace these with some simple top-level types:

  String
  StringList

and move the magic into a subtype.

The change is mechanical, for example:

    old                     --->    new
  FileIn "filename"               String (FileIn, "filename")
  DeviceList "devices"            StringList (Device, "devices")

Handling BufferIn is sufficiently different from a plain String
throughout all the bindings that it still uses a top-level type.
(Compare with FileIn/FileOut where the only difference is in the
protocol, but the bindings can uniformly treat it as a plain String.)

There is no semantic change, and the generated files are identical
except for a minor change in the (deprecated) Perl
%guestfs_introspection table.
2017-05-03 19:26:18 +01:00
Pavel Butsykin
a76e9040b2 v2v: bootloaders: search grub config for all distributions
This patch improves the search of grub config on EFI partition. This
means that the config will be found not only for rhel but also for
many other distributions.  Tests were performed on the following
distributions: centos, fedora, ubuntu, suse. In all cases, the config
path was /boot/efi/EFI/*distname*/grub.cfg

The main purpose of the patch is to improve support for converting of
vm with UEFI for most distributions. Unfortunately this patch does not
solve the problem for all distributions, for example Debian does not
store grub config on the EFI partition, therefore for such
distributions another solution is necessary.

Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
2017-05-03 10:54:29 +01:00
Matteo Cafasso
f3d3136a17 yara_scan: added API tests
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:42:29 +01:00
Matteo Cafasso
e7a0face42 New API: yara_scan
The yara_scan API parses the file generated by the daemon counterpart
function and returns the list of yara_detection structs to the user.

It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted yara_detection structs.

It returns to the caller the list of yara_detection structs generated by
the internal_yara_scan command.

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:34:43 +01:00
Matteo Cafasso
d00dc913aa New API: internal_yara_scan
The internal_yara_scan runs the Yara engine with the previously loaded
rules against the given file.

For each rule matching against the scanned file, a struct containing
the file name and the rule identifier is returned.

The gathered list of yara_detection structs is serialised into XDR format
and written to a file.

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:34:42 +01:00
Matteo Cafasso
09bab5d38c New API: yara_destroy
The yara_destroy API allows to claim resources back via the removal of
the previously loaded Yara rules.

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:34:42 +01:00
Matteo Cafasso
27f175b717 New API: yara_load
The yara_load API allows to load a set of Yara rules contained within a
file on the host.

Rules can be in binary format, as when compiled with yarac command, or
in source code format. In the latter case, the rules will be first
compiled and then loaded.

Subsequent calls of the yara_load API will result in the discard of the
previously loaded rules.

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:34:42 +01:00
Matteo Cafasso
2e24129da3 appliance: add yara dependency
libyara3 on Debian/Ubuntu
yara on SUSE/RedHat

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:01:11 +01:00
Matteo Cafasso
29af0500c2 daemon: expose file upload logic
Allows other modules to use the same logic for uploading files.

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-05-02 13:01:11 +01:00
Richard W.M. Jones
d0344f3522 v2v: -o rhv: Add Windows 2016 Server type in OVF output (RHBZ#1447202).
Thanks: Kun Wei

See also:
d233325d7b
2017-05-02 10:59:33 +01:00
Pavel Butsykin
f885fd227c appliance: more reliable check for the supermin appliance
At least two control files (packages and base.tar.gz) are necessary for the
supermin appliance.

Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
2017-04-28 19:21:17 +01:00
Pino Toscano
c657c93753 v2v: -o glance: factorize common properties
Many of the properties for disks are the same for all of them, so
collect them only once, instead of doing that for every disk.

Should be just code motion, with no behaviour change.
2017-04-28 10:17:18 +02:00
Pino Toscano
7409c9d9d3 v2v: -o glance: add property for UEFI firmware (RHBZ#1445659)
When converting a guest with UEFI firmware, set the also
hw_firmware_type=uefi property for all the disks of the guest, so Nova
can properly boot the guest.
2017-04-28 10:17:17 +02:00
Pino Toscano
79c5acc13d appliance: add cdrkit-cdrtools-compat on openSUSE
It looks like isoinfo is in that package since openSUSE Leap 42.1, so
make sure to include it to not break the isoinfo* APIs.
2017-04-26 12:05:43 +02:00
Richard W.M. Jones
311eb4c7e1 Version 1.37.12. v1.37.12 2017-04-25 22:52:47 +01:00
Richard W.M. Jones
965d253c74 generator: python: Omit unused parameter names in match cases. 2017-04-25 13:08:32 +01:00
Richard W.M. Jones
9243910ea2 generator: perl: Omit unused parameter names in match cases. 2017-04-25 13:08:32 +01:00
Richard W.M. Jones
5a1257a886 daemon: Use CLEANUP_* functions to avoid an explicit free in stub functions. 2017-04-25 12:50:02 +01:00
Matteo Cafasso
ffbf1b5605 daemon: ignore unused return value in upload function
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
2017-04-25 09:33:26 +01:00
Richard W.M. Jones
99813d1195 daemon: stubs.h is generated, so should be listed in generator_built.
Fixes commit 20f190617a.
2017-04-21 15:20:18 +01:00
Richard W.M. Jones
ca12aaa90c generator: daemon: Move *RESOLVE* macros to an ordinary header file.
It's clumsy to define these in the generator, as they are essentially
just static text.
2017-04-21 15:20:17 +01:00
Richard W.M. Jones
3f3dfe68de generator: daemon: Replace ‘cancel_stmt’ with ‘is_filein’ flag.
In every instance where we used the ‘cancel_stmt’ parameter of
these macros:

  ABS_PATH
  NEED_ROOT

the value was only ever ‘cancel_receive ()’ or empty.  We only use
‘cancel_receive’ for FileIn functions, so replace it with a simple
flag for whether the current function is a FileIn function.

This also removes some incorrect comments about macros that cannot be
used with FileIn functions when in fact they can.
2017-04-21 15:20:17 +01:00
Richard W.M. Jones
5200e51719 generator: daemon: Remove fail_stmt parameter from *RESOLVE* macros.
As a simple consequence of the previous commit, every instance of the
‘fail_stmt’ parameter to one of the following macros:

  RESOLVE_DEVICE
  RESOLVE_MOUNTABLE
  REQUIRE_ROOT_OR_RESOLVE_DEVICE
  REQUIRE_ROOT_OR_RESOLVE_MOUNTABLE

is now ‘return’ and therefore we can replace it in the macro and drop
the parameter completely.
2017-04-21 15:20:17 +01:00
Richard W.M. Jones
671b1d10a0 generator: daemon: Remove done label.
As a simple consequence of the previous commit, the ‘done’ label can
be removed, and any instance of ‘goto done’ can be replaced by
‘return’.
2017-04-21 15:20:17 +01:00
Richard W.M. Jones
378cb6f818 generator: daemon: Use a cleanup function to free XDR args struct.
Previously stubs were generated like this:

  void
  fn_stub (XDR *xdr_in)
  {
    struct guestfs_fn_args args;
    ...
    // on error paths, ‘goto done’
    ...
  done:
    xdr_free ((xdrproc_t) xdr_guestfs_fn_args, (char *) &args);
  }

This replaces the call to xdr_free with a generated cleanup function.
2017-04-21 15:20:17 +01:00
Richard W.M. Jones
b2cd4367b1 generator: daemon: Get rid of useless done_no_free label.
The generated code had:

    ...
  done_no_free:
    return;
  }

There was no possible code between the label and the return statement,
so both the label and the return statement are redundant.  The
instances of ‘goto done_no_free’ can simply be replaced by ‘return’.

The only small complication is that there is a label just before this
which contains some optional code.  So we might have ended up with
generated code looking like:

  done:
    // if there was no optional code, this would be empty
  }

which provokes a parse error in C.  Therefore I had to add a semicolon
after the ‘done:’ label.  This will be removed in a subsequent commit,
so it's just to make the current commit bisectable.
2017-04-21 15:20:16 +01:00
Richard W.M. Jones
99ac577a5b tests: Replace test-max-disks with several tests.
Replace the monolithic 'test-max-disks.pl' script with a test program
written in C.  The program is completely equivalent to the old script,
except for the enhancement that it is able to detect if disks are
added to the appliance in the wrong order.

The tests themselves are split out into some shell scripts:

 - test-27-disks: Fully tests 27 disks.
   This is the minimum supported configuration for all backends.

 - test-255-disks: Fully tests 255 disks.
   This is the most disks that libguestfs up to 1.36 supported.

 - test-add-lots-of-disks: Add ‘a lot’ of disks and exit (without
   any further testing).  This is meant to try to exercise > 255
   disk case but without being as slow as a test of the max number
   of disks (which takes ages, even for as few as 4000 disks).

 - test-max-disks: Test the maximum possible number of disks.
   The maximum depends on several factors, notably which backend
   is in use, and the limit on the number of open files.

   ‘test-max-disks’ is a slow test.
2017-04-21 15:20:16 +01:00
Richard W.M. Jones
8b68d00977 daemon: Add an assertion to suppress a warning with GCC.
swap.c: In function 'do_mkswap_U':
swap.c:62:9: error: argument 1 null where non-null expected [-Werror=nonnull]
     if (strlen (label) > SWAP_LABEL_MAX) {
         ^~~~~~~~~~~~~~
2017-04-20 14:38:08 +01:00
Richard W.M. Jones
2de5e317ad daemon: Remove use of fixed-size stack buffers.
GCC 7 complains that the fixed size buffers are not large enough
(at least in theory) when using ‘-O3 -mtune=broadwell’.
2017-04-20 14:38:08 +01:00
Richard W.M. Jones
9e0294f88f lib: direct: Remove support for virtio-blk as the default.
virtio-scsi has been supported in qemu since 2012, and it is superior
in every respect to virtio-blk.  There's no reason to still be using
virtio-blk.

virtio-scsi support was initially added in 2012
(commit 0c0a7d0d86).

You can still use virtio-blk using the (deprecated) iface parameter,
but don't do that in new code.
2017-04-20 14:38:08 +01:00
Richard W.M. Jones
067d442cdc lib: Reimplement is_zero.
As suggested by Eric Blake.
2017-04-20 14:38:07 +01:00
Richard W.M. Jones
c0fdeb36c7 daemon: Move the useful 'is_zero' function into common code.
This is largely a simple refactoring, but it combines another
definition of this function from virt-builder which had a slightly
different prototype.
2017-04-19 17:30:58 +01:00
Richard W.M. Jones
7eb26f03b9 lib: libvirt: Document virtio-scsi controller/bus/target/unit fields.
This adds comments documenting what the fields correspond to in qemu,
and the range of each field.
2017-04-19 10:59:39 +01:00
Richard W.M. Jones
661fa752bb v2v: ovf: Use push_back instead of append.
Microscopically more efficient.

Updates commit dac5fc53ac.
2017-04-15 12:31:08 +01:00
Tomáš Golembiovský
31f1fe7bd2 v2v: ovf: store CPU model for oVirt
If CPU model is available store it in OVF for oVirt. This affects
-o rhev and -o vdsm.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
2017-04-14 11:09:20 +01:00
Pavel Butsykin
684aa26115 daemon: add udev_settle_file to is_root_device
It fixes a fail with ENOENT when device is not yet present after system boot.

Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
2017-04-14 11:08:07 +01:00
Pavel Butsykin
93664d38d2 daemon: run 'udevadm settle' with --exit-if-exists option
Add udev_settle_file() to run 'udevadm settle' with --exit-if-exists option. It
will slightly reduce the waiting-time for pending events if we need to wait
for events related to a particular device/file.

Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>

RWMJ:
 - Use local variable for MAX_ARGS.
 - Use commandv instead of commandrv, fix checking of return code.
2017-04-14 11:07:58 +01:00
Richard W.M. Jones
bf547aac70 Version 1.37.11. v1.37.11 2017-04-13 17:08:17 +01:00
Richard W.M. Jones
d8e1c4bb47 v2v: windows: Install both legacy and modern virtio keys in the registry (RHBZ#1431579).
Thanks: Kun Wei (for finding the bug)
Thanks: Ladi Prosek (for diagnosing the problem and proposing the fix)
2017-04-13 13:27:05 +01:00
Richard W.M. Jones
9e0781f096 mllib: Bind %identity C primitive to Common_utils.identity. 2017-04-13 10:13:34 +01:00
Richard W.M. Jones
c509abef43 v2v: -o rhv: Document how to test RHV conversions without RHV. 2017-04-13 10:13:33 +01:00
Richard W.M. Jones
24a6c5b57c v2v: -o rhv: Quote parameter to rm -rf.
This was safe before, the change just prevents accidental errors.
2017-04-13 10:13:33 +01:00
Richard W.M. Jones
ec61873d39 v2v: tests: Fix -i vmx test so it is more stable.
When running under valgrind, the process takes a fraction of a second
to start up, changing the "Opening the guest" timestamp, which broke
the test.

Fixes commit ca40078cdd.
2017-04-12 08:32:58 +01:00
Richard W.M. Jones
ac894baf75 Version 1.37.10. v1.37.10 2017-04-11 23:41:51 +01:00
Pino Toscano
56c4387b38 builder: various improvements to the documentation
- convert the Fedora 20 examples to Fedora 25
- convert the yum examples to dnf
- convert the Debian 7 examples to Debian 8
- mention that --core updates all the installed packages, not "core"
  ones
- convert i386 to i686 for the Fedora 32bit examples
- mention that autogenerated root passwords are printed on stdout
- mention --get-kernel is deprecated in favour of virt-get-kernel
- update URL to the Puppet installation documentation
- edit the repository index documentation to make it clearer
  non-compressed templates are supported
- remove old wrong paragraph about customization specified in command
  line not processed in the same order
- use --install instead of --run-command '$pkgmgr install ...'
- other minor changes
2017-04-11 22:54:52 +01:00
Richard W.M. Jones
ca40078cdd v2v: Implement -i vmx to read VMware vmx files directly (RHBZ#1441197).
This is a mostly complete implementation of a VMX parser and input
class for virt-v2v.  It parses the name, memory size, CPU topology,
firmware, video, sound, hard disks, removable disks and network
interfaces from the VMX file.  It only omits support for floppies and
SCSI CD-ROMs.

The input class is split into two major parts: a generic VMX file
parser (Parse_vmx), and the Input_vmx module which translates the VMX
tree into the source device model.

This also contains tests.  There are simple unit tests of the
Parse_vmx module, and also some more complete parsing tests taken from
real guests.
2017-04-11 22:39:38 +01:00
Richard W.M. Jones
ef261d69ed mllib: Add Char.hexdigit function.
Same as the function defined in fish/fish.c.
2017-04-11 21:50:05 +01:00
Richard W.M. Jones
bb846887de ocaml: Add Bytes.sub_string function to Bytes compat module.
This is only used when OCaml < 4.02 (eg. RHEL 7).  For more recent
versions the real Bytes.sub_string function is used instead.
2017-04-11 21:50:05 +01:00