531 Commits

Author SHA1 Message Date
Pino Toscano
0a808d986a builder: fix EOF check with flex >= 2.6.1
It looks like flex 2.6.1 changed [1] the return code for EOF in
yyinput.  Therefore, use the right value depending on the version of
flex which generates the lexer.

[1] f863c9490e
2016-07-27 15:24:29 +02:00
Richard W.M. Jones
d5a8f82887 Use 'const' for stack integers where possible.
May improve optimization possibilities in a few cases.
2016-07-26 10:43:45 +01:00
Pino Toscano
7b90ff3fc8 Limit GCC 6 pragmas.
-Wnull-dereference and -Wshift-overflow are new warnings in GCC 6, so do
not try to disable them with pragmas on older GCC versions.

Fixes commit a8e15ea924.
2016-07-25 19:45:03 +02:00
Richard W.M. Jones
a8e15ea924 daemon: Ignore -Wnull-dereference & -Wshift-overflow warnings.
One -Wnull-dereference warning is real: we deliberately cause a
segfault in one of the tests.

There is a -Wshift-overflow bug in a Gtk 2 header.

The others are the result of shortcomings in GCC.

In all cases we have to add GCC diagnostic overrides to ignore
the warnings when compiling with ./configure --enable-werror.
2016-07-24 10:55:52 +01:00
Pino Toscano
8e57268dd4 static const char *str -> static const char str[]
Make all the static constant strings as char arrays, so they can be
fully stored in read-only memory.
2016-07-22 13:16:02 +02:00
Richard W.M. Jones
08e27451a6 tests: Add script to check documented tool options match actual options.
podcheck.pl is run as part of the tests to perform various checks on
the documentation and the tool.

Currently we check only that the documented options matches the
options that the tool implements and vice versa.  This commit would
also allow us (in future) to check --help, --long-options,
--short-options, --version output.

This commit includes scripts to run the tests and various fixes to the
manual pages to ensure that the tests pass.
2016-07-19 13:06:14 +01:00
Richard W.M. Jones
1fa3bf14e5 builder: Fix rhel-ppc64le guestroot to match reality. 2016-07-19 13:06:14 +01:00
Richard W.M. Jones
d03e8f461e builder: Fix URL for RHEL 7.2 ppc64le images. 2016-07-19 12:32:08 +01:00
Pino Toscano
34cdb1145e builder: improve the handling of list formats
Store them directly in List_entries, an adding helper function to
convert from string.

Also use Getopt.Symbol for them, so there is no need to manually reject
invalid formats.
2016-07-18 18:21:41 +02:00
Richard W.M. Jones
ba2cf9a2eb mllib: Use L"..." and S '...' for long and short options.
There is also an M"..." variant which acts the same way as
L but handles the annoying virt-v2v two letter options.
2016-07-18 17:01:22 +01:00
Richard W.M. Jones
c2fda2c1f2 build: Factor out definition of BEST and OCAMLLINKFLAGS.
No change, just refactoring.
2016-07-18 14:11:22 +01:00
Richard W.M. Jones
2c6bfccb06 build: Fix dependencies on mllib and customize.
Binaries should be rebuilt if mllib.cma/mllib.cmxa or
customize.cma/customize.cmxa change.

Fixes commit 1e2877c6f4.
2016-07-18 14:10:28 +01:00
Richard W.M. Jones
3bf9420f94 builder: Add new centos-aarch64.sh script to EXTRA_DIST.
Fixes commit c168638ae7.
2016-07-14 15:40:42 +01:00
Richard W.M. Jones
6a0fe553b9 customize: Build a customize.cma/customize.cmxa library.
Instead of linking with individual objects, which is very tedious,
build a proper library and link virt-builder, virt-customize and
virt-sysprep to it.

This makes the binaries a tiny bit smaller because .cmxa/.a files
allow unused code to be removed by the linker, whereas explicitly
linking .cmx/.o files does not.
2016-07-14 15:40:42 +01:00
Richard W.M. Jones
1e2877c6f4 mllib: Build a mllib.cma/mllib.cmxa library.
Instead of linking with individual objects, which is very tedious,
build a proper library and link the other tools with it.

This doesn't make the resulting binaries any larger.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
35bac3a650 lib: Deprecate old SELinux APIs, rewrite SELinux documentation (RHBZ#1152825).
Also turns the --selinux option of guestfish, guestmount and
virt-rescue into a no-op -- it didn't work before so this is
effectively no change.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
b6e92b1100 builder: Use virt-sysprep --selinux-relabel when preparing templates.
Don't need to be clever now.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
f3c69fe60b customize: Add module for doing SELinux relabel of filesystem (RHBZ#554829, RHBZ#983969, RHBZ#1089100).
This implements the --selinux-relabel option for virt-customize,
virt-builder and virt-sysprep.  There is no need to autorelabel
functionality now.

Thanks: Stephen Smalley
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
eb22e2d36e v2v: Move augeas_debug_errors (renamed debug_augeas_errors) to common code.
Since the function uses the StringMap module, I also had to
move this to mllib.

The function is renamed and the 'if verbose ()' part is moved into the
function, but apart from that, it's all just code motion.
2016-07-14 11:40:22 +01:00
Pino Toscano
0f7bf8f714 OCaml tools: add and use a Getopt module
Add a new Getopt module to mllib, to parse command line arguments with
handlers close to the ones used with Arg, but using getopt(3) (actually
getopt_long_only) to do the real parsing.  This allow us to provide
options for OCaml tools with a syntax similar to the C tools, and use
the additional features getopt offers and Arg does not.
Getopt now handles every part of the command line handling, including
the output of short & long options.

Do a single-step conversion of Common_utils and all the OCaml tools to
the syntax of Getopt.

Move a couple of utility functions from Common_utils to Getopt, since
they fit better there (and Common_utils cannot be used in Getopt, as
the former already uses the latter).

As side-change due to the conversion, extra arguments for sysprep
operation can have more keys for the same argument.
2016-07-13 18:43:32 +02:00
Richard W.M. Jones
c168638ae7 builder: Add CentOS aarch64 image. 2016-07-13 12:59:52 +01:00
Richard W.M. Jones
123dc018b8 mllib: Rename unshift -> push_front, push -> push_back etc.
The C++ function names are more regular.

Updates commit 1c57c71d46.
2016-07-08 09:56:47 +01:00
Richard W.M. Jones
1a3e986b61 builder: Add Fedora 24 (aarch64) image. 2016-07-07 20:36:42 +01:00
Richard W.M. Jones
7d99c2eb84 builder: downloader: More tweaks to curl code.
No functional change.
2016-07-07 18:22:09 +01:00
Richard W.M. Jones
442092a305 Replace 'xs := x :: !xs' with 'unshift x xs'.
In one case, I used prepend instead.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
e84382e9f8 builder, v2v: Use imperative list functions to simplify curl arg code.
No functional change in this commit.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
906c05c70f builder: Use the new Curl module for passing parameters to curl.
These are now passed using a curl configuration file, which is a
little bit safer than using command lines.  virt-builder doesn't need
to pass usernames and passwords to curl, but if it ever does in future
this will be a lot safer.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
044de389e7 builder: Add Fedora 24 images for x86-64, i686, armv7l.
aarch64 image isn't buildable at the moment because of
Anaconda bug RHBZ#1348980.

ppc64 & ppc64le images are not buildable because of delays updating
the Fedora mirrors.
2016-06-22 18:25:36 +01:00
Richard W.M. Jones
e4e197bb86 builder: Use newly open sourced edk2 from Fedora, replacing kraxel's builds. 2016-06-22 12:47:20 +01:00
Richard W.M. Jones
27b3b2cf7d builder: Use a different mirror when building armv7l image. 2016-06-22 12:41:57 +01:00
Richard W.M. Jones
b000d6bc68 virt tools: Add common --colours option.
This option (alternately spelled: --color, --colour, --colors, or
--colours) enables ANSI colour sequences output even if that would be
disabled becaues the output is not a TTY.
2016-06-18 21:15:59 +01:00
Richard W.M. Jones
6e79a3c84d Convert source so it can be compiled with OCaml '-safe-string' option.
OCaml 4.02 introduced the 'bytes' type, a mutable string intended to
replace the existing 'string' type for those cases where the byte
array can be mutated.  In future the 'string' type will become
immutable.  This is not the default now, but it can be forced using
the '-safe-string' compile option.

This commit changes the code so that it could be compiled using
'-safe-string' (but does not actually make that change).

If we detect OCaml < 4.02, we create a dummy 'Bytes' compatibility
module ((nearly) an alias for the 'String' module).  The only
significant difference from upstream OCaml is that you must write the
'bytes' type as 'Bytes.t' in interfaces, apart from that everything
else should work.
2016-06-16 15:22:04 +01:00
Andrea Bolognani
a2d0511904 builder: Fix description for --long option 2016-06-16 15:22:04 +01:00
Richard W.M. Jones
a3cb12ab2c ocaml: Move identical rules for building OCaml objects to common file. 2016-06-13 08:51:46 +01:00
Richard W.M. Jones
7274f11ad4 builder: Warn if --output is a host partition (RHBZ#1342337).
But allow the warning to be turned off using --no-warn-if-partition.

Ming Xie tried to create a bootable USB key using
'virt-p2v-make-disk -o /dev/sdX1'.  That works, but doesn't create a
bootable key because it puts everything into the first partition.

Emit a warning if someone tries to do this to try to prevent user
error:

  virt-builder: warning: output device (/dev/sdb1) is a partition.  If you
  are writing to a USB key or external drive then you probably need to write
  to the whole device, not to a partition.  If this warning is wrong then you
  can disable it with --no-warn-if-partition
2016-06-03 13:05:07 +01:00
Richard W.M. Jones
df7a33768e mllib: Add Common_utils.is_partition function.
Returns true if the device is a host partition.

The only tedious bit of this patch is that now Common_utils depends on
Dev_t so we have to add extra objects every time something links with
Common_utils.
2016-06-03 13:05:07 +01:00
Pino Toscano
c605765c95 mllib: add a new run_command helper
Add a simple helper to run a command from a sequence of arguments,
without using a shell: this should help reducing the amount of quoting
ineeded, since arguments are passed straight as such.

Make use of it in the places currently using shell_command, and which
don't assume they can run anything (so no shell redirections, `env`,
etc).
2016-05-24 11:26:48 +02:00
Pino Toscano
a52ee88f88 mllib: add an helper shell_command
Add a simple shell_command, which is mostly a wrapper around
Sys.command but with logging of the command run.
2016-05-24 11:10:42 +02:00
Pino Toscano
3dad36dab2 mllib: make external_command echo the command executed
Add an optional parameter to disable this behaviour, so the Curl module
in v2v won't print user-sensible data (like passwords).
2016-05-24 11:10:15 +02:00
Richard W.M. Jones
063af7f987 ocaml tools: Use a common debug function.
Add a common debug function for printing debugging messages.  It only
emits the debug message when the verbose (-v) flag is used on the
command line.

It sends the output to stderr, which is flushed immediately after the
message is printed (to help with debugging unexpected crashes).  There
are good arguments for sending the debug to either stdout or stderr,
and almost all existing debug messages replaced by this change went to
stdout.  However using stderr is consistent with libguestfs's own
debug messages which also go to stderr.

I only made simple changes to code of the form 'if verbose () then
printf ...'.  There are more places which could be changed in future.
In a few places I removed gettext calls since we probably should not
translate debug messages.
2016-05-23 10:30:56 +01:00
Cédric Bosdonnat
536ef6f34a customize: change windows firstboot path
Rename the C:\Program Files\Red Hat\Firstboot folder into the more vendor
independent C:\Program Files\Guestfs\Firstboot.
2016-05-20 10:48:50 +01:00
Cédric Bosdonnat
ee78c06b7f customize: add support for pvvxsvc
SUSE VMDP comes with a replacement for rhsrvany.exe named pvvxsvc.exe.
Check for either one of them instead of only rhsrvany.
2016-05-20 10:39:58 +01:00
Pino Toscano
ddcd9c2a10 builder: run/schedule a SELinux relabel if needed
If the guest uses SELinux, then make sure to run a relabel (or at least
schedule one) after the image build: this way the template is
relabelled, or at least it will do that at the next boot, without the
need for the user to ask for a relabel.

This just covers the case of building a new image with no additional
operations on it though.
2016-05-10 15:11:54 +02:00
Richard W.M. Jones
71a4ffdd05 builder: Add Ubuntu 16.04 (Xenial) image. 2016-04-21 13:41:52 +01:00
Richard W.M. Jones
d585aca1d5 builder: ubuntu: Lock the 'builder' account that we have to add to the template. 2016-04-21 13:41:00 +01:00
Richard W.M. Jones
f56a46d0b5 builder: ubuntu: Fix URL of archive.ubuntu.com.
The archive.ubuntu.net address appears to work only intermittently.
2016-04-21 13:40:37 +01:00
Richard W.M. Jones
dec30914e8 mllib: Rename uRI -> URI.
There is no difference in the OCaml module name, which remains 'URI'.
2016-04-20 22:48:37 +01:00
Richard W.M. Jones
0108240364 tests: Fix 'make check-slow'.
Since we started to use the parallel tests framework in automake,
'make check-slow' has been broken.  This is because parallel tests
doesn't allow you to run 'make check TESTS=...' with a set of test
scripts which do not also appear in the static list of tests in the
Makefile.am.  We would like to list and run only "fast" tests in the
Makefile.am, and have other scripts for slow tests.

The solution is to add the slow tests to Makefile.am, but condition
those tests on an environment variable SLOW=1 being set.

This commit fixes all the existing slow tests in this way, and updates
the documentation (guestfs-hacking(1)) to document how slow tests
should be written in future.
2016-04-16 20:40:49 +01:00
Richard W.M. Jones
64a04a4af0 builder: pxzcat: Remove ineffective POSIX_FADV_WILLNEED.
On Linux this will load the whole file into the page cache.  However
the output file is empty and zero sized just after it is opened, so
this has no effect.  Note that the advice is not persistent, so this
really does nothing.

I considered adding the call back after the file has been written,
just before the close, but:

 - If we do a virt-resize next then we will open and read the file mostly
   sequentially, so readahead will deal with any missing pages.

 - If we do a virt-customize next then we will only access a small
   part of the disk image, so loading it all into the page cache adds
   extra work.

 - In any case, since we have just written the file it's likely to
   still be in the page cache.
2016-04-14 17:00:44 +01:00
Richard W.M. Jones
816eb7605f builder: pxzcat: Close the output file.
After uncompressing the template we didn't close the output file,
which potentially could cause writes to the output file to be lost.
2016-04-14 16:10:47 +01:00