Commit Graph

544 Commits

Author SHA1 Message Date
Pino Toscano
49dd176974 builder: distribute fedora-s390x.sh
Followup of commit c25a898710.
2016-11-02 11:00:39 +01:00
Richard W.M. Jones
c25a898710 builder: Add script for starting Fedora/s390x.
However it does not currently work because of a bug in libvirt:

https://bugzilla.redhat.com/show_bug.cgi?id=1376547
2016-11-01 11:50:22 +00:00
Pino Toscano
75fbe4511e builder: consolidate handling of temporary files/dirs
Create a single temporary directory for all the files created during the
virt-builder run (except big disk images, which already use the
libguestfs cache directory).  A single directory means there is no need
to manually schedule all the temporary files and directories for removal
when the application quits.
2016-10-25 10:15:04 +02:00
Richard W.M. Jones
05f201a6e6 builder: Add test that a serial console is set up for guest templates we supply. 2016-10-19 14:50:36 +01:00
Richard W.M. Jones
a01c828f0c builder: Add virt-builder --notes for enabling serial console.
On debian-8 and ubuntu-16.04.
2016-10-19 14:50:36 +01:00
Richard W.M. Jones
fc35cfe361 builder: Allow test-virt-builder-planner to be skipped. 2016-10-19 12:19:44 +01:00
Richard W.M. Jones
d6c42a6c58 valgrind: Don't use "nested" run scripts.
When TESTS_ENVIRONMENT already uses 'run', the VG variable
doesn't also need to use 'run'.

The specific problem is that if the command contains newlines
then double invocations of the 'run' script fails (in libtool).
ie the following command failed causing errors in check-valgrind:

  $VG virt-builder phony-fedora \
      -v --no-cache --no-check-signature $no_network \
  ...
      --write '/etc/append4:line1
  ' \
  ...
2016-10-11 18:08:49 +01:00
Pino Toscano
a58dff49c5 mllib, builder: add and use Checksums.of_string
Add a simple way to turn a combination of checksum type and value into a
csum_t.  Use it in builder, even if still constrained.
2016-10-03 10:41:26 +02:00
Pino Toscano
7a09db5e48 mllib: move Checksums from builder
Move the Checksums module from virt-builder mostly as it is; the only
change is that on checksum mismatch an exception is raised rather than
invoking "error" directly: this way users of verify_checksum &
verify_checksums can do their own handling of the situation.
2016-10-03 10:41:26 +02:00
Richard W.M. Jones
5681db5ed8 customize: Add --append-line.
This appends a single line to a file, with some cleverness
involving guessing the right line endings to use.

Also adds a test.
2016-10-01 13:08:32 +01:00
Pino Toscano
eea210dbf7 Use the getprogname gnulib module
Make use of the recently added 'getprogname' module in gnulib: replace
our guestfs_int_program_name with the getprogname() provided by the
module, since it does the same thing, and in a portable way.
As consequence of the above, use gnulib in a couple of tests that use
getprogname().

Since guestfs_int_program_name is gone, drop the configure checks
associated with it.
2016-09-08 09:57:15 +02:00
Richard W.M. Jones
619905ac5d builder: Rename cmdline 'ops' field.
In OCaml 3.12, type inference was not as smart and could not use local
information to disambiguate same-named fields in different structures.
The easiest fix for this is just to rename the field so it doesn't
have the same name as a field in the Customize_cmdline.ops struct.
2016-08-26 12:52:27 +01:00
Richard W.M. Jones
76c0a67d30 build: Add common CLEANFILES and DISTCLEANFILES to common-rules.mk.
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.

Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
2016-08-25 16:54:34 +01:00
Pino Toscano
0a808d986a builder: fix EOF check with flex >= 2.6.1
It looks like flex 2.6.1 changed [1] the return code for EOF in
yyinput.  Therefore, use the right value depending on the version of
flex which generates the lexer.

[1] f863c9490e
2016-07-27 15:24:29 +02:00
Richard W.M. Jones
d5a8f82887 Use 'const' for stack integers where possible.
May improve optimization possibilities in a few cases.
2016-07-26 10:43:45 +01:00
Pino Toscano
7b90ff3fc8 Limit GCC 6 pragmas.
-Wnull-dereference and -Wshift-overflow are new warnings in GCC 6, so do
not try to disable them with pragmas on older GCC versions.

Fixes commit a8e15ea924.
2016-07-25 19:45:03 +02:00
Richard W.M. Jones
a8e15ea924 daemon: Ignore -Wnull-dereference & -Wshift-overflow warnings.
One -Wnull-dereference warning is real: we deliberately cause a
segfault in one of the tests.

There is a -Wshift-overflow bug in a Gtk 2 header.

The others are the result of shortcomings in GCC.

In all cases we have to add GCC diagnostic overrides to ignore
the warnings when compiling with ./configure --enable-werror.
2016-07-24 10:55:52 +01:00
Pino Toscano
8e57268dd4 static const char *str -> static const char str[]
Make all the static constant strings as char arrays, so they can be
fully stored in read-only memory.
2016-07-22 13:16:02 +02:00
Richard W.M. Jones
08e27451a6 tests: Add script to check documented tool options match actual options.
podcheck.pl is run as part of the tests to perform various checks on
the documentation and the tool.

Currently we check only that the documented options matches the
options that the tool implements and vice versa.  This commit would
also allow us (in future) to check --help, --long-options,
--short-options, --version output.

This commit includes scripts to run the tests and various fixes to the
manual pages to ensure that the tests pass.
2016-07-19 13:06:14 +01:00
Richard W.M. Jones
1fa3bf14e5 builder: Fix rhel-ppc64le guestroot to match reality. 2016-07-19 13:06:14 +01:00
Richard W.M. Jones
d03e8f461e builder: Fix URL for RHEL 7.2 ppc64le images. 2016-07-19 12:32:08 +01:00
Pino Toscano
34cdb1145e builder: improve the handling of list formats
Store them directly in List_entries, an adding helper function to
convert from string.

Also use Getopt.Symbol for them, so there is no need to manually reject
invalid formats.
2016-07-18 18:21:41 +02:00
Richard W.M. Jones
ba2cf9a2eb mllib: Use L"..." and S '...' for long and short options.
There is also an M"..." variant which acts the same way as
L but handles the annoying virt-v2v two letter options.
2016-07-18 17:01:22 +01:00
Richard W.M. Jones
c2fda2c1f2 build: Factor out definition of BEST and OCAMLLINKFLAGS.
No change, just refactoring.
2016-07-18 14:11:22 +01:00
Richard W.M. Jones
2c6bfccb06 build: Fix dependencies on mllib and customize.
Binaries should be rebuilt if mllib.cma/mllib.cmxa or
customize.cma/customize.cmxa change.

Fixes commit 1e2877c6f4.
2016-07-18 14:10:28 +01:00
Richard W.M. Jones
3bf9420f94 builder: Add new centos-aarch64.sh script to EXTRA_DIST.
Fixes commit c168638ae7.
2016-07-14 15:40:42 +01:00
Richard W.M. Jones
6a0fe553b9 customize: Build a customize.cma/customize.cmxa library.
Instead of linking with individual objects, which is very tedious,
build a proper library and link virt-builder, virt-customize and
virt-sysprep to it.

This makes the binaries a tiny bit smaller because .cmxa/.a files
allow unused code to be removed by the linker, whereas explicitly
linking .cmx/.o files does not.
2016-07-14 15:40:42 +01:00
Richard W.M. Jones
1e2877c6f4 mllib: Build a mllib.cma/mllib.cmxa library.
Instead of linking with individual objects, which is very tedious,
build a proper library and link the other tools with it.

This doesn't make the resulting binaries any larger.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
35bac3a650 lib: Deprecate old SELinux APIs, rewrite SELinux documentation (RHBZ#1152825).
Also turns the --selinux option of guestfish, guestmount and
virt-rescue into a no-op -- it didn't work before so this is
effectively no change.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
b6e92b1100 builder: Use virt-sysprep --selinux-relabel when preparing templates.
Don't need to be clever now.
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
f3c69fe60b customize: Add module for doing SELinux relabel of filesystem (RHBZ#554829, RHBZ#983969, RHBZ#1089100).
This implements the --selinux-relabel option for virt-customize,
virt-builder and virt-sysprep.  There is no need to autorelabel
functionality now.

Thanks: Stephen Smalley
2016-07-14 15:28:10 +01:00
Richard W.M. Jones
eb22e2d36e v2v: Move augeas_debug_errors (renamed debug_augeas_errors) to common code.
Since the function uses the StringMap module, I also had to
move this to mllib.

The function is renamed and the 'if verbose ()' part is moved into the
function, but apart from that, it's all just code motion.
2016-07-14 11:40:22 +01:00
Pino Toscano
0f7bf8f714 OCaml tools: add and use a Getopt module
Add a new Getopt module to mllib, to parse command line arguments with
handlers close to the ones used with Arg, but using getopt(3) (actually
getopt_long_only) to do the real parsing.  This allow us to provide
options for OCaml tools with a syntax similar to the C tools, and use
the additional features getopt offers and Arg does not.
Getopt now handles every part of the command line handling, including
the output of short & long options.

Do a single-step conversion of Common_utils and all the OCaml tools to
the syntax of Getopt.

Move a couple of utility functions from Common_utils to Getopt, since
they fit better there (and Common_utils cannot be used in Getopt, as
the former already uses the latter).

As side-change due to the conversion, extra arguments for sysprep
operation can have more keys for the same argument.
2016-07-13 18:43:32 +02:00
Richard W.M. Jones
c168638ae7 builder: Add CentOS aarch64 image. 2016-07-13 12:59:52 +01:00
Richard W.M. Jones
123dc018b8 mllib: Rename unshift -> push_front, push -> push_back etc.
The C++ function names are more regular.

Updates commit 1c57c71d46.
2016-07-08 09:56:47 +01:00
Richard W.M. Jones
1a3e986b61 builder: Add Fedora 24 (aarch64) image. 2016-07-07 20:36:42 +01:00
Richard W.M. Jones
7d99c2eb84 builder: downloader: More tweaks to curl code.
No functional change.
2016-07-07 18:22:09 +01:00
Richard W.M. Jones
442092a305 Replace 'xs := x :: !xs' with 'unshift x xs'.
In one case, I used prepend instead.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
e84382e9f8 builder, v2v: Use imperative list functions to simplify curl arg code.
No functional change in this commit.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
906c05c70f builder: Use the new Curl module for passing parameters to curl.
These are now passed using a curl configuration file, which is a
little bit safer than using command lines.  virt-builder doesn't need
to pass usernames and passwords to curl, but if it ever does in future
this will be a lot safer.
2016-07-07 18:13:07 +01:00
Richard W.M. Jones
044de389e7 builder: Add Fedora 24 images for x86-64, i686, armv7l.
aarch64 image isn't buildable at the moment because of
Anaconda bug RHBZ#1348980.

ppc64 & ppc64le images are not buildable because of delays updating
the Fedora mirrors.
2016-06-22 18:25:36 +01:00
Richard W.M. Jones
e4e197bb86 builder: Use newly open sourced edk2 from Fedora, replacing kraxel's builds. 2016-06-22 12:47:20 +01:00
Richard W.M. Jones
27b3b2cf7d builder: Use a different mirror when building armv7l image. 2016-06-22 12:41:57 +01:00
Richard W.M. Jones
b000d6bc68 virt tools: Add common --colours option.
This option (alternately spelled: --color, --colour, --colors, or
--colours) enables ANSI colour sequences output even if that would be
disabled becaues the output is not a TTY.
2016-06-18 21:15:59 +01:00
Richard W.M. Jones
6e79a3c84d Convert source so it can be compiled with OCaml '-safe-string' option.
OCaml 4.02 introduced the 'bytes' type, a mutable string intended to
replace the existing 'string' type for those cases where the byte
array can be mutated.  In future the 'string' type will become
immutable.  This is not the default now, but it can be forced using
the '-safe-string' compile option.

This commit changes the code so that it could be compiled using
'-safe-string' (but does not actually make that change).

If we detect OCaml < 4.02, we create a dummy 'Bytes' compatibility
module ((nearly) an alias for the 'String' module).  The only
significant difference from upstream OCaml is that you must write the
'bytes' type as 'Bytes.t' in interfaces, apart from that everything
else should work.
2016-06-16 15:22:04 +01:00
Andrea Bolognani
a2d0511904 builder: Fix description for --long option 2016-06-16 15:22:04 +01:00
Richard W.M. Jones
a3cb12ab2c ocaml: Move identical rules for building OCaml objects to common file. 2016-06-13 08:51:46 +01:00
Richard W.M. Jones
7274f11ad4 builder: Warn if --output is a host partition (RHBZ#1342337).
But allow the warning to be turned off using --no-warn-if-partition.

Ming Xie tried to create a bootable USB key using
'virt-p2v-make-disk -o /dev/sdX1'.  That works, but doesn't create a
bootable key because it puts everything into the first partition.

Emit a warning if someone tries to do this to try to prevent user
error:

  virt-builder: warning: output device (/dev/sdb1) is a partition.  If you
  are writing to a USB key or external drive then you probably need to write
  to the whole device, not to a partition.  If this warning is wrong then you
  can disable it with --no-warn-if-partition
2016-06-03 13:05:07 +01:00
Richard W.M. Jones
df7a33768e mllib: Add Common_utils.is_partition function.
Returns true if the device is a host partition.

The only tedious bit of this patch is that now Common_utils depends on
Dev_t so we have to add extra objects every time something links with
Common_utils.
2016-06-03 13:05:07 +01:00
Pino Toscano
c605765c95 mllib: add a new run_command helper
Add a simple helper to run a command from a sequence of arguments,
without using a shell: this should help reducing the amount of quoting
ineeded, since arguments are passed straight as such.

Make use of it in the places currently using shell_command, and which
don't assume they can run anything (so no shell redirections, `env`,
etc).
2016-05-24 11:26:48 +02:00