Commit Graph

650 Commits

Author SHA1 Message Date
Richard W.M. Jones
0eb23230fa common/mlstdutils: Move list functions into extended List module.
However some existing functions had names which shadowed existing
functions in the List module, so I had to rename them:

  assoc -> List.assoc_lbl
  append -> List.push_back_list
  prepend -> List.push_front_list

This is an extension of the previous commit.
2017-12-08 16:22:11 +00:00
Pino Toscano
b34f457cf2 builder: use the template arch when caching all templates
When caching all the templates, use the architecture of each template,
instead of the architecture passed as --arch (or the host architecture,
as default).  This way, the right destination filename will be used.

Fixes commit b1cf6246f3.

Thanks to: Erik Skultety.
2017-12-07 18:16:17 +01:00
Richard W.M. Jones
67b0de7399 builder: Add missing builder/repository_main.mli file.
Fixes commit a442d2c321.
2017-11-21 23:11:34 +00:00
Cédric Bosdonnat
a442d2c321 New tool: virt-builder-repository
virt-builder-repository allows users to easily create or update
a virt-builder source repository out of disk images. The tool can
be run in either interactive or automated mode.
2017-11-21 16:36:32 +00:00
Cédric Bosdonnat
efaf570aaa builder: add a template parameter to get_index
get_index now gets a new template parameter. Setting it to true will
make the index parsing less picky about missing important data. This
can be used to parse a partial index file.
2017-11-21 16:16:19 +00:00
Cédric Bosdonnat
448a61fbd5 builder: change arch type to distinguish guesses
Change Index.arch to the type (Arch of string | GuessedArch of string).

In a future commit, the index parser will allow arch not to be set
for some cases. Thus arch value will be guessed by inspecting the
image. However we need to distinguish between a set value and a guessed
one. Using this new type will help it:

    match arch with
    | Arch s        -> (* This is a set value *)
    | GuessedArch s -> (* This is a guessed value *)
2017-11-21 16:16:19 +00:00
Richard W.M. Jones
9dd81036ce builder: Add osinfo_config.mli file.
Every .ml file now needs a corresponding .mli file so that
dependencies are generated correctly.  The check-mli.sh script picks
up the problem:

  FAIL: check-mli.sh
  ==================
  ./builder/osinfo_config.ml: missing ./builder/osinfo_config.mli
  FAIL check-mli.sh (exit status: 1)

Fixes commit 46abca0ec3
and commit 02184f55f9.
2017-11-16 13:22:02 +00:00
Richard W.M. Jones
46abca0ec3 builder: Add osinfo to virt-builder and hence to EXTRA_DIST.
Fixes commit 02184f55f9.
2017-11-16 10:33:42 +00:00
Richard W.M. Jones
994ca8d873 builder: Add Fedora 27 templates for aarch64, armv7l, i686, ppc64, ppc64le. 2017-11-14 19:54:43 +00:00
Richard W.M. Jones
52f8e2b830 builder: Fedora 27 (x86_64) image.
Note this uses GPT, see commit a06e50e263.
2017-11-14 14:58:53 +00:00
Richard W.M. Jones
a52b7bdccc builder: template: Save the final virt-install command to a file.
For documentation purposes only.
2017-11-14 14:08:34 +00:00
Richard W.M. Jones
143d61745b builder: templates: Rebuild Fedora 26 ppc64 & ppc64le templates with GPT. 2017-11-14 14:08:34 +00:00
Richard W.M. Jones
a06e50e263 builder: templates: For RHEL, CentOS >= 7 and all Fedora, default to GPT.
The immediate issue is with Fedora/ppc64 and /ppc64le which currently
use extended partitions, breaking the virt-builder ‘--size’ parameter,
eg:

  $ virt-builder --arch ppc64le fedora-26 --size 20G
  ...
  [  21.6] Resizing (using virt-resize) to expand the disk to 20.0G
  virt-resize: error: /dev/sda5: partition not found in the source disk image
  (this error came from '--expand' option on the command line).  Try running
  this command: virt-filesystems --partitions --long -a /var/tmp/vbf67b8c.img

However more generally MBR is broken and should die.  GPT is supported
by all modern virtual bootloaders, so just default to it.

Notes:

* This is different from mandating a UEFI bootloader.

* I am not planning to rebuild any existing images except the
  F26 ppc64 & ppc64le ones.
2017-11-14 14:07:56 +00:00
Cédric Bosdonnat
7e3689bfe0 builder: add Index.write_entry function
Add a function to properly write virt-builder source index entries.
Note that this function is very similar to Index.print_entry that is
meant for debugging purposes.
2017-11-07 12:51:28 +00:00
Cédric Bosdonnat
0933d2d818 builder: add Utils.get_image_infos function
This helper function calls qemu-img info on an image file and
returns the output as a JSON Yajl tree.

This function will be used in future commits.
2017-11-07 12:41:31 +00:00
Pino Toscano
02184f55f9 builder: add simple OCaml osinfo-db reader
Add a simple OCaml-based implementation of reader of the osinfo-db:
the only interface is an iterator that invokes an user-supplied
function with each XML file found.

This implementation behaves like the current C implementation, and
still supports the old libosinfo db.

[RWMJ: Fixed trailing whitespace]
2017-11-07 12:24:42 +00:00
Pino Toscano
0d4136c69f builder: ignore spaces after repo identifiers (RHBZ#1506511) 2017-10-27 16:30:21 +02:00
Richard W.M. Jones
f484403213 builder: template: Quote parameters when printing.
This is for documentation only so the quoting doesn't actually matter,
but it's better for the user to understand what is really being run.
2017-10-18 11:16:31 +01:00
Richard W.M. Jones
c4a6f2a23f builder: template: Factor out function for printing virt-install command.
Pure refactoring, no change.
2017-10-18 11:16:31 +01:00
Richard W.M. Jones
b92f74458f common/mlstdutils: Introduce Option submodule.
Inspired by ocaml-extlib, introduce a module for handling option
types.

We already had the ‘may’ function (which becomes ‘Option.may’).  This
adds also ‘Option.map’ (unused), and ‘Option.default’ functions.

Note this does *not* introduce the unsafe ‘Option.get’ function from
extlib.
2017-10-12 16:31:47 +01:00
Cédric Bosdonnat
7291fed7fb builder: rename docs test script
Rename test-virt-builder-docs.sh into test-docs.sh to include test
for another tool's documentation.
2017-10-09 10:38:56 +01:00
Richard W.M. Jones
85388f09a3 builder: Add FreeBSD 11.1 template. 2017-10-06 14:47:12 +01:00
Richard W.M. Jones
83d6ad3790 builder: template: Add minimal support for creating FreeBSD templates.
These cannot be automated yet.
2017-10-06 14:47:12 +01:00
Richard W.M. Jones
92fa2d3794 builder: make-template: Don't need --os-type=linux.
The --os-type option is obsolete.  All the information is conveyed in
the --os-variant option.
2017-10-06 14:47:12 +01:00
Richard W.M. Jones
319fddb54b builder: make-template: Use the virt-install --transient option.
Ensures that we don't need to clean up the libvirt domain.
2017-10-06 14:47:12 +01:00
Richard W.M. Jones
457bdb4e2f common/mlstdutils: Drop our implementations of functions now in OCaml 4.01.
We reimplemented some functions which can now be found in the OCaml
stdlib since 4.01 (or earlier).  The functions I have dropped are:

 - String.map
 - |>
 - iteri  (replaced by List.iteri)
 - mapi   (replaced by List.mapi)

Note that our definition of iteri was slightly wrong: the type of the
function parameter was too wide, allowing (int -> 'a -> 'b) instead of
(int -> 'a -> unit).

I also added this new function to the Std_utils.String module as an
export from stdlib String:

 - String.iteri

Thanks: Pino Toscano
2017-10-05 11:32:54 +01:00
Richard W.M. Jones
e44562dd6f ocaml: Avoid Warning 52 for Planner.plan function.
Change the Planner.plan function so it returns an optional type.  This
means it no longer raises Failure "plan" on error, so we can both
force the caller to deal with the error case and avoid Warning 52.
2017-10-05 11:32:54 +01:00
Richard W.M. Jones
0970bd0608 ocaml: Avoid Warning 52 for URI.parse_uri function.
This avoids warning 52 in OCaml code such as:

  try URI.parse_uri arg
  with Invalid_argument "URI.parse_uri" -> ...

which prints:

  Warning 52: Code should not depend on the actual values of
  this constructor's arguments. They are only for information
  and may change in future versions. (See manual section 8.5)

In the long term we need to change fish/uri.c so that we can throw
proper errors.
2017-10-05 11:32:54 +01:00
Richard W.M. Jones
c7651744da ocaml: Replace pattern matching { field = field } with { field }.
If you have a struct containing ‘field’, eg:

  type t = { field : int }

then previously to pattern-match on this type, eg. in function
parameters, you had to write:

  let f { field = field } =
    (* ... use field ... *)

In OCaml >= 3.12 it is possible to abbreviate cases where the field
being matched and the variable being bound have the same name, so now
you can just write:

  let f { field } =
    (* ... use field ... *)

(Similarly for a field prefixed by a Module name you can use
‘{ Module.field }’ instead of ‘{ Module.field = field }’).

This style is widely used inside the OCaml compiler sources, and is
briefer than the long form, so it makes sense to use it.  Furthermore
there was one place in virt-dib where we are already using this new
style, so the old code did not compile on OCaml < 3.12.

See also:
https://forge.ocamlcore.org/docman/view.php/77/112/leroy-cug2010.pdf
2017-10-05 11:32:54 +01:00
Richard W.M. Jones
a6a982f004 builder: Choose better weights in the planner. 2017-10-04 15:38:24 +01:00
Richard W.M. Jones
286a365166 mltools: planner: Documentation and minor refactoring of types and parameters.
Just documentation and code refactoring, no functional change.
2017-10-02 22:39:38 +01:00
Richard W.M. Jones
cd304ad838 common/mltools: Rename Common_utils to Tools_utils.
Reflecting the purpose of this module now, which is to act as a place
for utility functions shared only by the OCaml virt tools.
2017-09-28 14:39:23 +01:00
Richard W.M. Jones
80fa8a91e3 Rename mllib -> common/mltools.
This directory which previously contained random modules and functions
now has an official purpose: to be the place for any OCaml utility
needed by the OCaml virt tools.

This is just code movement, I didn't (yet) rename or move any of the
modules.
2017-09-28 14:39:23 +01:00
Richard W.M. Jones
2b00983f23 common/mlgettext: Move common_gettext.ml{,i} to common/mlgettext.
Mostly just code motion, but common_gettext.mli was the same whether
or not ocaml-gettext exists, so instead of generating it, add the file
to git.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
08ee044be1 builder: Simplify PCRE regular expression by using case-insensitive matching.
Updates commit e5182b87cf.
2017-09-28 14:39:22 +01:00
Hilko Bengen
0b3079ba39 builder: Fix problem about index-parse.h not being generated
If configured with --without-ocaml, the build might fail because the
fix added in df5bd5741b was not active.
According to the automake documentation, it should be enough to set
BUILT_SOURCES.
2017-09-26 18:50:53 +01:00
Richard W.M. Jones
a706ecb8f7 Make sure every *.ml file has a corresponding *.mli file. 2017-09-21 18:05:07 +01:00
Cédric Bosdonnat
5d323461b4 builder: remove useless fish dependency 2017-09-20 13:18:20 +02:00
Richard W.M. Jones
fe23cee926 builder: Add centos-7.4 image.
Requested by Gal Ben Haim.
2017-09-18 16:06:04 +01:00
Richard W.M. Jones
def7e10b60 builder: Fix rhel-7.X Server source baseurl (again).
Updates commit 91c950dfba.
2017-09-07 09:06:56 +01:00
Richard W.M. Jones
91c950dfba builder: Fix rhel-7.X Server source baseurl.
Thanks: Jiri Denemark
2017-09-06 18:17:58 +01:00
Florian Klink
84c9ba2898 builder/templates/debian.preseed: enable serial console
RWMJ:

 - Rebuilt the images, to fix RHBZ#1484957

 - Regenerated the index-fragment files as part of rebuilding the images.
2017-08-29 13:36:15 +01:00
Florian Klink
b7fef61f46 builder: templates: debian: use single-partition layout
The previously selected 'atomic' recipe resulted in 2GB swap in a 6GB
volume. Also, we don't really need the boot partition, so just create a
partition using the whole disk space.
2017-08-18 10:36:11 +01:00
Richard W.M. Jones
692195c6ba build: Add a common script for generating OCaml dependencies correctly.
These are generated in many different ways in the various
subdirectories, and sometimes not generated correctly.  Introduce a
script to do this in one place, and hopefully correctly.

This is mostly simple refactoring, but I got rid of a couple of
things:

(1) The ‘make depend’ rule doesn't appear to be needed.  automake (or
make?)  seems to rebuild the ‘.depend’ file automatically just because
it is included.

(2) I got rid of the hairy path rewriting sed expression.  Possibly
that is needed for srcdir != builddir.
2017-08-09 14:45:48 +01:00
Richard W.M. Jones
e5182b87cf builder: Replace small usage of Str with new PCRE module. 2017-08-03 17:36:09 +01:00
Richard W.M. Jones
ff83a88b4c builder: Added RHEL 7.4 kickstarts. 2017-08-02 10:35:36 +01:00
Pino Toscano
0028b99415 builder: templates: disable Debian support for NVRAM
It isn't ready to be pushed yet (needs more testing), and it was
accidentally committed by me with
commit ef53cd5619.

My mistake, my fix, apologies.
2017-07-31 11:19:33 +02:00
Pino Toscano
ef53cd5619 daemon: ntfs: fix name of the ntfs-3g.probe tool
Fixes commit 381c8b68c4, which forgot to
add the dash in the name.
2017-07-31 11:16:17 +02:00
Richard W.M. Jones
60408cecd6 common/mlutils: Remove bogus suffix parameter from Mkdtemp.temp_dir.
The C function mkdtemp(3) requires that the string ends with 6 'X'
characters, so appending a non-empty suffix causes the function to
raise EINVAL.

Luckily we only ever called this function with the last parameter "".
2017-07-24 11:48:24 +01:00
Richard W.M. Jones
8548729cf2 common/mlstdutils: Implement StringSet. 2017-07-24 11:46:22 +01:00