Commit Graph

10450 Commits

Author SHA1 Message Date
Richard W.M. Jones
7e73fdece4 generator: Replace use of Str.split with String.nsplit.
Faster and equivalent.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
93851042f2 v2v: parse VMX: Replace Str with PCRE. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
49208f243f v2v: parse OVF: Replace Str with PCRE. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
9f0aad1f37 v2v: vCenter: Replace Str with PCRE. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
d0c8f0ffe1 v2v: -o libvirt: Use PCRE to verify arch is sane. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
9c191bb15f v2v: -i ova: Use PCRE to match lines in manifest files. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
6db4606948 v2v: windows: Convert the Windows-related conversion modules from Str to PCRE.
This is all new code since the virt-v2v conversion from Perl so there
was no need to check back with the original code.

See also commit 9d920732a3.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
524bf19661 v2v: linux: Convert the Linux-related conversion modules from Str to PCRE.
For each regular expression I went back to the original Perl code to
ensure that the regexp is correct.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
d279d602be v2v: linux: Properly ignore rpm/dpkg-move-aside kernels.
The old virt-v2v code ignored boot kernels with names like
"/boot/vmlinuz-*.rpmsave".  The transscribed code did not because the
Str module requires ‘|’ to be escaped as ‘\|’.

This changes the code to use a simpler test.

Thanks: Pino Toscano
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
3a4410000e v2v: linux: Fix rewriting of grub2 GRUB_CMDLINE=...resume=<device>...
Commit dbe0b69f24 transscribed the Perl
regexp incorrectly so that it only matched the impossible case of
‘resume=/dev/X’ for a single non-whitespace character X.

This fixes the regular expression, referencing back to the original
Perl code in virt-v2v.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
95b0da8f37 v2v: linux: Fix uninstallation of kmod-xenpv from /etc/rc.local script.
In the original conversion of virt-v2v from Perl
(commit 0131d6f666), the Perl regular
expression was incorrectly transscribed.  ‘\b’ was not converted to
‘\\b’ so the new regexp was looking for an ASCII BEL character, not a
word boundary.  Also ‘|’ was used, but Str requires ‘\|’ (ie. "\\|"
in the final source).

To fix these problems I converted the code to use PCRE, and went back
to the original virt-v2v code (virt-v2v.git:
lib/Sys/VirtConvert/Converter/Linux.pm) to find out what the Perl
regular expression should have been.

Note I have also removed ‘.*’ at the beginning and end of the regexp
because PCRE regexps are not anchored.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
843ea53af0 mllib: Convert Common_utils.parse_size and parse_resize to use PCRE.
This also involved enhancing/fixing the test so that parse_size has
test coverage.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
508f3b29b8 customize: Remove use of Str module from virt-customize code. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
08ee044be1 builder: Simplify PCRE regular expression by using case-insensitive matching.
Updates commit e5182b87cf.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
75e7d7e36c valgrind: Ignore PCRE.compile regexps stored in a global variable. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
45ef3545d4 common/mlpcre: Add split and nsplit functions.
These work like our String.split and String.nsplit functions.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
8bd5933cc7 common/mlstdutils: Add String.nsplit ?max parameter, and tests.
This idea was previously proposed by Tomáš Golembiovský in
https://www.redhat.com/archives/libguestfs/2017-January/msg00138.html
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
430029db0a common/mlpcre: Allow some PCRE_* flags to be passed to pcre_compile.
Only five simple flags are allowed so far, and not all of them are
actually used in any code.  They are:

  ~anchored / PCRE_ANCHORED   - implicit ^...$ around regexp
  ~caseless / PCRE_CASELESS   - fold upper and lower case
  ~dotall / PCRE_DOTALL       - ‘.’ matches anything
  ~extended / PCRE_EXTENDED   - extended regular expressions
  ~multiline / PCRE_MULTILINE - ^ and $ match lines within the subject string
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
727b2a41ed common/mlpcre: Add PCRE.replace function.
Similar to Perl s/// but lacks backreferences.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
de3ffa2774 common/mlpcre: Add PCRE.subi to return indexes instead of the substring. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
d2daafe7b7 common/mlpcre: Add const-correctness to guestfs_int_pcre_sub. 2017-09-28 14:39:22 +01:00
Richard W.M. Jones
cbd20c0f01 common/mlpcre: Raise Invalid_argument if PCRE.sub n parameter is negative. 2017-09-28 14:00:42 +01:00
Hilko Bengen
0b3079ba39 builder: Fix problem about index-parse.h not being generated
If configured with --without-ocaml, the build might fail because the
fix added in df5bd5741b was not active.
According to the automake documentation, it should be enough to set
BUILT_SOURCES.
2017-09-26 18:50:53 +01:00
Richard W.M. Jones
ecef1708a5 diff: Remove bogus perror when guestfs_* functions return error.
Neither guestfs_copy_statns nor guestfs_copy_xattr_list set errno, so
it's wrong to call perror(3) here.
2017-09-25 22:22:37 +01:00
Hilko Bengen
4bbf8a321f Fix determining release date for builddir != srcdir 2017-09-25 22:17:01 +01:00
Richard W.M. Jones
5a2db925ec mllib: registry: Make with_hive_readonly, with_hive_write clearer.
Tiny refactoring to make the code clearer.
2017-09-25 14:49:17 +01:00
Richard W.M. Jones
c10b7aaf36 v2v: Refactor convert functions.
This pure refactoring changes the convert () functions so that the
conversion operations are listed first, followed by the
implementations of those operations.
2017-09-25 14:49:17 +01:00
Richard W.M. Jones
deadf7c415 p2v: test: Send sshd debugging messages to stderr.
So that we can see errors/problems in sshd from the test in the
ordinary test-suite.log file.
2017-09-25 14:49:17 +01:00
Pino Toscano
531316cc3f build: improve and simplify distro detection
Add a --with-distro=ID argument for configure, so it is possible to
manually specify the distro to use for the packages (in case os-release
does not provide ID=.., or the ID is not recognized yet).

In the case when --with-distro is not set, keep doing the autodetection,
but using os-release only, i.e. dropping the checks for all the other
-release files -- since there is --with-distro, older distros with no
os-release can still be used.

RWMJ: Add documentation to guestfs-building(1).
2017-09-25 14:49:17 +01:00
Richard W.M. Jones
2cc70fca71 Version 1.37.27. v1.37.27 2017-09-22 09:45:24 +01:00
Richard W.M. Jones
4a7aba23c1 common/mlaugeas: Add readme file to EXTRA_DIST.
Fixes commit 2ffb8a6b25.
2017-09-22 09:42:39 +01:00
Richard W.M. Jones
e6cc355ae8 build: Add comment to OCaml dependencies .depend files.
Also mark them readonly, since they are generated.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
d79df649e0 build: Since .mli files are now required, always build .cmi first.
Here is an example of a failure with the previous dependency
calculation:

  $ make -j5
  make: Entering directory '/home/rjones/d/libguestfs/common/mlstdutils'
    OCAMLOPT guestfs_config.cmx
    CC       libmlstdutils_a-dummy.o
    OCAMLOPT libdir.cmx
    OCAMLCMI stringMap.cmi
    OCAMLCMI stringSet.cmi
    OCAMLCMI guestfs_config.cmi
    OCAMLCMI std_utils.cmi
    OCAMLC   guestfs_config.cmo
    OCAMLC   libdir.cmo
    OCAMLC   stringMap.cmo
    OCAMLC   stringSet.cmo
    OCAMLOPT stringMap.cmx
    OCAMLOPT stringSet.cmx
    OCAMLOPT std_utils.cmx
    OCAMLC   std_utils.cmo
  ocamlfind ocamlc -package str,unix -I . -a guestfs_config.cmo libdir.cmo stringMap.cmo stringSet.cmo std_utils.cmo -o mlstdutils.cma
  ocamlfind ocamlopt -package str,unix -I . -a guestfs_config.cmx libdir.cmx stringMap.cmx stringSet.cmx std_utils.cmx -o mlstdutils.cmxa
    AR       libmlstdutils.a
  File "_none_", line 1:
  Error: Files std_utils.cmx and guestfs_config.cmx
         make inconsistent assumptions over interface Guestfs_config
  make: *** [Makefile:2523: mlstdutils.cmxa] Error 2
  make: Leaving directory '/home/rjones/d/libguestfs/common/mlstdutils'

What seems to be happening is that there is a rule:

  std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]

In this case, make chose to build guestfs_config.cmx and
guestfs_config.cmi in parallel (see the first 5 rules above).  However
building guestfs_config.cmx also creates guestfs_config.cmi
(implicitly - this is not expressed in make dependencies, and make
doesn't "know" that guestfs_config.cmi has already been created).

Unfortunately the OCaml compiler doesn't create output files
atomically.  Worse than that, it creates an intermediate version of
the output file, reads it back and then creates the final version.  It
seems if the build of std_utils.cmi reads this intermediate version.
In any case, Std_utils sees the wrong hash of the Guestfs_config
module.

The above only happens where we have a *.ml file without a
corresponding *.mli file.  That is because if there is a *.mli file,
ocamldep generates slightly different dependencies:

  guestfs_config.cmx [...] : guestfs_config.cmi guestfs_config.ml
  std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]

std_utils.cmx still depends on both files, but there is an extra rule
which ensures that guestfs_config.cmx isn't built in parallel with
guestfs_config.cmi.

I tested this change by running this command:

  $ while ( rm common/mlstdutils/.depend; make -C common/mlstdutils/ clean && make -C common/mlstdutils/ ) >& /tmp/log; do echo -n .; done

Before the change it would fail after about 100 iterations.  After the
change it ran for 10000s iterations and did not fail ever.

Updates commit 6d0ad49d5e.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
d38fcf280a tests: Check that every *.ml file has a corresponding *.mli file.
This is so that we can finally express OCaml dependencies correctly.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
a706ecb8f7 Make sure every *.ml file has a corresponding *.mli file. 2017-09-21 18:05:07 +01:00
Richard W.M. Jones
f04561a1fe common/mlstdutils: Remove unused Libdir module.
Last used in commit fd63d89644 (in 2013).
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
290e4608f4 daemon: Remove vestigial daemon/types.ml file.
This file was empty and unused, likely left over from refactorings
while the daemon inspection code was being written.

Fixes commit d5b6f1df5f.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
7e974b509c lib: Use guestfs_int_make_temp_path in a few more places.
There were various places in the library where we open coded making
temporary filenames.  This uses the utility function instead.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
0734afd41f lib: Fix guestfs_int_download_to_tmp.
Since commit 65cfecb0f5,
‘guestfs_int_download_to_tmp’ was buggy because it did not deal with
the case where a guest had multiple roots.  It cached the downloaded
file under a single name which was not distinguished by which root we
were looking at.  As a result, if you inspected (eg.) the icon on such
a guest, then in some circumstances the same icon could be returned
for all roots (incorrectly).

This changes the function in a few ways:

 - Don't cache downloaded files.  It wasn't a useful feature of the
   function in most scenarios.  Instead a new name is generated for
   every call.

 - Use guestfs_int_make_temp_path.

 - Allow an extension to be specified.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
0ec0af27c5 lib: Allow an extension to be specified in guestfs_int_make_temp_path.
In the function ‘guestfs_int_make_temp_path’, allow an optional
extension to be specified.  For example:

  r = guestfs_int_make_temp_path (g, "ls", "txt");

will create paths like ‘<tmpdir>/ls123.txt’.

NULL can also be passed for the extension, which means no extension.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
41df9aaf90 lib: Move guestfs_int_download_to_tmp and remove inspect.c.
Move the last remaining function ‘guestfs_int_download_to_tmp’ to
lib/inspect-icon.c (the main, but not only user of this function).
Then remove lib/inspect.c entirely.

This is not quite code motion because I updated the comment for the
function to reflect what it does in reality.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
26a5916a24 lib: Move guestfs_int_parse_unsigned_int to version.c
Just code motion, but this function seems to make more sense in
version.c since that is the main (but not only) user.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
439a5cf57b lib: Remove unused function guestfs_int_parse_unsigned_int_ignore_trailing.
Not used by any current code, removed.

Last use of the function was seen in
commit 65cfecb0f5.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
9fb5376db4 v2v: linux: Small fixes to documentation.
Also combine the match cases into one statement.

No functional change.
2017-09-21 11:52:23 +01:00
Pino Toscano
6037cb39d4 build: remove extra SUFFIXES from Makefile.am in 'po'
Commit c226656519 added them to
common-rules.mk, so they are redundant here.  Also, this assignment
overwrites the value set in common-rules.mk.
2017-09-21 09:22:29 +02:00
Richard W.M. Jones
64f49df747 docs: hivex >= 1.3.14 is required (RHBZ#1493048).
Bug reported by Jean-Christophe Manciot.
2017-09-20 17:57:47 +01:00
Richard W.M. Jones
8ae2281971 daemon: Fix configure check so it requires hivex.
Commit 4d3601eb4e made it clear that
ocaml-hivex is required, implying also that hivex is required.
However the configure test was still optional, and if you built
libguestfs without hivex you got a very long and confusing error in
the daemon/ subdirectory.

Thanks: Cédric Bosdonnat
2017-09-20 17:49:27 +01:00
Richard W.M. Jones
6a12065f34 daemon: inspect: Simplify parse_version_from_major_minor.
Thanks: Pino Toscano.

Updates commit cb7696a5d2.
2017-09-20 17:49:27 +01:00
Cédric Bosdonnat
5d323461b4 builder: remove useless fish dependency 2017-09-20 13:18:20 +02:00
Richard W.M. Jones
fe23cee926 builder: Add centos-7.4 image.
Requested by Gal Ben Haim.
2017-09-18 16:06:04 +01:00