=head1 NAME

virt-v2v-test-harness - Used to test virt-v2v against real test cases

=head1 SYNOPSIS

 open V2v_test_harness
 
 let test = "rhel_45_i386_fv"
 let test_plan = {
   default_plan with
     boot_plan = Boot_to_screenshot (test ^ "-login.ppm")
 }
 
 let () = run ~test ~test_plan ()

=head1 DESCRIPTION

L<virt-v2v(1)> converts guests from a foreign hypervisor to run on
KVM, managed by libvirt, OpenStack, oVirt, Red Hat Virtualisation
(RHV) or several other targets.

Virt-v2v-test-harness is a small library (module name:
C<V2v_test_harness>) used to run virt-v2v against a set of test cases
consisting of real virtual machines.

It acts as a test harness, taking a test case, running virt-v2v on it
(non-destructively), then test-booting the result.  It can ensure that
the test case converts successfully, boots successfully, and reaches a
milestone (such as a particular screenshot).  It can also test that
the conversion created, modified or deleted the expected files from
within the guest.

=head2 GETTING THE TEST CASES

Because the test cases are actual virtual machines, we split them into
two groups: test cases which are freely redistributable and those
which are proprietary.  The former are things like Fedora or CentOS
images, which are free software.  The latter are things like Windows
or Red Hat Enterprise Linux.

The freely redistributable test cases can be downloaded from:
L<http://git.annexia.org/?p=virt-v2v-test-cases-free.git>
I<B<not available yet>>

The proprietary test cases are available at
L<http://git.annexia.org/?p=virt-v2v-test-cases-nonfree.git> This
I<does not> contain the proprietary images themselves, which are not
made available to the public for licensing reasons.

The test cases consist of disk images which are very large, from 250
MB through to tens of gigabytes I<each>.  This means that distributing
test cases can be very time-consuming and expensive.  We use
L<git-annex(1)> to distribute the test images.

=head2 REQUIREMENTS

It’s recommended to use an idle machine for testing.  You will need
B<a lot of disk space> to run the tests, in excess of S<100 GB>.  You
should also ensure the test machine has plenty of RAM, at least
S<16 GB>.

=head2 GETTING THE TEST HARNESS

To run the test cases you must have the virt-v2v test harness.

The OCaml module is C<V2v_test_harness>.

The easiest way is to compile libguestfs from source (I<note> do not
install it).  The test harness will be in
C<libguestfs/v2v/test-harness>

It is also possible to install test harness as an OCaml module.

=head2 RUNNING THE TEST CASES

Once you have checked out the freely redistributed test cases from the
repository, do:

 ./configure [--with-test-harness=/path/to/libguestfs/v2v/test-harness]
 make
 make check -k

Using the I<-k> option is recommended so the test doesn't stop at the
first failure.

=head2 PARALLEL TESTS

You can run test cases in parallel by doing:

 make check -k -j<N>

(eg. I<-j2> for running up to 2 tests in parallel).  Be careful about
running too many parallel tests, as it can slow down each test enough
to cause false failures.

=head2 RUNNING TEST CASES AGAINST UPSTREAM VIRT-V2V

Using C<make check> picks up whatever C<virt-v2v> binary is on your
C<$PATH>.

If you have compiled libguestfs from source and want to test that
version of virt-v2v, use the libguestfs C<run> script (in the
top-level build directory of the libguestfs sources).  eg:

 ../libguestfs/run make check -k

=head1 WRITING NEW TEST CASES

If you are interested in writing test cases, it is suggested that you
start by downloading the freely redistributable test cases, or at
least look at them online.

Also you must have the virt-v2v test harness - see
L</GETTING THE TEST HARNESS> above.

=head2 FILES IN EACH TEST CASE

Each test case consists of:

=over 4

=item I<test>.img.xz

The disk image of the virtual machine before conversion.  Usually this
should be converted to raw format and xz-compressed.

=item I<test>.ova

Alternatively, an OVA, exported from VMware, may be used.

=item I<test>.xml

The libvirt XML used as input to virt-v2v.  See the discussion of
I<-i libvirtxml> in L<virt-v2v(1)>.

=item I<test>.ppm

An optional screenshot or screenshots.

You can supply zero or more "known good" screenshots which represent
intermediate steps where the guest is booting.  This is useful where a
guest sits for some time doing something, and lets the test harness
know that it should allow the guest to continue to boot.

You can supply zero or one "final" screenshot.  This is often a
screenshot of the login page which indicates that the guest booted
successfully.

The screenshots are captured using L<virsh(1)>.  Comparison of
screenshots against the test images is done using the ImageMagick
L<compare(1)> program.

=item I<test>.ml

The test itself - see below.

=back

=head2 WRITING THE TEST

The test file (F<*.ml>) is used to control the test harness, and
minimally it would look something like this:

 open V2v_test_harness
 
 let test = "short_name"
 
 let () = run ~test ()

That would instruct the test harness to:

=over 4

=item *

Uncompress F<I<short_name>.img.xz>

=item *

Run C<virt-v2v -i libvirtxml I<short_name>.xml [...]>

=item *

Boot the resulting guest and check that it writes to its disk and then
the disk becomes idle.

=back

The above is a rather simplistic test.  A more realistic test is to
ensure the guest reaches a final milestone (screenshot), eg. a login
page.  To do that you have to supply a C<~test_plan> parameter:

 open V2v_test_harness
 
 let test = "short_name"
 let test_plan = {
   default_plan with
     boot_plan = Boot_to_screenshot (test ^ ".ppm")
 }
 
 let () = run ~test ~test_plan ()

For an even better test, you can supply post-conversion and post-boot
test cases which examine the disk image (using libguestfs) to verify
that files have been created, modified or deleted as expected within
the disk image.  See F<V2v_test_harness.mli> for more information on
how to do that.

=head2 FILES GENERATED BY RUNNING THE TEST

When you run each test, the following files can be created:

=over 4

=item I<test>-I<yyyymmdd-hhmmss>.scrn

Screenshot(s) of the guest’s graphical console.  These are helpful
when writing tests or debugging test failures.

The screenshot format is Portable Pixmap (PPM).

=item I<test>.img

The uncompressed original disk image (before conversion).

=item I<test>-converted-sda

=item I<test>-converted.xml

The result of conversion, ie. after running virt-v2v but before
test-booting the guest.  See the L<virt-v2v(1)> manual page
description of I<-o local>.

The disk image format is qcow2.

=item I<test>-booted-sda

The disk image after test-booting.  This is a qcow2 file which uses
the I<test>-converted-sda file as a backing disk, in order to save
disk space.

=back

=head1 FILES

=over 4

=item F<$ocamllibdir/v2v_test_harness/v2v_test_harness.mli>

The test library interface.  Read this for detailed programming
documentation.

=item C<$ocamllibdir/v2v_test_harness/META>

The findlib META file allowing you to use the library from
L<ocamlfind(1)>.

=back

NB: To find the value of C<$ocamllibdir>, run C<ocamlc -where>

=head1 SEE ALSO

L<virt-v2v(1)>,
L<virt-p2v(1)>,
L<guestfs(3)>,
L<virsh(1)>,
L<compare(1)>,
L<git-annex(1)>,
L<http://libguestfs.org/>.

=head1 AUTHORS

Richard W.M. Jones L<http://people.redhat.com/~rjones/>

=head1 COPYRIGHT

Copyright (C) 2014-2018 Red Hat Inc.
