libguestfs/builder/virt-builder.pod

=encoding utf8

=begin html

<img src="virt-builder.svg" width="250"
  style="float: right; clear: right;" />

=end html

=head1 NAME

virt-builder - Build virtual machine images quickly

=head1 SYNOPSIS

 virt-builder [-o|--output DISKIMAGE] [--size SIZE] [--format raw|qcow2]
    [--attach ISOFILE]
    [--root-password SELECTOR]
    [--hostname HOSTNAME]
    [--install PKG,[PKG...]]
    [--mkdir DIR]
    [--write FILE:CONTENT]
    [--upload FILE:DEST]
    [--edit FILE:EXPR]
    [--delete FILE] [--scrub FILE]
    [--run SCRIPT] [--run-command 'CMD ARGS ...']
    [--firstboot SCRIPT] [--firstboot-command 'CMD ARGS ...']
    [--firstboot-install PKG,[PKG...]]
    os-version

 virt-builder -l|--list [--long]

 virt-builder --notes os-version

 virt-builder --print-cache

 virt-builder --cache-all-templates

 virt-builder --delete-cache

 virt-builder --get-kernel DISKIMAGE
    [--format raw|qcow2] [--output OUTPUTDIR]

=head1 DESCRIPTION

Virt-builder is a tool for quickly building new virtual machines.  You
can build a variety of VMs for local or cloud use, usually within a
few minutes or less.  Virt-builder also has many ways to customize
these VMs.  Everything is run from the command line and nothing
requires root privileges, so automation and scripting is simple.

Note that virt-builder does not install guests from scratch.  It takes
cleanly prepared, digitally signed OS templates and customizes them.
This approach is used because it is much faster, but if you need to do
fresh installs you may want to look at L<virt-install(1)> and
L<oz-install(1)>.

The easiest way to get started is by looking at the examples in the
next section.

=head1 EXAMPLES

=head2 List the virtual machines available

 virt-builder --list

will list out the operating systems available to install.  A selection
of freely redistributable OSes is available as standard.  You can add
your own too (see below).

After choosing a guest from the list, you may want to see if there
are any installation notes:

 virt-builder --notes fedora-20

=head2 Build a virtual machine

 virt-builder fedora-20

will build a Fedora 20 image.  This will have all default
configuration (minimal size, no user accounts, random root password,
only the bare minimum installed software, etc.).

You I<do not> need to run this command as root.

The first time this runs it has to download the template over the
network, but this gets cached (see L</CACHING>).

The name of the output file is derived from the template name, so
above it will be C<fedora-20.img>.  You can change the output filename
using the I<-o> option:

 virt-builder fedora-20 -o mydisk.img

You can also use the I<-o> option to write to existing devices or
logical volumes.

 virt-builder fedora-20 --format qcow2

As above, but write the output in qcow2 format to C<fedora-20.qcow2>.

 virt-builder fedora-20 --size 20G

As above, but the output size will be 20 GB.  The guest OS is resized
as it is copied to the output (automatically, using
L<virt-resize(1)>).

=head2 Setting the root password

 virt-builder fedora-20 --root-password file:/tmp/rootpw

Create a Fedora 20 image.  The root password is taken from the file
C</tmp/rootpw>.

Note if you I<don't> set I<--root-password> then the guest is given
a I<random> root password.

You can also create user accounts.  See L</USERS AND PASSWORDS> below.

=head2 Set the hostname

 virt-builder fedora-20 --hostname virt.example.com

Set the hostname to C<virt.example.com>.

=head2 Installing software

To install packages from the ordinary (guest) software repository
(eg. yum or apt):

 virt-builder fedora-20 --install "inkscape,@Xfce Desktop"

(In Fedora, C<@> is used to install groups of packages.  On Debian
you would install a meta-package instead.)

=head2 Customizing the installation

There are many options that let you customize the installation.  These
include: I<--run>/I<--run-command>, which run a shell script or
command while the disk image is being generated and lets you add or
edit files that go into the disk image.
I<--firstboot>/I<--firstboot-command>, which let you add
scripts/commands that are run the first time the guest boots.
I<--edit> to edit files.  I<--upload> to upload files.

For example:

 cat <<'EOF' > /tmp/yum-update.sh
 yum -y update
 EOF
 
 virt-builder fedora-20 --firstboot /tmp/yum-update.sh

or simply:

 virt-builder fedora-20 --firstboot-command 'yum -y update'

which makes the L<yum(8)> C<update> command run once the first time
the guest boots.

Or:

 virt-builder fedora-20 --edit '/etc/yum.conf: s/gpgcheck=1/gpgcheck=0/'

which edits C</etc/yum.conf> inside the disk image (during disk image
creation, long before boot).

You can combine these options, and have multiple options of all types.

=head1 OPTIONS

=over 4

=item B<--help>

Display help.

=item B<--attach> ISOFILE

During the customization phase, the given disk is attached to the
libguestfs appliance.  This is used to provide extra software
repositories or other data for customization.

You probably want to ensure the volume(s) or filesystems in the
attached disks are labelled (or use an ISO volume name) so that you
can mount them by label in your run-scripts:

 mkdir /tmp/mount
 mount LABEL=EXTRA /tmp/mount

You can have multiple I<--attach> options, and the format can be any
disk format (not just an ISO).

See also: I<--run>,
L</Installing packages at build time from a side repository>,
L<virt-make-fs(1)>.

=item B<--attach-format> FORMAT

Specify the disk format for the next I<--attach> option.  The
C<FORMAT> is usually C<raw> or C<qcow2>.  Use C<raw> for ISOs.

=item B<--cache> DIR

=item B<--no-cache>

I<--cache> DIR sets the directory to use/check for cached template
files.  If not set, defaults to either
C<$XDG_CACHE_HOME/virt-builder/> or C<$HOME/.cache/virt-builder/>.

I<--no-cache> disables template caching.

=item B<--cache-all-templates>

Download all templates to the cache and then exit.  See L</CACHING>.

Note this doesn't cache everything.  More templates might be uploaded.
Also this doesn't cache packages (the I<--install> option).

=item B<--check-signature>

=item B<--no-check-signature>

Check/don't check the digital signature of the OS template.  The
default is to check the signature and exit if it is not correct.
Using I<--no-check-signature> bypasses this check.

See also I<--fingerprint>.

=item B<--curl> CURL

Specify an alternate L<curl(1)> binary.  You can also use this to add
curl parameters, for example to disable https certificate checks:

 virt-builder --curl "curl --insecure" [...]

=item B<--delete> FILE

=item B<--delete> DIR

Delete a file from the guest.  Or delete a directory (and all its
contents, recursively).

See also: I<--upload>, I<--scrub>.

=item B<--delete-cache>

Delete the template cache.  See L</CACHING>.

=item B<--edit> FILE:EXPR

Edit C<FILE> using the Perl expression C<EXPR>.

Be careful to properly quote the expression to prevent it from
being altered by the shell.

Note that this option is only available when Perl 5 is installed.

See L<virt-edit(1)/NON-INTERACTIVE EDITING>.

=item B<--fingerprint> 'AAAA BBBB ...'

Check that the index and templates are signed by the key with the
given fingerprint.  (The fingerprint is a long string, usually written
as 10 groups of 4 hexadecimal digits).

You can give this option multiple times.  If you have multiple source
URLs, then you can have either no fingerprint, one fingerprint or
multiple fingerprints.  If you have multiple, then each must
correspond 1-1 with a source URL.

The default fingerprint (if none are supplied) is
S<F777 4FB1 AD07 4A7E 8C87 67EA 9173 8F73 E1B7 68A0>
(which is S<Richard W.M. Jones's> key).

You can also set the C<VIRT_BUILDER_FINGERPRINT> environment variable.

=item B<--firstboot> SCRIPT

=item B<--firstboot-command> 'CMD ARGS ...'

Install C<SCRIPT> inside the guest, so that when the guest first boots
up, the script runs (as root, late in the boot process).

The script is automatically chmod +x after installation in the guest.

The alternative version I<--firstboot-command> is the same, but it
conveniently wraps the command up in a single line script for you.

You can have multiple I<--firstboot> and I<--firstboot-command>
options.  They run in the same order that they appear on the command
line.

See also I<--run>.

=item B<--firstboot-install> PKG[,PKG,...]

Install the named packages (a comma-separated list).  These are
installed when the guest first boots using the guest's package manager
(eg. apt, yum, etc.) and the guest's network connection.

For an overview on the different ways to install packages, see
L</INSTALLING PACKAGES>.

=item B<--format> qcow2

=item B<--format> raw

For ordinary builds, this selects the output format.  The default is I<raw>.

With I<--get-kernel> this specifies the input format.

=item B<--get-kernel> IMAGE

This option extracts the kernel and initramfs from a previously built
disk image called C<IMAGE> (in fact it works for any VM disk image,
not just ones built using virt-builder).

The kernel and initramfs are written to the current directory, unless
you also specify the I<--output> C<outputdir> B<directory> name.

The format of the disk image is automatically detected unless you
specify it by using the I<--format> option.

In the case where the guest contains multiple kernels, the one with
the highest version number is chosen.  To extract arbitrary kernels
from the disk image, see L<guestfish(1)>.  To extract the entire
C</boot> directory of a guest, see L<virt-copy-out(1)>.

=item B<--gpg> GPG

Specify an alternate L<gpg(1)> (GNU Privacy Guard) binary.  You can
also use this to add gpg parameters, for example to specify an
alternate home directory:

 virt-builder --gpg "gpg --homedir /tmp" [...]

=item B<--hostname> HOSTNAME

Set the hostname of the guest to C<HOSTNAME>.  You can use a
dotted hostname.domainname (FQDN) if you want.

=item B<--install> PKG[,PKG,...]

Install the named packages (a comma-separated list).  These are
installed during the image build using the guest's package manager
(eg. apt, yum, etc.) and the host's network connection.

For an overview on the different ways to install packages, see
L</INSTALLING PACKAGES>.

=item B<-l>

=item B<--list>

=item B<--list --long>

List available templates.

The alternative I<--list --long> form shows lots more details about
each operating system option.

See also: I<--source>, I<--notes>, L</CREATING YOUR OWN TEMPLATES>.

=item B<--no-logfile>

Scrub C<builder.log> (log file from build commands) from the image
after building is complete.  If you don't want to reveal precisely how
the image was built, use this option.

See also: L</LOG FILE>.

=item B<-m> MB

=item B<--memsize> MB

Change the amount of memory allocated to I<--run> scripts.  Increase
this if you find that I<--run> scripts or the I<--install> option are
running out of memory.

The default can be found with this command:

 guestfish get-memsize

=item B<--mkdir> DIR

Create a directory in the guest.

This uses S<C<mkdir -p>> so any intermediate directories are created,
and it also works if the directory already exists.

=item B<--network>

=item B<--no-network>

Enable or disable network access from the guest during the installation.

Enabled is the default.  Use I<--no-network> to disable access.

The network only allows outgoing connections and has other minor
limitations.  See L<virt-rescue(1)/NETWORK>.

If you use I<--no-network> then certain other options such as
I<--install> will not work.

This does not affect whether the guest can access the network once it
has been booted, because that is controlled by your hypervisor or
cloud environment and has nothing to do with virt-builder.

Generally speaking you should I<not> use I<--no-network>.  But here
are some reasons why you might want to:

=over 4

=item 1.

Because the libguestfs backend that you are using doesn't support the
network.  (See: L<guestfs(3)/BACKEND>).

=item 2.

Any software you need to install comes from an attached ISO, so you
don't need the network.

=item 3.

You don't want untrusted guest code trying to access your host network
when running virt-builder.  This is particularly an issue when you
don't trust the source of the operating system templates.  (See
L</SECURITY> below).

=item 4.

You don't have a host network (eg. in secure/restricted environments).

=back

=item B<--no-sync>

Do not sync the output file on exit.

Virt-builder fsync's the output file or disk image when it exits.

The reason is that qemu/KVM's default caching mode is C<none> or
C<directsync>, both of which bypass the host page cache.  Therefore
these would not work correctly if you immediately started the guest
after running virt-builder - they would not see the complete output
file.  (Note that you should not use these caching modes - they are
fundamentally broken for this and other reasons.)

If you are not using these broken caching modes, you can use
I<--no-sync> to avoid this unnecessary sync and gain considerable
extra performance.

=item B<--notes> os-version

List any notes associated with this guest, then exit (this does not do
the install).

=item B<-o> filename

=item B<--output> filename

Write the output to C<filename>.  If you don't specify this option,
then the output filename is generated by taking the C<os-version>
string and adding C<.img> (for raw format) or C<.qcow2> (for qcow2
format).

Note that the output filename could be a device, partition or logical
volume.

When used with I<--get-kernel>, this option specifies the output
directory.

=item B<--password-crypto> password-crypto

Set the password encryption to C<md5>, C<sha256> or C<sha512>.

C<sha256> and C<sha512> require glibc E<ge> 2.7 (check crypt(3) inside
the guest).

C<md5> will work with relatively old Linux guests (eg. RHEL 3), but
is not secure against modern attacks.

The default is C<sha512> unless libguestfs detects an old guest that
didn't have support for SHA-512, in which case it will use C<md5>.
You can override libguestfs by specifying this option.

=item B<--print-cache>

Print information about the template cache.  See L</CACHING>.

=item B<--quiet>

Don't print ordinary progress messages.

=item B<--root-password> SELECTOR

Set the root password.

See L</USERS AND PASSWORDS> below for the format of the C<SELECTOR>
field, and also how to set up user accounts.

Note if you I<don't> set I<--root-password> then the guest is given
a I<random> root password.

=item B<--run> SCRIPT

=item B<--run-command> 'CMD ARGS ...'

Run the shell script (or any program) called C<SCRIPT> on the disk
image.  The script runs virtualized inside a small appliance, chrooted
into the guest filesystem.

The script is automatically chmod +x.

If libguestfs supports it then a limited network connection is
available but it only allows outgoing network connections.  You can
also attach data disks (eg. ISO files) as another way to provide data
(eg. software packages) to the script without needing a network
connection (I<--attach>).  You can also upload data files (I<--upload>).

The alternative version I<--run-command> is the same, but it
conveniently wraps the command up in a single line script for you.

You can have multiple I<--run> and I<--run-command> options.  They run
in the same order that they appear on the command line.

See also: I<--firstboot>, I<--attach>, I<--upload>.

=item B<--scrub> FILE

Scrub a file from the guest.  This is like I<--delete> except that:

=over 4

=item *

It scrubs the data so a guest could not recover it.

=item *

It cannot delete directories, only regular files.

=back

=item B<--size> SIZE

Select the size of the output disk, where the size can be specified
using common names such as C<32G> (32 gigabytes) etc.

Virt-builder will resize filesystems inside the disk image
automatically.

If the size is not specified, then one of two things happens.  If the
output is a file, then the size is the same as the template.  If the
output is a device, partition, etc then the size of that device is
used.

=item B<--smp> N

Enable N E<ge> 2 virtual CPUs for I<--run> scripts to use.

=item B<--source> URL

Set the source URL to look for indexes.

You can give this option multiple times to specify multiple sources.
If not specified it defaults to
L<http://libguestfs.org/download/builder/index.asc>

See also L</CREATING YOUR OWN TEMPLATES> below.

You can also set the C<VIRT_BUILDER_SOURCE> environment variable.

Note that you should not point I<--source> to sources that you don't
trust (unless the source is signed by someone you do trust).  See also
the I<--no-network> option.

=item B<--upload> FILE:DEST

Upload local file C<FILE> to destination C<DEST> in the disk image.
File owner and permissions from the original are preserved, so you
should set them to what you want them to be in the disk image.

C<DEST> could be the final filename.  This can be used to rename
the file on upload.

If C<DEST> is a directory name (which must already exist in the guest)
then the file is uploaded into that directory, and it keeps the same
name as on the local filesystem.

See also: I<--mkdir>, I<--delete>, I<--scrub>.

=item B<-v>

=item B<--verbose>

Enable debug messages and/or produce verbose output.

When reporting bugs, use this option and attach the complete output to
your bug report.

=item B<-V>

=item B<--version>

Display version number and exit.

=item B<--write> FILE:CONTENT

Write C<CONTENT> to C<FILE>.

=back

=head1 REFERENCE

=head2 INSTALLING PACKAGES

There are several approaches to installing packages or applications in
the guest which have different trade-offs.

=head3 Installing packages at build time

If the guest OS you are installing is similar to the host OS (eg.
both are Linux), and if libguestfs supports network connections, then
you can use I<--install> to install packages like this:

 virt-builder fedora-20 --install inkscape

This uses the guest's package manager and the host's network
connection.

=head3 Installing packages at first boot

Another option is to install the packages when the guest first boots:

 virt-builder fedora-20 --firstboot-install inkscape

This uses the guest's package manager and the guest's network
connection.

The downsides are that it will take the guest a lot longer to boot
first time, and there's nothing much you can do if package
installation fails (eg. if a network problem means the guest can't
reach the package repositories).

=head3 Installing packages at build time from a side repository

If the software you want to install is not available in the main
package repository of the guest, then you can add a side repository.
Usually this is presented as an ISO (CD disk image) file containing
extra packages.

Create a script that mounts the ISO and sets up the repository.  For
yum, create /tmp/install.sh containing:

 mkdir /tmp/mount
 
 # Assume the volume label of the CD is 'EXTRA':
 mount LABEL=EXTRA /tmp/mount
 
 cat <<'EOF' > /etc/yum.repos.d/extra.repo
 [extra]
 name=extra
 baseurl=file:///tmp/mount
 enabled=1
 EOF
 
 yum -y install famousdatabase

For apt, create /tmp/install.sh containing:

 mkdir /tmp/mount
 
 # Assume the volume label of the CD is 'EXTRA':
 mount LABEL=EXTRA /tmp/mount
 
 apt-cdrom -d=/tmp/mount add
 apt-get -y install famousdatabase

Use the I<--attach> option to attach the CD:

 virt-builder fedora 20 --attach extra.iso --run /tmp/install.sh

=head2 USERS AND PASSWORDS

The I<--root-password> option is used to change the root password
(otherwise a random password is used).  This option takes a password
C<SELECTOR> in one of the following formats:

=over 4

=item B<--root-password> file:FILENAME

Read the root password from C<FILENAME>.  The whole first line
of this file is the replacement password.  Any other lines are
ignored.  You should create the file with mode 0600 to ensure
no one else can read it.

=item B<--root-password> password:PASSWORD

Set the root password to the literal string C<PASSWORD>.

B<Note: this is not secure> since any user on the same machine can
see the cleartext password using L<ps(1)>.

=item B<--root-password> random

Choose a random password, which is printed on stdout.  The password
has approximately 120 bits of randomness.

This is the default.

=item B<--root-password> disabled

The root account password is disabled.  This is like putting C<*>
in the password field.

=item B<--root-password> locked:file:FILENAME

=item B<--root-password> locked:password:PASSWORD

=item B<--root-password> locked:random

The root account is locked, but a password is placed on the
account.  If first unlocked (using C<passwd -u>) then logins will
use the given password.

=item B<--root-password> locked

=item B<--root-password> locked:disabled

The root account is locked I<and> password is disabled.

=back

=head3 Creating user accounts

To create user accounts, use the L<useradd(8)> command with
L<--firstboot-command> like this:

 virt-builder --firstboot-command \
    'useradd -m -p "" rjones ; chage -d 0 rjones'

The above command will create an C<rjones> account with no password,
and force the user to set a password when they first log in.  There
are other ways to manage passwords, see L<useradd(8)> for details.

=head2 LOG FILE

Scripts and package installation that runs at build time (I<--run>,
I<--run-command>, I<--install>, but I<not> firstboot) is logged in one
of the following locations:

=over 4

=item C</tmp/builder.log>

On Linux, BSD and other guests.

=item C<C:\Temp\builder.log>

On Windows, DOS guests.

=item C</builder.log>

If C</tmp> or C<C:\Temp> is missing.

=back

If you don't want the log file to appear in the final image, then
use the I<--no-logfile> command line option.

=head2 INSTALLATION PROCESS

When you invoke virt-builder, installation proceeds as follows:

=over 4

=item *

The template image is downloaded.

If the template image is present in the cache, the cached version
is used instead.  (See L</CACHING>).

=item *

The template signature is checked.

=item *

The template is uncompressed to a tmp file.

=item *

The template image is resized into the destination, using
L<virt-resize(1)>.

=item *

Extra disks are attached (I<--attach>).

=item *

A new random seed is generated for the guest.

=item *

The hostname is set (I<--hostname>).

=item *

The root password is changed (I<--root-password>).

=item *

Packages are installed (I<--install>).

=item *

Directories are created (I<--mkdir>).

=item *

Files are written (I<--write>).

=item *

Files are uploaded (I<--upload>).

=item *

Files are edited (I<--edit>).

=item *

Files are deleted (I<--delete>, I<--scrub>).

=item *

Firstboot scripts are installed (I<--firstboot>,
I<--firstboot-command>, I<--firstboot-install>).

Note that although firstboot scripts are installed at this step, they
do not run until the guest is booted first time.  Firstboot scripts
will run in the order they appear on the command line.

=item *

Scripts are run (I<--run>, I<--run-command>).

Scripts run in the order they appear on the command line.

=back

=head2 IMPORTING THE DISK IMAGE

=head3 Importing into libvirt

Import the disk image into libvirt using L<virt-install(1)>
I<--import> option.

 virt-install --import \
   --name guest --ram 2048 --disk path=disk.img,format=raw

Notes:

=over 4

=item 1.

You I<must> specify the correct format.  The format is C<raw> unless
you used virt-builder's I<--format> option.

=item 2.

You can run virt-install as root or non-root.  Each works slightly
differently because libvirt manages a different set of virtual
machines for each user.  In particular virt-manager normally shows the
root-owned VMs, whereas Boxes shows the user-owned VMs, and other
tools probably work differently as well.

=back

=begin comment

=head3 Importing into OpenStack

XXX

=end comment

=head3 Booting directly using qemu or KVM

The qemu command line is not very stable or easy to use, hence libvirt
should be used if possible.  However a command line similar to the
following could be used to boot the virtual machine:

 qemu-system-x86_64 \
   -machine accel=kvm:tcg \
   -cpu host \
   -m 2048 \
   -drive file=disk.img,format=raw,if=virtio

As with libvirt, it is very important that the correct format is
chosen.  It will be C<raw> unless the I<--format> option was used.

=head2 DEBUGGING BUILDS

If virt-builder fails with an error, then enable debugging (I<-v>) and
report a bug (see L</BUGS> below).

If virt-builder is successful but the image doesn't work, here are
some things to try:

=over 4

=item Use virt-rescue

Run L<virt-rescue(1)> on the disk image:

 virt-rescue -a disk.img

This gives you a rescue shell.  You can mount the filesystems from the
disk image on C</sysroot> and examine them using ordinary Linux
commands.  You can also chroot into the guest to reinstall the
bootloader.  The virt-rescue man page has a lot more information and
examples.

=item Use guestfish

Run L<guestfish(1)> on the disk image:

 guestfish -a disk.img -i

Use guestfish commands like C<ll /directory> and C<cat /file> to
examine directories and files.

=item Use guestmount

Mount the disk image safely on the host using FUSE and L<guestmount(1)>:

 mkdir /tmp/mp
 guestmount -a disk.img -i /tmp/mp
 cd /tmp/mp

To unmount the disk image do:

 fusermount -u /tmp/mp

=item Add a serial console

If the guest hangs during boot, it can be helpful to add a serial
console to the guest, and direct kernel messages to the serial
console.  Adding the serial console will involve looking at the
documentation for your hypervisor.  To direct kernel messages to the
serial console, add the following on the kernel command line:

 console=tty0 console=ttyS0,115200

=back

=head2 CREATING YOUR OWN TEMPLATES

For serious virt-builder use, you may want to create your own
repository of templates.

=head3 Libguestfs.org repository

Out of the box, virt-builder downloads the file
L<http://libguestfs.org/download/builder/index.asc> which is an index
of available templates plus some information about each one, wrapped
up in a digital signature.  The command C<virt-builder --list> lists
out the information in this index file.

The templates hosted on libguestfs.org were created using shell
scripts, kickstart files and preseed files which can be found in the
libguestfs source tree, in C<builder/website>.

=head3 Setting up the repository

You can set up your own site containing an index file and some
templates, and then point virt-builder at the site by using the
I<--source> option:

 virt-builder --source https://example.com/builder/index.asc \
    --fingerprint 'AAAA BBBB ...' \
    --list

(Note setting the environment variables C<VIRT_BUILDER_SOURCE> and
C<VIRT_BUILDER_FINGERPRINT> may be easier to type!)

You can host this on any web or FTP server, or a local or network
filesystem.

=head3 Setting up a GPG key

If you don't have a GnuPG key, you will need to set one up.  (Strictly
speaking this is optional, but if your index and template files are
not signed then virt-builder users will have to use the
I<--no-check-signature> flag every time they use virt-builder.)

To create a key, see the GPG manual
L<http://www.gnupg.org/gph/en/manual.html>.

Export your GPG public key and add it to the keyring of all
virt-builder users:

 gpg --export -a "you@example.com" > pubkey
 
 # For each virt-builder user:
 gpg --import pubkey

Also find the fingerprint of your key:

 gpg --list-keys --fingerprint

=head3 Create the templates

There are many ways to create the templates.  For example you could
clone existing guests (see L<virt-sysprep(1)>), or you could install a
guest by hand (L<virt-install(1)>).  To see how the templates were
created for virt-builder, look at the scripts in C<builder/website>

For best results when compressing the templates, use the following xz
options (see L<nbdkit-xz-plugin(1)> for further explanation):

 xz --best --block-size=16777216 disk

=head3 Creating and signing the index file

The index file has a simple text format (shown here without the
digital signature):

 [fedora-18]
 name=Fedora® 18
 osinfo=fedora18
 file=fedora-18.xz
 checksum[sha512]=...
 format=raw
 size=6442450944
 compressed_size=148947524
 expand=/dev/sda3
 
 [fedora-19]
 name=Fedora® 19
 osinfo=fedora19
 file=fedora-19.xz
 checksum[sha512]=...
 revision=3
 format=raw
 size=4294967296
 compressed_size=172190964
 expand=/dev/sda3

The part in square brackets is the C<os-version>, which is the same
string that is used on the virt-builder command line to build that OS.

After preparing the C<index> file in the correct format, clearsign it
using the following command:

 gpg --clearsign --armor index

This will create the final file called C<index.asc> which can be
uploaded to the server (and is the I<--source> URL).  As noted above,
signing the index file is optional, but recommended.

The following fields can appear:

=over 4

=item C<name=NAME>

The user-friendly name of this template.  This is displayed in the
I<--list> output but is otherwise not significant.

=item C<osinfo=ID>

This optional field maps the operating system to the associated
libosinfo ID.  Virt-builder does not use it (yet).

=item C<file=PATH>

The path (relative to the index) of the xz-compressed template.

Note that absolute paths or URIs are B<not> permitted here.  This is
because virt-builder has a "same origin" policy for templates so they
cannot come from other servers.

=item C<sig=PATH>

B<This option is deprecated>.  Use the checksum field instead.

The path (relative to the index) of the GPG detached signature of the
xz file.

Note that absolute paths or URIs are B<not> permitted here.  This is
because virt-builder has a "same origin" policy for templates so they
cannot come from other servers.

The file can be created as follows:

 gpg --detach-sign --armor -o disk.xz.sig disk.xz

=item C<checksum[sha512]=7b882fe9b82eb0fef...>

The SHA-512 checksum of the B<compressed> file is checked after it is
downloaded.  To work out the signature, do:

 sha512sum disk.xz

Note if you use this, you don't need to sign the file, ie. don't use
C<sig>.  This option overrides C<sig>.

=item C<checksum=7b882fe9b82eb0fef...>

C<checksum> is an alias for C<checksum[sha512]>.

If you need to interoperate with virt-builder = 1.24.0 then you have
to use C<checksum> because that version would give a parse error with
square brackets and numbers in the key of a field.  This is fixed in
virt-builder E<ge> 1.24.1.

=item C<revision=N>

The revision is an integer which is used to control the template
cache.  Increasing the revision number causes clients to download the
template again even if they have a copy in the cache.

The revision number is optional.  If omitted it defaults to C<1>.

=item C<format=raw>

=item C<format=qcow2>

Specify the format of the disk image (before it was compressed).  If
not given, the format is autodetected, but generally it is better to
be explicit about the intended format.

Note this is the source format, which is different from the
I<--format> option (requested output format).  Virt-builder does
on-the-fly conversion from the source format to the requested output
format.

=item C<size=NNN>

The virtual size of the image in bytes.  This is the size of the image
when uncompressed.  If using a non-raw format such as qcow2 then it
means the virtual disk size, not the size of the qcow2 file.

This field is required.

Virt-builder also uses this as the minimum size that users can request
via the I<--size> option, or as the default size if there is no
I<--size> option.

=item C<compressed_size=NNN>

The compressed size of the disk image in bytes.  This is just used for
information (when using I<--list --long>).

=item C<expand=/dev/sdaX>

When expanding the image to its final size, instruct L<virt-resize(1)>
to expand the named partition in the guest image to fill up all
available space.  This works like the virt-resize I<--expand> option.

You should usually put the device name of the guest's root filesystem here.

It's a good idea to use this, but not required.  If the field is
omitted then virt-resize will create an extra partition at the end of
the disk to cover the free space, which is much less user-friendly.

=item C<lvexpand=/dev/VolGroup/LogVol>

When expanding the image to its final size, instruct L<virt-resize(1)>
to expand the named logical volume in the guest image to fill up all
available space.  This works like the virt-resize I<--lv-expand> option.

If the guest uses LVM2 you should usually put the LV of the guest's
root filesystem here.  If the guest does not use LVM2 or its root
filesystem is not on an LV, don't use this option.

=item C<notes=NOTES>

Any notes that go with this image, especially notes describing what
packages are in the image, how the image was prepared, and licensing
information.

This information is shown in the I<--notes> and I<--list> I<--long> modes.

You can use multi-line notes here by indenting each new line with at
least one character of whitespace (even on blank lines):

 notes=This image was prepared using
  the following kickstart script:
                                <-- one space at beginning of line
  timezone Europe/London
  part /boot --fstype ext3

=item C<hidden=true>

Using the hidden flag prevents the template from being listed by the
I<--list> option (but it is still installable).  This is used for test
images.

=back

=head3 Running virt-builder against the alternate repository

Ensure each virt-builder user has imported your public key into
their gpg keyring (see above).

Each virt-builder user should export these environment variables:

=over 4

=item *

C<VIRT_BUILDER_SOURCE> to point to the URL of the C<index.asc> file.

=item *

C<VIRT_BUILDER_FINGERPRINT> to contain the fingerprint (long hex
string) of the user who signed the index file and the templates.

=back

Now run virt-builder commands as normal, eg:

 virt-builder --list --long

 virt-builder os-version

To debug problems, add the C<-v> option to these commands.

=head3 Running virt-builder against multiple sources

It is possible to use multiple sources with virt-builder.  Use either
multiple I<--source> and/or I<--fingerprint> options, or a
comma-separated list in the C<VIRT_BUILDER_SOURCE> /
C<VIRT_BUILDER_FINGERPRINT> environment variables:

 virt-builder \
   --source http://example.com/s1/index.asc \
   --source http://example.com/s2/index.asc

or equivalently:

 export VIRT_BUILDER_SOURCE=http://example.com/s1/index.asc,http://example.com/s2/index.asc
 virt-builder [...]

You can provide N, 1 or 0 fingerprints.  In the case where you
provide N fingerprints, N = number of sources and there is a 1-1
correspondence between each source and each fingerprint:

 virt-builder \
   --source http://example.com/s1/index.asc --fingerprint '0123 ...' \
   --source http://example.com/s2/index.asc --fingerprint '9876 ...'

In the case where you provide 1 fingerprint, the same fingerprint
is used for all sources.

In the case where you provide no fingerprints, the default fingerprint
built into virt-builder is used for all sources.

=head3 Licensing of templates

You should be aware of the licensing of images that you distribute.
For open source guests, provide a link to the source code in the
C<notes> field and comply with other requirements (eg. around
trademarks).

=head3 Formal specification of the index file

The index file format has a formal specification defined by the flex
scanner and bison parser used to parse the file.  This can be found in
the following files in the libguestfs source tree:

 builder/index-scan.l
 builder/index-parse.y

A tool called L<virt-index-validate(1)> is available to validate the
index file to ensure it is correct.

Note that the parser and tool can work on either the signed or
unsigned index file (ie. C<index> or C<index.asc>).

The index is always encoded in UTF-8.

=head2 CACHING

Since the templates are usually very large, downloaded templates are
cached in the user's home directory.

The location of the cache is C<$XDG_CACHE_HOME/virt-builder/> or
C<$HOME/.cache/virt-builder>.

You can print out information about the cache directory, including
which guests are currently cached, by doing:

 virt-builder --print-cache

The cache can be deleted if you want to save space by doing:

 virt-builder --delete-cache

You can download all (current) templates to the local cache by doing:

 virt-builder --cache-all-templates

To disable the template cache, use I<--no-cache>.

Only templates are cached.  The index and detached digital signatures
are not cached.

Virt-builder uses L<curl(1)> to download files and it also uses the
current C<http_proxy> (etc) settings when installing packages
(I<--install>).  You may therefore want to set those environment
variables in order to maximize the amount of local caching that
happens.  See L</ENVIRONMENT VARIABLES> and L<curl(1)>.

=head2 DIGITAL SIGNATURES

Virt-builder uses GNU Privacy Guard (GnuPG or gpg) to verify that the
index and templates have not been tampered with.

The source points to an index file, which is optionally signed.

Virt-builder downloads the index and checks that the signature is
valid and the signer's fingerprint matches the specified fingerprint
(ie. I<--fingerprint>, C<VIRT_BUILDER_FINGERPRINT>, or a built-in
fingerprint, in that order).

For checking against the built-in public key/fingerprint, this
requires importing the public key into the user's local gpg keyring
(that's just the way that gpg works).

When a template is downloaded, its signature is checked in the same
way.

Although the signatures are optional, if you don't have them then
virt-builder users will have to use I<--no-check-signature> on the
command line.  This prevents an attacker from replacing the signed
index file with an unsigned index file and having virt-builder
silently work without checking the signature.  In any case it is
highly recommended that you always create signed index and templates.

=head2 ARCHITECTURE

Virt-builder can build a guest for any architecture no matter what the
host architecture is.  For example an x86-64 guest on an ARM host.

However certain options may not work correctly, specifically options
that require running commands in the guest during the build process:
I<--install>, I<--run>, I<--run-command>.  You may need to replace
these with their firstboot-equivalents.

An x86-64 host building 32 bit i686 guests should work without any
special steps.

=head2 SECURITY

Virt-builder does not need to run as root (in fact, should not be run
as root), and doesn't use setuid, C<sudo> or any similar mechanism.

I<--install>, I<--run> and I<--run-command> are implemented using an
appliance (a small virtual machine) so these commands do not run on
the host.  If you are using the libguestfs libvirt backend and have
SELinux enabled then the virtual machine is additionally encapsulated
in an SELinux container (sVirt).

However these options will have access to the host's network and since
the template may contain untrusted code, the code might try to access
host network resources which it should not.  You can use
I<--no-network> to prevent this.

Firstboot commands run in the context of the guest when it is booted,
and so the security of your hypervisor / cloud should be considered.

Virt-builder injects a random seed into every guest which it builds.
This helps to ensure that TCP sequence numbers, UUIDs, ssh host keys
etc are truly random when the guest boots.

You should check digital signatures and not ignore any signing errors.

=head2 PERFORMANCE

The most important aspect of getting good performance is caching.
Templates gets downloaded into the cache the first time they are used,
or if you use the I<--cache-all-templates> option.  See L</CACHING>
above for further information.

Packages required for the I<--install> option are downloaded using the
host network connection.  Setting the C<http_proxy>, C<https_proxy>
and C<ftp_proxy> environment variables to point to a local web cache
may ensure they only need to be downloaded once.  You can also try
using a local package repository, although this can be complex to set
up and varies according to which Linux distro you are trying to
install.

=head3 Skipping virt-resize

Virt-builder can skip the virt-resize step under certain conditions.
This makes virt-builder much faster.  The conditions are:

=over 4

=item *

the output must be a regular file (not a block device), B<and>

=item *

the user did B<not> use the I<--size> option, B<and>

=item *

the output format is the same as the template format (usually raw).

=back

=head3 pxzcat

Virt-builder uses an internal implementation of pxzcat (parallel
xzcat) if liblzma was found at build time.  If liblzma was not found
at build time, regular C<xzcat> is used which is single-threaded.

=head3 User-Mode Linux

You can use virt-builder with the User-Mode Linux (UML) backend.  This
may be faster when running virt-builder inside a virtual machine
(eg. in the cloud).

To enable the UML backend, read the instructions in
L<guestfs(3)/USER-MODE LINUX BACKEND>.

Currently you have to use the I<--no-network> option.  This should be
fixed in a future version.

The qcow2 output format is not supported by UML.  You can only create
raw-format guests.

=head1 ENVIRONMENT VARIABLES

For other environment variables which affect all libguestfs programs,
see L<guestfs(3)/ENVIRONMENT VARIABLES>.

=over 4

=item C<http_proxy>

=item C<https_proxy>

=item C<no_proxy>

Set the proxy for downloads.  These environment variables (and more)
are actually interpreted by L<curl(1)>, not virt-builder.

=item C<HOME>

Used to determine the location of the template cache.  See L</CACHING>.

=item C<VIRT_BUILDER_FINGERPRINT>

Set the default value for the GPG signature fingerprint or
comma-separated list of fingerprints (see I<--fingerprint> option).

=item C<VIRT_BUILDER_SOURCE>

Set the default value for the source URL (or comma-separated list of
URLs) for the template repository (see I<--source> option).

=item C<XDG_CACHE_HOME>

Used to determine the location of the template cache.  See L</CACHING>.

=back

=head1 EXIT STATUS

This program returns 0 if successful, or non-zero if there was an
error.

=head1 SEE ALSO

L<guestfs(3)>,
L<guestfish(1)>,
L<guestmount(1)>,
L<virt-copy-out(1)>,
L<virt-install(1)>,
L<virt-rescue(1)>,
L<virt-resize(1)>,
L<virt-sysprep(1)>,
L<oz-install(1)>,
L<gpg(1)>,
L<curl(1)>,
L<http://libguestfs.org/>.

=head1 AUTHOR

Richard W.M. Jones L<http://people.redhat.com/~rjones/>

=head1 COPYRIGHT

Copyright (C) 2013 Red Hat Inc.