Commit Graph

11368 Commits

Author SHA1 Message Date
Csaba Henk
83474887ed ruby: change value of 'readonly' drive toption to Boolean in doc/example/test
Seeing `g.add_drive_opt :readonly => 1` allows one to imply
that ensuring writable access to drive should happen via
`g.add_drive_opt :readonly => 0`. However, the passed option
value gets passed down to C according to Ruby Boolean semantics,
that is, any value apart from `false` and `nil` will be true
(see RTEST in Ruby C API).

So its more idiomatic and provides a better hint if we use
`g.add_drive_opt :readonly => true` in Ruby samples.
2020-02-19 18:27:02 +00:00
Richard W.M. Jones
b68a67e1a9 Revert "golang: Don't run launch test if appliance has not been built."
This causes a strange error with modern golang:

  go: parsing $GOFLAGS: non-flag "appliance"

This reverts commit d57d8f22de.
2020-02-19 18:24:51 +00:00
Csaba Henk
95244fd2ca golang: make API idiomatic so that functions return (<val>, error)
Go API functions returned (<val>, *GuestfsError) that made
code like this fail to build:

    n, err := os.Stdin.Read(buf)
    if err != nil {
        log.Fatal(err)
    }
    n, err = g.Pwrite_device(dev, buf[:n], off)
    ...

As err should be of error (interface) type as of the stdlib call,
and should be of *GuestfsError type as of the libguestfs call.

The concrete error value that libguestfs functions return can be
a *GuestfsError, but the function signature should have (<val>, error)
as return value.
2020-02-19 18:12:15 +00:00
Nikolay Ivanets
c509420be7 virt-get-kernel: add '--blocksize' option support
This patch adds '--blocksize' command line option for virt-get-kernel
tool.  This option allows specifying disk sector size as described in
'guestfs_add_drive_opts' libguestfs API.
2020-02-17 18:04:11 +00:00
Richard W.M. Jones
e6764a5415 appliance: Add ntfs-3g-system-compression (RHBZ#1703463).
This package in Fedora enables optional support for Windows 10
"CompactOS" (file-level compression), read-only, which is sufficient
for inspecting Windows guests and doing certain types of modifications
to them.  Virt-v2v appears to work, but it may be that anything that
involves modifying a compressed file might not work.

I couldn't find the equivalent package in Debian or SUSE.  It's
available in Arch AUR although I didn't verify that part of the change
actually works there (but should be safe because supermin ignores
packages that are not known about on the target system).
2020-02-17 12:38:27 +00:00
Nikolay Ivanets
6b6bb44ebc tools: add '--blocksize' option for C-based tools
This patch adds '--blocksize' command line option for guestfish and
other C-based tools.  This option allows specifying disk sector size.
2020-02-17 12:32:03 +00:00
Nikolay Ivanets
94843f155a lib: add support for disks with 4096 bytes sector size
Nowadays there are hard drives and operating systems which support
"4K native" sector size.  In this mode physical and logical block size
exposed to the operating system is equal to 4096 bytes.

GPT partition table (as a known example) being created in this mode will
place GPT header at LBA1 which is 4096 bytes.  libguetfs is unable to
recognize partition table on such physical block devices or disk images.
The reason is that libguestfs appliance will look for a GPT header at
LBA1 which is seen at 512 byte offset.

In order to fix the issue we need a way to provide correct logical block
size for attached disks.  Fortunately QEMU and libvirt already provides
a way to specify physical/logical block size per disk basis.

After discussion in a mailing list we agreed that physical block size is
rarely used and is not so important.  Thus both physical and logical
block size will be set to the same value.

In this patch one more optional parameter 'blocksize' is added
to add_drive_opts API method.  Valid values are 512 and 4096.

add_drive_scratch has the same optional parameter for a consistency and
testing purpose.

add-domain and add_libvirt_dom will pass logical_block_size value from
libvirt XML to add_drive_opts method.
2020-02-11 15:20:09 +00:00
Richard W.M. Jones
815eab8a66 tests: Use explicit backing format for all backing disks.
Libvirt 6.0 now requires that every disk in the backing chain has an
explicit backing format.  For example this will be rejected by
libvirt:

  qemu-img create -f qcow2 -b backing-disk disk.qcow2

with the error:

  Original error from libvirt: Requested operation is not valid:
  format of backing image 'backing-disk' of image 'disk.qcow2' was not
  specified in the image metadata (See
  https://libvirt.org/kbase/backing_chains.html for troubleshooting)
  [code=55 int1=-1]

Instead you have to use the -F option to specify the format, eg:

  qemu-img create -f qcow2 -b backing-disk -F raw disk.qcow2
2020-02-06 16:06:19 +00:00
Daria Phoebe Brashear
56834875b2 properly initialize error_data_lock_list before use
when a handle is allocated, the error_data_list_lock must be initialized
2020-02-06 13:23:31 +00:00
Richard W.M. Jones
d6fcf519f2 builder: Fix const correctness for OCaml 4.10.
String_val now returns a const char *.
2020-02-06 10:33:39 +00:00
Richard W.M. Jones
9f3148c791 ocaml: Use caml_alloc_initialized_string instead of memcpy.
See this commit in libguestfs-common:
398dc56a6c
2020-02-06 10:32:08 +00:00
Richard W.M. Jones
378b49152d m4: ocaml: Move definition of HIVEX_OPEN_UNSAFE_FLAG closer to test.
Minor code cleanup, no change in semantics.
2020-02-06 10:28:34 +00:00
Richard W.M. Jones
4e199494c4 cat: Fix GCC 10 warning.
I believe this warning is bogus, but simply initializing the local
variable is enough to avoid it.

log.c: In function 'do_log':
log.c:390:7: error: 'comm_len' may be used uninitialized in this function [-Werror=maybe-uninitialized]
  390 |       printf (" %.*s", (int) comm_len, comm);
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-02-06 10:28:34 +00:00
Pino Toscano
b3ef101a0f daemon: fix/enhance error reporting of Augeas exceptions
The current code was broken, as the field 1 of the exception value is
the error code (int), not an error string, and thus it would have
crashed.  This did not happen in practice, as all the usage of
ocaml-augeas were only in the inspection code with ad-hoc exception
catching blocks.

Other than fixing the aforementioned issue, enhance the error reporting
to be as close as possible to what the current AUGEAS_ERROR() macro
does: error message, error minor message (if available), error details
(if available).
2020-01-27 14:53:35 +01:00
Richard W.M. Jones
a883a2f0c9 Update BUGS and PO files. 2020-01-22 13:07:08 +00:00
Richard W.M. Jones
d4a1c3a778 build: Add build-aux/config.rpath from gettext.
This is somehow required after updating gnulib.  I lack the energy to
find out exactly why, but this fixes it.
2020-01-22 13:07:01 +00:00
Richard W.M. Jones
538f728d5f Update gnulib. 2020-01-22 13:03:03 +00:00
Richard W.M. Jones
f7bb3eef19 Update common submodule. 2020-01-22 13:03:03 +00:00
Richard W.M. Jones
bd4d9264e5 docs: Update release notes in preparation for 1.42 release. 2020-01-22 13:03:03 +00:00
Richard W.M. Jones
54ccd4fee9 docs: Document that Python 2.7 or above is now required.
Updates commit 444750bfd1.
2020-01-22 10:45:37 +00:00
Pino Toscano
a754cd4307 launch: libvirt: use machine type when querying for domcaps
On some architectures a certain machine type is used, so use it when
querying the libvirt domain capabilities.
2020-01-20 13:00:12 +01:00
Pino Toscano
e636e64d34 lib: uefi: use the efi libvirt firmware if available
In case libvirt supports the firmware autoselection and there is an EFI
firmware available, use it directly instead of handling the firmware
manually.
2020-01-20 13:00:12 +01:00
Pino Toscano
029901113c lib: allow to use libvirt firmware autoselection
Enhance the UEFI firmware lookup function with the information on the
libvirt firmware autoselection, allowing it to return a value to use for
the appliance.

At the moment no firmware is selected this way, so there is no behaviour
change.
2020-01-20 13:00:06 +01:00
Pino Toscano
777a21b16b lib: uefi: reset out parameters earlier
Make sure they are always reset, no matter the code branches later on.

This is mostly code motion.
2020-01-20 13:00:06 +01:00
Pino Toscano
9484136fd0 launch: libvirt: parse firmware autoselection
Parse from the domain capabilities whether libvirt supports the
autoselection of firmware, and which values are supported.
2020-01-20 13:00:06 +01:00
Pino Toscano
5d8cca738e build: define HAVE_PYCODESTYLE in all the cases
Fixes commit cad3ea9e74 in case Python is
disabled, or not available.
2020-01-16 11:02:25 +01:00
Richard W.M. Jones
711fab172c ocaml: Fix test for -runtime-variant _pic.
This fails with Fedora 32, and possibly earlier versions:

  checking if OCaml ‘-runtime-variant _pic’ works... no

The reason is:

  $ ocamlc -runtime-variant _pic test.ml -o test
  File "test.ml", line 1:
  Error: Cannot find file camlheader_pic

which may even be a packaging error in the Fedora package.  However it
makes no sense to test the bytecode compiler since we don't use it on
any architecture we care about and bytecode doesn't even contain a
linked runtime.  Changing the test to use ocamlopt instead of ocamlc
fixes the problem.
2020-01-15 12:52:29 +00:00
Pino Toscano
cad3ea9e74 python: add a pycodestyle test
Look for pycodestyle, and use it to check all the Python sources (tests
and auxiliary scripts included) of the Python bindings.
2020-01-14 16:21:22 +01:00
Pino Toscano
da4c7f9112 python: skip line length check in license line
Properly mark the long line with the license classifier in setup.py:
the classifier is long on its own, and changing the code to stay within
79 characters would result in worse code.
2020-01-14 16:21:22 +01:00
Pino Toscano
92759efbf5 python: improve errors in inspect_vm example
When validating user input, print an error message and exit, instead of
either asserting or raising a non-existing exception.
2020-01-14 16:21:22 +01:00
Pino Toscano
7d0f8e1b22 python: tests: remove unused imports
No functional changes.
2020-01-14 16:21:22 +01:00
Pino Toscano
4719cca7f5 python: tests: improve variable naming
Use a slightly more descriptive name, as also pointed out by
pycodestyle.

No functional changes.
2020-01-14 16:21:22 +01:00
Pino Toscano
6d0e9531c8 python: tests: catch specific exception
When trying to import libvirt, catch the specific exception that is
raised when an importing fails.
2020-01-14 16:21:22 +01:00
Pino Toscano
0703ec9c8c python: PEP 8: adapt whitespaces in lines
Fix continuation indentation, and whitespaces around operators.

This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
2020-01-14 16:21:22 +01:00
Pino Toscano
df3e693b1b python: PEP 8: adapt empty lines
Add or remove empty lines to match the needed ones around
blocks/functions/etc.

Adapt the generation of guestfs.py to emit the separating empty line
before adding a new function/alias, rather than after it.

This is just formatting, no behaviour changes.
2020-01-14 16:21:22 +01:00
Pino Toscano
40251e4b3c python: replace guestfs_int_py_fromstringsize in Py2 branches
In Python 2 code branches use PyString_FromStringAndSize directly
instead of the guestfs_int_py_fromstringsize wrapper (which calls
PyString_FromStringAndSize anyway on Python < 3).
2020-01-09 17:49:16 +01:00
Pino Toscano
a68a0298b5 python: remove compile time check for PyString_AsString
The PyString API is specific to Python < 3, replaced by PyBytes and
PyUnicode in Python 3+; hence, drop the compile time check, and use the
right API depending on the Python version.
2020-01-09 17:45:26 +01:00
Pino Toscano
c3728416fb python: assume support for Capsules
They were introduced in Python 2.7 and 3.1, so we can assume they exist
(as Python 2.7 is the minimum version required, and 3.0 is long gone).
2020-01-09 17:29:15 +01:00
Pino Toscano
444750bfd1 python: drop code for Python < 2.5
Python >= 2.7 is required.
2020-01-09 17:22:16 +01:00
Pino Toscano
d8cb64dbe7 build: enforce a minimum Python version
In case the Python bindings are enabled, enforce a minimum version.
Set this minimum version to 2.7, as it has been de-facto for a while
now.
2020-01-09 17:15:20 +01:00
Pino Toscano
2c4aec6dcf build: update miscellaneous files
C_SOURCE_FILES
POTFILES

Followup of commit cb0edd4354.
2020-01-09 15:47:56 +01:00
Pino Toscano
eb43478f83 inspect: avoid returning "unknownX.Y" for unknown Linux distros
If it is not possible to detect the distribution of a Linux OS, do not
propose "unknownX.Y" (where X is the major version number, and Y the
minor) as short osinfo ID. Just return "unknown" instead.
2020-01-09 14:57:59 +01:00
Pino Toscano
cb0edd4354 build: use split stringlist functions from common/utils
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list.  That source
contained also functions using gnulib functions, however without
linking to gnulib.  When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.

As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
  needed
2020-01-09 14:57:38 +01:00
Pino Toscano
e789a070de python: remove also __pycache__ directories on clean 2019-12-19 15:34:37 +01:00
Pino Toscano
a420e535d5 daemon: reorder internal static libs to fix linking
Move the interal static libraries as the last items in the list of
libraries of guestfsd, to make sure their symbols are used for all the
other libraries.  This is because GCC resolves the symbols looking at
the arguments from the beginning to the end of the command line.
This currently does not cause failures, however it "just works" because
of the tricky situation set up.

The situation is the following:

1) common/utils contains few utility sources: one of them is utils.c,
which contains various functions -- for example
guestfs_int_free_string_list and guestfs_int_drive_name --, it is built
as utils.o, and bundled in the static library libutils.a

2) common/mlutils builds a OCaml library with bindings for some utility
functions in libutils.a, in particular guestfs_int_drive_name (but not
guestfs_int_free_string_list); there are two versions of this library,
one OCaml library (dllmlcutils.so) that links with libutils.a, and one
static library (libmlcutils.a), which cannot specify the libraries it
links to (as it is static)

3) when the daemon is linked, the command line was the following
(simplified):
  $ gcc [...] -o guestfsd guestfsd-9p.o other_daemon_object.o [...] \
      ../common/utils/.libs/libutils.a [...] -lmlcutils [...]
Some of the objects of the daemon itself use
guestfs_int_free_string_list, and thus the compiler opens libutils.a
(it is after the objects in the command line) and picks utils.o, which
contains also guestfs_int_drive_name (not used directly in the daemon);
when linking later on with libmlcutils.a, the symbols for this static
library (like guestfs_int_drive_name) are already resolved, and thus
all the symbols are resolved, and the linking succeeds

This fragile situation can be easily broken by moving e.g.
guestfs_int_drive_name out of common/utils/utils.c to a new source (say
utils2.c) still built as part of libutils.a: since nothing before
-lmlcutils actually needs to pick utils2.o from libutils.a for symbols,
then GCC will not be able to resolve all the symbols in libmlcutils.a.

As solution, move libutils.a (and other internal static libraries) as
last libraries to link guestfsd to: this way, GCC knows where to find
all the symbols needed by all the objects and libraries specified in
the command line.
2019-12-19 13:10:41 +01:00
Pino Toscano
d8540adffb inspect: document OpenMandriva Lx as detected distro
Also extend the virt-inspector RNG schema to allow it.

Followup of commit a4ef6716b4.
2019-12-18 13:14:59 +01:00
Pino Toscano
1cc8293935 po: reduce the list of extracted sources
Prune from the list of sources where to extract messages various sources
with no messages:
- .pl and .pm files, as they do not contain messages: almost all the
  .pl files are tests, and the only .pm file is the Perl Sys::Guestfs
  module, which wraps the XS extension
- dummy.c sources; they are empty sources used to build OCaml-only
  targets using automake
- gperf generated sources
- C/OCaml tests
2019-12-18 11:06:40 +01:00
Pino Toscano
98700b6d8c docs: exclude dummy.c sources
They are empty sources used to build OCaml-only targets using automake,
so there is no documentation to extract.
2019-12-18 11:01:58 +01:00
Pino Toscano
af3b8624b7 build: exclude common/mlv2v from distcheck
It is part of the common submodule but not used in libguestfs, so it is
not included in distribution tarballs.
2019-12-17 16:31:09 +01:00
Pino Toscano
4cd801dcff build: fix make implicit dependencies on ml/mli files
As proposed in the Debian bug #946594 [1], the implicit .cmi/.cmo/.cmx
dependencies are tied to files in the source directory only, and thus
they break when either the .ml or the .mli files are in the build
directory. Since there is already VPATH set up by automake, rely on it
to locate the right source.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946594#36

Solution proposed by Vincent Danjean, thanks!

This reverts commit 802c5d2055 in this
file.
2019-12-17 15:38:00 +01:00