Acquire the per-handle lock on entering each public API function.
The lock is released by a cleanup handler, so we only need to use the
ACQUIRE_LOCK_FOR_CURRENT_SCOPE macro at the top of each function.
Note this means we require __attribute__((cleanup)). On platforms
where this is not supported, the code will probably hang whenever a
libguestfs function is called.
The only definitive list of public APIs is found indirectly in the
generator (in generator/c.ml : globals).
hivex has a function hivex_value_string. We were not calling it under
the mistaken belief that because hivex implements this using iconv,
the function wouldn't work inside the daemon. Instead we
reimplemented the functionality in the library.
This commit deprecates hivex_value_utf8 and removes the library side
code. It replaces it with a plain wrapper around hivex_value_string.
Thanks: Pino Toscano
These are generated in many different ways in the various
subdirectories, and sometimes not generated correctly. Introduce a
script to do this in one place, and hopefully correctly.
This is mostly simple refactoring, but I got rid of a couple of
things:
(1) The ‘make depend’ rule doesn't appear to be needed. automake (or
make?) seems to rebuild the ‘.depend’ file automatically just because
it is included.
(2) I got rid of the hairy path rewriting sed expression. Possibly
that is needed for srcdir != builddir.
All sorts of strings might be passed here hoping to make them
canonical LV names. We cannot be sure that the strings passed will be
devices which exist in the appliance.
This also reimplements the lv_canonical function in OCaml. We cannot
call the original C function because it calls reply_with_perror which
would break the OCaml bindings.
Move the list_filesystems API into the daemon, reimplementing it in
OCaml. Since this API makes many other API calls, it runs a lot
faster in the daemon.
The previously library-side ‘file_architecture’ API is reimplemented
in the daemon, in OCaml.
There are some significant differences compared to the C
implementation:
- The C code used libmagic. That is replaced by calling the ‘file’
command (because that is simpler than using the library).
- The C code had extra cases to deal with compressed files. This is
not necessary since the ‘file’ command supports the ‘-z’ option
which transparently looks inside compressed content (this is a
consequence of the change above).
This commit demonstrates a number of techniques which will be useful
for moving inspection code to the daemon:
- Moving an API from the C library to the OCaml daemon.
- Calling from one OCaml API inside the daemon to another (from
‘Filearch.file_architecture’ to ‘File.file’). This can be done and
is done with C daemon APIs but correct reply_with_error handling is
more difficult in C.
- Use of Str for regular expression matching within the appliance.
This change allows parts of the daemon to be written in the OCaml
programming language. I am using the ‘Main Program in C’ method along
with ‘-output-obj’ to create an object file from the OCaml code /
runtime, as described here:
https://caml.inria.fr/pub/docs/manual-ocaml/intfc.html
Furthermore, change the generator to allow individual APIs to be
implemented in OCaml. This is picked by setting:
impl = OCaml <ocaml_function>;
The generator creates ‘do_function’ (the same one you would have to
write by hand in C), with the function calling the named
‘ocaml_function’ and dealing with marshalling/unmarshalling the OCaml
parameters.
Even if ocamlopt is available, always build a bytecode version of
‘common/mlstdutils’.
Furthermore, because this library is pure OCaml, we should not be
using ‘ocamlmklib’. We should use ‘ocaml{c,opt} -a’ instead. This
doesn't make any difference for native code, but for bytecode it was
building a broken library.
The original reason for making this change is because the generator is
always built as bytecode, and it depended on
‘../common/mlstdutils/guestfs_config.cmo’ and
‘../common/mlstdutils/std_utils.cmo’. On native code platforms these
were not built before the generator and so the generator races to
build the .cmi and .cmo files. Since the generator doesn't have
correct dependencies covering the ‘common/mlstdutils’ directory you
can get a broken link on fast machines:
File "../common/mlstdutils/std_utils.ml", line 1:
Error: Corrupted compiled interface
../common/mlstdutils/guestfs_config.cmi
make[2]: *** [Makefile:1993: ../common/mlstdutils/std_utils.cmo] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/builddir/build/BUILD/libguestfs-1.37.17/generator'
Previously the OCaml compiler was only required if building from git
but was at least theoretically optional if building from tarballs
(although this was never tested). Since we want to write parts of the
daemon in OCaml, this makes OCaml required for all builds.
Note that the ‘--disable-ocaml’ option remains, but it now only
disables OCaml bindings and OCaml virt tools. Using this option does
not disable the OCaml compiler requirement.
Also note that ‘HAVE_OCAML’ changes meaning slightly, so it now means
"build OCaml bindings and tools" (analogous to ‘HAVE_PERL’ and
others). The generator, daemon [in a future commit], and some utility
libraries needed by the generator or daemon do not test for this macro
because we can assume OCaml compiler availability.
Place the Bytes fallback module in the right place (mlstdutils), with no
need to make it available directly also for generation, since it uses
mlstdutils now.
Fixes commit 61d4891ef4.
The gobject bindings are adequately covered in the usual manual pages:
guestfs(3). There is no need for separate generation of gtk-doc.
Also generating gtk documentation is the slowest part of the build,
and the tooling around gtk-doc is broken
(https://bugzilla.redhat.com/show_bug.cgi?id=1465665).
Note this removes the configure ‘--enable-gtk-doc’ option. Using this
option now gives a warning, but is otherwise ignored:
configure: WARNING: unrecognized options: --enable-gtk-doc
After the previous refactoring, we are able to link the daemon to
common/utils, and also remove some of the "duplicate" functions that
the daemon carried ("duplicate" in quotes because they were often not
exact duplicates).
Also this removes the duplicate reimplementation of (most) cleanup
functions in the daemon, since those are provided by libutils now.
It also allows us in future (but not in this commit) to move utility
functions from the daemon into libutils.
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
Verify the returned values of Python Object constructor functions
are not NULL before adding them to a collection.
This is particularly relevant when constructing Unicode strings in
Python 3 as they will return NULL if non UTF-8 characters are present.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The current need for #ifdef's based on the presence of
PyString_FromString makes both the OCaml code of the generator, and the
generated C code a mess to read.
Hence, add three simple wrappers to make both the OCaml, and C code more
readable, and easier to tweak in the future.
This should be just refactoring, with no actual behaviour changes.
Thanks to: Matteo Cafasso
Previously the generator did not change any string returned from the
daemon. Thus guestfs_list_devices (for example) might return internal
device names like /dev/vda (if virtio-blk was in use).
This changes calls to the daemon so that returned strings are
annotated as plain strings, devices or mountables:
old ---> new
RString "uuid" RString (RPlainString "uuid")
RString "device" RString (RDevice "device")
RString "fs" RString (RMountable "fs")
For hash tables, keys and values must be annotated separately. For
example a hash table of mountables (keys) -> plain strings (values)
would be annotated like this:
old ---> new
RHashtable "fses" RHashtable (RMountable, RPlainString, "fses")
The daemon calls reverse_device_name_translation (currently a no-op)
for devices and mountables.
Note that this has no effect for calls which are handled on the
library side.
(cherry picked from commit 6b77cc196ecb8d7e1d73592ef65a189a7412c97c)
It is not something standard, and actually unused by libguestfs itself.
Possibly going to be used by the old virt-v2v (which was in Perl), but
never used for that either.
After the previous commit it become possible to construct various
"impossible" argument types, such as lists of FileIn strings. This
commit prevents these from happening.
