When checking the return value of guestfs_int_py_fromstring for string
fields of structs, add a newline to generated C code, so it is properly
indented.
Fixes commit 401c445636.
We reimplemented some functions which can now be found in the OCaml
stdlib since 4.01 (or earlier). The functions I have dropped are:
- String.map
- |>
- iteri (replaced by List.iteri)
- mapi (replaced by List.mapi)
Note that our definition of iteri was slightly wrong: the type of the
function parameter was too wide, allowing (int -> 'a -> 'b) instead of
(int -> 'a -> unit).
I also added this new function to the Std_utils.String module as an
export from stdlib String:
- String.iteri
Thanks: Pino Toscano
If you have a struct containing ‘field’, eg:
type t = { field : int }
then previously to pattern-match on this type, eg. in function
parameters, you had to write:
let f { field = field } =
(* ... use field ... *)
In OCaml >= 3.12 it is possible to abbreviate cases where the field
being matched and the variable being bound have the same name, so now
you can just write:
let f { field } =
(* ... use field ... *)
(Similarly for a field prefixed by a Module name you can use
‘{ Module.field }’ instead of ‘{ Module.field = field }’).
This style is widely used inside the OCaml compiler sources, and is
briefer than the long form, so it makes sense to use it. Furthermore
there was one place in virt-dib where we are already using this new
style, so the old code did not compile on OCaml < 3.12.
See also:
https://forge.ocamlcore.org/docman/view.php/77/112/leroy-cug2010.pdf
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
Verify the returned values of Python Object constructor functions
are not NULL before adding them to a collection.
This is particularly relevant when constructing Unicode strings in
Python 3 as they will return NULL if non UTF-8 characters are present.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The current need for #ifdef's based on the presence of
PyString_FromString makes both the OCaml code of the generator, and the
generated C code a mess to read.
Hence, add three simple wrappers to make both the OCaml, and C code more
readable, and easier to tweak in the future.
This should be just refactoring, with no actual behaviour changes.
Thanks to: Matteo Cafasso
Previously we had lots of types like String, Device, StringList,
DeviceList, etc. where Device was just a String with magical
properties (but only inside the daemon), and DeviceList was just a
list of Device strings.
Replace these with some simple top-level types:
String
StringList
and move the magic into a subtype.
The change is mechanical, for example:
old ---> new
FileIn "filename" String (FileIn, "filename")
DeviceList "devices" StringList (Device, "devices")
Handling BufferIn is sufficiently different from a plain String
throughout all the bindings that it still uses a top-level type.
(Compare with FileIn/FileOut where the only difference is in the
protocol, but the bindings can uniformly treat it as a plain String.)
There is no semantic change, and the generated files are identical
except for a minor change in the (deprecated) Perl
%guestfs_introspection table.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
For a very long time we have maintained two sets of utility functions,
in mllib/common_utils.ml and generator/utils.ml. This changes things
so that the same set of utility functions can be shared with both
directories.
It's not possible to use common_utils.ml directly in the generator
because it provides several functions that use modules outside the
OCaml stdlib. Therefore we add some lightweight post-processing which
extracts the functions using only the stdlib:
(*<stdlib>*)
...
(*</stdlib>*)
and creates generator/common_utils.ml and generator/common_utils.mli
from that. The effect is we only need to write utility functions
once.
As with other tools, we still have generator-specific utility
functions in generator/utils.ml.
Also in this change:
- Use String.uppercase_ascii and String.lowercase_ascii in place
of deprecated String.uppercase/String.lowercase.
- Implement String.capitalize_ascii to replace deprecated
String.capitalize.
- Move isspace, isdigit, isxdigit functions to Char module.
This mostly mechanical change moves all of the libguestfs API
lists of functions into a struct in the Actions module.
It also adds filter functions to be used elsewhere to get
subsets of these functions.
Original code Replacement
all_functions actions
daemon_functions actions |> daemon_functions
non_daemon_functions actions |> non_daemon_functions
external_functions actions |> external_functions
internal_functions actions |> internal_functions
documented_functions actions |> documented_functions
fish_functions actions |> fish_functions
*_functions_sorted ... replacement as above ... |> sort
In the case where we are building a Python dist (which can be
distributed separately), we generated calls to non-existent optargs
structures if an API was once no-optargs but had optargs added. A
typical error was this (library: 1.32.5, Python sdist: 1.33.42):
guestfs-py.c: In function ‘guestfs_int_py_glob_expand’:
guestfs-py.c:9648:40: error: storage size of ‘optargs_s’ isn’t known
struct guestfs_glob_expand_opts_argv optargs_s;
This is a simple solution to this, although it has the possible
disadvantage that such functions will not be bound at all.
python/guestfs.py:136:37: E712 comparison to True should be 'if cond is True:' or 'if cond:'
python/t/tests_helper.py:42:8: E713 test for membership should be 'not in'
No functional changes, as the new versions follow the suggested Python
coding style to do the same things.
Reindent Python scripts to make sure lines are not longer than 80
columns.
Regarding autogenerated code (guestfs.py and bindtests.py): add an
helper function to make sure comma-separated lists are wrapped at the
wanted length.
This produces only differences in the indentation of long Python lines,
with no behaviour changes.
Add (after comma) or remove (before opening round bracket, and around
'=' in arguments) whitespaces according to the PEP 8 specification.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
When a struct containing FChar fields is defined
in generator/structs.ml, the Python code generator will generate
code referencing a dirent struct instead of the correct one
thus breaking at compile time.
This bug was left unnoticed because the dirent struct is the only
one using FChar fields. Yet it prevents any other struct
from using such fields.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Document which feature, if any, is needed for a function; this should
help users in properly checking feature availability when using certain
functions.
As in commit e102bcf3cf (for Perl), any
Python program has the handle program name field set to 'python'. Set
it to the true name (derived from sys.argv[0]).
When compiling the libguestfs bindings as a pip module, it's helpful
if you can compile the bindings with a different version of libguestfs
(eg. installed 1.28, pip module 1.30).
That broke the previous assumption that we were always compiling
against precisely the same version of libguestfs (which made sense
because the python bindings are part of the libguestfs source tree).
Python distutils has no way to check a C library to find out if
functions are defined. Therefore we have to rely on GUESTFS_HAVE_*
macros from <guestfs.h>.
We also need to be more careful about PyString_FromString and similar
functions returning NULL on failure. Currently we don't check this
every time. This commit adds more checks, but is still not complete.
libguestfs has used double and triple underscores in identifiers.
These aren't valid for global names in C++.
The first step is to replace all guestfs___* (3 underscores) with
guestfs_int_*. We've used guestfs_int_* elsewhere already as a prefix
for internal identifiers.
This is an entirely mechanical change done using:
git ls-files | xargs perl -pi.bak -e 's/guestfs___/guestfs_int_/g'
Reference: http://stackoverflow.com/a/228797
This allows the Python binding of guestfs_add_libvirt_dom to work.
There is a regression test to ensure this keeps working.
Note this requires a patched libvirt-python, supporting the
c_pointer() method.
This implements Pointer arguments properly, at least for certain
limited definitions of "implements" and "properly".
'Pointer' as an argument type is meant to indicate a pointer passed to
an API. The canonical example is the following proposed API:
int guestfs_add_libvirt_dom (guestfs_h *g, virDomainPtr dom, ...);
where 'dom' is described in the generator as:
Pointer ("virDomainPtr", "dom")
Pointer existed already in the generator, but the implementation was
broken. It is not used by any existing API.
There are two basic difficulties of implementing Pointer:
(1) In language bindings there is no portable way to turn (eg.) a Perl
Sys::Virt 'dom' object into a C virDomainPtr.
(2) We can't rely on <libvirt/libvirt.h> being included (since it's an
optional dependency).
In this commit, we solve (2) by using a 'void *'.
We don't solve (1), really. Instead we have a macro
POINTER_NOT_IMPLEMENTED which is used by currently all the non-C
language bindings. It complains loudly and passes a NULL to the
underlying function. The underlying function detects the NULL and
safely returns an error. It is to be hoped that people will
contribute patches to make each language binding work, although in
some bindings it will always remain impossible to implement.
When using optional arguments of type OStringList, the code free'ing
the member in the optargs_s struct corresponding to that optional
argument would just check for a non-PyNone PyObject for that argument.
If before that optional argument there are other arguments which can
cause an earlier error return from that binding function, the free'ing
code will then act on garbage values.
Enhance the check by also checking whether the optargs struct has the
bitmask with the element for that argument, meaning that the
corresponding struct member was initialized.
Look for use of external_functions and fish_functions and replace with
use of external_functions_sorted and fish_functions_sorted where
possible. This ensures that the output of the generator is sorted as
far as possible.
I also checked for uses of internal_functions and documented_functions
but those are not used. The *_sorted versions are always used
instead.
The visibility field in action replaces in_fish, in_docs and internal.
The defined types are:
VPublic:
A public API. This is exported and documented in all language
bindings, and in guestfish.
VStateTest:
A public API which queries the library state machine. It is exported
and documented in all language bindings, but not guestfish.
VBindTest:
An internal API used only for testing language bindings. It is
guarded by GUESTFS_PRIVATE in the C api, but exported by all other
language bindings as it is required for testing. If language
bindings offer any way to guard use of these apis, that mechanism
should be used. It is not documented anywhere.
VDebug:
A debugging API. It is exported by all language bindings, and in
guestfish, but is not documented anywhere.
VInternal:
An internal-only API. It is guarded by GUESTFS_PRIVATE in the C api,
and not exported at all in any other language binding. It is not
documented anywhere.
Certain functions are intended to be internal only, but we currently
export them anyway. This change moves them into a separate section of
guestfs.h protected by a GUESTFS_PRIVATE variable. This change also
enables private structs, but doesn't implement any.
This change only affects the C api. Language bindings aren't affected,
but probably should be in the future.
