This reverts one part of commit 85235aec83 related to reference
counts. Py_BuildValue always increments the reference count (when it
succeeds) and I could not reproduce the Python 3 problem that was
described there.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
(cherry picked from commit e07304de86)
The event callback gets a buffer parameter which is usually something
like a log message. However as it comes from C it is not necessarily
well-formed (eg) UTF-8 but could contain any old sequence of bytes.
In the test case provided by the reporter, we failed to encode the
buffer as 'str' with this error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 137: unexpected end of data
Use 'bytes' instead. Strictly speaking this changes the type
signature of the callbacks, but our existing Python tests which just
print the buffer using '%s' don't fail and in any case we don't
guarantee the stability of non-C APIs.
Reported-by: Yonatan Shtarkman
See: https://listman.redhat.com/archives/libguestfs/2023-February/030653.html
(cherry picked from commit bbf396fc55)
In the case that building the parameters to the Python event callback
fails, args was returned as NULL. We immediately tried to call
Py_INCREF on this which crashed. Returning NULL means the Python
function threw an exception, so print the exception and return (there
is no way to return an error here - the event is lost).
Reported-by: Yonatan Shtarkman
See: https://listman.redhat.com/archives/libguestfs/2023-February/030653.html
(cherry picked from commit 6ef5837e2d)
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Python 2 reached end of life on 2020-01-01:
https://python3statement.org/https://pythonclock.org/
The minimum version required is now Python 3.4 (since that is the
version in Debian oldoldstable), but 3.6 is the minimum version that
I actually test.
This symbol is not present in Python 2.7 or 3.6. It's not really
necessary to call this, it just avoids a crash in certain corner cases
when the interpreter is shutting down. So make the call conditional
on the function existing.
Fixes commit e6f9e0b0f6.
Properly mark the long line with the license classifier in setup.py:
the classifier is long on its own, and changing the code to stay within
79 characters would result in worse code.
Fix continuation indentation, and whitespaces around operators.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
Add or remove empty lines to match the needed ones around
blocks/functions/etc.
Adapt the generation of guestfs.py to emit the separating empty line
before adding a new function/alias, rather than after it.
This is just formatting, no behaviour changes.
The PyString API is specific to Python < 3, replaced by PyBytes and
PyUnicode in Python 3+; hence, drop the compile time check, and use the
right API depending on the Python version.
The OCaml and Python bindings directly use the utils.c source in
common/utils, mostly for guestfs_int_free_string_list. That source
contained also functions using gnulib functions, however without
linking to gnulib. When building with default build flags (e.g. without
as-needed mode), the gnulib symbols cannot be resolved, leading to
unusable OCaml and Python libraries.
As solution, update the common submodule to get the split of the split
of the stringlist functions in common/utils, and adapt the OCaml and
Python bindings:
- both now use stringlists-utils.c instead of utils.c
- fix the Python distutils setup to include only the sources really
needed
If you've registered a callback in Python and the handle is implicitly
closed when the Python interpreter exits, then it can be that the
following happens:
- Python interpreter is finalized.
- guestfs_close is called
- callback is invoked (eg. close event or to display a debug message)
- Python code runs from the callback wrapper
This cause a segfault on shutdown. Catch this case and stop the
callback from running (we lose the event but given the above sequence
of events there's not much we can do about it).
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1773520#c7
All Py_* functions should be protected by the GIL.
Otherwise internal python data structures can get corrupted.
Move PyGILState_Ensure to the beginning of the block and
PyGILState_Release to the bottom.
Signed-off-by: Sam Eiderman <sameid@google.com>
Previously to work around some problems in Python 2 header files we
had to include <Python.h> before any other config file.
For Python 3 which is all we really care about now this is no longer
needed. We can move the include from three files into the local
"actions.h" file, bringing all the Python definitions and workarounds
into a single place.
Since we already assume Python >= 2.7, modernize this example to make it
work also on Python 3:
- use print() as function
- sort the mount points using a key for sorted(), instead of a
comparison function
- remove extra newline escape
- reident two lines according to the PEP 8 style
Make sure to reference the arguments, to make sure they are kept alive
during the function call; this is visible when setting an event handler
for the CLOSE event, and testing it with Python 3.
This does not seem to create a memory leak e.g. with Python 2.
Also, switch away from the quasi-internal PyEval_CallObject to the
public PyObject_CallObject, which takes care of doing safety checks.
Currently test820RHBZ912499.py fails with libvirt because libvirt
still doesn't work around qemu locking properly. Allow this test to
be skipped on a temporary basis using SKIP_TEST820RHBZ912499_PY=1
This refactoring change just moves the cleanup functions around in the
common/utils directory.
libxml2 cleanups are moved to a separate object file, so that we can
still link to libutils even if the main program is not using libxml2
anywhere. Similarly gnulib cleanups.
cleanup.c is renamed to cleanups.c.
A new header file cleanups.h is introduced which will replace
guestfs-internal-frontend-cleanups.h (fully replaced in a later commit).
Remove (before opening round bracket) whitespaces in the documentation
of the Python binding, according to the PEP 8 specification.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
Verify the returned values of Python Object constructor functions
are not NULL before adding them to a collection.
This is particularly relevant when constructing Unicode strings in
Python 3 as they will return NULL if non UTF-8 characters are present.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Make use of functions and types that fit more, and that do the same job:
- use PyErr_NoMemory() on malloc failure
- use PyErr_SetFromErrno when setting an exception from an errno
- throw TypeError if not getting a list when required
The current need for #ifdef's based on the presence of
PyString_FromString makes both the OCaml code of the generator, and the
generated C code a mess to read.
Hence, add three simple wrappers to make both the OCaml, and C code more
readable, and easier to tweak in the future.
This should be just refactoring, with no actual behaviour changes.
Thanks to: Matteo Cafasso
In case there are no event handlers registered with the handle,
get_all_event_callbacks will count 0 elements, trying to malloc a buffer
of that size. POSIX says that this can result in either a null pointer,
or an unusable pointer.
Short-circuit get_all_event_callbacks to allocate nothing when there are
no events, making sure to use its results only when there were events.
