Search for DC by 'storage.name=' to make it explicit.
I.e. "storage=" uses regex, so similar names can be
found in the search query. For example, searching for
a domain named FCSD, will find FCSD1 as well.
Due to a conflict with the IDs of the OVF standard, and the existing
implementation in ovirt-engine, the ID of QXL devices changed to a
different value.
As a consequence, change the ResourceType of QXL devices, but only in
OVirt flavour to avoid breaking vdsm mode.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1598715#c5
(cherry picked from commit aa9e18f6d1)
For direct upload, a suitable host must be in status 'Up'
and belong to the same datacenter as the created disk.
Added these criteria to the host search query.
(cherry picked from commit 4ed1bc5a79)
The optimization to start the transfer on the local host makes sense
only when using the rhv-direct=true option. When using a proxy, let the
engine choose a host.
(cherry picked from commit 891b5a0ec0)
When optimizing the connection using unix socket, we handle these cases:
- The local host is not an oVirt host (no /etc/vdsm/vdsm.id).
- The local host is an oVirt host, but is not registered with engine.
- Creating UnixHTTPConnection() fails. Unlikely and probably a bug in
the plugin, but we can recover by using the https connection.
The current code handle these cases silently, making it harder to
understand why the unix socket optimization did no happen. Add debug
message to make this clear.
Also comment in the error handler why we take this path instead of
failing the operation.
(cherry picked from commit f5442d2f04)
Old imageio proxy was using Authorization header for GET and PUT
requests. Remove unneeded authorization when sending OPTIONS request.
Remove unneeded duplicated comments about authorization for old
imageio, and replace them with a comment when we set needs_auth.
(cherry picked from commit d5f36bacf9)
In the case where virt-v2v runs on the same server as the imageio
daemon that we are talking to, it may be possible to optimize access
using a Unix domain socket.
This is only an optimization. If it fails or if we're not running on
the same server it will fall back to the usual HTTPS over TCP
connection.
Thanks: Nir Soffer, Daniel Erez.
(cherry picked from commit b7a2e6270d)
Previously we lazily requested the server options in the can_*
callbacks. The can_* callbacks are always called by nbdkit straight
after open, so this just adds complexity for no benefit. This change
simply makes the code always fetch the server options during the open
callback.
This is — functionally at least — mostly just refactoring. However I
also added a useful debug message so we can see what features the
imageio server is offering.
(cherry picked from commit a1e1f6ec88)
"http" and "transfer" variables were missing in emulate_zero, so the
code would fail with NameError. This can happen only when communicating
with old imageio versions not supporting the "zero" feature.
Testing with qemu-img 2.12 show that we never send emulated zero request
because of the highestwrite mechanism, but it can break with older
qemu-img-rhev used on RHEL.
(cherry picked from commit 0ae61ce99c)
Python manual warns[1]:
Note that you must have read the whole response before you can send
a new request to the server.
The reason for this warning is exposed only when the server is using
keep alive connections. When the response is not read, sending a new
request will fail with:
httplib.ResponseNotReady
Even if Content-Length was 0 or the request has no content. The failure
looks like this when using --verbose:
nbdkit: python[1]: debug: zero count=33554432 offset=0 may_trim=1 fua=0
nbdkit: python[1]: debug: zero count=33554432 offset=33554432 may_trim=1 fua=0
nbdkit: python[1]: error: /home/nsoffer/src/libguestfs/tmp/rhvupload.Au2B5I/rhv-upload-plugin.py: zero: error: Request-sent
nbdkit: python[1]: debug: sending error reply: Input/output error
qemu-img: error writing zeroes at offset 0: Input/output error
Change all requests to read the whole response.
Tested with imageio patch supporting keep alive connections:
https://gerrit.ovirt.org/#/c/92296/
[1] https://docs.python.org/3.8/library/http.client.html#http.client.HTTPConnection.getresponse
(cherry picked from commit f4e0a8342d)
When sending request with small or no payload, it is simpler and
possibly more efficient to use the high level HTTPSConnection.request(),
instead of the lower level APIs.
The only reason to use the lower level APIs is to avoid copying the
payload, or on python 2, to use a bigger buffer size when streaming a
file-like object.
(cherry picked from commit 77a412c0a1)
Output also the osinfo guess for each OS in the generated XML output;
adapt the RNG schema, and the test data to it.
(cherry picked from commit 2f8ec91fc1)
This increases the inactivity timeout for transfers from the default
(60 seconds in oVirt < 4.2.3, 600 seconds in >= 4.2.3), up to 1 hour,
so that we should never hit it for ordinary transfers.
Note this requires oVirt >= 4.2.3. The corresponding oVirt fix was
https://bugzilla.redhat.com/1563278
I also replaced the deprecated ‘image’ parameter with ‘disk’.
Thanks: Nir Soffer, Daniel Erez.
(cherry picked from commit f25404c65f)
Add an helper to convert an OVF flavour to string, and use it in
-o vdsm to print the actual value of the vdsm-ovf-flavour option.
Thanks to Ming Xie.
(cherry picked from commit c0da02f90d)
Use the $(srcdir) variable where needed, to make sure it builds also
with srcdir != builddir.
Furthermore, make sure that the OCaml dependencies calculation depend on
the generated output_rhv_upload_*_source.ml files, otherwise there will
be incomplete OCaml rules for them in the generated .depend.
Fixes commit cc04573927.
(cherry picked from commit a4e3b7c059)
Because we checked the limitation of raw+sparse during command line
processing, it had the effect of forcing you to use ‘-of raw’ even if
the input was already in raw format. Move the checking to the output
mode to avoid this.
Fixes commit cc04573927.
(cherry picked from commit b3769afaec)
This adds a new output mode to virt-v2v. virt-v2v -o rhv-upload
streams images directly to an oVirt or RHV >= 4 Data Domain using the
oVirt SDK v4. It is more efficient than -o rhv because it does not
need to go via the Export Storage Domain, and is possible for humans
to use unlike -o vdsm.
The implementation uses the Python SDK (‘ovirtsdk4’ module). An
nbdkit Python 3 plugin translates NBD calls from qemu into HTTPS
requests to oVirt via the SDK.
(cherry picked from commit cc04573927)
When writing the OVF in OVirt flavour, add a ovirt:id attribute to the
OperatingSystemSection tag: this attribute represents the numeric value
of the ostype ID, which is ignored by oVirt when parsing OVFs in API
mode.
(cherry picked from commit 593a19cc86)
Currently we have a bunch of ad hoc options like --vddk* and --vdsm*
(and proposed to add --rhv*) to handle extra parameters for input and
output modes/transports. This complicates the command line parsing
and also the clarity of the command line (becauseit's not very obvious
which options apply to which side of the conversion).
Replace these with a general mechanism for handling input and output
options.
Thus (for example):
--vddk-thumbprint=... becomes -io vddk-thumbprint=...
--vdsm-compat=0.10 -oo vdsm-compat=0.10
The responsibility for parsing input and output options moves into the
input and output drivers.
This improves error checking so it's harder now for wrong flags to be
included on the command line when they don't apply to the current mode.
The old option names are preserved for compatibility.
(cherry picked from commit 6327e716cd)
Currently unused, in a future commit this will allow you to pass in a
password to be used when connecting to the target hypervisor.
(cherry picked from commit a4e181137a)
Updates commit a52ed4b445
("v2v: ovf: Create OVF more aligned with the standard") with some
small code refactoring and formatting.
(cherry picked from commit 9e83f3a2cc)
Add option for -o vdsm that enables output of the modified OVF. oVirt
engine should already be able to consume the OVF, but let's not take any
chances and enable it only by command line argument. It can be made
default later when it receives proper testing.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
(cherry picked from commit 285014b290)
For historical reasons the OVF used in RHV export domain contains some
deviations from the OVF standard. The format used in -o rhv has to
remain fixed but for -o vdsm and we could produce much nicer OVF. This
patch serves as a preparatory step to this.
The main reason for creating different OVF is that it can be used to
create VM by oVirt REST API. The RHV export domain flavor cannot be used
that way.
For now the virt-v2v behavior is unchanged. The modified output will be
enabled in some later patch.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
(cherry picked from commit a52ed4b445)
While YAJL mostly works fine, it did not see any active development in
the last 3 years. OTOH, Jansson is another JSON C implementation, with
a very liberal license, and a much nicer API.
Hence, switch all of libguestfs from YAJL to Jansson:
- configure checks, and buildsystem in general
- packages pulled in the appliance
- actual implementations
- contrib scripts
- documentation
This also makes use of the better APIs available (e.g. json_object_get,
json_array_foreach, and json_object_foreach). This does not change the
API of our OCaml Yajl module.
(cherry picked from commit bd1c5c9f4d)
If the /etc/fstab of a guest contains devices specified with UUID or
LABEL, then the new OCaml inspection code will report the findfs failure
as general failure of the inspection. OTOH, the old C inspection code
simply ignored all the devices that cannot be resolved.
Hence, restore the old behaviour by ignoring unresolvable devices.
(cherry picked from commit 67c36779c3)
