avutil/hwcontext_vulkan: reject unsupported semi-planar CUDA imports

Cuda currently only supports packed and single-component planar formats, but
fails to import semi-planer (eg: NV12, P010). Even though there are now
semi-planar cuda array formats, which the latest nvdec can use, these formats
are not used when mapping Vulkan imports. Maybe they'll fix that some day.

But until then, let's explicitly detect the case and return a clear error
message for the user.

Exercising this error path revealed that vulkan_free_internal() frees
f->internal on a transfer error and then runs again when the frame is
destroyed, dereferencing the freed pointer; let's make it idempotent.
This commit is contained in:
Philip Langdale
2026-07-06 12:36:02 -07:00
committed by philipl
parent 30ba440386
commit c29d710cd5

View File

@@ -2309,6 +2309,10 @@ static void vulkan_free_internal(VulkanDevicePriv *p, AVVkFrame *f)
{
av_unused AVVkFrameInternal *internal = f->internal;
// Make this function safe to call repeatedly
if (!internal)
return;
#if CONFIG_CUDA
if (internal->cuda_fc_ref) {
AVHWFramesContext *cuda_fc = (AVHWFramesContext *)internal->cuda_fc_ref->data;
@@ -3898,6 +3902,18 @@ static int vulkan_export_to_cuda(AVHWFramesContext *hwfc,
if (nb_images != planes) {
for (int i = 0; i < planes; i++) {
/* Cuda now defines array formats for semi-planar, but these are
* not currently supported for imported Vulkan images. */
if (desc->comp[i].step / elem_size > 1) {
av_log(ctx, AV_LOG_ERROR,
"Cannot map a multiplane Vulkan image (%d image(s) "
"for %d plane(s)) to CUDA; create the Vulkan device "
"with the disable_multiplane=1 option (one image per "
"plane) for CUDA interop.\n", nb_images, planes);
err = AVERROR(ENOSYS);
goto fail;
}
VkImageSubresource subres = {
.aspectMask = i == 2 ? VK_IMAGE_ASPECT_MEMORY_PLANE_2_BIT_EXT :
i == 1 ? VK_IMAGE_ASPECT_MEMORY_PLANE_1_BIT_EXT :