vf_libplacebo is unique in that it can function as both a Vulkan filter, and a
software filter, depending on how it the filter graph is configured. While this
is very flexible, it create a problem in situations where the filter does not
receive a Vulkan hw frames context up front. When that doesn't happen, it will
initialise its own standalone pl context, and then fail to interoperate with
a context provided via the input link. This then leads to graph failures.
There are two primary scenarios where this existing logic breaks what should
be valid configurations:
* When the global filter hw device is a cuda device and we use something like
`hwupload=derive_device=vulkan` to pass frames from cuda to vulkan
* In mpv (and probably other media players) which don't set the global filter
hw device at all. In this case, it's impossible to configure vf_libplacebo
to use a hw frames ctx, even if it's using Vulkan for everythng else. This
prevents the use of vf_libplacebo in any fully hardware accelerated pipeline
in mpv
There are various ways we could imagine addressing it - such as allowing the
filter to discard the initial pl context and recreating it based on the passed
in device, but it's easier to reason about if we add a flag that explicitly
tells the filter that it should inherit the device context from the input link.
This puts the filter into a mode that works like all the other Vulkan filters.
This requires explicit configuration from the user, but the intent is clearer,
and the user can always know when it's necessary as they define the filter
graph.
A format-only conversion leaves the auto use_filters value unresolved.
Regression since 5d0748243f.
Fix issue #23737
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Two runtime paths could compute out-of-range source coordinates for
degenerate projection geometry, causing heap-buffer-overflow reads
Fixes: out of array read
Fixes: assertion failure
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: assertion failure
Fixes: mQzloVqnivHQ
Found-by: Anthony Hurtado
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
```
vf_quirc.c: In function 'filter_frame':
vf_quirc.c:104:46: warning: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64_t' {aka 'long long int'} [-Wformat=]
104 | "Found count %d codes in image #%ld\n", codes_count, inl->frame_count_out);
| ~~^ ~~~~~~~~~~~~~~~~~~~~
| | |
| long int int64_t {aka long long int}
| %lld
```
C23 made these generic functions that no longer cast const
away for you, leading to warnings when compiling with C23
and a recent enough toolchain (glibc supports this since 2.43).
This commit fixes all the warnings that can simply be fixed
by adding const, without adding casts.
(the latter excludes parse_forced_key_frames() in ffmpeg_mux_init.c).
Reviewed-by: Kacper Michajłow <kasper93@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
When the blending factor was calculated to be 0, the hot path skipped
updating variables later emitted in metadata.
Fix the flow to ensure they are populated consistently.
Extend the FATE test suite to check metadata output.
This commit attempts to address feedback from this filter's users, by
introducing a new option which controls the amelioration mechanism.
The "blend" option is a factor which is multiplied by the difference
in badness (between the threshold and the currently accumulated
badness). This difference normally controls how much of the next frame
we can let through without making it exceed the badness threshold.
Setting the option to zero effectively puts the filter into a mode
where it always duplicates the last frame (which did not put
accumulated badness over the threshold) instead of attempting to blend
in new frames. I have received reports that this mode is preferable
to users for some types of media.
This commit attempts to address feedback from this filter's users, by
improving the filter's behavior at the start of playback or
immediately after seeking.
In these situations, the history buffer is empty, but because we did
not previously track its size, we were calculating the weighted
average as if the corresponding frames had zero badness. This caused
the filter to behave differently and possibly produce false negatives
when the history was not fully populated.
Address this by instead taking into account the history size when
calculating cumulative badness. To accomplish this:
- Add a history_size field to to PhotosensitivityContext, tracking how
much of the history buffer is populated.
- Change the semantics of PhotosensitivityContext::badness_threshold.
Previously, it was premultiplied by the maximum history size. This
is no longer done, so that we can multiply it to the live
history_size on the fly instead.
- Calculate the badness threshold on a per-frame basis. The result is
now stored in a badness_threshold local variable.
This commit only changes the filter's behavior for the first
PhotosensitivityContext::nb_frames (configurable as the "frames"
filter option) frames. For successively filtered frames, the behavior
is unchanged.
ff_dnn_start_inference_async() return value was not stored in ret,
causing execute_model_th() to return success on async startup failure.
This left vf_dnn_processing stuck in its flush loop waiting on a
task that was never started.
Currently, such filter graphs just fail with a nebulous:
[fc#0 @ 0x2a7b3c0] [error] Error while filtering: Cannot allocate memory
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
It's been years, but took another look at the bwdif_cuda implementation and
there are a couple of typos sitting in there. Found them when I was doing a
comparison with the Vulkan implementation.
This probably explains the small PSNR differences we've noted in the past.
Since the input and output format can differ (e.g. 444 -> 420), we need to
reference the correct subsampling for the partially applied filter.
Keep track of this in the CUDATex itself.
Signed-off-by: Niklas Haas <git@haasn.dev>
Instead of going via an AVFrame at all. This will allow us to fix the
intermediate chroma plane size for split downscaling.
Signed-off-by: Niklas Haas <git@haasn.dev>
I want to disentangle the internal logic from AVFrame, because some
intermediate states (e.g. for partially subsampled chroma with simultaneous
scaling) may not directly map to a valid AVPixelFormat.
Signed-off-by: Niklas Haas <git@haasn.dev>
The hand written boundary mirroring reflected an out of range index only
once, which is insufficient when the image dimension is smaller than the
filter half width (filt_w/2 == 8). A 1x1 input made the index reach 8
and -7, reading out of the src[]/temp[] arrays. Use avpriv_mirror(),
which mirrors repeatedly and stays in range for any dimension.
Fixes: out of array access
Fixes: repro.sh
Fixes: HuQn51lLiVJX
Fixes: 38aea9b041 (avfilter: add vif filter)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
NVDEC and CUVID now output AV_PIX_FMT_P012 (12-bit 4:2:0), AV_PIX_FMT_P212
(12-bit 4:2:2) and AV_PIX_FMT_YUV444P10MSB / AV_PIX_FMT_YUV444P12MSB
(10/12-bit 4:4:4) for high-bit-depth content, but these CUDA filters
rejected the formats in their supported-format lists, breaking pipelines
such as "-hwaccel cuda ... -vf scale_cuda" on 12-bit input.
These formats use 16-bit sample storage, and the filters select their CUDA
kernel by byte-storage size and plane layout, not by the number of valid
bits, so they can reuse the existing 16-bit kernels:
- scale_cuda: P012/P212 -> "semiplanar16", YUV444P10MSB/YUV444P12MSB ->
"planar16".
- transpose_cuda: the ushort/ushort2 kernels are chosen from the pixel
descriptor (byte size + channel count); just allow the new formats.
- thumbnail_cuda: P012 reuses the P010/P016 path and the MSB 4:4:4 formats
reuse the YUV444P16 path; P012 is added to the 4:2:0 chroma-histogram
scaling as well. (thumbnail has no 4:2:2 path, so P212 is not added.)
The CUDA deinterlacers (bwdif_cuda, yadif_cuda) already accept any format
with <= 2 bytes per sample and <= 2 channels, so they need no change. The
8-bit-only filters (overlay_cuda, pad_cuda, bilateral_cuda, chromakey_cuda,
colorspace_cuda) do not support high bit depths and are left untouched.
Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
This may be faster or slower than the existing specialized kernels,
so I opted not to prefer it by default. I also deliberately didn't expose
additional filter function capabilites yet.
The main motivating reason here is to get correct anti-aliasing behavior
when downscaling, which is currently completely broken.
Signed-off-by: Niklas Haas <git@haasn.dev>
Useful for GPU-based filters, which may also need to compute filter weights.
Since we cannot cross-link to internal functions, we need to recompile this
helper inside libavfilter.c.
Signed-off-by: Niklas Haas <git@haasn.dev>
This fallback function is used if external MMX is available,
while inline MMX and intrinsics for emitting emms are unavailable.
It is implemented as an avpriv function, which has several
drawbacks for shared builds:
1. The function is so small (3 bytes; 16 with padding)
that the overhead of exporting and importing it dwarfs
the gains from code deduplication.
2. A call to an external library has more overhead than
a library-internal one.
3. It may cause linking failures when a libavutil not exporting
avpriv_emms_asm() is paired with a library needing it
(if inline assembly and intrinsics were unavailable when building
the dependent library). I am not aware of this ever happening.
4. We would be forced to keep avpriv_emms_asm() around for ABI stability
even after it is no longer needed.
This commit therefore uses the STLIBOBJS, SHLIBOBJS approach
to duplicating it into each library on its own if needed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
libnpp and the corresponding filters have been deprecated
in commit 994a368451
on 2025-09-26. By the time of our next release,
a year will have passed, so they are removed immediately.
Note: Passing --enable-libnpp to configure results in
a warning about the deprecation and is otherwise a no-op.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The ass filter exposes libass' shaping mode selection so callers can
request complex shaping for scripts such as Arabic. The subtitles filter
uses the same renderer path but did not expose the option.
This left the zero-initialized shaping field to select
ASS_SHAPING_SIMPLE implicitly.
Expose the same shaping option for subtitles and default it to auto,
matching the ass filter. This allows subtitles=...:shaping=complex to
render Arabic lam-alef correctly when libass is built with HarfBuzz
support.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
This patch adds ONNX Runtime as a new DNN backend for FFmpeg's dnn_processing
filter, enabling hardware-accelerated neural network inference on multiple
GPU and NPU platforms.
Execution Providers Supported:
- CPU execution provider (default)
- CUDA execution provider (NVIDIA GPUs)
- DirectML execution provider (AMD/Intel/NVIDIA GPUs on Windows)
- VitisAI execution provider (AMD Ryzen AI NPU)
The options for dnn_processing with dnn_backend=onnx:
- device: execution provider — cpu, cuda, dml, or vitisai (default: cpu)
- device_id: GPU device index (default: 0)
- threads_per_operation: inference thread count for CPU EP (default: 0, auto)
- input: input tensor name. When omitted the backend resolves it from loaded session
- output: output tensor name. When omitted the backend resolves it from loaded session
Example usage:
# CPU inference
ffmpeg -i input.mp4 -vf "format=rgb24,dnn_processing=dnn_backend=onnx:model=model.onnx:input=image_in:output=image_out" output.mp4
# CUDA GPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=cuda:device_id=0" output.mp4
# DirectML GPU inference (Windows)
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=dml:device_id=0" output.mp4
# VitisAI NPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=vitisai" output.mp4
Note: depending on the model, you may need a format filter (e.g. format=rgb24 or format=grayf32) before dnn_processing to convert the frames to the pixel format the model's input tensor expects.
Signed-off-by: younengxiao <steven.xiao@amd.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
