This patch adds ONNX Runtime as a new DNN backend for FFmpeg's dnn_processing
filter, enabling hardware-accelerated neural network inference on multiple
GPU and NPU platforms.
Execution Providers Supported:
- CPU execution provider (default)
- CUDA execution provider (NVIDIA GPUs)
- DirectML execution provider (AMD/Intel/NVIDIA GPUs on Windows)
- VitisAI execution provider (AMD Ryzen AI NPU)
The options for dnn_processing with dnn_backend=onnx:
- device: execution provider — cpu, cuda, dml, or vitisai (default: cpu)
- device_id: GPU device index (default: 0)
- threads_per_operation: inference thread count for CPU EP (default: 0, auto)
- input: input tensor name. When omitted the backend resolves it from loaded session
- output: output tensor name. When omitted the backend resolves it from loaded session
Example usage:
# CPU inference
ffmpeg -i input.mp4 -vf "format=rgb24,dnn_processing=dnn_backend=onnx:model=model.onnx:input=image_in:output=image_out" output.mp4
# CUDA GPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=cuda:device_id=0" output.mp4
# DirectML GPU inference (Windows)
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=dml:device_id=0" output.mp4
# VitisAI NPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=vitisai" output.mp4
Note: depending on the model, you may need a format filter (e.g. format=rgb24 or format=grayf32) before dnn_processing to convert the frames to the pixel format the model's input tensor expects.
Signed-off-by: younengxiao <steven.xiao@amd.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
libcelt, which it depends on, was not updated in a very long time and is
considered deprecated, as Opus exists which has a CELT mode. Therefore
remove standalone CELT decoding support.
It was already broken since b8604a9761,
11 years ago, and no one noticed and complained.
If incoming packets contain Smpte2094App5 metadata, retain them
so that they are passed through to the output.
Signed-off-by: Vignesh Venkat <vigneshv@google.com>
This patch adds the transpose_cuda video filter.
It's similar to the existing transpose filter but accelerated by CUDA.
It supports the same pixel formats as the scale_cuda filter.
This also supersedes the deprecated transpose_npp filter.
Example usage:
ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i <INPUT> -vf "transpose_cuda=dir=clock" <OUTPUT>
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
Add a muxer for the Playdate PDV container format.
The muxer writes the frame table and packet layout required by the
Playdate runtime. It requires seekable output and a predeclared
maximum number of frames (-max_frames).
Includes validation for single video stream input, dimension and
framerate checks, and bounded payload/table offset checks. The frame
entry table is allocated once in write_header() using max_frames + 1.
Document the muxer in doc/muxers.texi and add a Changelog entry.
As defined in ITU-T H.222.0 v9, LCEVC streams use the "Byte stream format"
defined in Annex B of ISO/IEC 23094-2:2021.
Signed-off-by: James Almer <jamrial@gmail.com>
refs does not belong to AVCodecParameters, so require a decoder:
it should only be showed when frames are actually processed by ffprobe.
Signed-off-by: Nicolas Gaullier <nicolas.gaullier@cji.paris>
The drawvg filter can draw vector graphics on top of a video, using libcairo. It
is enabled if FFmpeg is configured with `--enable-cairo`.
The language for drawvg scripts is documented in `doc/drawvg-reference.texi`.
There are two new tests:
- `fate-filter-drawvg-interpreter` launch a script with most commands, and
verify which libcairo functions are executed.
- `fate-filter-drawvg-video` render a very simple image, just to verify that
libcairo is working as expected.
Signed-off-by: Ayose <ayosec@gmail.com>
This commit adds a structure to contain parsed EXIF metadata, as well
as code to read and write that struct from/to binary EXIF buffers. Some
internal functions have been moved to exif_internal.h. Code to read
from this new struct and write to an AVDictionary **dict has been added
as well in order to preserve interoperability with existing callers.
The only codec changes so far as of this commit are to call these
interop functions, but in future commits there will be codec changes to
use the new parsing routines instead.
Signed-off-by: Leo Izen <leo.izen@gmail.com>
Commit f256487cd8 bumped requirement to
1.1.0 for OPENSSL_init_ssl.
Bump this again to 1.1.1, because it was an LTS version. Although it has
no mainline support anymore, it still has paid/premium support. 1.1.0 has
no support at all.
Motivated for use of BIO_read_ex() for next commits.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
