Until now decode_seq_header() refused profile_id 0x48, so nothing the
preceding commits added was reachable. The annex A de-emulation and the
field decoding among them deliberately apply to JiZhun as well.
A Broadcasting sequence that is not 8 bit 4:2:0 is refused rather than
decoded with the wrong macroblock layer.
The SD AVS+ mux attached to trac ticket 6948 decodes with this: both
video streams, frame coded AEC pictures throughout.
Fixes: https://trac.ffmpeg.org/ticket/6948
Signed-off-by: Rainbaby <r@bgr.jp>
Forgejo CI runs FATE on ppc64, mips64 and riscv64 with GCC 16 under
QEMU, and FATE has native ppc64le (GCC 15) and riscv64 (GCC 14) clients,
so these architectures get the same coverage as x86, arm and aarch64.
On loongarch GCC 14.0-14.3 and 15.0-15.1 miscompile vector permutations
(GCC PR121064), fixed in 14.4, 15.2 and newer, so vectorization is
disabled only for those versions.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
A DTS:X stream in the standard presentation is DTS-HD Master Audio with a 7.1
bed and an extension at the end of every XLL frame, behind the syncword the
decoder already recognises (DCA_SYNCWORD_XLL_X). The extension is a 22-byte
header followed by a regular XLL channel set: four channels, every one coded
in full (no residual against the core), no speaker mask, its own NAVI table
with the frame's segment geometry, then a two-byte trailer. Its channels are
the front and rear height pairs, and the encoder folds each of them into the
bed speaker below it (L, R, Lsr, Rsr) at -3 dB, gain code 23170/32768, so the
7.1 bed plays them on players that ignore the extension.
The decoder now parses that channel set with the existing channel set code,
outputs 7.1.4 (TFL, TFR, TBL, TBR), and removes the fold from the bed at
output resolution. A "downmix" request for 7.1 keeps the bed as coded, heights
folded in, which is what the decoder output before; the requests for 5.1 and
stereo skip the extension as well, so their output is unchanged. Streams
without the extension are not affected. A frame whose extension does not
decode keeps the 7.1.4 layout with silent heights and the bed as coded, and
the frame still ends where it did before. The alternate presentations, which
carry objects behind DCA_SYNCWORD_XLL_X_IMAX, are not touched.
The format facts (header size, channel order, the fold) come from the public
analysis in harletty-bridge's dca crate and kristofferR's SpatialAudioBridge;
this is an implementation on FFmpeg's own XLL code, not a translation.
Signed-off-by: Vincent Herbst <superuser404@tuta.com>
This format is only auto-probed when we can be certain that the image is a
JPEG MPF file; as evidenced either by the extension (.mpo), or by the
presence of an MPF APP2 marker or an ISO21496 gain map stub.
This does require parsing possibly arbitrarily far into the file, since
the MPF header can be after a > 2 KB EXIF header; and this matches files
produced in the wild (by e.g. Apple's jpeg ultra hdr encoder).
The probe function returns AVERROR(EAGAIN) to request more data until it
reaches either the MPF marker or the start of the entropy coded data. If
the probe limit or EOF is hit first, the best scoring other format wins,
e.g. image2 for regular .jpg files.
A few additional notes:
- libultrahdr produces quite spec-incompliant files which we unfortunately
have to work around
- since the MPF format table can be arbitrarily far from the header, we
also need to store possibly a lot of data and re-return it to the caller;
I have followed the convention established by e.g. fitsdec.c of just keeping
a spare AVPacket for bytes already read during header parsing (cf. also
jpegxl_anim_dec.c)
Signed-off-by: Niklas Haas <git@haasn.dev>
Add a filter that removes side data of the given types from packets and from
the stream, or keeps only the packets carrying side data of any of the given
types, as the select mode of the sidedata frame filter does for frames, but
with a list of types. The types are named by their handle, as option
constants joined with '+'.
This commit enables the DNN video filter to run entirely on the GPU
for the full D3D12 pipeline (decode, AI inference, and encode)
without ever copying frames back to system memory, making it
significantly faster on D3D12 hardware.
Benchmarked the new zero-copy D3D12/DirectML DNN pipeline against
the existing CPU round-trip pipeline on three major GPU vendors; up
to 10x throughput improvement was measured across every vendor.
Add dnn_onnx_d3d12, a helper that binds the DirectML execution
provider to a D3D12VA input frame's own device and does NV12
texture<->tensor conversion on the GPU (CopyTextureRegion + compute
shaders), avoiding a host round-trip.
dnn_processing negotiates this automatically for D3D12 NV12 input
with device=dml. Software-frame input retains the host I/O path.
D3D12 input requires successful DirectML negotiation; unsupported
hardware-input configurations fail during configuration. Currently
limited to single-channel FLOAT/NCHW models; chroma bypasses the
model and is copied/upscaled on the GPU. At runtime, DirectML.dll
and d3dcompiler_47.dll must be loadable from the application
directory or System32.
Example usage:
# Zero-copy pipeline
ffmpeg.exe -y -loglevel info \
-hwaccel d3d12va -hwaccel_output_format d3d12 \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_zero_copy.mp4
# CPU round-trip pipeline
ffmpeg.exe -y -loglevel info \
-init_hw_device d3d12va=dev -filter_hw_device dev -hwaccel d3d12va \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output,hwupload" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_round_trip.mp4
Signed-off-by: younengxiao <steven.xiao@amd.com>
asf_o is a second, non-default implementation of the ASF demuxer next
to asfdec_f.c. Security issues keep being reported against it, so it
is a maintenance burden without a benefit over the default demuxer.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Inject APP0..APP15 segments from a length-prefixed sidecar into each
MJPEG packet after SOI. A zero-length record passes the packet through.
An APP1-only payload retains the existing APP0; a payload starting with
APP0 replaces an APP0 located immediately after SOI.
Consume records in packet arrival order, with a 16 MiB record limit.
Missing or invalid records return an error. Remember sidecar read and
validation errors so later packets cannot consume a rejected record's
payload as metadata. Flushing neither rewinds the sidecar nor clears
this error; closing the context does not read ahead.
Count every consumed input packet in diagnostics, distinguish I/O errors
from truncated records, include the open error text for a sidecar that
cannot be opened, and document the sidecar and lifecycle contract.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Reinhard Nißl <rnissl@gmx.de>
Assisted-by: Fairy
This is a new library, separate from the existing fdk-aac (but
built from the same upstream repo), for encoding xHE-AAC.
This library has the same traits as fdk-aac; it's being distributed
from Fraunhofer to Android Open Source Project, and further
repackaged and distributed through opencore-amr [1] (and in a
personal repo on github at [2]). It is distributed under the
same license as fdk-aac.
This is an initial, barebones wrapper around the library - not
all features of the library are hooked up currently.
In particular, do note that the wrapper currently only handles
loudness in the live operating mode (single pass), where encoding
is done with a single loudness level set before encoding. This may
affect the dynamics of the input.
The preferred and intended mode of use for the fdk-xhe-aac encoder
is for it to be used in two pass mode, which handles the loudness
properly. The two pass operation mode is not set up in the
encoder wrapper yet, but may be added later.
[1] https://sourceforge.net/projects/opencore-amr/
[2] https://github.com/mstorsjo/fdk-aac
Supports RawFormat 8, used by the S1II, S5II and GH6. Outputs the Bayer
mosaic, with the sensor borders as cropping and the black and white
levels as raw color params side data.
The as-shot white balance (tags 0x24 - 0x26) is exported as
AVRW2RawColorParams, and the TIFF and EXIF tags of IFD0 and its sub-IFDs
as EXIF side data, gathered during the single header walk and attached
without re-parsing. The orientation becomes a display matrix; Panasonic's
raw parameters, the previews and the strip pointers are left out.
The whole file is given as one packet.
Bit-exact with LibRaw on 801 S1II files, at 3x its speed per thread.
AV_SAMPLE_FMT_DSD frames are already the bitstream, so the dsd_msbf
"encoder" is a plain copy of the samples into packets.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
A capture device could previously be selected only by index or by name.
Both are unreliable when multiple audio/video devices share a name or
across reboots. AVFoundation reorders the device indices, and the USB
video uniqueID embeds the macOS locationID, which can be reassigned on
reboot or replug and can then resolve to a different physical device.
Add -video_device_id and -audio_device_id, which take a prefixed
identifier: uid:<unique ID> or serial:<USB serial number>.
The USB serial number is the only identifier that stays pegged to a
given physical unit. For video it is resolved to the device's current
locationID via IOKit. For audio it is matched against the uniqueID,
which already embeds it. The unique ID covers devices that have no
serial, such as virtual camera/audio devices.
-list_devices additionally prints each device's uniqueID and USB serial
so the values can be discovered. IOKit is detected in configure and used
only when available.
Signed-off-by: Gabriel Balaich <ffmpeg@ninbura.com>
Decodes the LPCM variant found in DVD-Audio AOB streams. Unlike
DVD-Video LPCM it supports up to 192 kHz sample rates and splits
channels in two channel groups, which may use different quantization.
Samples are grouped in sets of 2 samples over all channels, with the
second channel group's data stored first within each set, as 16-bit
big-endian most significant parts followed by the remaining bits.
Based on information from expired US 6,580,671 patent et al.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
Convert packed 24/32-bit RGB/BGR/RGBA/BGRA input to PAL8 using a per-frame palette
whose colors are placed on a face-centered cubic lattice (realized as the
scaled D3/D4 checkerboard lattice), with a user-supplied density controlling
the number of lattice steps spanning one color axis.
Only lattice points actually used by a frame enter its palette; if a frame
needs more than 256 of them, the filter will itearatively drop palettte
entries and reassign affected pixels until 256 color remain
lookup uses the Conway-Sloane rounding algorithm. Supported dithering
modes: none, ordered 8x8 bayer (swscale), Cluster & Void blue noise and
Floyd-Steinberg error diffusion.
Co-Authored-by: Fable-5
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The format stores video-only H.264 in a fixed header + index table +
chunk layout. Each chunk is a self-contained GOP, enabling simple
random access via the index.
Fix#23540
Add a muxer that wraps encoded image in the iTerm2 inline image protocol
(OSC 1337) so ffmpeg can play video directly in an iTerm2 terminal. The
output is a self-contained byte stream: it can be played live or saved
to a file and replayed with cat.
This patch adds ONNX Runtime as a new DNN backend for FFmpeg's dnn_processing
filter, enabling hardware-accelerated neural network inference on multiple
GPU and NPU platforms.
Execution Providers Supported:
- CPU execution provider (default)
- CUDA execution provider (NVIDIA GPUs)
- DirectML execution provider (AMD/Intel/NVIDIA GPUs on Windows)
- VitisAI execution provider (AMD Ryzen AI NPU)
The options for dnn_processing with dnn_backend=onnx:
- device: execution provider — cpu, cuda, dml, or vitisai (default: cpu)
- device_id: GPU device index (default: 0)
- threads_per_operation: inference thread count for CPU EP (default: 0, auto)
- input: input tensor name. When omitted the backend resolves it from loaded session
- output: output tensor name. When omitted the backend resolves it from loaded session
Example usage:
# CPU inference
ffmpeg -i input.mp4 -vf "format=rgb24,dnn_processing=dnn_backend=onnx:model=model.onnx:input=image_in:output=image_out" output.mp4
# CUDA GPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=cuda:device_id=0" output.mp4
# DirectML GPU inference (Windows)
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=dml:device_id=0" output.mp4
# VitisAI NPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=vitisai" output.mp4
Note: depending on the model, you may need a format filter (e.g. format=rgb24 or format=grayf32) before dnn_processing to convert the frames to the pixel format the model's input tensor expects.
Signed-off-by: younengxiao <steven.xiao@amd.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
libcelt, which it depends on, was not updated in a very long time and is
considered deprecated, as Opus exists which has a CELT mode. Therefore
remove standalone CELT decoding support.
It was already broken since b8604a9761,
11 years ago, and no one noticed and complained.
If incoming packets contain Smpte2094App5 metadata, retain them
so that they are passed through to the output.
Signed-off-by: Vignesh Venkat <vigneshv@google.com>
This patch adds the transpose_cuda video filter.
It's similar to the existing transpose filter but accelerated by CUDA.
It supports the same pixel formats as the scale_cuda filter.
This also supersedes the deprecated transpose_npp filter.
Example usage:
ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i <INPUT> -vf "transpose_cuda=dir=clock" <OUTPUT>
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
Add a muxer for the Playdate PDV container format.
The muxer writes the frame table and packet layout required by the
Playdate runtime. It requires seekable output and a predeclared
maximum number of frames (-max_frames).
Includes validation for single video stream input, dimension and
framerate checks, and bounded payload/table offset checks. The frame
entry table is allocated once in write_header() using max_frames + 1.
Document the muxer in doc/muxers.texi and add a Changelog entry.
James Almer