Without engine_type, frc_amf uses the runtime's default DX12 engine even
on a DX11 AMF context. The interpolated frames then come back as DX12
surfaces, and the filter runs about three times slower. Select
FRC_ENGINE_DX11 on a DX11 context whose device has feature level 11_1 or
higher, and keep an explicit engine_type. The DX11 engine fails Init()
on 11_0, which is what hwcontext_d3d11va creates, so those devices keep
the default.
Assisted-by: Claude Opus 5.5
Signed-off-by: Julius Bairaktaris <julius@bairaktaris.de>
Set the timestamp_map option of the webvtt demuxer for the subtitle
segments in place of prefer_hls_mpegts_pts, which no demuxer has, so the
cue times land on the MPEG-TS timeline of the audio and video. That time
is on the 33 bit PES clock, so each cue is put on the period of that clock
nearest to the position of the media, which is read first when both are
enabled. A cue read without any media does not become that position, and
a cue still buffered when the media is enabled is placed relative to it
before it is returned.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
HLS WebVTT segments carry an X-TIMESTAMP-MAP header that maps a cue time
to an MPEG-TS time of the other renditions (RFC 8216, 3.5). With
timestamp_map set, add the offset it gives to the cue timestamps, and use
the 90 kHz MPEG-TS time base for the stream so that the mapping stays
exact. Skip the header when it comes in a block of its own instead of
ending the parsing there.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
Until now decode_seq_header() refused profile_id 0x48, so nothing the
preceding commits added was reachable. The annex A de-emulation and the
field decoding among them deliberately apply to JiZhun as well.
A Broadcasting sequence that is not 8 bit 4:2:0 is refused rather than
decoded with the wrong macroblock layer.
The SD AVS+ mux attached to trac ticket 6948 decodes with this: both
video streams, frame coded AEC pictures throughout.
Fixes: https://trac.ffmpeg.org/ticket/6948
Signed-off-by: Rainbaby <r@bgr.jp>
Extend scale_d3d12 from NV12/P010 resizing to full pixel format and
color space conversion, matching what the D3D12 video processor is
actually capable of.
- Support converting to/from the semi-planar YUV formats (nv12, p010,
p012) and the packed RGB formats (bgra, rgba, x2bgr10, rgbaf16),
when the hardware supports it.
- Add out_color_matrix, out_range, out_color_primaries and
out_color_transfer options.
- Recalculate sample_aspect_ratio and drop size-dependent side data
on resize.
- Harden GPU teardown/reconfiguration.
Example usage:
Convert a d3d12 NV12 surface to BGRA:
scale_d3d12=format=bgra
Convert an HDR10 (P010) surface to 10-bit RGB, without altering
its HDR color properties:
scale_d3d12=format=x2bgr10
Scale and convert to full-range BGRA:
scale_d3d12=format=bgra:out_range=full
Signed-off-by: younengxiao <steven.xiao@amd.com>
Replace the deprecated pl_tone_map_params.param field with
pl_color_map_params.tone_constants
This allows configuring each tonemapping algorithm independently, rather
than relying on a single algorithm-specific parameter.
Update the documentation to describe the new options and their defaults.
Also deprecate the vf_libplacebo tonemapping_param option as well.
The tls and dtls protocols verify the peer by default. Without ca_file,
OpenSSL only looks in the default locations configured when it was
built, which on Windows usually do not exist, so every connection fails
certificate verification unless the caller provides a CA file. The
Schannel backend uses the system store and does not have this problem.
OpenSSL 3.2 added a store loader for the Windows ROOT system store.
When no ca_file is given, register it in addition to the default
locations, unless OpenSSL was built without it. Its certificates are
looked up during verification, with the usual OpenSSL checks; trust
restrictions that Windows keeps as certificate properties are not
enforced by the loader.
OpenSSL 3.5.0 crashes in this loader (openssl/openssl#27355, fixed in
3.5.1); no workaround is added for that superseded release.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: J-RAP <srrapero720@hotmail.com>
This filter takes a scene graph in the libplacebo scene JSON format and
exposes it as a filter. The number of inputs and their properties are
determined by the scene graph.
I decided to make this a separate filter because it invalidates most of
the existing options, including e.g. the existing scaling and fitting logic.
For simplicity, scene graphs are always assumed to have a fixed canvas size,
which the user continues specifying at init time.
Requires libplacebo API version 386 or newer, which we probe the existence
for as a separate capability so we can selectively expose the filter.
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
Forces the output sample aspect ratio, instead of deriving it from the
input.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
The AVS decoder handles the video stream used by the Creature Shock
multimedia format, not the Chinese AVS codec. The current "Audio Video
Standard" expansion conflates the two.
Rename the decoder long name to "Argonaut Games Creature Shock video",
using the same identifying name as the existing demuxer while making
clear that this is the video decoder. Update the codec descriptor and
codec list as well.
No short codec names, codec IDs, or decoding behavior are changed.
Signed-off-by: Ruonan <yu_ruonan@outlook.com>
The so far default one pass encoding requires a hardcoded
loudness level parameter (which defaults to -24 dB).
By doing two pass encoding, the first pass measures the loudness
in order to be able to preserve the dynamics of the stream
properly.
This allows callers to (optionally) set a specific limit for audio and
video frames, separately.
Since these are not really stream options, I had to add a new parsing logic
to handle the :a and :v suffixes, but I don't think it's too ugly.
As an aside, also fixes a memleak if setting the option somehow fails.
Signed-off-by: Niklas Haas <git@haasn.dev>
This is a new API that offers two improvements over the existing field:
1. It allows setting a different limit per media type
2. It allows updating the limit after the graph has been configured, as
long as this would not cause already existing frames to exceed the limit
I'm not sure if the second point is ever useful, but it's easy to implement
regardless.
Because the old field will be deprecated, the new API just overrides the old
value wholesale. The alternative was to respect both (i.e. use the smaller
of the two limits); but this way, callers can use the new API to e.g.
selectively uncap the limit on audio frames while remaining backwards
compatible with older libavfilter.
Explicitly set limits are tracked separately, so that avfilter_graph_config()
re-derives all other limits from the old field on every call.
Signed-off-by: Niklas Haas <git@haasn.dev>
This additionally fixes a prior silent corruption when a parent `offset`
option was implicitly passed down to the inner protocol, resulting in a
stream that did not start at 0, yet the inner protocol offset tracking
did not account for this.
By just always handling this option ourselves, we avoid this class of
problem.
Fixes: https://github.com/nxtedition/FFmpeg/issues/80
See-Also: https://github.com/nxtedition/casparcg/pull/103
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
The phase mode is the RDFT display with the blue component raised and
the red and green components lowered by phase(L) - phase(R), wrapped to
(-pi, pi], times the smaller of the red and green components. This shows
channels in antiphase, which look the same as channels in phase in the
RDFT display.
Basically it looks pretty
A DTS:X stream in the standard presentation is DTS-HD Master Audio with a 7.1
bed and an extension at the end of every XLL frame, behind the syncword the
decoder already recognises (DCA_SYNCWORD_XLL_X). The extension is a 22-byte
header followed by a regular XLL channel set: four channels, every one coded
in full (no residual against the core), no speaker mask, its own NAVI table
with the frame's segment geometry, then a two-byte trailer. Its channels are
the front and rear height pairs, and the encoder folds each of them into the
bed speaker below it (L, R, Lsr, Rsr) at -3 dB, gain code 23170/32768, so the
7.1 bed plays them on players that ignore the extension.
The decoder now parses that channel set with the existing channel set code,
outputs 7.1.4 (TFL, TFR, TBL, TBR), and removes the fold from the bed at
output resolution. A "downmix" request for 7.1 keeps the bed as coded, heights
folded in, which is what the decoder output before; the requests for 5.1 and
stereo skip the extension as well, so their output is unchanged. Streams
without the extension are not affected. A frame whose extension does not
decode keeps the 7.1.4 layout with silent heights and the bed as coded, and
the frame still ends where it did before. The alternate presentations, which
carry objects behind DCA_SYNCWORD_XLL_X_IMAX, are not touched.
The format facts (header size, channel order, the fold) come from the public
analysis in harletty-bridge's dca crate and kristofferR's SpatialAudioBridge;
this is an implementation on FFmpeg's own XLL code, not a translation.
Signed-off-by: Vincent Herbst <superuser404@tuta.com>
Windows Graphics Capture only delivers a frame when the captured content
changes, so a static source makes gfxcapture output nothing at all. An
fps filter after it cannot help, since it has no later frame to
duplicate up to, and anything muxed alongside stalls as well.
Add a dup_frames option, like ddagrab has. When no new frame arrived
within one 1/max_framerate interval, output a new reference to the last
frame, or a black frame before the source delivered its first one.
Repeats are stamped with their deadline in the QPC clock, the same clock
as SystemRelativeTime, and the next deadline counts from the previous
one, so timer overshoot shows up neither in the timestamps nor in the
rate. A captured frame timestamped at or before the last output is
bumped to keep timestamps increasing.
The new option is on by default, since the current behaviour is arguably
surprising and clashes with how a lot of the filter chain and the
fftools work.
Signed-off-by: Rafael Henrique Gonçalves de Oliveira <rafaelh9six@gmail.com>
Co-authored-by: Timo Rothenpieler <timo@rothenpieler.org>
Otherwise this demuxer would regress relative to image2 for basic JPEG
inputs that happen to contain an HDR gain map, which are often produced
by e.g. smartphones.
Signed-off-by: Niklas Haas <git@haasn.dev>
This format is only auto-probed when we can be certain that the image is a
JPEG MPF file; as evidenced either by the extension (.mpo), or by the
presence of an MPF APP2 marker or an ISO21496 gain map stub.
This does require parsing possibly arbitrarily far into the file, since
the MPF header can be after a > 2 KB EXIF header; and this matches files
produced in the wild (by e.g. Apple's jpeg ultra hdr encoder).
The probe function returns AVERROR(EAGAIN) to request more data until it
reaches either the MPF marker or the start of the entropy coded data. If
the probe limit or EOF is hit first, the best scoring other format wins,
e.g. image2 for regular .jpg files.
A few additional notes:
- libultrahdr produces quite spec-incompliant files which we unfortunately
have to work around
- since the MPF format table can be arbitrarily far from the header, we
also need to store possibly a lot of data and re-return it to the caller;
I have followed the convention established by e.g. fitsdec.c of just keeping
a spare AVPacket for bytes already read during header parsing (cf. also
jpegxl_anim_dec.c)
Signed-off-by: Niklas Haas <git@haasn.dev>
After avformat_seek_file(), the decoder is not told what to discard
from what it decodes: the frames over the pre-roll are not reliable,
and a seek that lands inside the priming at the start of the stream
outputs the rest of it. Only some demuxers, such as mov, set the skip
for the priming themselves.
Mark the streams after a successful seek, and on the first packet of
each audio stream, add to the samples to skip, which the decoder applies
through AV_PKT_DATA_SKIP_SAMPLES:
- on the first packet of the stream, the priming, as when decoding from
the start: its exact count when it is known;
- on any other packet, the larger of the rest of the priming, which ends
at the start time of the stream, and the seek pre-roll of the codec,
as the decoder misses what came before. After the priming of a stream
whose start the seek targets, no pre-roll is added, as which packet
comes first then depends on the demuxer. Each stream compares the
target with its own start.
The first packet is only recognised when the demuxer knows the first
timestamp of the stream, such as after avformat_find_stream_info();
otherwise, the packet gets the pre-roll as after any seek. The priming
needs the timestamp of the packet and the start time of the stream, the
pre-roll does not. The skip may span several packets, as the decoder
carries it over the frames it outputs. Side data the demuxer set on the
packet is kept: the larger skip is used, and its padding at the end of
the stream is not overwritten.
The skip supplies the pre-roll the stream declares, whatever the demuxer
landed on; recovery beyond it depends on the codec and the bitstream.
The caller keeps or trims what is decoded after it as it wishes.
The decoding of the tests without accurate seek now starts one frame
later for AAC in MP4, CAF and ADTS, and two 2048 samples frames later
for HE-AAC in ADTS and with implicit SBR. A seek that lands on the last
packet of the priming of AAC in CAF, with 64 samples of it left, now
skips the 1024 samples of pre-roll instead of outputting that priming.
For Opus in Matroska, 3840 samples are skipped instead of the 312 the
decoder drops from its first frame, and the decoding with accurate seek
is cut on frames 24 samples, half a millisecond, earlier, within the
precision of the Matroska timestamps. A seek past the end of AAC in CAF
and Opus in Matroska skips the pre-roll on the last packet and keeps its
padding. A seek on the later of two streams, close to its start, adds
the pre-roll to the other stream, which is already past its own start.
A seek back to the start of Matroska now skips the exact priming on the
first packet: 312 samples for Opus and 481 for MP2.
Demuxers and decoders declare the pre-roll an audio stream needs in
AVCodecParameters.seek_preroll, but nothing used it, so a seek landed on
the frame that holds the target and the first frame decoded there was not
reliable.
Move the target back by the largest pre-roll of the audio streams that
are not discarded, whichever stream the seek is on, as they all land
near the target, without leaving the range the caller asked for or going
before the start of the stream. A finite end of the range moves back as
much as the target, and an open end stays open. The pre-roll is rounded up so that it is never shorter
than declared. The caller still discards what it decodes before the
target, as it already had to for a seek that does not land on it. Seeks
by byte or by frame number are left alone. When the start of the stream
is unknown, the earlier target may come before the first packet, so if
the seek fails, it is done again to the target the caller asked for.
The seeks of the tests now land one frame earlier for AAC, and about
80 ms earlier for Opus in Matroska. A forward seek still lands forward,
one frame earlier. A seek on the video stream just after a keyframe now
lands on the keyframe before, to leave room for the pre-roll of AAC. A
seek through concat lands earlier, as concat and the file it reads both
move the target back. A seek close to the start of ADTS, whose start is
unknown, lands at the start instead of failing.
AAC decoding overlaps each frame with the previous one, so the first
frame decoded after a seek is only reliable once the frame before it has
been decoded. Opus declares its pre-roll in AVCodecParameters.seek_preroll,
AAC did not.
Have the decoder export its pre-roll, as it knows the frame length: 1024
or 960 samples, half of it for LD and ELD, and 1024 or 768 for USAC.
ELD needs three frames, as its low delay filterbank uses a window four
frames long. SBR keeps the state of the frame before, in its
filterbanks, its high-frequency generator and its gain smoothing, and
codes its envelopes and noise floors relative to the frame before, so
it needs one more frame, of twice as many output samples when it
doubles the sample rate. Without it, the first 256 samples decoded
after a seek into HE-AAC differ from continuous decoding by -36.6 dB
with one frame of pre-roll, and not at all with two.
Implicit SBR is only known once found in a frame, so the pre-roll is
raised then, as it is for USAC, whose frame length depends on its SBR
ratio. Demuxers get it through the stream probing, whatever the
container. A larger pre-roll declared by the container is kept, and the
documentation of AVCodecContext.seek_preroll now says a decoder may
raise it.
The output of an AAC-LC stream decoded after a seek has an error of
-9.5 dB in the first 1024 samples, against -85 dB elsewhere, which one
frame of pre-roll removes.
The pre-roll tests now show the declared value, for AAC in every
container, 4096 samples for HE-AAC, explicit or implicit, 2048 for SBR
that does not double the sample rate, and 1536 for ELD.
avcodec_parameters_to_context() copied AVCodecParameters.initial_padding
into both AVCodecContext.initial_padding and AVCodecContext.delay. Since
ff98352c3d, avcodec_open2() sets skip_samples from AVCodecContext.delay
for every decoder, so a decoder opened on a stream with container
priming, such as AAC in MP4, started out with that many samples pending
to be discarded.
The two are different things. initial_padding is the encoder priming at
the start of the stream, which the demuxer already attaches to the first
packet as skip-samples side data. delay is a property of the decoder
instance, which only decoders that know their own delay export in init.
Applying the container's value at open discards it from whatever the
first decoded packet turns out to be, so seeking before the first
decode drops real samples. For a mono 44.1 kHz AAC-LC file with 2048
priming, ffmpeg -ss 530 -t 10 -i in.m4a returned 439152 instead of
441000 samples, with the audio starting 1848 samples late.
Stop copying initial_padding into delay. Decoders that export a delay,
such as libopus, native opus, wma, lc3, ape and mpegh, set it
themselves and are unaffected, and decoding from the start of a stream
is unchanged since the side data carries the priming. The documentation
of AVCodecContext.delay now says it is discarded when the decoder is
opened, and points to seek_preroll for seeking.
The seek tests of AAC in MP4 and CAF now start at the requested time
instead of late by the container's priming.
Show AVCodecParameters.seek_preroll next to initial_padding in the
stream section, and add the attribute to the XSD. Demuxers that know
a pre-roll, such as Opus, report it there; everything else prints 0.
Add a filter that removes side data of the given types from packets and from
the stream, or keeps only the packets carrying side data of any of the given
types, as the select mode of the sidedata frame filter does for frames, but
with a list of types. The types are named by their handle, as option
constants joined with '+'.
h264_nvenc, hevc_nvenc and av1_nvenc had no entry in encoders.texi.
Document them in one section: hardware input and devices, the generic
options they map, the options common to all three (presets and tuning,
rate control, lookahead, AQ and B-frames, SEI and side data), and the
options specific to each codec.
Option names, named values, ranges and defaults follow the AVOption
tables in nvenc_h264.c, nvenc_hevc.c and nvenc_av1.c, checked against
ffmpeg -h encoder=<name> from a build against nv-codec-headers 13.1.
Options that depend on the SDK version state the version they need.
Signed-off-by: lusoris <lusoris@proton.me>
The whisper filter exposes segment timestamps and text in JSON but omits the language selected by whisper.cpp. Clients consuming multilingual transcripts cannot distinguish the language of each transcription chunk without detecting it again.
Read whisper_full_lang_id() after each successful transcription and include its language code in JSON segments when available. This leaves the configured language and audio-frame metadata unchanged. Document that the code applies to the whole chunk, not to individual segments.
Tested with a minimal --enable-whisper build, make fate (without external SAMPLES), and a local WAV transcription using a multilingual tiny model.
Signed-off-by: Stefan Singer <stefan.singer@quartr.se>
It allocates the buffers of a software frame used for a transfer to or
from a hardware frames context out of a pool kept by that context.
Assisted-by: Claude
The OpenGL output device was removed, but configure still exposed --enable-opengl and probed OpenGL headers and functions.
Remove the stale option, dependency checks, configure probes, and related device metadata.
Signed-off-by: Ruonan <yu_ruonan@outlook.com>
Document the output_file_index:stream_index argument to -dec, including
zero-based numbering and the lack of general stream specifier support.
Fixes#24064.
Assisted-by: Fairy
Documenting that the directories must not be writable by untrusted users
was suggested in the report.
Documentation for umsIH5gcyBXl (AISLE-2026-0111-00168)
Found-by: Joshua Rogers <joshua.rogers@aisle.com>
This commit enables the DNN video filter to run entirely on the GPU
for the full D3D12 pipeline (decode, AI inference, and encode)
without ever copying frames back to system memory, making it
significantly faster on D3D12 hardware.
Benchmarked the new zero-copy D3D12/DirectML DNN pipeline against
the existing CPU round-trip pipeline on three major GPU vendors; up
to 10x throughput improvement was measured across every vendor.
Add dnn_onnx_d3d12, a helper that binds the DirectML execution
provider to a D3D12VA input frame's own device and does NV12
texture<->tensor conversion on the GPU (CopyTextureRegion + compute
shaders), avoiding a host round-trip.
dnn_processing negotiates this automatically for D3D12 NV12 input
with device=dml. Software-frame input retains the host I/O path.
D3D12 input requires successful DirectML negotiation; unsupported
hardware-input configurations fail during configuration. Currently
limited to single-channel FLOAT/NCHW models; chroma bypasses the
model and is copied/upscaled on the GPU. At runtime, DirectML.dll
and d3dcompiler_47.dll must be loadable from the application
directory or System32.
Example usage:
# Zero-copy pipeline
ffmpeg.exe -y -loglevel info \
-hwaccel d3d12va -hwaccel_output_format d3d12 \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_zero_copy.mp4
# CPU round-trip pipeline
ffmpeg.exe -y -loglevel info \
-init_hw_device d3d12va=dev -filter_hw_device dev -hwaccel d3d12va \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output,hwupload" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_round_trip.mp4
Signed-off-by: younengxiao <steven.xiao@amd.com>
Appends to already serialized init info side data, growing it with
av_fast_realloc(), so a demuxer can collect pssh records in linear time
without depending on the side data format.
The o key lists the directory of the input, local or remote, in the
list. Enter or a click lists the chosen directory or plays the chosen
file instead of the input once it opened, so a file that is no media
leaves the input playing. Directories come first, hidden entries are
left out, and the popup of a file shows its size.
Assisted-by: Claude
A directory listing that blocks in a protocol could not be
interrupted; avio_open_dir2() passes an interrupt callback to the
protocol like avio_open2() does.
Assisted-by: Claude
The strips were rendered by the ass filter over a color source in a
filter graph, whose generic blending took most of the time. libass now
renders each strip and its glyphs are composited over the strip
background where they cover it; a row strip at 1080p takes 0.47 instead
of 1.29 ms. ffplay suggests libass instead of the ass filter and no
longer selects the color filter.
Assisted-by: Claude
The filter picked its output format in config_props(), after format
negotiation had already settled the output link. The frames it emitted
could therefore disagree with the negotiated link format, for example
when a downstream zscale had negotiated something else.
Switch to FILTER_QUERY_FUNC2() and decide the output format during
negotiation instead:
- with format=, the output is constrained to that single format, which
is now also validated in init() against the supported output list;
- otherwise, when no half-float candidate remains on the input, the
output list is the input list, so the filter passes the format
through and downstream constraints can still narrow the input;
- otherwise the output is restricted to float, and negotiation picks
GBRPF32 or GBRAPF32 to match the input's alpha.
Half-float input has to be promoted because swscale cannot read or
write half-float in its new backend, and its legacy code clamps
half-float to [0,1], which would destroy values outside that range.
Half-float is therefore no longer offered as an output format.
The default output depends on the input format, so the query waits for
the input list to be merged with the upstream one, which it detects by
comparing it against the input link's other end. The merged list need
not be a singleton. Waiting for a singleton would reject valid graphs,
because a filter such as zscale offers half-float and integer formats
on its output independently of its input, so that list never narrows
to one of the two.
While the query is delayed it publishes the input color space, color
range, alpha mode and chroma location lists, because a converter
inserted on the input requires them, but the matching output lists are
only published together with the output pixel formats, as a converter
inserted on the output would otherwise find them without a format list
and crash.
Finally, supply opaque input alpha when the output format has alpha and
the input format does not. Prepend an alpha-initialization matrix to the
resolved OCIO processor, allowing OCIO to optimize it with the user's
transform. This avoids a separate per-frame alpha fill and ensures that
transforms which use alpha in RGB calculations or modify alpha receive
the opaque value before processing and retain their transformed alpha.
Filling output alpha after processing would instead leave alpha-dependent
RGB incorrect and overwrite alpha produced by the user's transform.
Note, this includes Fairys fairy/pr21799-replace patch (Thank-you Fairy!).
Signed-off-by: Sam Richards <Sam.Richards@taurich.org>
Assisted-by: Fairy
The rx and tx threads no longer use pthread_cancel(), so gate the
circular buffer on HAVE_THREADS instead of HAVE_PTHREAD_CANCEL. This
also drops the w32threads override.
On targets that have threads but no pthread_cancel(), such as Android,
udp inputs now start a receive thread and allocate the default 7*4096
by 188 byte fifo, as on other platforms. Set fifo_size=0 to get the
previous behavior.
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Julius Bairaktaris