A level works in place on the lowpass lattice of the previous one, so
the mode needs 5 planes instead of 9 + 3 * levels and about an order of
magnitude less computation. Shrinkage is less effective without the
shift invariance of the undecimated transform.
Assisted-by: Claude
Every pass over a plane, the row and column passes of the transforms as
well as the box filters and the per coefficient operations of the
shrinkage, is split into slices of lines. The output does not depend on
the number of threads.
Assisted-by: Claude
Lowpass coefficients are stored at even and highpass ones at odd
positions of the line, so a level works in place on the lattice of the
previous level's lowpass coefficients. Reconstruction from a single
polyphase set needs no averaging, hence no factor 0.5.
Assisted-by: Claude
Dividing by the attenuation amplifies the noise and the quantization
steps of the dark areas by the same factor. The wavelet mode performs
the division in the pixel domain, then transforms the result and its
per pixel noise variance, propagated through the squared filter taps as
if the samples were uncorrelated, with the undecimated 9/7 wavelet.
Each detail coefficient is Wiener shrunk toward 0 with a signal
variance estimated from a 7x7 box of coefficients at the dilation of
the level, weighted by their precision and counted only when it exceeds
the noise by 3 standard errors of independent coefficients. The lowpass
of the top level averages over large areas and is kept, so the shade of
heavily attenuated areas survives.
Assisted-by: Claude
Makes the transform usable by other filters. The analysis filter taps become
a parameter so the same code can also propagate variances with squared taps.
Assisted-by: Claude
The size and checksum of the encoded file pin the slice layout and the
remap coding of a 704x576 float frame, which the 360x288 default slice
limit splits into 2x2 slices.
The escape code of a Golomb-Rice residual is 12 + bits long and
put_bits() writes at most 31 bits, so bits must stay below 20. The RCT
planes of remapped 32bit floats need log2 of the summed remap counts of
two planes, which a slice of at most 1<<18 pixels keeps within 19 bits.
The default slice size of 360x288 is within that, larger slices requested
with -slices were written with escape codes longer than put_bits()
supports. The assertion on k follows the same limit.
The testcases reproduce since b0c77e5a12.
Fixes: out of array read
Fixes: fAak9zecyttL/testcase/poc.vvc
Fixes: fAak9zecyttL/testcase/poc-entry-points.vvc
Fixes: fAak9zecyttL
Found-by: mzfr <security@mzfr.me>
Fixes: heap-use-after-free
Fixes: YtyT0QH1InrQ
Fixes: ANT-2026-13DK7NZD
Fixes: poc.264
Found-by: Claude and Ada Logics
Use after free Replicated through UnModified FFmpeg with ASAN
Fixes: negative-size-param
Fixes: assertion failure
Fixes: wzomtdUBffdP
Fixes: ANT-2026-G2YFQ5N2
Fixes: poc.exr
Found-by: Claude and Ada Logics
Negative size memcpy and assertion failure Replicated through UnModified FFmpeg with ASAN
Fixes: heap-buffer-overflow (read)
Fixes: BIqbuSbTJqAH
Fixes: ANT-2026-V2EPGE9M
Fixes: poc.avi
Found-by: Claude and Ada Logics
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: heap-buffer-overflow (write)
Fixes: GITTfi6VsK4u
Fixes: ANT-2026-5KX1BXM6
Fixes: poc.flv
Found-by: Claude and Ada Logics
Out of array write Replicated through UnModified FFmpeg with ASAN
Found during triage/review of the security report NzELYSvufGfQ
(ANT-2026-324B3XNF), which does not report this.
Fixes: pointer overflow
Pointer overflow Replicated through UnModified FFmpeg with UBSAN
Fixes: heap-buffer-overflow (read)
Fixes: NzELYSvufGfQ
Fixes: ANT-2026-324B3XNF
Fixes: ant-324b3xnf-avrn-64x32-flipped.avi
Found-by: Claude and Ada Logics
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: negative-size-param
Fixes: left shift of negative value
Fixes: 0IMcpiRlVAb5
Fixes: ANT-2026-QY2HYS5A
Fixes: poc.mkv
Found-by: Claude and Ada Logics
Negative size memcpy Replicated through UnModified FFmpeg with ASAN and UBSAN
Fixes: negative-size-param
Fixes: left shift of negative value
Fixes: 0IMcpiRlVAb5
Fixes: ANT-2026-QY2HYS5A
Fixes: poc.mkv
Found-by: Claude and Ada Logics
Negative size memcpy Replicated through UnModified FFmpeg with ASAN and UBSAN
Fixes: heap-buffer-overflow (read)
Fixes: q0RFgfMenefI
Fixes: ANT-2026-HRT03RV0
Fixes: vvc-subpic-tiles.mkv
Found-by: Claude and Ada Logics
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: heap-buffer-overflow (read and write)
Fixes: signed integer overflow
Fixes: IhY7QcsV5nva
Fixes: ANT-2026-RYWQTXHH
Fixes: poc-median-65600x4.pgm
Found-by: Claude and Ada Logics
Out of array access Replicated through UnModified FFmpeg with ASAN and UBSAN
Fixes: heap-buffer-overflow (read)
Fixes: G8jPOZLbzYtj
Fixes: ANT-2026-22YEN9NA
Fixes: lcl-zlib-mt-shortlace.mkv
Found-by: Claude and Ada Logics
Out of array read Replicated through UnModified FFmpeg with ASAN
Exercise the short DVD PCM tail, one-sample TTA frame, early RoQ flush and mono Microsoft ADPCM trellis paths fixed by PR 25694. Existing audio round trips do not cover these boundaries.
Generate input with sine and atrim, and compare packets through the framecrc muxer. No new samples or bitstream generator are needed.
Assisted-by: Fairy
Fixes: signed integer overflow
Fixes: infinite loop
Fixes: dVqGncTUAyg9
Fixes: AISLE-2026-0150-00591
Signed integer overflow and infinite loop Replicated through API with UBSAN
Found-by: Joshua Rogers (AISLE)
Fixes: use of uninitialized memory
Fixes: 98u4sU59NEf4
Fixes: AISLE-2026-0150-00113
Use of uninitialized memory Replicated through UnModified FFmpeg with Valgrind
Found-by: Joshua Rogers (AISLE)
Encode the buffered samples once and return no packet when the buffer
is empty.
Fixes: use of uninitialized memory
Fixes: X43TLCpjYK88
Fixes: AISLE-2026-0150-00397
Use of uninitialized memory Replicated through UnModified FFmpeg with Valgrind
Found-by: Joshua Rogers (AISLE)
The fix was suggested in the report.
24 bit samples are stored in blocks of samples_per_block samples
and the sample count of a frame is rounded down to whole blocks. A last
frame with fewer samples than a block gives zero blocks, the do/while
loops still run and read past the frame until the block count wraps.
Return no packet for such a frame.
Fixes: out of array read
Fixes: ZplUzGvg2XP9
Fixes: AISLE-2026-0150-00112
Out of array read Replicated through UnModified FFmpeg with ASAN
Found-by: Joshua Rogers (AISLE)
Fixes: out of array read
Fixes: 2dyV2eNJkKBV
Fixes: AISLE-2026-0150-00092
Out of array read Replicated through API with ASAN
Found-by: Joshua Rogers (AISLE)
This is not a security issue: the leak needs the frame allocation to
fail.
Fixes: memleak
Fixes: 9UXawhjlJguX
Fixes: AISLE-2026-0150-00349
Memory leak Replicated through API with LSAN
Found-by: Joshua Rogers (AISLE)
Fixes: out of array write
Fixes: MymIHidCHVJu
Fixes: AISLE-2026-0150-00348
Out of array write Replicated through UnModified FFmpeg (32-bit) with ASAN
Found-by: Joshua Rogers (AISLE)
Fixes: out of array write
Fixes: fglxAK3pz5sP
Fixes: AISLE-2026-0150-00462
Out of array write Replicated through UnModified FFmpeg (32-bit) with ASAN
Found-by: Joshua Rogers (AISLE)
Fixes: out of array write
Fixes: GGsIHTeUwf0U
Fixes: AISLE-2026-0150-00642
Out of array write Replicated through API with ASAN
Found-by: Joshua Rogers (AISLE)
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Use of uninitialized memory Replicated through API with MSAN
Fixes: use of uninitialized memory
Fixes: ln0NC22XPto3
Fixes: AISLE-2026-0150-00420
Found-by: Joshua Rogers (AISLE)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Fixes: use of uninitialized memory
Fixes: aWhU4kuts0tn
Fixes: AISLE-2026-0150-00040
Found-by: Joshua Rogers (AISLE)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Excessive memory allocation (peak RSS 2.58 GB from a 52 byte file)
Replicated through UnModified FFmpeg with ASAN
Fixes: out of memory
Fixes: 2cNjGsVIMOVX
Fixes: AISLE-2026-0150-00053
Found-by: Joshua Rogers (AISLE)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Excessive memory allocation (peak RSS 2.3 GB from a 69 byte script)
Replicated through UnModified FFmpeg with ASAN
Fixes: out of memory
Fixes: 6m4Av2yQJEhE
Fixes: AISLE-2026-0150-00550
Found-by: Joshua Rogers (AISLE)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The limit is libacm's MAX_BLOCK_LEN (1024 * 1024 samples, decode.c);
libacm is the decoder this one derives from and is what the games'
modding tools use, so no file decodable by it is rejected. Real files
from Baldur's Gate, Baldur's Gate II, Icewind Dale, Icewind Dale II and
Planescape: Torment use level 7 or 8 with 16 rows, blocks of 2048 or
4096 samples, and the snd2acm / DLTCEP encoder defaults to level 7 with
16 rows, 256 times below the limit.
Excessive memory allocation (peak RSS 944 MB from a 78 byte file)
Replicated through UnModified FFmpeg with ASAN
Fixes: out of memory
Fixes: fi1TMFhPx2mP
Fixes: AISLE-2026-0150-00274
Found-by: Joshua Rogers (AISLE)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
A code table whose lengths add up to more than the block size advanced
dst and dst_size past the end of the block. The copies were already
limited to the block, so no memory was written out of bounds, but forming
a pointer beyond one past the end of an array is undefined behaviour.
This is not a security issue.
Found during triage of the security report V9Z2ZdGQsUo6
Fixes: V9Z2ZdGQsUo6
Fixes: use of uninitialized memory
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Fixes: V9Z2ZdGQsUo6
Fixes: AISLE-2026-0150-00048
Fixes: use of uninitialized memory
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Found-by: Joshua Rogers (AISLE)
Fixes: Wq9WI8WzfMaj
Fixes: AISLE-2026-0150-00106
Fixes: use of uninitialized memory
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Found-by: Joshua Rogers (AISLE)
Fixes: Ao8AXNwB2RlH
Fixes: AISLE-2026-0150-00011
Fixes: use of uninitialized memory
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Found-by: Joshua Rogers (AISLE)
Fixes: 8VUuZb6oh1t5
Fixes: AISLE-2026-0150-00187
Fixes: disclosure of previous frame content
Disclosure of previous frame content Replicated through UnModified FFmpeg
(image sequence of 40 lossy images with alpha 0xAA followed by one with a
4 byte uncompressed ALPH chunk, the alpha plane of the last frame shows
0xAA)
Found-by: Joshua Rogers (AISLE)
Fixes: pqQviaXoqzKF
Fixes: AISLE-2026-0150-00236
Fixes: use of uninitialized memory
Use of uninitialized memory Replicated through UnModified FFmpeg with MSAN
Found-by: Joshua Rogers (AISLE)
Add a NEON implementation of the fixed-point DCA LFE FIR interpolation
filter.
Each vector lane holds one of four consecutive LFE samples, so the
filter coefficients are used as by-element multipliers and can be
loaded contiguously without any permutes. One load of c[8j..8j+7] and
c[248-8j..255-8j] covers outputs j, 32+j, 31-j and 63-j. The 8-tap
sums are accumulated in 64 bits with SMULL/SMLAL, rounded with SQRSHRN,
clipped to 23 bits with SQSHL/SSHR and written with ST2 lane stores.
Recent clang versions auto-vectorize the C version, so the speedup
depends a lot on the compiler. On Neoverse-N1 (64 LFE samples):
C compiler speedup
clang 14 4.84x
gcc 11 1.75x
clang 20 1.21x
clang 22 1.21x
Signed-off-by: Jeongkeun Kim <variety0724@gmail.com>
Michael Niedermayer