avfilter/vf_nnedi: fix a heap overflow in the prescreener buffer

process_new() writes the prescreener flags four at a time, but the
buffer was only as wide as the plane. If the width is not a multiple
of four, up to three flags were written past the end of it.

Found with ASan using the default settings, for example on gray
321x120.

Found during a code review sweep of ffmpeg using an LLM.

Signed-off-by: romainhedouin <romain.hedouin@epitech.eu>
This commit is contained in:
romainhedouin authored and michaelni committed 2026-11-01 04:00:54 +00:00
1 parent 0783904188
commit dec1770703
1 file changed
+1 -1
+1 -1
View File
@@ -1096,7 +1096,7 @@ static int config_input(AVFilterLink *inlink)
return AVERROR(ENOMEM);
for (int i = 0; i < s->nb_threads; i++) {
s->prescreen_buf[i] = av_calloc(s->planewidth[0], sizeof(**s->prescreen_buf));
s->prescreen_buf[i] = av_calloc(FFALIGN(s->planewidth[0], 4), sizeof(**s->prescreen_buf));
if (!s->prescreen_buf[i])
return AVERROR(ENOMEM);
}