swscale/ops: fix merge_comp_flags() for SWS_COMP_SWAPPED

This violates the documentation (monoid property).

It's a bit arbitrary whether to consider this an OR-type or AND-type flag,
since mixing swapped and non-swapped components is almost surely a bug, but
keeping it as an OR-type makes sure such cases at least show up in the
result.

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This commit is contained in:
Niklas Haas
2026-06-21 16:51:03 +02:00
committed by Niklas Haas
parent 2a6ed5e29c
commit 08c90c60f7

View File

@@ -281,7 +281,7 @@ enum {
/* merge_comp_flags() forms a monoid with SWS_COMP_IDENTITY as the null element */
static SwsCompFlags merge_comp_flags(SwsCompFlags a, SwsCompFlags b)
{
const SwsCompFlags flags_or = SWS_COMP_GARBAGE;
const SwsCompFlags flags_or = SWS_COMP_GARBAGE | SWS_COMP_SWAPPED;
const SwsCompFlags flags_and = SWS_COMP_IDENTITY;
return ((a & b) & flags_and) | ((a | b) & flags_or);
}