The CLEAR codegen iterated components by value[i].den, while the
constants pass iterates by clear.mask; unmasked components may hold
leftover values with a nonzero denominator, consuming more constant
IDs than were registered and emitting ID 0 into the instruction
stream.
The image handle arrays were also sized by the number of planes an
op touches, but plane_src/plane_dst contain actual frame plane
indices, so a pass writing only e.g. the alpha plane references
handle 3 while only handle 0 was loaded.
Either results in invalid SPIR-V, which crashes RADV inside
spirv_to_nir when creating the shader object.
(cherry picked from commit b2b0429d15)
No longer needed with AVRational64. Now we can just apply_op_q(), on
everything except floating point types (which are still not representable
in AVRational64 and likely never will be, though this is mostly a moot
point as the uops layer can still optimize this further).
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
1. This is currently impossible to trigger
2. We're about to switch to AVRational64, eliminating this concern
3. The AVRational64 API intentionally doesn't expose av_reduce64()
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This formulation is generally preferred as it avoids the risk of (1 << depth)
overflowing when depth is 32/64/etc.
I also wanted to have these in a common place to make the upcoming changes
easier.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
These come with some challenging ambiguity in the following patch to switch
from AVRational to AVRational64, so best just avoid them and have individual
usage sites define them locally with the correct type.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This is needed by the ops code, to represent intermediate values for 32-bit
formats, which can exceed the value range of int32_t (especially for
intermediate products).
I copied the math almost 1:1 from rational.c, but adapted to use the 128-bit
integer wrappers defined by int128.h. I added a generous amount of tests in
any case.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This fallback function is used if external MMX is available,
while inline MMX and intrinsics for emitting emms are unavailable.
It is implemented as an avpriv function, which has several
drawbacks for shared builds:
1. The function is so small (3 bytes; 16 with padding)
that the overhead of exporting and importing it dwarfs
the gains from code deduplication.
2. A call to an external library has more overhead than
a library-internal one.
3. It may cause linking failures when a libavutil not exporting
avpriv_emms_asm() is paired with a library needing it
(if inline assembly and intrinsics were unavailable when building
the dependent library). I am not aware of this ever happening.
4. We would be forced to keep avpriv_emms_asm() around for ABI stability
even after it is no longer needed.
This commit therefore uses the STLIBOBJS, SHLIBOBJS approach
to duplicating it into each library on its own if needed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This updates uops_macros.h and the graph.c implementation in lockstep,
otherwise we'd have an intermediate commit with a bunch of broken formats.
Overall speedup=1.008x faster, min=0.144x max=5.550x
The min/max numbers are mostly measurement noise, but the real speedup for
affected formats is anywhere from 0.9x to around 2x-3x.
It's worth noting that the speedup for the formats which currently regress
is because we don't yet refcopy the planes, but I have another series in the
works which will take care of this soon.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This already helps performance as-is, but will help performance massively
once we add the ability for the memcpy backend to do a refcopy instead of
an actual copy.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
Can be used to extract a reduced subset of operations affecting only certain
output planes, e.g. splitting an op list into a "memcpy" and a "non-memcpy"
part, or splitting apart op lists for independent or subsampled planes.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
If the filter cannot actually be optimized into the read (for whatever
reason), this code would previously loop infinitely. Bail out cleanly
instead.
The FFSWAP is there to make the error message print the remainder (the one
containing unsplittable ops), rather than the noop list.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
Instead of a loop with fixed structure, this function now recursively calls
itself as many times as needed to satisfy all criteria.
This is absolutely needed for the upcoming refactor which will allow for
also splitting apart ops lists as needed to e.g. handle partially subsampled
ops lists, which may need a complex sequence of filtering and merge steps
to be fully satisfied.
This does modify the way in which subpasses are compiled slightly, in that
each new subpass first tried again un-split, rather than a single split
resulting in all subsequent passes being split as well. This is mostly a
benign change, though it might matter one day.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
We already have the unoptimized reference ops; printing each intermediate
stage here is just noise that makes this file harder to scroll through IMO.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This will make it easier to keep passing around these parameters in helper
functions in the upcoming refactor.
Take the opportunity to also rename the plain `compile` function to
`compile_single`.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
Otherwise, this will false negative if the redundant operations haven't
been optimized away yet, resulting in unnecessary memcpy operations.
Fixes: a534156083
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
Makes ff_sws_compile_pass() more robust; will be needed for plane splitting.
Besides, it's perfectly valid to have an operation list that starts with
e.g. SWS_OP_CLEAR.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This prevents the generation of a few more duplicate functions (where
there would be both f32 and u32 functions).
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Ramiro Polla <ramiro.polla@gmail.com>
There is no easy optimization that can be triggered by knowing that the
offset is exactly 1. This led to identical functions being instantiated
for different params.
Also simplified the AVRational comparisons a bit.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Ramiro Polla <ramiro.polla@gmail.com>
The mask for swizzle ops assumed that merely having a component assigned
to itself was enough to detect whether the swizzle was needed for that
component, but that wasn't correct. We should also take into account
whether the component is needed for the next operation or not.
Additionally, prevent duplicate functions from being generated by
clearing the swizzle index for unused components.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Ramiro Polla <ramiro.polla@gmail.com>
These functions are essentially the same as single-component planar
read/write, and are actually never instantiated. This was left over
from the initial implementation.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Ramiro Polla <ramiro.polla@gmail.com>
The AVX2 is a fairly straightforward vpgatherdd + 4x4 transpose. The SSE4
fallback is an unrolled scalar loop, for lack of anything better to do.
checkasm:
- CPU: AMD Ryzen 9 9950X3D 16-Core Processor (00B40F40)
- Timing source: x86 (rdtsc)
- Bench duration: 10000 µs per function (45898205 cycles)
- Random seed: 2518020648
Benchmark results:
name cycles (vs ref)
u8_read_palette_xyzw_c: 2877.5
u8_read_palette_xyzw_x86_sse4: 1951.9 ( 1.47x)
u8_read_palette_xyzw_x86_avx2: 1051.6 ( 2.74x)
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This is handled using the new SWS_RW_PALETTE read op mode. We need to be a bit
careful to use the correct pixfmt descriptor downstream, because the descriptor
for PAL8 itself merely describes the *index*, rather than the actual data
values.
Accomplish this by introducing a new function to map the palette format to the
resulting pixel format after applying the palette (explicitly documented as
AV_PIX_FMT_RGB32).
+pal8 16x16 -> rgb24 16x16:
+ [ u8 +++X] SWS_OP_READ : 4 elem(s) palette >> 0
+ min: {0 0 0 _}, max: {255 255 255 _}
+ [ u8 +++X] SWS_OP_SWIZZLE : 2103
+ min: {0 0 0 _}, max: {255 255 255 _}
+ [ u8 XXXX] SWS_OP_WRITE : 3 elem(s) packed >> 0
+ (X = unused, z = byteswapped, + = exact, 0 = zero)
+ translated micro-ops:
+ u8_read_palette_xyzw
+ u8_permute_xz_zx
+ u8_write_packed_xyz
...
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This does not actually generate any code yet as the macro is still empty,
but that will change once I add support for generated palette reads to
the format handling code. This logic merely needs to be in place first
to avoid introducing broken intermediate states where palette uops are
generated but not implemented by the reference backend.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
This commit only adds the uop itself; it does not yet add any implementations.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
