# SPDX-License-Identifier: AGPL-3.0-or-later # Copyright (C) 2026 David (Corey) Herrera # attention_float16b.cbtable # The CB descriptor contract the FLASH-DECODE ATTENTION compute kernel is compiled # against (resident_attention_compute.c). It generates ft_tensix_cb_formats.h and MUST # match, byte for byte, the L1 CB config array ft_tt_op_attention_place_cbs() writes at # launch. There is no runtime check (docs/tt_kernel_model.md §11). # # EVERY BUFFER IS Float16_b, AND THAT IS WHAT LICENSES THE ABSENT reconfig_data_format(). # The kernel mixes a matmul (reversed source order) with row reduces, broadcast-column # multiplies and eltwise adds. Upstream's own sdpa kernel reconfigures the unpacker # between those phases -- because it carries operands of different formats. Here one # format serves every buffer, exactly as in reduce_float16b.cbtable, so the reconfig has # nothing to reprogram. If a shape ever scores wrong, that is the first line to add back. # # dst_accum_mode 1 IS REQUIRED HERE, NOT PREFERRED. # The running max, the running row sum and the output accumulator are all carried across # EVERY KV block, so a 16-bit destination accumulator would re-round all three once per # block -- docs/tt_bringup_log.md §19's failure mode multiplied by the block count (128 # blocks at 32k context, against the 144 K-tiles that grew a matmul's error 159x). §19 # also measured the cost of fp32 accumulate at zero wall clock on a data-movement-bound # op, which this is. # # math_fidelity 4 is HiFi4, recorded rather than inherited (§16). It is free here for the # same reason it is free on the matmul: one HiFi4 tile matmul is 64 cycles = 80 ns at # 800 MHz, and the reader delivers a tile pair every ~104 ns. math_fidelity 4 approx 0 dst_accum_mode 1 # in index format tile geometry cb 0 Float16_b 32 32 16 4 # cb_q the packed query, as the reader built it cb 1 Float16_b 32 32 16 4 # cb_k one K block cb 2 Float16_b 32 32 16 4 # cb_v one V block cb 3 Float16_b 32 32 16 4 # cb_const scaler, 1/sqrt(d), tail mask, dead mask cb 14 Float16_b 32 32 16 4 # cb_qs the scaled query # intermediates cb 4 Float16_b 32 32 16 4 # cb_s scores, then the exponentials in place cb 5 Float16_b 32 32 16 4 # cb_scratch the mask bounce cb 6 Float16_b 32 32 16 4 # cb_m_a running max, ping cb 7 Float16_b 32 32 16 4 # cb_m_b running max, pong cb 8 Float16_b 32 32 16 4 # cb_l running row sum cb 9 Float16_b 32 32 16 4 # cb_lsum this block's row sum cb 10 Float16_b 32 32 16 4 # cb_c exp(m_old - m_new), then 1/l cb 11 Float16_b 32 32 16 4 # cb_o the output accumulator cb 12 Float16_b 32 32 16 4 # cb_pv this block's P.V # out cb 16 Float16_b 32 32 16 4 # cb_out