52 lines
2.8 KiB
Plaintext
52 lines
2.8 KiB
Plaintext
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
# Copyright (C) 2026 David (Corey) Herrera
|
|
# attention_float16b.cbtable
|
|
# The CB descriptor contract the FLASH-DECODE ATTENTION compute kernel is compiled
|
|
# against (resident_attention_compute.c). It generates ft_tensix_cb_formats.h and MUST
|
|
# match, byte for byte, the L1 CB config array ft_tt_op_attention_place_cbs() writes at
|
|
# launch. There is no runtime check (docs/tt_kernel_model.md §11).
|
|
#
|
|
# EVERY BUFFER IS Float16_b, AND THAT IS WHAT LICENSES THE ABSENT reconfig_data_format().
|
|
# The kernel mixes a matmul (reversed source order) with row reduces, broadcast-column
|
|
# multiplies and eltwise adds. Upstream's own sdpa kernel reconfigures the unpacker
|
|
# between those phases -- because it carries operands of different formats. Here one
|
|
# format serves every buffer, exactly as in reduce_float16b.cbtable, so the reconfig has
|
|
# nothing to reprogram. If a shape ever scores wrong, that is the first line to add back.
|
|
#
|
|
# dst_accum_mode 1 IS REQUIRED HERE, NOT PREFERRED.
|
|
# The running max, the running row sum and the output accumulator are all carried across
|
|
# EVERY KV block, so a 16-bit destination accumulator would re-round all three once per
|
|
# block -- docs/tt_bringup_log.md §19's failure mode multiplied by the block count (128
|
|
# blocks at 32k context, against the 144 K-tiles that grew a matmul's error 159x). §19
|
|
# also measured the cost of fp32 accumulate at zero wall clock on a data-movement-bound
|
|
# op, which this is.
|
|
#
|
|
# math_fidelity 4 is HiFi4, recorded rather than inherited (§16). It is free here for the
|
|
# same reason it is free on the matmul: one HiFi4 tile matmul is 64 cycles = 80 ns at
|
|
# 800 MHz, and the reader delivers a tile pair every ~104 ns.
|
|
|
|
math_fidelity 4
|
|
approx 0
|
|
dst_accum_mode 1
|
|
|
|
# in index format tile geometry
|
|
cb 0 Float16_b 32 32 16 4 # cb_q the packed query, as the reader built it
|
|
cb 1 Float16_b 32 32 16 4 # cb_k one K block
|
|
cb 2 Float16_b 32 32 16 4 # cb_v one V block
|
|
cb 3 Float16_b 32 32 16 4 # cb_const scaler, 1/sqrt(d), tail mask, dead mask
|
|
cb 14 Float16_b 32 32 16 4 # cb_qs the scaled query
|
|
|
|
# intermediates
|
|
cb 4 Float16_b 32 32 16 4 # cb_s scores, then the exponentials in place
|
|
cb 5 Float16_b 32 32 16 4 # cb_scratch the mask bounce
|
|
cb 6 Float16_b 32 32 16 4 # cb_m_a running max, ping
|
|
cb 7 Float16_b 32 32 16 4 # cb_m_b running max, pong
|
|
cb 8 Float16_b 32 32 16 4 # cb_l running row sum
|
|
cb 9 Float16_b 32 32 16 4 # cb_lsum this block's row sum
|
|
cb 10 Float16_b 32 32 16 4 # cb_c exp(m_old - m_new), then 1/l
|
|
cb 11 Float16_b 32 32 16 4 # cb_o the output accumulator
|
|
cb 12 Float16_b 32 32 16 4 # cb_pv this block's P.V
|
|
|
|
# out
|
|
cb 16 Float16_b 32 32 16 4 # cb_out
|