An operation count describes an algorithm. Runtime also depends on where values live, how often they move, and which execution units can process them.
An FFT has stages—and a memory story
A radix-2 FFT of length N has log₂N stages with N/2 butterflies per stage. A butterfly mixes two complex values with a phase factor. Its arithmetic is small; loading and storing those values may be a substantial part of its cost.
Suppose every stage is a separate kernel that reads and writes a complete complex FP32 buffer. Each complex value occupies eight bytes, so stage traffic is 16N log₂N bytes per transform. An ideal external-I/O floor is one input read and one output write: 16N bytes. These are a deliberately simple comparison, not bounds on every implementation's total traffic.
Interactive experiment
Count traffic before timing it
How much data could a stage-by-stage FFT move?
Radix-2 complex FP32 model: each stage reads and writes the whole buffer once. The ideal column counts only one input read and output write. Neither is a measured runtime or a complete convolution pipeline.
Interpret the gap carefully
For length 1024, there are ten stages. Our model gives 163,840 bytes of stage traffic versus 16,384 bytes at the ideal floor per transform. Increasing batch multiplies both counts. This gap identifies an opportunity for reuse; it does not predict a tenfold speedup. Caches may already serve intermediate values, and fused kernels can be constrained by register pressure or local memory.
Arithmetic intensity is a useful question
Arithmetic intensity measures operations per byte moved at a specified memory boundary. Low intensity can limit throughput even when a device advertises enormous arithmetic capacity. The relevant bandwidth might be device memory, shared memory or another level; stating the boundary prevents misleading comparisons.
Conversely, replacing scalar operations with a larger matrix formulation can increase arithmetic while using specialised matrix units much more effectively. Extra work can be worthwhile when it raises utilisation enough and avoids expensive data movement.
What FlashFFTConv contributes
FlashFFTConv (2023; ICLR 2024) factorises FFT convolution into matrix operations suitable for tensor cores and arranges fusion to reduce memory I/O. It also studies partial and frequency-sparse convolutions. Exact convolution and sparse variants are different numerical contracts; skipping frequency blocks can change the operation.
The paper's measured improvements belong to its hardware, workloads and baselines. Our traffic figure implements neither its decomposition nor its kernels. It explains why moving beyond asymptotic arithmetic can change performance.
Apply the idea to an STFT
In an STFT pipeline, framing, windowing, padding, FFT and magnitude extraction may each introduce a buffer. Begin with a correct reference and profile the complete path. Fusing windowing into preparation may remove a write/read pair; reusing a buffer may avoid allocation; keeping features on device may remove a transfer. Each optimisation addresses a distinct cost.
A reproducible comparison
Hold operation, shapes, dtype and tolerance fixed. Warm up plans and kernels, time completed device work, repeat trials, and report dispersion as well as a typical value. Measure both individual kernels and the intended end-to-end use. A faster isolated transform is not sufficient if layout conversion consumes the saving.
The next experiment should compare implementations on available hardware. Until those measurements exist, these byte counts remain analytical estimates.