An STFT is easy to write as an equation. Making thousands of overlapping transforms fast requires equal attention to shapes, traffic, allocation and scheduling.
Local spectra
A DFT of an entire recording says which frequencies occurred but not when. The short-time Fourier transform extracts overlapping frames, applies a window, and transforms each frame:
L is frame length, H hop length, N FFT size, m frame index and k frequency bin. Without padding, frame count is ⌊(S−L)/H⌋+1. A one-sided real transform yields ⌊N/2⌋+1 complex bins.
Interactive experiment
Shapes through an STFT pipeline
How do framing choices determine the output tensor and its storage?
The estimate assumes an unpadded, one-sided transform and stores real and imaginary FP32 values. It is an analytical size estimate, not a runtime benchmark.
Resolution is a choice with consequences
A longer frame places DFT bins closer together but smears events over a longer interval. A shorter hop samples the time axis more densely without changing the frame's intrinsic frequency resolution; it increases overlap and work. Zero-padding to a larger FFT interpolates spectral samples but does not recover detail absent from the observation.
Map the equation to memory
The direct conceptual tensor is [batch, frame, sample], followed by [batch, frame, frequency]. Materialising every overlapping frame repeats input values in memory. A GPU pipeline can instead gather frames into a contiguous reusable workspace, fuse gathering with window multiplication, then submit a batched real FFT.
Where time goes
The FFT is only one cost. Host-to-device transfers, temporary allocation, frame preparation, kernel launch overhead, complex-output writes and later magnitude/log scaling can dominate different regimes. Small signals may not provide enough parallel work to amortise launch overhead. Large batches may become limited by memory traffic rather than arithmetic.
Optimisation hypotheses
- Keep input and downstream features on the device.
- Reuse buffers and transform plans.
- Use contiguous batched layouts expected by the FFT library.
- Fuse windowing, padding and type conversion when measurement justifies it.
- Consider fusing magnitude and log scaling to avoid storing an intermediate complex tensor.
- Validate reduced precision against task-relevant error, not throughput alone.
Benchmark integrity
This article reports no runtime numbers because no controlled GPU benchmark was executed for it. A useful harness must disclose device, runtime and library versions; warm up plans and kernels; synchronise correctly; repeat measurements; report a distribution; and validate output against a reference. WebGPU timings would describe that browser and adapter, not CUDA performance in general.
Where the model breaks
The byte count in the experiment covers output storage only. Real peak memory includes input, window, workspaces, plans and downstream tensors. Vendor FFT implementations use algorithm-dependent work areas, and cache reuse makes a simple bytes-divided-by-bandwidth estimate incomplete. Treat analytical counts as a map for measurement, not a substitute for it.