An STFT is easy to write as an equation. Making thousands of overlapping transforms fast requires equal attention to shapes, traffic, allocation and scheduling.
Local spectra
A DFT of an entire recording says which frequencies occurred but not when. The short-time Fourier transform extracts overlapping frames, applies a window, and transforms each frame:
L is frame length, H hop length, N FFT size, m frame index and k frequency bin. Without padding, frame count is ⌊(S−L)/H⌋+1. A one-sided real transform yields ⌊N/2⌋+1 complex bins.
Interactive experiment
Shapes through an STFT pipeline
How do framing choices determine the output tensor and its storage?
The estimate assumes an unpadded, one-sided transform and stores real and imaginary FP32 values. It is an analytical size estimate, not a runtime benchmark.
Resolution is a choice with consequences
A longer non-zero window improves frequency discrimination but smears events over a longer interval. Bin spacing itself is set by FFT size and sample rate. A shorter hop samples the time axis more densely without changing the frame's intrinsic frequency resolution; it increases overlap and work. Zero-padding to a larger FFT interpolates spectral samples but does not recover detail absent from the observation.
Map the equation to memory
The direct conceptual tensor is [batch, frame, sample], followed by [batch, frame, frequency]. Materialising every overlapping frame repeats input values in memory. A GPU pipeline can instead gather frames into a contiguous reusable workspace, fuse gathering with window multiplication, then submit a batched real FFT.
Where time goes
The FFT is only one cost. Host-to-device transfers, temporary allocation, frame preparation, kernel launch overhead, complex-output writes and later magnitude/log scaling can dominate different regimes. Small signals may not provide enough parallel work to amortise launch overhead. Large batches may become limited by memory traffic rather than arithmetic.
Optimisation hypotheses
- Keep input and downstream features on the device.
- Reuse buffers and transform plans.
- Use contiguous batched layouts expected by the FFT library.
- Fuse windowing, padding and type conversion when measurement justifies it.
- Consider fusing magnitude and log scaling to avoid storing an intermediate complex tensor.
- Validate reduced precision against task-relevant error, not throughput alone.
Benchmark integrity
This article reports no runtime numbers because no controlled GPU benchmark was executed for it. A useful harness must disclose device, runtime and library versions; warm up plans and kernels; synchronise correctly; repeat measurements; report a distribution; and validate output against a reference. WebGPU timings would describe that browser and adapter, not CUDA performance in general.
Where the model breaks
The byte count in the experiment covers output storage only. Real peak memory includes input, window, workspaces, plans and downstream tensors. Vendor FFT implementations use algorithm-dependent work areas, and cache reuse makes a simple bytes-divided-by-bandwidth estimate incomplete. Treat analytical counts as a map for measurement, not a substitute for it.
Work through one second of audio
At 48,000 samples per second, a 1024-sample frame covers 21.33 ms. With a 256-sample hop, frame starts are 5.33 ms apart. An unpadded one-second input produces 184 frames and 513 bins per frame. The final frame starts at sample 46,848 and ends at 47,871; the remaining 128 samples do not form a complete frame.
The one-sided complex output contains 94,392 values, occupying 755,136 bytes when each complex value uses two FP32 numbers. The prepared frame tensor contains 188,416 real samples—almost four times the original input count. These are analytical counts, useful for identifying buffers worth measuring.
Choose conventions before choosing kernels
Specify whether frame times refer to starts or centres, whether the signal is padded at its edges, which Hann convention you use, whether the transform is normalised, and whether features use magnitude, power or logarithms. A periodic Hann window and a symmetric Hann window differ in their denominator; both can look plausible while disagreeing numerically.
Our calculator uses no signal-edge padding and requires FFT size at least frame length. A library may pad or centre frames by default, producing a different frame count. Larger FFT sizes interpolate frequency samples; longer non-zero windows improve the ability to separate nearby tones.
A reference pipeline before optimisation
- Enumerate complete frames at starts
mH. - Multiply each frame by the same window.
- Append zeros to reach the FFT size.
- Apply a real FFT along the last axis.
- Compute the downstream feature with its specified scaling.
Validate against an impulse, a constant input and a bin-centred tone before comparing speeds. Check complex outputs before log scaling: a floor applied before a logarithm can conceal substantial errors near zero.
Measure an asynchronous device correctly
A host timer around a GPU call may measure submission rather than completion. Use device events or explicit synchronisation at the measurement boundaries. Report preparation, transfer and transform costs separately when diagnosing a pipeline, and also measure the complete path relevant to the application. A faster FFT can still yield a slower end-to-end pipeline if it requires more copying.
Further reading
PyTorch's STFT reference documents centring, padding, window and complex-output choices. Compare those conventions with the calculator before using its dimensions to size a library pipeline.