A new technique called Dust enables the pretraining of transformer models without backpropagation by utilizing local, asynchronous optimization methods that are more biologically plausible. This approach bypasses certain mathematical limitations of traditional backpropagation, such as the conditioning of the Hessian matrix. However, researchers caution that it may currently suffer from lower computational and sample efficiency on modern GPU hardware compared to standard methods.