Use attention, positional information, encoder-decoder structures, and transformer training objectives. Implement a small transformer and compare its computational and modeling tradeoffs with recurrent networks.
