Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation

This work develops an Actor-Learner Distillation procedure that leverages a continual form of distillation that transfers learning progress from a large capacity learner model to a small capacity actor model in the context of partially-observable environments.