Deformable Grouped Query Attention
- Implemented deformable attention with grouped-query support. This is the first known instance of deformable attention enhanced with grouped-query attention, demonstrating a novel architectural combination for improved efficiency.
- Experiments on improving TEP-Net, the current SOTA in rail vehicle ego path detection. Added confidence scores, improved out-of-distribution errors when a leading vehicle is close, worked on adding memory to handle otherwise ambiguous cases.
- Contributed
torch.compile
integration back to the original TEP-Net, reducing training time by 17.7% on RTX 4090, with potentially greater speedups on datacenter GPUs.
- Solutions to most of the questions and many of the practical exersises in the textbook Deep Reinforcent Learning (2022) by Aske Plaat. Made as a reference for people working through these exercises. At the time of writing, the only one on the internet.