We have learned a lot about DQN. We started off with vanilla DQN and then we saw various improvements such as double DQN, dueling network architecture, and prioritized experience replay. We have also learned to build DQN to play Atari games. We stored the agent's interactions with the environment in the experience buffer and made the agent learn from those experiences. But the problem was, it took us a lot of training time to improve performance. For learning in simulated environments, it is fine, but when we make our agent learn in a real-world environment it causes a lot of problems. To overcome this, a researcher from Google's DeepMind introduced an improvement on DQN called deep Q learning from demonstrations (DQfd).

If we already have some demonstration data, then we can directly add those demonstrations to the experience replay...