example_1 DQN won't learn Pong

I'm trying to run the examples as a sanity check but they don't seem to be learning. I tried examples 1 and 3.

The only change I made to example 1 was a smaller replay buffer so it'd fit in memory:

algo = DQN(min_steps_learn=1e3, replay_size=int(3e5)) # Run with defaults.

I also ran example_3 using the serial sampler for over 100k iterations without any apparent improvement

Hope you might be able to point out what's wrong, or maybe the examples broke for some reason?

Provide feedback