How do I become better at counter play when I'm down in material? How do I practice counter play when I'm down material?
Counter play, especially when down material, is often about creating dynamic chances. So you should probably work in that area:


Study the games of some attacking geniuses like Tal, Shirov, Morphy, Anderssen.
Work through some good books on attacking chess.
Play some gambits in training games. 
Train tactics, but thats a given anyway.