The knowing whether it hit the upper or lower pipe combined with the generalization that if it hits the top it needs to flap less and if it hits the bottom it needs to flap more were the parts I was missing.
The simple environment and neural network help a whole lot here. By knowing just how tweaking the weights will effect the output behavior and that it's a matter of only signal frequency you can make progress toward error reduction.
As you start using more complex networks, perhaps with many inputs/outputs and multiple wide layers, or in more complicated environments, perhaps even ones where some or all outputs have magnitude and are not just binary, knowing how to tweak what and when quickly becomes astronomically difficult.
Have you done anything with larger networks in richer environments?