There's a couple problems with VR:
1: 40% of the population cannot process 3D correctly, rendering VR pointless to them.
2: Unless you're in a cockpit or a seat, VR can be a nauseating experience. Tricking your brain into thinking you're moving without actually moving can be a jarring experience.
3: This goes along with 2. The number of truly viable applications for VR are limited in the entertainment sector. Now, medicine and other simulations might be able to use it, but it's treated as a gimmick right now, and that's going to hamper the legitimacy of the sciences picking it up. Then again, the USAF networked a bunch of PS3s together to make a supercomputer, so anything is possible...
You make great points, I myself find it difficult to process VR as I have a problem with my eyes focusing at different levels. But I still find the concept exciting, especially for education and re-education.