The scaling problem is very different for PoW and (D)PoS chains. For PoW chains it is much more difficult because security and permissionlessness (in the sense that on a PoW chain anyone can start producing blocks) are possible because of PoW. Security and scaling are at odds as design choices there. In (D)PoS chains, we can arbitrarily increase the block size without changing anything except increasing the hardware requirements because it's up to the stakeholders to choose the block producers.
Whether or not everything needs to be stored on the main chain is of course a valid question. Our only current side chain stores everything on the main chain as JSON data while Scotbot is the only server that does anything in response to detecting them on the chain. There are different side chain models and I believe there will be many kinds used in the ecosystem.
I like to imagine it this way: say two people are playing a competitive game against one another. If both players can play the game using their posting key encryption, that data doesn't even need to go on chain. All of of sudden, now it doesn't even matter that Steem has a 3 second block limit. Both players can post actions as fast as the latency of the node they are playing on, just like a normal centralized game.
However, due to the private key encryption, the node running the game doesn't have to worry about things like security. If someone cheats at the game the opposing player can prove it using the rules of consensus of whatever game they are playing. No one would be able to counterfeit gameplay because that gameplay is secured locally on said user's machine.
We are going to see some crazy applications over the next ten years that blow everyone's mind. I guarantee it.