Nah is more casual and softer. Kind of like いいえ and いや in Japanese.
You can use either in most situations. If it's a serious situation where you want to say ダメ！ or いやだ！ you should use no.
Like this:
A: I think I'm going to buy some Bitcoin.
B: Noooooo! It's at two million yen right now. Wait for the dip.
A: What happens if I push this button?
B: No! That's the emergency alarm!