It is possible to lose weight without going to the gym, but from personal experience, you are more likely to gain the weight back easily if you deviate from a strict diet.
Maintaining a healthy balanced diet, with regular exercise, gives you the freedom to eat a little more of the food you like, without having to live on such a strict diet.