Yah, I think it is normal to feel a bit awkward once you noticed someone unfollowed you. It's a fine line between taking it personal or rational. I remember feeling awkward the first time too, but then again it's just numbers as everyone can still interact without following.
When I decluttered my feed I didn't even thought about how it may be perceived by some. Just because I knew that it is nothing personal from my side. But the others do not know. So I thought it might be a good question to ask and to see what others think about this topic.