Yup, introvert. I relate to this very much:
Shouldn't you go home already?''. Interacting with people tires and drains me very quickly.
I prefer to go to other people's places, so that when I become tired, I can leave when I need to. You are who you are, so you need to be okay with that because it is completely okay. A little bit of connection with people to the degree you need it is most likely enough for you.
Probably when you have something like a basketball game, it creates enough of a focus within a group where you can feel more comfortable with less of having to socialize.