I would walk over to him. Alone of course. Vet him out. You need to know for your family, for your safety. If You have a bad vibe follow thru. A short "hello my name is"
Convo will quickly lay to rest if it is nothing, and may gain you a local friend you can feel good about.
We both know the other direction it could go. But it needs to be done. That is Your Territory now and for your family and neighbors. It needs to be done in my opinion. It is just a conversation after all. He will dictate where it goes from there.
He had already been spoken to - and came up "weird vibe".