Many years ago I used to think that the best response to button pushing was to ignore completely. Keep quiet or walk away. But I got to realize that there is a time to keep quite and a time to speak.
Because I ignore I got more my buttons pushed often.
“I will try, he does nothing in reply. That means he is cool with it.”
Not anymore though. Letting people get away with their shenanigans never benefited me. Anyone that warrants same reply I do so immediately.
It seems like you've thought it through and choose to be selective with whom you allow to affect your life more because of it.