Hello.
The truth is I congratulate you for having so many comments, it was bad that you had the misunderstanding with the person to whom you did not reply the comment and they were upset.
Personally, I think that you do not have an obligation to respond to all of us and we should not bother about that.
The comments are the expression of the affection we have for your work and effort, you have a lot of it because your work is phenomenal. I know that as an author you want to give everyone the response time but think about the celebrities, if they will spend it answering each of the comments of their fans. When would they make the new content? Will it be as good as the one we are used to consuming? Time is valuable, you decide how to invest it.
This is where a lot of people make a mistake, because they do not well consider what is worth investing time into.