People of color were subjected to grave injustices in the past. Fortunately, this has diminished today. Those who form opinions about people based on their skin color clearly have intellectual disabilities.
As you said, we are all the same species. What sets us apart should be our behavior, our morality, our diligence, and our sense of empathy.