This is a misquote and is often misquoted by those conflating racism with prejudice and bigotry.

Read the full quote and the context of this quote and you will understand that he was talking about his little Black children, who have grown up in an American society judged for the colour of their skin, through the lens of white privilege and white supremacy.