You have to go even further than that. You'd have to switch the geographic history of Black people and White people. Black people would have been the colonizers of Africa, Asia and South America. You also have to reverse colorism so that dark skin would be more enviable around the world. THEN you get to the trans-Atlantic slave trade which completely separates White people from their roots and makes them subhuman in another land. After enduring all that and finally being "free" I think they would be in a similar situation to Black people as there would still be repercussions of the hundreds of years of oppression in America and they would also face the demonization of their skin color on a global scale.
Now I'm just thinking about all the examples of Black people stealing culture from White people in this scenario. And denying Black privilege. And Black Donald Trump!