Two Times a Lover

When mixed race Alex Crossman enters a wrinkle in time and is hurled back eighty years, he lands in the hell of racial segregation. He commits himself to use his knowledge of the future to help the black community and he is successful. He gets married but is yanked back into his own time before he finds out his wife is pregnant. His life achieves some sense of normalcy until a woman near his age shows up claiming to be his granddaughter.
And there is a parade of similar incidents coming to his door.