Carver was born into slavery in Diamond Grove, Newton County, near Crystal Place, now known as Diamond, Missouri, possibly in 1864 or 1865, though the exact date is not known.
His mother, Mary, was a slave owned by Moses Carver, and his father was probably Giles, a slave owned by James Grant. According to report, Giles was killed in an accident when George was an infant.
Carver himself became the kidnap victim of night riders while still a baby. With his mother and brother, James, he was held for ransom, but before they were rescued, his mother died. Merely a babe in arms, Carver was ransomed for a $300 racehorse by Moses Carver, a German farmer. Thus he was orphaned and left in the custody of a white guardian from early childhood.