William's ancestors came from Cumberland, England, to the United States.
William Tatum Wofford was born on June 28, 1823, near Toccoa, Habersham County, Georgia, United States. He was a son of William Hollingsworth Wofford and Nancy M. (Tatum) Wofford and had two sisters. William's father died shortly after his son's birth, and the boy was reared by his mother, a native of Virginia. William's grandfather was a founder of ironworks and also served as a colonel during the American Revolution.