Tucker was born near Port Royal, Bermuda, to English colonists Anne Butterfield (1797) and Henry Tucker (1713–1787). His father was the great-grandson of George Tucker, who emigrated to Bermuda from England in 1662. The Tuckers were well-regarded in Port Royal. St. George's older brother Thomas Tudor Tucker migrated to Virginia in the 1760s after completing medical school in Scotland, and settled in South Carolina before the American Revolutionary War. George Tucker, a politician and author, was a relative of theirs. The name St. George had been in the family since his great-great-grandfather George Tucker married Frances St. George.