Robert Henry was born on December 6, 1792, in Charleston, South Carolina, United States. He was the son of Peter Henry, a native of Banffshire, Scotland, who went to the West Indies and became a successful merchant. In Jamaica he met Anne Adelaide (Schwiers) Angel, a widow with one child, whom he married. In May 1792 they came to Charleston. But he went back to the West Indies to settle up his business affairs, and as he was returning to Charleston, the ship on which he traveled was captured by a French privateer and he was so harshly treated that he died in Savannah in September 1794. His wife opened a drygoods store for the support of herself and her children. In 1803 she took Robert to England that he might have the best educational advantages.