Background
He was born on January 17, 1820 in Gloucester, Massachussets, United States. His father, Winthrop, was descendant of William Sargent, who came to Gloucester from England before 1678, many of whose descendants distinguished themselves in the civil and military affairs of the colonies and some in literary or artistic paths. Winthrop Sargent succeeded his father in the management of large shipping interests belonging to his family, but the business failed in 1829, and in the following year he removed to Philadelphia, where he engaged in the commission business. He returned to Massachusetts in 1843, but in 1862 went back to Pennsylvania and lived at Morgan's Corner (now Radnor), near Philadelphia, until his death in 1874.
His mother, Emily Haskell, was a descendant of William Haskell, who came to New England about 1637.