Background
Wharton was born in Philadelphia, Pennsylvania, the son of Fishbourne Wharton (1778–1846), merchant, and his second wife Mary Ann Shoemaker, and the grandson of Governor Thomas Wharton Junior. He studied at the Pennsylvania Academy of Fine Arts, and later in Paris and Dresden.