Jessie Benton Frémont was born on May 31, 1824, near Lexington, Virginia, and was the daughter of Senator Thomas Hart Benton. She was the second of five children.
Her mother was Elizabeth McDowell, and Jessie was born at her grandfather McDowell’s estate near Lexington, Virginia. She grew up traveling between her three homes, Cherry Grove, a home in St. Louis, and a home in Washington, D.C. While living in St. Louis at age eight, Jessie experienced the horrific cholera outbreak that struck the city.