He graduated from the College in the English Colony of Rhode Island and Providence Plantations (the former name of Brown University) at Providence in 1774. He then studied law and was admitted to the bar association in 1778.
He served in the Massachusetts House of Representatives, the United States House of Representatives and the United States Senate. He remained in Rhode Island to practice law, beginning his law practice in Providence. He received his Master"s degree from Harvard University in 1784.
After returning to Massachusetts, Foster held various positions in the government.
He served as justice of the peace for Worcester County from 1781-1823, as special justice of the court of common pleas in 1792, and as sheriff of Worcester County in 1792. In 1791, he was elected as a Federalist candidate to the Massachusetts House of Representatives.
He was elected as a United States House of Representatives to the 3rd United States Congress in 1793, and was reelected to the three succeeding Congresses. He served in Congress from March 4, 1793 until his resignation on June 6, 1800.
While in Congress, he was Chairman of the United States Senate Committee on Claims.
In 1799, he was a delegate to the State constitutional convention and on June 6, 1800 he was elected to the United States Senate to fill the vacancy caused by Samuel Dexter"s resignation. He served in the Senate until his resignation on March 2, 1803. Foster died in Brookfield on April 29, 1823, aged 65, and is interred in Brookfield Cemetery there.
He was a member of the State House from 1808-1809 and a member of the Massachusetts Governor"s Council in 1818. In 1813 he was elected a member of the American Antiquarian Society.