On account of the poverty of his parents, he received little formal schooling. In 1808 he was given the honorary degree of M. A. by Yale College.
A shrewd trader and a wandering Yankee peddler in his younger days, he soon followed in the footsteps of his brother Nathaniel and read law with Judge Tapping Reeve at Litchfield. Overcoming the obstacles of poverty and a fragmentary knowledge of books, he was admitted to the Litchfield County bar in 1792, and commenced the practice of law in New Haven.
He was one of the framers of the reformed constitution of 1818. He disagreed, however, with the policy of popularizing the state judiciary, for he was no democrat and was completely unsympathetic with the experiments of the radical group. He was prosecuting attorney for New Haven County from 1817 to 1835, an unsuccessful candidate for the governorship against Oliver Wolcott in 1825, and United States attorney for Connecticut by appointment of President John Quincy Adams in 1828, removed in the following year by President Jackson.
In May 1832 he was chosen as a Whig to the United States Senate, to succeed Samuel A. Foot, but his senatorial career was cut short by a heart attack, in Washington, three years later.
A prominent Episcopalian, a vestryman of Trinity Church in New Haven, Smith was an outstanding Tolerationist who fought stoutly for the separation of Church and State.
Initially he was Anti-Jacksonian, later - Whig. Smith had not been so aggressive a partisan as to arouse personal hostility.
He won a reputation as an able lawyer of sound judgment, and a clever politician, recognized as a man of high principle.
He was the father of six children.