Born in Stamford in the Colony of Connecticut, after the American Revolutionary War his Loyalist family emigrated to New Brunswick, Canada, settling in a rural area on the west bank of the Missaguash River near the present-day town of Sackville. Trained in medicine, Doctor Rufus Smith had a large practice, serving the residents across the Isthmus of Chignecto. In 1817, he was elected to the 6th New Brunswick Legislative Assembly as a representative for the County of Westmorland.
He was the representative again in the 7th and 8th Assemblies, serving until 1827, then again in the 10th Assembly from 1831 to 1834.
Rufus Smith died in 1844 and is buried in the Methodist Church cemetery at Point de Bute.