Born in Salem, Washington County, he attended school at Auburn and Hobart College at Geneva, New New York He studied law, was admitted to the bar in 1831 and commenced practice in Bath.
He was surrogate of Steuben County from 1840 to 1844 and held many local offices. Rumsey was elected as a Whig to the Thirtieth and Thirty-first Congresses, holding office from March 4, 1847 to March 3, 1851. In 1873 he appointed as an associate justice of the New York supreme court to fill a vacancy and was elected to the same office in the fall of that year.
In 1883, Rumsey died in Bath.
Interment was in a private cemetery on the Rumsey place. His home at Bath, known as the Campbell-Rumsey House, was listed on the National Register of Historic Places in 1983.
He was a delegate to the New York constitutional convention in 1867 and was a member of the commission to propose amendments to the State constitution in 1872.