He was educated by private tutors and graduated from Williams College in 1797. Mills studied law, was admitted to the bar, and commenced practice in Northampton, Massachusetts.
He was the district attorney for Hampshire County, Massachusetts, and opened Northampton Law School in 1823. Mills was elected as a Federalist to the United States House of Representatives (March 4, 1815 - March 3, 1819). In 1819 he returned to the Massachusetts House of Representatives, where he became Speaker of the House in 1820.
He was elected to the United States Senate in 1820 to fill the vacancy caused by the resignation of Prentiss Mellen.
Mills was reelected and served from June 12, 1820, to March 4, 1827. He was an unsuccessful candidate for reelection in 1826.
He retired from public life due to ill health. Mills died in Northampton, and was interred in the Bridge Street Cemetery.
One of Mills" daughters, Harriette Blake Mills, married Admiral Charles Henry Davis.
Another of Mills" daughters, Sarah Hunt Mills, married Benjamin Peirce, the father of Charles Sanders Peirce.
Mills was also a founding member of the American Antiquarian Society in 1812. He was a member of the Massachusetts House of Representatives (1811–1814).