He was educated in Manchester, Vermont, first attending Burr and Burton Seminary and then graduating with a Bachelor of Arts from Middlebury College in 1862.
With the American Civil War being under way at the time of his graduation, Knapp enlisted as a private in the Company I, 16th Vermont Infantry and was quickly promoted to captain of Company F, 17th Vermont Infantry. During his military service, he was wounded at Gettysburg, Spotsylvania, and St. Petersburg. Knapp rose to the rank of lieutenant colonel and was brevetted a colonel for his gallantry during the Siege of St. Petersburg.
On January 23, 1865, Knapp married Martha A Severance of Middlebury, Vermont.
The couple had four children. Following the war, Knapp returned to Vermont where he published the Middlebury Register from 1865 till 1878.
In addition to the work on his own newspaper, he submitted articles to the Chicago Inter-Ocean and the American Law Register. Beyond his journalistic efforts, Knapp served as Clerk of the Vermont Legislature during the 1872-1873 session.
After being admitted to the bar in 1876, he served as a probate judge for the Addison district from 1879-1889.
Knapp was also elected for a term in the Vermont House of Representatives from 1884 to 1885. Knapp was appointed Governor of the District of Alaska by President Benjamin Harrison, his term beginning on April 20, 1889. At the time of his appointment, many Alaskans desired a resident governor and were disappointed receiving an "outsider".
The district did however see a number of improvements during his administration.
Postal service was extended with the creation of over 1,600 miles (2,600 km) of new mail routes. Knapp also helped establish a historical society and library.
On the legal front, Knapp asked for Alaska to be granted a Delegate to the United States. Congress. He also supported creation of a commission to revise the district"s legal code and, arguing that lack of clear land possession discouraged improvements, lobbied for a revision of the land possession laws.
To help maintain law and order, Knapp help organize both a militia and Indian police.
The former turned into a source of amusement for some area residents due to the governor"s predilection for dressing in his military uniform and watch the militia parade before him. During his term, an international dispute over seal fisheries reached a zenith. The United States, claiming that seals from Alaska were being killed by Canadian and other ships on the high seas, had seized several ships and brought them to Sitka.
Arbitration to resolve the dispute was agreed to in 1891 between Canada, the United Kingdom, and the United States.
As a result a set of restrictions was imposed on seal hunting near Alaska. Knapp was replaced as governor on August 29, 1893 following the appointment of James Sheakley by President Grover Cleveland.
Following his term as governor, Knapp moved to Seattle and opened a law practice. He was active in civil organizations such as the Institute of Civics and National Geographic Society and received an Honorary Doctor of Laws from Whitman College in 1893.
He remained active in the practice of law until his death on October 9, 1904.
He was also a member of the Vermont House of Representatives from 1884 to 1885.
Married Martha A. Severance, January 23, 1865.