He graduated from Cornell University in 1916. He enlisted for World War I, completed officer training at Madison Barracks, New York, and received his commission as a second lieutenant of Infantry.
Kilburn served in France with the 26th Infantry Regiment, 1st Infantry Division, and later returned to the United States as an instructor in the use of trench mortars. He was discharged at Camp Gordon, Georgia as a captain in 1919. After the war Kilburn worked for a Malone ice cream and candy wholesaler before beginning a career in banking.
In 1930 he was appointed president of the People's Trust Company of Malone.
He was elected to Congress in 1940 to fill the vacancy caused by the death of Wallace East. Pierce and served from February 13, 1940 until January 3, 1965. He died in Malone, New York, and was buried at Morningside Cemetery.
He was also a member of the board of directors of the Marine Midland Trust Company of Northern New New York During his years in Congress, Kilburn was one of the more conservative members of the New York Republican delegation, but was liberal on the issue of foreign aid, like most members of the New York delegation. He is particularly notable for being the only member of the New York Congressional delegation to vote against the Civil Rights Acting of 1964.