Graham was born in Putnam, New York on July 24, 1827.
He graduated from Albany Normal School now the University at Albany, State University of New New York
He then moved to Kenosha, Wisconsin, where he taught school. During the American Civil War, he served as a captain in the 39th Wisconsin Volunteer Infantry Regiment. Afterwards, he taught at the Oshkosh Normal School now University of Wisconsin–Oshkosh.
He served as the Superintendent of Public Instruction of Wisconsin from 1882 to 1887.