He was a Congregationalist and studied at the public schools in Massachusetts, Connecticut, and New New York
He had six children, John Milton Holley, William Ralston Holley, Alexander Lyman Holley (1832 - 1882), John Coffing Holley (1837 - 1865), Maria Holley Rudd (1842 - 1914). Holley served in the state militia, rising through the ranks to Lieutenant Colonel. He entered politics in 1844, serving as a delegate to the Whig National Convention.
Holley became president of the Holley Manufacturing Company in 1854 and held that position for life.
He also became director of the Salisbury Iron Bank and Connecticut Western Railroad. Holley was nominated by the Republican Party and elected the 25th Lieutenant Governor of Connecticut in 1854, and became the Governor of Connecticut in 1857.
During his term, his administration endorsed the requirement for recently naturalized citizens to wait one year before being eligible to vote, and the Supreme Court ruled on the Dred Scott case. He left office on May 5, 1858.
He was a delegate to Republican National Convention from Connecticut, 1860, and retired from public service.
Holley died on October 2, 1887, Lakeville. He is interred at Salisbury Cemetery, Salisbury, Litchfield County, Connecticut.
Whig Party, Republican Party.