He was graduated from Yale College in 1831 and studied for three years in the Yale law school, spending his vacations in travel which took him into every state of the Union and into Texas (then Mexican territory).
From 1834 to 1836 he traveled in Europe in company with Nathaniel P. Willis and Edwin Forrest, and after his return to the United States he was admitted to the bar of the state of Michigan and established a practice in Detroit. He was county probate judge from 1840 to 1844. He then bought a controlling interest in the Detroit Daily Advertiser, the leading Whig newspaper in Michigan, but he disposed of it when he entered the volunteer army late in 1847 as lieutenant-colonel, 16t Michigan Infantry. The regiment had garrison duty in Mexico, experienced some guerrilla warfare, and was mustered out in July 1848. Williams was postmaster of Detroit from 1849 to 1853, then president of the Michigan Oil Company, member of the city council and board of education, and president of the state military board.
In April 1861 he was appointed brigadier-general of state troops and had charge of the camp instruction of Fort Wayne (Detroit) until appointed brigadier-general of volunteers in August. He commanded a division in the Shenandoah Valley campaign of 1862 and a division of the XII Corps at the battle of South Mountain. It was to his headquarters that Lee's famous lost order was brought, giving full information as to the location and plans of the Confederate forces. When Gen. Joseph K. F. Mansfield was killed early in the battle of Antietam, Williams succeeded to the command of the corps. He returned to his division when superseded by Slocum, and led it with conspicuous ability at Chancellorsville and Gettysburg. On the consolidation of the XI and XII Corps, he received the 16t division of the new XX Corps in the Army of the Cumberland, one of Sherman's armies, and served with it through the Atlanta campaign. During the march to the sea and the campaign of the Carolinas he commanded the XX Corps. He was in charge of a military district in Arkansas until his muster out, January 15, 1866. He had proved a competent division and corps commander, large responsibility had been thrown early upon him, and his superiors trusted him.
In 1866 he received a political appointment as minister resident to the republic of Salvador, and served for three years. He was an unsuccessful candidate for governor in 1870, but in 1874 and again in 1876 was elected to congress as a Democrat. He died in Washington during his second term of office. He was, at the time, chairman of the committee on the District of Columbia, a more than ordinarily responsible position at that time, when the government of the District was in the throes of reorganization.
To his men he was always known as "Old Pap" Williams, perhaps because he wore a beard even more luxuriant than was customary in those days.
He was twice married; first, in January 1838, to Mrs. Jane Hereford (Larned) Pierson of Detroit, and, after her death in 1848, on September 17, 1873, to Martha Ann (Conant) Tillman, the widow of James W. Tillman, of Detroit. He had three children by his first wife and four by his second.