At the age of seven he accompanied his father to a farm near Lebanon, Illinois where he was educated in the public schools and at McKendree College. After leaving college he settled in Belleville, St. Clair County, Illinois. There he studied law and in 1846 was admitted to the bar.
In 1849 he became coeditor of the Belleville Daily Advocate. He took an increasing interest in politics and by 1861 had gained rank in the Republican party and a wide publicity through his speeches. In that year he entered public service as master in chancery for St. Clair County. At the end of his chancellorship in 1865 he began the first of four terms in the House of Representatives in Washington, Democratic predominance in Illinois politics had persisted during the Civil War until Lincoln's reelection in 1864, when the Republicans carried all but three of the Illinois congressional districts. Baker was selected in 1864 as a Republican to run against William R. Morrison, a strong Democratic candidate in central Illinois. He defeated Morrison in that year, in 1866, and again in 1886, becoming known in state politics as the "great Jehu Baker. "
At the expiration of his second term in 1869, he retired temporarily to private life. In 1876 he completed a literary work, translated from the French of Montesquieu, which was published in 1882 as Montesquieu's Considerations on the Causes of the Grandeur and Decadence of the Romans. The copious annotation of this volume reflects his command of the classics and his wide reading of ancient and modern history in English, French, German, and Spanish. His growing reputation earned his appointment as minister to Venezuela from 1878 to 1881, and from 1882 to 1885. He also acted, during the latter years, as consul general at Caracas. He became interested in the Anglo-Venezuelan boundary dispute, then approaching a critical stage, and won the confidence of the Venezuelan government, which through him pressed the Department of State to accept an alliance with Venezuela, or, failing that, to accept the cession of the exclusive fluvial navigation of Venezuela. Neither of these projects, however, was successful. Upon Cleveland's election he returned to Illinois to defeat Morrison in 1886 for Congress for the third time, but he failed of reelection in 1888. Long a supporter of an expanded currency, he left the Republican party in 1896 and was elected congressman for the fourth time as a Fusionist. Since his eyesight was already impaired, he declined to run again. He became totally blind in 1899 and died of a paralytic stroke in 1903.
Baker plunged into the contest of the "radical" Republicans in Congress against President Johnson's policies. He spoke often on Reconstruction, denouncing the southern "aristocracy" and advocating that the proposed Fourteenth Amendment remove all discriminations against Negroes. He was unsparing in his censure of what he considered Johnson's abuse of executive power and was strongly in favor of the President's impeachment. During these first two consecutive terms he tried also to further the interests of his Illinois constituents. He resisted efforts of the Illinois Central Railroad to profit at the expense of the state, attempted to save part of the public domain from a land company, favored an expanded currency, and helped create a House committee on education and labor.
He was twice married. His first wife was Olive Starr Wait, to whom he was married on April 28, 1856, and by whom he had one daughter. She died in 1865 and in 1874 he was married to Mary (West) Robertson, by whom he had a daughter who died at Caracas.