Oliver grew up on the family's small farm and began his education at a country school. He attended Alabama Presbyterian College from 1907 to 1909, but he received his B. A. in 1911 and his M. A. in 1914 from the University of Alabama.
Carmichael taught French and German at the University of Alabama in 1911-1912, and in 1912-1913 he was acting professor of modern languages at the Florence Normal School in Birmingham. He attended Oxford University as a Rhodes Scholar from 1913 to 1917, earning a B. S. and a diploma in anthropology. These studies were interrupted by service in the Commission for Relief in Belgium, the British Young Men's Christian Association, the British army, and the United States Army.
Following World War I, Carmichael returned to Alabama and became head of the foreign-language department at Birmingham Central High School. He was principal of Henley Grammar School (1920 - 1921) and Woodlawn High School (1921 - 1922), and in 1922 he was appointed dean and assistant to the president of Alabama State College for Women (now the University of Montevallo). As its president from 1926 to 1935, Carmichael directed a fund-raising drive to improve the college and provide for its continued growth. Carmichael began his association with Vanderbilt University in 1935, when he became dean of the graduate school and senior college. He assumed the additional post of vice-chancellor in 1936 and served as chancellor from 1937 to 1946. Carmichael's tenure in Nashville was characterized by expansion of the school's curriculum and the addition of research and academic facilities. Vanderbilt University joined with Scarritt College and George Peabody College to endow and construct a central university library. A $9 million campaign for building and endowment funds for Vanderbilt's liberal arts college was launched, and Carmichael helped raise money for a school of law. During 1946-1953, Carmichael was president of the Carnegie Foundation for the Advancement of Teaching. Carmichael expressed his concerns with international studies in his annual reports to the foundation's board of trustees. During these years he was also vice-chairman of the Temporary Commission on the Need for a State University of New York, and he was chairman of the board of trustees of the New York State university system from 1948 to 1953. As president of the University of Alabama in Tuscaloosa, a post he assumed in 1953, Carmichael faced perhaps his greatest challenge. His leadership was tested in the turmoil brought about by the civil rights movement. Labeled a "southern moderate" by the New York Times, Carmichael was caught in the middle of the conflict over racial segregation. In 1956, after attending classes for three days, Autherine Lucy, the university's first black student, was expelled for accusing university officials of conspiring with rioters. Following an investigation, several white students believed to have incited mob outbursts on the Tuscaloosa campus were also expelled. In public, Carmichael spoke of maintaining the order and decorum befitting an educational institution and of obeying the law of the land. While Carmichael denied published reports of his differences with University of Alabama trustees, one member of the board was quoted as saying, "If that says 'integration' again, we'll fire him. " Carmichael resigned in 1957; for the remainder of his life he was a consultant to the Fund for the Advancement of Education. Carmichael's first major book was The Changing Role of Higher Education (1949), in which he reviewed the expansion of higher education since 1900. He found growth in the numbers attending colleges and universities, as well as in support, subsidies, and endowments. Additionally, he recorded an increase in the number of American junior colleges and professional schools and a broadening of the curriculum. Universities, he stated, were assuming a more strategic role and accepting broader responsibilities than were envisioned in 1900. He saw increased growth in every aspect of higher education in the future. Four problem areas he discussed were balance in the curriculum; the recruitment, selection, training, and improvement of teachers for college; international studies; and appraising educational results.
His clarion note was the role of higher education as "society's Number One agency for promoting fundamental social progress. " For his second major book, Universities, Commonwealth and American: A Comparative Study (1959), Carmichael visited eight nations of the British Commonwealth and fifty-six universities. After talking with hundreds of faculty members and administrators, he recommended the establishment of a Commonwealth-American Commission on University Education to explore the needs of higher education and serve as a clearinghouse for the exchange of ideas.
He saw graduate school as the most strategic segment of higher education, as well as the most inefficient and ineffective part of the university. Graduate schools, he said, were badly organized, too amorphous, and prone to "the cult of objectivity, " which leads to an ethical neutrality and the proliferation of courses and of degrees. He stressed that the most urgent need of American education was reform of graduate education. In 1960, Carmichael was appointed to the commission established by the Southern Regional Educational Board to chart a course for higher education in the South. The commission's report, Within Our Reach (1961), contains many of Carmichael's tocsins: broadening educational opportunity; educating people to be citizens responsive to their time's social, economic, and political needs; achieving excellence in teaching, scholarship, and research; helping the South to advance economically; and providing guidance in the solution of social problems. Carmichael, known to friends and family as "Mike, " died in Asheville, N. C.
(Book by Carmichael, Oliver Cromwell)
He was the president of the Carnegie Foundation for the Advancement of Teaching.
On July 13, 1918, Carmichael married Mae Crabtree; they had two children.