Born in Siler City, North Carolina, Whitley attended Siler City High School, graduating in 1943. He served in the United States Army from 1944 to 1946, then attended Wake Forest University, earning his undergraduate degree in 1949 and a law degree in 1950.
He earned an Master of Arts from the George Washington University in 1974. Whitley became a lawyer and practiced in Mount Olive, North Carolina. He became the town"s attorney in 1961.
That same year, he joined the staff of Congressman David North. Henderson.
In 1976 Whitley ran for Congress himself and was elected as a Democrat to the 95th United States Congress. Re-elected four times (serving in Congress from January 3, 1977 to December 31, 1986).
He did not seek re-election in 1986, but retired to political consulting. Whitley died in Durham, North Carolina in 2002.
Served with Army of the United States, 1944-1946. Member of Masons, Woodmen of World.
Married Audrey Kornegay, June 11, 1949. Children: Charles Orville, Martha, Sara.