During the reformist Liberal administration of Alfonso López Michelson (1934-1938), Olaya assumed leadership of the moderate wing of the party. He served López briefly as minister of foreign relations and ambassador to the Vatican.
He served on the Liberal side against the Conservatives in the Civil War of the Thousand Days (1899-1903), and later joined the opposition to the government of General Rafael Reyes.He played a leading role in the overthrow of Reyes in 1909.
Olaya was an early advocate of cooperation between the parties. He accepted the fact that the Liberals were a minority, but argued that they should have a role in the government. Despite his own party’s opposition, he accepted the post of minister in Washington during the Conservative administration of Pedro Nel Ospina (1922-1926) and served until 1929.