He earned a bachelor"s degree from the Curtis Institute of Music, where he studied with Gregor Piatigorsky. Olefsky subsequently studied with Pablo Casals. He studied conducting with Herbert von Karajan and Pierre Monteux.
During the 1940s, Olefsky performed as the principal cellist of the Philadelphia Orchestra while maintaining a worldwide career as a solo artist. In 1950, he performed the world premiere of the Cello Concerto by Virgil Thomson. Olefsky left his position in Philadelphia in 1950 to join the United States Navy.
Upon completing his military service, Olefsky became the principal cellist of the Detroit Symphony Orchestra.
In 1974, he took a position as professor of cello at the University of Texas at Austin, where he taught for several decades and was a professor emeritus at the time of his death. He died in Austin, Texas, at the age of 87.