In the mid-1950s he joined the Alabamians. He formed the Dixie Gentlemen in 1956 together with Jake Landers and Herschel Sizemore. In the beginning they called themselves the Country Gentlemen but when they found out that another group already had that name, they quickly changed theirs to the Dixie Gentlemen.
They later recorded with fiddler Tommy Jackson.
The group disbanded in 1966 and Yarbrough found work with Jimmy Martin"s Sunny Mountain Boys. He continued working with Flatt & Scruggs and the Foggy Mountain Boys, Jim & Jesse and the Virginia Boys and Bobby Smith and the Boys From Shiloh.
When Yarbrough was performing in Columbus, Ohio with the Boys From Shiloh, he met Monroe who offered him a job with the Bluegrass Boys since his banjo player Vic Jordan had just left. Yarbrough was hired and made his first recordings with Monroe two days later, on March 26, 1969.
Between 1969 and 1970 he made 21 recordings with Monroe.
Yarbrough also recorded a number of solo albums, and also founded the group, The Dixiemen. Among these records with The Dixiemen was "The Old Oak Tree," in 1974, on Old Homestead Records in Brighton, Michigan, of which about 500 copies were pressed and few survived, though the recordings have been preserved in digital form. Yarbrough also opened Rual"s Music Service in Muscle Shoals, Alabama where he became well known for his ability to repair and build stringed instruments.
Due his shop"s proximity to FAME Studios, Yarbrough would serve as a session performer for many of the acts who performed at the studio during the 1960s and 1970s.
He also performed as session player in Nashville, and was featured on records by the likes of Hank Williams, Junior, Mac Davis, and The Pointer Sisters. Rual Yarbrough died from complications from pulmonary fibrosis in Florence, Alabama on September 21, 2010.
Yarbrough was honored as an Alabama Music Achiever by the Alabama Music Hall of Fame both as a solo artist. In 2011, Yarbrough was inducted into the Alabama Bluegrass Music Hall of Fame.
And as a member of the Dixie Gentlemen.