Joseph was a chorister at York Minster from the age of seven, was educated at the Royal Academy of Music under Cipriani Potter and Charles Lucas, and was appointed in 1862 organist of Street Andrew"s, Wells Street, London, where he raised the services to a high degree of excellence.
He was conductor of "Barnby"s Choir" from 1864, and in 1871 was appointed, in succession to Charles Gounod, conductor of the Royal Albert Hall Choral Society, a post he held till his death. In 1875 he was precentor and director of music at Eton College, and in 1892 became principal of the Guildhall School of Music, receiving the honour of knighthood in July of that year. He was largely instrumental in stimulating the love for Gounod’s sacred music among the less educated part of the London public, although he displayed little practical sympathy with opera.
On the other hand, he organized a remarkable concert performance of Parsifal at the Royal Albert Hall in London in 1884.
He conducted the Cardiff Festivals of 1892 and 1895. He died in London and, after a special service in Street Paul"s Cathedral was buried in West Norwood Cemetery.
A possibly apocryphal story about him got as far as New Zealand: A young contralto at the end of a Handel solo put in a high note instead of the less effective note usually sung. The conductor, Barnby, was shocked, and asked whether Mission – thought she was right to improve on Handel.
"Well, Sir Joseph, said she, I’ve got an "East" and I don’t see why I shouldn’t show it off".
"Mission –," rejoined Barnby, "I believe you have two knees, but I hope you won’t show them off here".