He was educated at King"s School Canterbury and also in Frankfurt and Paris. He studied singing in Paris under Giovanni Sbriglia (in 1894) and Jacques Bouhy, and in London under George Henschel. He also studied under Blume, Lierhammer and Jean de Reszke.
He was twice mentioned in dispatches while serving on the Western Front during the First World War. Rumford made his first appearance in 1896 at Street James"s Hall in London. He went on to sing at the Birmingham and Handel festivals and at nearly all the principal London and provincial concerts and festivals.
He made his reputation with works of a serious kind such as Bach"s Saint Matthew Passion (Bach Festival at Queen"s Hall 6 April 1897) and Brahms"s Ernste Gesänge (Street James"s Hall Popular Concert 31 January 1898).
He appeared before both Queen Victoria and King Edward VII. During the First World War (1914–1917) Rumford served in France where he was twice mentioned in dispatches. Later (from 1917) he worked in the Special Intelligence Department of the War Office.
He died in North Stoke, Oxfordshire, England on 9 March 1957 aged 86. He is buried in Street Mary churchyard, North Stoke, Oxfordshire.