He studied at University College London for a Bachelor of Science, and furthered those studies at the universities of Edinburgh with an Master of Arts graduating in 1885, and Heidelberg where he completed a Doctor of Philosophy From there he went on to study at Saint Andrews Biological Station in New Brunswick and Stazione Zoologica in Naples.
Scharff died in Worthing on 13 September 1934. Scharff was appointed as an Assistant in the Natural History Division of the National Museum of Ireland in 1887, becoming Keeper in 1890. In the years after 1916 and the resignation of George Noble Plunkett as Director of the National Museum of Ireland Scharff became the acting Director until his retirement in 1922.
Scharff was one of the main organisers and contributors to the Clare Island Survey, serving as a chair of the organising committee.
He is best known for his work on biogeography, particularly the recolonisation of Ireland after the ice age. He was a vocal proponent of the land-bridge hypothesis of animal migration.
This work was informed by his experience excavating caves in Kesh. He collected zoological specimens extensively, many of which are housed with the National Museum of Ireland.
In 1899 a species of land planarian was named in his honour, Microplana scharffi (Graff, 1899).
Following his retirement, Scharff moved back to the United Kingdom, to Worthing.
He was an active member of many societies in Ireland and the United Kingdom. Whilst a member of the Royal Irish Academy over a forty year period, he served as vice-president, secretary for foreign correspondence, and chairman of the flora and fauna committee. He was also an active member of the Royal Zoological Society of Ireland, the Royal Dublin Society, the Conchological Society of Great Britain & Ireland, the Linnean Society and the Zoological Society of London. He still remained an active member of Irish zoological groups until his death in 1934.