He was educated at Blackheath School, London.
He entered the Royal Navy in 1848, and was made a commander in 1865. He was a Deputy Lieutenant and Justice of the Peace for Banffshire (from April 1894) and Kincardineshire (from January 1900). He joined Robert Lowe as one of the Adullamites opposing the parliamentary Reform Bill of 1866, which led to the Reform Acting 1867.
He was a junior Lord of the Treasury and Liberal whip from 1882-1885, and Civil Lord of the Admiralty in 1886.
He was appointed a Privy Counsellor in 1892. Duff was appointed in March 1893 and was subsequently awarded the Knight Grand Cross of the Order of Street Michael and Street George. He reached Sydney to take up his duties on 29 May 1893.
His term of office was chiefly marked by his permitting the premier, Sir George Dibbs, to obtain the prorogation of parliament on 8 December 1893, after that minister had incurred a vote of censure. In July 1894, after his ministry had failed to carry the general election, Dibbs desired Duff to nominate several persons to the legislative council on his recommendation.
Duff declined to accede to his wish on the ground that the ministry had been condemned by the colony, and in consequence Dibbs and his colleagues resigned.
In February 1895, he became ill while visiting Hobart and returned to Government House in Sydney, where he died of multiple hepatic abscesses and septicaemia on 15 March aged 59. He was interred in Waverley Cemetery. He is the only NSW Governor to have died in Office.
He was a freemason.
During his term as Governor (1893-1895), he was also Grand Master of the Grand Lodge of New South Wales. On 21 February 1871, Duff married Louisa, youngest daughter of Sir William Scott, ninth bart. of Ancrum in Roxburghshire.
18th United Kingdom Parliament. 19th United Kingdom Parliament. 20th United Kingdom Parliament.
21st United Kingdom Parliament.
22nd United Kingdom Parliament. 23rd United Kingdom Parliament.
24th United Kingdom Parliament. 25th United Kingdom Parliament]
Duff served as Liberal Member of Parliament (Member of Parliament) for Banffshire from 1861-1893.