Born at Exeter on 16 December 1714, he was the fourth son of Robert Walker of Withycombe Raleigh near Exmouth, Devon, by his wife Margaret, daughter of Richard Hall, rector of Saint Edmund and All Hallows, Exeter. Robert Walker (1699–1789), his elder brother, made manuscript collections for the history of Cornwall and Devon, which went to Sir Thomas Phillipps.
He was educated at Exeter grammar school from 1722 to 1731, and matriculated at Exeter College, Oxford, on 4 November 1732, graduating Bachelor of Arts on 25 June 1736.
In 1737 Walker was appointed curate of Doddiscombe Leigh, near Exeter, but resigned his position in August 1738 to accompany Lord Rolle"s youngest brother to France as tutor. Returning early in 1740, he became curate of Lanlivery in Cornwall. On the death of the vicar, Nicholas Kendall, a few weeks later, he succeeded him on 3 March 1740.
In 1746 he resigned the vicarage, which he had only held in trust, and was appointed curate of Truro and vicar of Talland.
He experienced an evangelical conversion, and his well-attended sermons emphasised central parts of evangelical theology. In 1752 he resigned the vicarage of Talland over a scruple on being a pluralist.
In 1754 Walker tried to set up religious society bound by rules of conduct. In 1755 he also formed an association of the neighbouring clergy who met monthly.
Walker died unmarried on 19 July 1761 at Blackheath, at the house of William Legge, 2nd Earl of Dartmouth, a supporter.
He was buried in Lewisham churchyard. Risdon Darracott and Thomas Adam were among his correspondents.
