He was educated at the King"s School, Canterbury, where he was a king"s scholar, and at Street John"s College, Cambridge, entering in 1711, and taking the degrees of Bachelor of Arts in 1715, Master of Arts
In 1719. The rest of his life was passed in or near Canterbury. Gostling served in the diocese of Canterbury as curate or parish priest from 1720. He was instituted to the rectory of Brook near Wye, Kent, on 23 September 1722.
He held a minor canonry at Canterbury from 1727 until his death.
His father died on 17 July 1733, and vacated the vicarage of Littlebourne, to which Gostling succeeded. He gave it up in 1753, on being appointed to Stone in Oxney.
Gostling acted as a guide in Canterbury, but during the last two decades of his life was largely confined to his room. He was visited in 1772 and again in 1773 by Sir John Hawkins, who was then researching his History of Music.
He died at his house in the Mint Yard, Canterbury, on 9 March 1777, and was buried in the cloisters on 15 March.
He possessed a significant collection of manuscripts of works by Henry Purcell. These were sold by Messrs. Langford. His library was sold by William Flackton of Canterbury in 1778.
Gostling married in Canterbury Cathedral, on 3 October 1717, Hester Thomas.
They were both then described as of the precincts. She died on 24 February 1760, aged 64, and was buried in the cloisters on 3 March.
They were commemorated on an oval marble tablet on the west side of the cloisters at Canterbury.