Pelham was educated at Westminster School and went then to Christ Church, Oxford, where he graduated with a Bachelor of Arts in 1832 and Master of Arts four years thereafter.
In 1857, he received a Doctor of Divinity by the University of Oxford. Pelham was ordained by Charles James Blomfield, at that time Bishop of London, in 1834 and assumed the post as deacon of Eastergate, befriending Henry Edward Manning. In 1837, he was appointed rector at Bergh Apton until 1852, when he was transferred as curate to Christ Church, Hampstead.
After three years, he became rector of Marylebone and in 1857 on the resignation of Samuel Hinds, he was consecrated the 64th Bishop of Norwich.
From 1847, he served as chaplain to Queen Victoria. Pelham retired as bishop in 1893 and spent the next year in Thorpe, Norfolk.
He died in 1894 and already one year later a monument to his reminiscence was positioned in one of the transepts of Norwich Cathedral.