Rust was born at Stowmarket, Suffolk, on 25 March 1808, the son of Thomas Rust (1774–1842),a leading tradesman and a prominent member of the Baptist Congregation in Stowmarket, and Ann Bridge (d 1810), and was educated in a boarding school at Halesworth.
He was placed while a youth with a Messrs Spooner, Loggatt & Company, woollen merchants. His leisure, however, was occupied with linguistic studies, and he arranged in parallel columns, for comparative purposes, translations of the Scriptures in Hebrew, Greek, Latin and Syriac. Rust joined the Baptist Church worshipping in Salem Chapel, Soho, and was baptised by John Stevens, the pastor, 30 June 1831.
About 1836 he began to preach in various rooms and small chapels in the London suburbs, together with some other young men who devoted themselves on Sundays to this work.
He entered the Particular Baptist ministry in 1837, and about this time married Elizabeth Maria Warren (1808–1887), the only daughter of the late John Willing Warren, the author of Ten Thousand a Year. There was one son, John Cyprian Rust, to the union.
He became a Baptist preacher in London, and in 1838 was ordained pastor of the baptist chapel, Eld Lane, Colchester. In 1842 he resigned his pastorate on account of ill health, but remained at Colchester, taking literary and occasional Ministerial work, till 1849.
He had previously been licensed to the perpetual curacy of Saint Michael at Thorn, Norwich, and in 1860 he was presented by John Thomas Pelham, bishop of Norwich, to the rectory of Heigham.
Its huge parish was subsequently divided into three, and Rust chose for himself the newly constituted parish of Holy Trinity, South Heigham, to the rectory of which he was admitted on 2 April 1868. During the time of his residence at Norwich, he took an active part in most of the religious and philanthropic movements and societies which were carried on in the City. In 1875 he was presented by the Bishop to the Rectory of Westerfield, a small village 2 miles from Ipswich, where he remained till 1890.
In that year he resigned under the Incumbents Resignation Acting, and came to live at Soham, Cambridgeshire.
He was buried at Westerfield on 13 March.
