He was also a Baptist minister: from 1836 to 1840 at Llanelian-yn-Rhos and Llanddulas, Denbighshire. From 1838-1840 Glyn Ceiriog in the Ceiriog Valley. From 1847 to 1862 at Tredegar, Monmouthshire.
And from 1862 until his death in 1875, at Caernarfon.
His poem Year Adgyfodiad was published in 1849, whilst he was a minister at Tredegar in the Sirhowy Valley. Many other poems, biographies, an autobiography, and a dictionary followed.
His dictionary, Geiriadur Cymreig Cymraeg, published in Caernarfon in 1868 was one of the first dictionaries to be published only in Welsh (rather than English and Welsh).