From 4 August 1778 until April 1803, he was colonel of the East Devon Militia, which was in active service in the Army in 1779. In 1792 he was appointed Lord Lieutenant of Somerset, a post he held until his death. He was also a Recorder of Bridgwater.
The East Devon Militia was again embodied, under his colonelcy, for active service in March 1794, and he was also commissioned colonel of the Somersetshire Fencible Cavalry.
On 30 May, he was invested a Knight of the Thistle. He was appointed a Lord of the Bedchamber to George III on 19 November 1795, an office he held until his death.
Poulett was commissioned colonel of the 1st (East) Somersetshire Militia and the East Somerset Regiment of Yeomanry Cavalry on 17 September 1803. Lord Poulett married firstly Sophia Pocock, daughter of Admiral Sir George Pocock, in 1782.
They had ten children, including Vice-Admiral the Honourable George Poulett, father of William Poulett, 6th Earl Poulett.
One daughter, Lady Sophia Poulett was the wife of Henry Vane, 2nd Duke of Cleveland, while another daughter, Lady Mary Poulett, was the second wife of Lord Charles Somerset. After Sophia"s death in January 1811 Poulett married secondly Margaret Burges, daughter of Ynyr Burges and widow of Sir John Smith-Burges, 1st Baronet, in 1816. The Countess Poulett died at Brighton in May 1838.