Hurt was born in Chesterfield, Derbyshire, the son of Phyllis (née Massey), an amateur actress and engineer, and Arnould Herbert Hurt, a mathematician who became an Anglican clergyman and served as vicar of Shirebrook. Hurt's father was also a vicar at St John's Church in Sunderland. In 1937, he moved his family to Derbyshire, where he became Perpetual Curate of Holy Trinity Church. When Hurt was five, his father became the vicar of St. Stephen's Church in Woodville, Derbyshire, and remained there until 1952.
In 1945, Hurt's father founded 1st Woodville (St. Stephen's) Scout Group, which is still going today. Hurt had a strict upbringing; the family lived opposite a cinema, but he was not allowed to see films there. He was also not permitted to mix with local children because his parents saw them as "too common".