Cadbury was born at Great Yarmouth in Norfolk, the son of Sir Egbert Cadbury and his wife, Mary Forbes, the daughter of Review Forbes Phillips of Gorleston in Suffolk. His father was a World War I flying ace and managing director of Cadbury Brothers, the chocolate enterprise.
He followed his father into flying, with an early career as a test pilot for jet fighters.
Cadbury was educated at Leighton Park School, a Quaker school in Reading, Berkshire founded by his grandfather, George Cadbury, and at Trinity College, Cambridge. He stood as a Liberal candidate in Stroud at the 1945 General Election and finished third.
Cadbury never worked in the family business. He qualified as a barrister, playing a minor role in the Nuremberg War Crimes trials, before deciding his future did not lie in law. Borrowing £75,000 from his father, Cadbury purchased the Keith Prowse theatre booking agency.
After this, he was involved as a company director in the establishment of Tyne Tees Television and led the consortium responsible for Westward Television, the first Independent Television franchise holder for the southwest of England, becoming its Executive Director.
He also owned his own airline and travel business.
