Freeman John Dyson was born on December 15, 1923, in Crowthorne, Berkshire, United Kingdom. His father, George Dyson, was a musician and composer, and his mother, Mildred Atkey, a lawyer. Dyson’s elder sister, Alice, remembered him as a little boy, sitting surrounded by encyclopedias and sheets of paper on which he was calculating things. At the time of Freeman's birth, George was teaching music at Wellington College in Berkshire. He had earlier taught at Marlborough College where he was a colleague and close friend of Mildred's brother Freeman Atkey. After the death of Freeman Atkey, who was killed in action during World War I, both George and Mildred were shattered. It brought them close together and they married in 1916. Their first child Alice was born in 1919, then their second child was Freeman Dyson who was named after Freeman Atkey.
Shortly after Dyson was born, his father accepted the post of Master of Music at Winchester College, and so Freeman spent his early years in Winchester. He was closer to his mother than to his father, for she was the more serious of the two being extremely talented and well-read. The family was well off and employed a cook, gardener, housemaid, and nursemaid.