Julia Elizabeth Wells was born on 1 October 1935, in Walton-on-Thames, Surrey, England. Her mother, Barbara Ward Wells (née Morris; 1910–1984) was born in Chertsey and married Edward Charles "Ted" Wells (1908–1990), a teacher of metalwork and woodwork in 1932. However, Andrews was conceived as a result of an affair her mother had with a family friend believed to be Alfred Westmacott, a boat builder who designed the popular XOD sailboat. Andrews discovered her true parentage from her mother in 1950, although it was not publicly disclosed until her 2008 autobiography.
With the outbreak of World War II, Barbara and Ted Wells went their separate ways and were soon divorced. Each remarried: Barbara to Ted Andrews, in 1943, and Ted Wells, in 1944, to Winifred Maud (Hyde) Birkhead, a war widow and former hairstylist working a lathe at a war work factory that employed them both in Hinchley Wood, Surrey. Ted Wells assisted with evacuating children to Surrey during the Blitz, while Barbara joined Ted Andrews in entertaining the troops through the Entertainments National Service Association (ENSA).
Andrews lived briefly with Ted Wells and her brother John in Surrey. In 1940, Ted Wells sent young Julia to live with her mother and stepfather, who, the elder Wells thought, would be better able to provide for his talented daughter's artistic training. According to her 2008 autobiography Home, while Julie had been used to calling Ted Andrews "Uncle Ted", her mother suggested it would be more appropriate to refer to her stepfather as "Pop", while her father remained "Dad" or "Daddy" to her. Julie disliked this change.
The Andrews family was "very poor and we lived in a bad slum area of London," Andrews recalled, adding, "That was a very black period in my life." According to Andrews, her stepfather was violent and an alcoholic. Ted Andrews twice, while drunk, tried to get into bed with his stepdaughter, resulting in Andrews fitting a lock on her door. But, as the stage career of Ted and Barbara Andrews improved, they were able to afford to move to better surroundings, first to Beckenham and then, as the war ended, back to the Andrews' hometown of Hersham. The Andrews family took up residence at the Old Meuse, in West Grove, Hersham, a house (now demolished) where Andrews' maternal grandmother had served as a maid.