Kilby grew up and attended school in Great Bend, Kansas, graduating from Great Bend High School. Road signs at the entrances to the town commemorate his time there, and the Commons Area at Great Bend High School has been named The Jack Kilby Commons Area.
Kilby received his bachelor of science degree from the University of Illinois at Urbana-Champaign, where he is an honorary member of Acacia Fraternity. In 1947, he received a degree in Electrical Engineering. He obtained his master of science in Electrical Engineering from the University of Wisconsin-Extension in Milwaukee, which later became the University of Wisconsin–Milwaukee in 1950, while simultaneously working at Centralab in Milwaukee.