Tony Smith was born on September 23, 1912, in South Orange, New Jersey, United States. He was a second from seven children into a family of a mechanical engineer owning the family waterworks factory, and his spouse, a housewife.
When Tony was three years old, the family visited the Panama-Pacific International Exposition in San Francisco. The neo-classical Palace of Fine Arts by Bernard Maybeck and the ancient Taos Pueblo in New Mexico impressed the boy a lot and influenced his future art.
A year later, Tony was diagnosed tuberculosis which remained during his school years. All this time, the boy was cured by a nurse and tutored by a private tutor at the one-room prefabricated construction in the backyard of the home.
It was this time when Smith created the prototypes of his future sculptural compositions using the medicine boxes.