Émilie Claudette Chauchoin was born on September 13, 1903, in Saint-Mandé, France, to Jeanne Marie (née Loew, 1877–1970) and Georges Claude Chauchoin (1867–1925).
Although christened "Émilie", she was called "Lily". because she had an aunt living with her by the name of Émilie. The aunt was her maternal grandmother's adopted child, Emilie Loew (1878–1954), who was not a blood relative, worked as a dressmaker, and never married. Colbert's nickname "Lily" came from Jersey-born actress Lillie Langtry. Jeanne, Emilie Loew, and Colbert's grandmother, Marie Augustine Loew (1842–1930), were born in the Channel Islands between England and France, and thus were already fluent English speakers before coming to the U.S., though French and English were spoken in the family circle.
Colbert's brother, Charles Auguste Chauchoin (1898–1971), was also born in the Bailiwick of Jersey. Jeanne held various occupations. While Georges Chauchoin had lost the sight in his right eye and hadn't settled into a profession, he worked as investment banker, suffering business setbacks. Marie Loew had already been to the U.S., and Georges' brother-in-law (surname Vedel) was already living in New York City. Marie was willing to help Georges financially but also encouraged him to try his luck in the U.S.
In order to pursue more employment opportunities, Colbert and her family, including Marie and Emilie Loew, emigrated to Manhattan in 1906.
They lived in a fifth-floor walk-up at 53rd Street. Colbert stated that climbing those stairs to the fifth floor every day until 1922 made her legs beautiful. Her parents formally changed her legal name to Lily Claudette Chauchoin. Georges Chauchoin worked as a minor official at First National City Bank.
Her family was naturalized in the U.S. in 1912. Her mother wanted to become an opera singer. After all, Jeanne joined two silent films Journey's End (1918) and The Poison Pen (1919) in supporting roles as Jeanne Loew.
Her father, Georges, died in 1925 and her grandmother, Marie Loew, died in New York in 1930.