Harry Golden was born on May 6, 1902 to Leib and Anna Klein Goldhirsch. Although many references identify Golden's birthplace as New York City, his autobiography says that his father and his oldest brother left their home in Milulintsy, a village in eastern Galicia, then part of the Austria-Hungary Empire, and migrated to Canada and, after brief stays in other places, saved money to bring Golden and his mother and sister to New York City in 1905. Immigration officials at Ellis Island changed the surname to Goldhurst.
Golden grew up in the poor but culturally rich Jewish ghetto of New York City's Lower East Side, where his father earned a living as a teacher of Hebrew and as a freelance writer. Leib, a rationalist in religion, nonetheless kept strict Sabbath observances.