Arbus was born Diane Nemerov to David Nemerov and Gertrude Russek Nemerov, a Jewish couple who lived in New York City and owned Russek's, a famous Fifth Avenue department store. Because of her family's wealth, Arbus was insulated from the effects of the Great Depression while growing up in the 1930s.
Arbus's parents were not deeply involved in parenting their children. She and her siblings were raised by maids and governesses while her mother suffered from depression and her father was busy with work. She separated herself from her family and her lavish childhood.