Philo was probably born with the name Julius Philo. His ancestors and family were contemporaries to the rule of the Ptolemaic dynasty and the rule of the Seleucid Empire. Although the names of his parents are unknown, Philo came from a family which was noble, honourable and wealthy. It was either his father or paternal grandfather who was granted Roman citizenship from Roman dictator Gaius Julius Caesar. Jerome wrote that Philo came "de genere sacerdotum" (from a priestly family). His ancestors and family had social ties and connections to the priesthood in Judea, the Hasmonean Dynasty, the Herodian Dynasty and the Julio-Claudian dynasty in Rome.
Philo had two brothers, Alexander the Alabarch and Lysimachus. Through Alexander, Philo had two nephews Tiberius Julius Alexander and Marcus Julius Alexander. The latter was the first husband of the Herodian Princess Berenice. Marcus died in 43 or 44.