Ross Granville Harrison was born on January 13, 1870 in Philadelphia, Pennsylvania. He was the only son of Samuel Harrison, a native of Philadelphia, and Catherine Diggs.
Harrison's early boyhood was spent in Germantown, where his interest in the natural environment was fostered in his first school. Usually reserved about his family, Harrison enjoyed telling of his father's professional life as a mechanical engineer. After an apprenticeship as pattern maker, his father spent ten years (1844-1854) in Russia with the American engineers Andrew Eastwick, Joseph Harrison and Thomas Winans. There they equipped and ran the first railroad between St. Petersburg and Moscow under a contract awarded by the czar to George Washington Whistler, father of James McNeill Whistler. From his father Harrison acquired a fine collection of Whistler etchings and his lifelong interest in railroads.
After the death of his mother, he lived with his mother's brother, John Diggs, in Baltimore, Maryland.