Gilliam began playing on a local semi-pro team at age 14 and dropped out of high school in his senior year to pursue his baseball career.
While Gilliam was an adolescent he hung around baseball parks in Nashville, where he was picked up by one of the traveling Negro League teams, the Baltimore Elite Giants, who placed him on their roster in 1945. He roomed with Roy Campanella while playing in Baltimore. A second baseman, Gilliam was tutored by Sammy Hughes, who taught him to be a switch-hitter. After playing in the Negro League he was signed by the Brooklyn Dodgers as they vigorously set the pace for integrating major league baseball between 1947 and 1953. He was the eighth black player chosen by the Dodger organization and he was selected specifically because of his potential to replace the aging Jackie Robinson at second base. Gilliam, who was three times an all-star in the eastern circuit of the Negro National League, acquired while playing in Baltimore the sobriquet "Junior" because he was the youngest member of the team. His youth belied his seriousness about the sport, however; he had an avid interest in baseball and preferred to listen to the older players reminisce than carouse with the younger ones after games were completed. Signed by the Dodgers in late 1950, Gilliam played two years in the International League; he led the league in runs scored in 1951 and 1952. In 1952 he also led in fielding. He was called up to the Dodgers in 1953, where he at first alternated with Jackie Robinson at second base, thereby enabling Robinson to play other less taxing positions and lengthen his career. Gilliam remained with the Dodgers from 1953 through 1966, moving with the team from Brooklyn to Los Angeles in 1958. From 1964 through 1966 while a player-coach, he insisted on hard play, hustles, and spirit, setting a tone around which younger athletes could rally and assuring Dodger victories in the late-season pennant races and in the World Series, which the Dodgers won in 1965. Gilliam's career batting average was . 265; in seven World Series he batted . 211. His greatest strength lay in the fact that he was a versatile athlete who, in the course of his career, played six of the nine baseball positions. A reliable and consistent player, Gilliam selflessly underpinned the more spectacular play of others who starred on the successful Dodger teams of the era. Gilliam, whose salary was modest, especially by later standards, worked in the off-season as a salesman for Hiram Walker Breweries, promoting their beverages. Gilliam had ambitions to become a major league manager, but despite some experience as manager of the San Juan club in the Winter League, these remained unfulfilled at his death. He died in Englewood, California, on October 8, 1978, after suffering a stroke.
Early in his career Gilliam married Edwina A. Fields; the couple had four children.