Born in New York City, he was signed as an amateur free agent by the New York Yankees in 1948. His Major League debut was on April 18, 1954 for the Yankees. He wore uniform number 55 for the Yankees during his entire period on the team
He played in two World Series for the Yankees, in 1955 (against the Brooklyn Dodgers) and in 1957 (against the Milwaukee Braves).
By 1957, because of arm troubles, he became an exclusive relief pitcher. He has been retroactively listed as leading the American League in saves in 1957 with 19.
(At the time, saves were not a regularly calculated statistic)
Grim got the final out of the 1957 All-Star Game, being brought in from the bullpen with the American League leading 6–5 and getting pinch-hitter Gil Hodges on a game-ending fly out to left field On June 15, 1958, Grim was traded, along with Harry Simpson, by the New York Yankees to the Kansas City Athletics for Duke Maas and Virgil Trucks.
He had records of 7–6 and 6–10 for the Athletics the next two seasons.
He played with three teams in 1960. On April 5, 1960, he was traded by Kansas City to the Cleveland Indians for Leo Kiely. On May 18, the Cincinnati Reds purchased his contract from Cleveland, and on July 29, the Saint Louis Cardinals purchased his contract from Cincinnati.
Grim was released by the Cardinals before the 1962 season, but on April 9, 1962 he was signed as a free agent with the Kansas City Athletics.
His final MLB game was played on May 24, and the A"s released him on May 31. He died in Shawnee, Kansas.