He attended the Hebrew University of Jerusalem, gaining an Master of Arts in Humanities.
He made aliyah to Mandatory Palestine in 1935 and worked as a teacher in Jerusalem. He went on to study at the New York Theological Seminary, where he was awarded a doctorate in literature. He worked for the Jewish Agency and was director of the department of Craftsman and Small Business.
Between 1940 and 1947, he edited a small journal.
In 1951, he was elected to the Knesset on the Hapoel HaMizrachi list. He was re-elected in 1955, 1959 (by which time Hapoel HaMizrachi had merged into the National Religious Party) and 1961.
In November 1961 he was appointed Deputy Minister of Health, serving until David Ben-Gurion resigned as Prime Minister in June 1963. He was re-appointed to the post in July 1963, serving until March 1965.
He was re-elected in 1965 and 1969, and following Haim-Moshe Shapira"s death in July 1970, became the leader of the NRP. He was re-elected again in 1973, and was appointed Minister of Religions by Golda Meir in March 1974.
However, when Yitzhak Rabin formed a new government in June 1974 following Meir"s resignation, the NRP were not included in the coalition and Rafael left the cabinet. When the party joined the coalition in October that year, Rafael was re-appointed Minister of Religions, serving until all NRP ministers were sacked in December 1976 for abstaining in a motion of no confidence vote. In the same year he lost the party leadership when Zevulun Hammer and Yosef Burg combined forces to oust him.
He lost his seat in the 1977 elections and retired from politics.
During his youth he was a member of the Torah and Work youth movement, and founded a local branch of the Bnei Akiva youth movement. He was a member of the Hagana and represented Hapoel HaMizrachi in the Assembly of Representatives in 1944. During the 1948 Arab–Israeli War, he was a member of the Jerusalem Emergency Committee.
Between 1948 and 1953, he was a member of the Jewish Agency"s board, and headed its aliyah department, where he was responsible for the large influx of Jewish refugees (numbering over 685,000 between 1948 and 1951), rejecting calls to put a limit on numbers.