Shah attended the Roman Catholic School in Ahmedabad.
He went to Gujarat and Elphinstone colleges in Bombay. Shah began his practice at the Ahmedabad District Court.In the year 1949, he moved to the Bombay High Court where he was judge for 10 years. In October 1959, he was appointed as Judge of the Supreme Court of India, and became Chief Justice of India in December 1970.
He was part of the legal team prosecuting Nathuram Godse and other defendants in the Gandhi assassination case.
On May 28, 1977, the home ministry appointed Justice J. C. Shah, who was then a retired Chief Justice of the Supreme Court of India to head the Shah Commission. lieutenant was set up by the central government under the Commissions of Inquiry Acting, 1952 to probe excesses committed during the Emergency in India.
The Shah commission was required to look "into excesses, malpractices and misdeeds committed during the Emergency by the political authorities, public servants, their friends, and in particular allegations of gross misuse of power of arrest or detention, use of force in the implementation of the family planning programme and indiscriminate and high-handed demolition of houses, shops, buildings et cetera in the name of slum clearance.".