He grew up at Lonaconing and passed through its public schools, and in 1876 entered the Wyoming Seminary, near Wilkes-Barre, Pennsylvania.
The following year the family moved to Osage City, Kansas, where his father became a coal dealer. Having decided to enter the ministry of the Methodist Episcopal Church, in which he had been brought up, Murray became a student at Drew Theological Seminary, Madison, New Jersey, in the fall of 1879, expecting to spend four years there.
Before he had finished the course, however, his father died, and to help support the family Murray abandoned his preparation for the ministry and entered upon a business career.
He became the trusted agent and confidential clerk of Col. T. J. Peter, principal owner of the Carbon Coal Company of Osage City, whose business interests were later extended to central Alabama.
In 1882 Murray was transferred to Brierfield, Alabama, and made secretary and treasurer of the Brierfield Iron Company. Seven years later he established a home at Selma, Alabama, where he became a wholesale grocer and a dealer in real estate, in both of which ventures he was successful. While at Brierfield he had transferred his church connection from the Methodist Episcopal Church to the Protestant Episcopal Church, being confirmed July 4, 1886. Throughout his business career his desire to become a minister persisted and he held religious services as he had opportunity.
In 1891 he was licensed to act as lay reader in the Episcopal Church; in 1893 he was made deacon by Coadjutor Bishop Henry M. Jackson, and in 1894 was ordained priest by Bishop Richard Hooker Wilmer. He was now thirty-six years old.
For two years he served as diocesan missionary and had charge of the Alabama River Mission, consisting of eight small stations which he reached on horseback. So signal was his success that in 1896 he was elected rector of the Church of the Advent, Birmingham, the largest parish in the Diocese of Alabama, and here soon took a leading place in the city as well as the Church. He remained there nearly seven years. When Bishop Wilmer died in 1900, Murray came within a few votes of being elected his successor.
In January 1903 he was chosen rector of the largest parish in the Diocese of Maryland, the Church of St. Michael and All Angels, Baltimore, and at once gained prominence because of his great popularity, especially among men. He continued as rector of this church for six years. The Diocese of Kentucky and the Diocese of Mississippi each elected him to be its bishop; but he declined both offices.
In May 1909 he was elected almost unanimously to be bishop coadjutor of Maryland, and was consecrated in his own church on September 29 by Bishops Paret, Adams, Randolph, Nelson, and DuMoulin of Canada. Upon the death of Bishop Paret, January 18, 1911, Murray succeeded at once to the office of bishop of Maryland. Owing to his exceptional business ability and his personal popularity the diocese made great progress during his episcopate of twenty years.
At the General Convention of the Protestant Episcopal Church held at New Orleans, Louisiana, in October 1925, Bishop Murray was elected presiding bishop of the Church. Previously the senior bishop had automatically succeeded to that office; Murray was the first to hold it through the choice of his brother bishops. To share his labors as bishop of Maryland he was granted a coadjutor the following year.
John Gardner Murray died on October 3, 1929, in New York City. His funeral took place in the Church of St. Michael and All Angels, Baltimore, and he was interred in the Druid Ridge Cemetery.
In 1881, John G. Murray married Harriet M. Sprague, by whom he had a daughter.
On December 4, 1889, John G. Murray married Clara Alice Hunsicker of Osage City. The couple had six children.