After graduating from the Naval Academy he served on the frigate Saint Lawrence, flagship to the Brazil Squadron. In 1859 he was promoted to Master, and in 1861 served on the sloop Jamestown. That same year, he was commissioned as a Lieutenant, and then served at the Naval Academy from 1862 to 1863.
He was then assigned to the ironclad monitor Lehigh, part of the South Atlantic Blockading Squadron.
In 1864 he was assigned to another ironclad, New Ironsides and continued to serve on blockading duty until the end of the war in 1865. From 1866 to 1869 he served as an instructor again at the Naval Academy.
On July 13, 1870, Phythian was commissioned a commander and became chief-of-staff of the Pacific Squadron, remaining there until 1872. He assigned to the Boston Navy Yard from 1873 until 1874, then was transferred to the New York Nautical School (now State University of New York Maritime College) as its first superintendent.
Phythian would remain the superintendent of the school until 1878, after which he was assigned for special duty in 1879 and later torpedo instruction in 1880.
In November 1881 he was promoted to Captain. He then returned to the Boston Navy Yard where he served from 1882 to 1883. After this, he was assigned to the ship Trenton on the Asiatic Station where he served from 1883 to 1886.
Upon his return he served as Superintendent of the Naval Observatory from 1886 to 1890, and then as Superintendent of the Naval Academy from 1890 to 1894.
On September 7, 1894 he was commissioned Commodore and then returned to the Naval Observatory where he served as superintendent until 1897. He retired the next year.
Phythian died in 1917.