He played three matches for Victoria as a right-hand batsman, scoring 65 runs at an average of 13.00 with a highest score of 28. He also kept wicket, taking two catches. Swift umpired eight Test matches, and was the first Australian to regularly officiate.
He made his debut in the match between Australia and England in Melbourne on 31 December 1881 to 4 January 1882, officiating with James Lillywhite.
In 1882-1883 he and Ted Elliott stood in all four Test matches, the first time two umpires had officiated throughout an entire series. Moyes commented that "apparently we had reached the time when some kind of qualification was required and some consistency in appointment was observed." This was the series that saw presentation of The Ashes urn to the English captain, Ivo Bligh.
Swift"s last match was between Australia and England in Sydney from 25 February to 1 March 1887.