Williams joined the newly formed 9th Cavalry in 1866. Having been illiterate before joining the army, he learned to read and write, reaching the rank of First Sergeant just two years later. By August 16, 1881, Williams was serving in Company I of the 9th Cavalry Regiment.
On that day he participated in an engagement in the foothills of the Cuchillo Negro Mountains in New Mexico.
Williams became one of the first African-American Ordnance Sergeants in 1886, and starting in 1895, served at Fort Stevens, Oregon, where he was charged with the care of large coastal gun emplacements. He left the Army in 1898 due to health issues and died the next year at the age of 53.
He was buried in Fort Vancouver Military Cemetery, Vancouver, Washington.