Since his father was dying poor, the boy received only a meager common school education.
Bryant early displayed a mechanical bent, and, at fifteen, was apprenticed by his mother to a Boston contractor. By 1808 he was in complete charge of his employer's affairs and two years later himself became a contractor.
He was for a time a contractor for the United States Government and in 1823 built the branch of the United States Bank in Boston, at which time he invented the portable derrick that afterward came into general use.
In 1825 he suggested to the Bunker Hill Monument Association that a railroad be built for the transportation of granite over the rough country from the Quincy quarry to the Neponsit River, a little less than three miles.
Bryant and his associates, chief among whom was Thomas H. Perkins, received a charter from the Massachusetts General Court, March 4, 1826, which incorporated them as the Granite Railway Company, although the enterprise was popularly known as the Quincy Railroad.
Work was begun April 1, 1826, Bryant acting as engineer. Wooden rails, six inches thick and twelve inches in height, were laid on stone sleepers, which were placed eight feet apart.
The rails were covered with iron plates, four inches wide and one-fourth of an inch thick. At crossings stone rails were used. Bryant then built four-wheeled trucks, each with a projecting platform, two trucks being joined to form an eight-wheeled car, and erected a switch and a turntable. These inventions he never patented.
Others improved the inventions and they became standard railroad equipment. The road was completed at a cost of $50, 000 and was opened October 7, 1826, by a train of horse-drawn cars. It was one of the pioneer American railroads, but it was not, as is alleged, the first railroad in the United States, although it was probably the first American railroad to cover wooden rails with iron plates. Bryant's later years, devoted to the care of the Quincy quarry and the Granite Railway Company, were featured by the eight-wheeled car controversy.
Ross Winans adapted the eight-wheeled car to high-speed freight and passenger transportation, and for his adaptation he received a patent in 1834. Sometime after the renewal of the patent in 1848 Winans began suit against the railroads for violation of patent rights. Litigation continued for years, during which Bryant assisted the railroads, whose chief defense consisted of the Bryant car used on the Quincy Railroad.
In 1858 the Supreme Court of the United States refused to sustain the Winans patent on the ground that it was too broad. Bryant never received the promised financial settlement from the railroads and he died poor at Scituate at the age of seventy-eight.
He had, as he later declared, "abandoned [them] to the public. "
In 1859 he declared that, "Every railroad in the country is now using my eight-wheeled car, and I have never received one cent for the invention".
On December 3, 1815, he married Maria Winship Fox, of Boston, by whom he had ten children. His son, Gridley James Fox Bryant, was a famous 19th-century architect and builder.