Joseph Bloomfield was born on October 18, 1753, in Woodbridge, New Jersey, United States. He was descended from Thomas Bloomfield, major in Cromwell's army, who emigrated from Woodbridge, on the river Deben in Suffolk, by way of Massachusetts to Woodbridge on Woodbridge Creek, Middlesex County, New Jersey. His descendants were noted for public spirit and in 1796 certain rural settlements in Essex County, New Jersey, were named "Bloomfield" after his great-great-grandson. His father was Dr. Moses Bloomfield, a founder of the New Jersey Medical Society and a member of the colonial assembly and of the provincial congress, who freed fourteen slaves on July 4, 1783, to prove his belief in the Declaration of Independence. Joseph's mother was Sarah (Ogden) Bloomfield.