He took up the writing of haiku, studying under the poet Takarai (Enomoto) Kikaku and adopting the literary name Senzan, but with the fall from power of the statesman Yanagisawa Yoshiyasu, his fortunes rapidly declined and he lived out his last years in retirement in the Fukagawa section of Edo.
The region of Kii was famous for its tangerines, which commanded a very high price in Edo because of the difficulty of transporting them there by sea. In spite of the risks, however, Bunzaemon took a load of them to Edo by ship and succeeded in turning a very large profit. On the return voyage he loaded his ship with salted salmon, an item in short supply in western Japan, and further added to his wealth. Later he set up a lumber business in the Hat- chobori district of Edo and in time became a supplier of lumber for the shogunate, providing the material used hi the building of the Komponchu-do of the Kan’ei-ji temple in Ueno.
He ranked with Naraya Mozaemon as one of the wealthiest merchants of the time and, under the name Kibun Daijin, was frequently made the subject of kabuki plays and other forms of Edo popular literature. There is even a popular song of the period that mentions "white sails seen in the darkness offshore, the tangerine boats from the province of Kii.”
The facts of his life have become so entangled with legend that they are often difficult to make out, and it has been suggested that, while the original Kinokuniya made his money by shipping tangerines, it was a successor by the same name who became a wealthy lumber merchant. It is also possible that the story of Kinokuniya has become confused with that of another wealthy lumber dealer and shipping agent of approximately the same period, Kawamura Zuiken.