His family had immigrated from Flensburg, Southern Schleswig in 1787. Peter Feilberg was a dyer by education, and operated a printing press He also ran a book shop, and, later, a rental library.
In 1830 Feilberg started the first newspaper in Skien, Skiens Ugeblad.
He was notably convicted of libel in 1837 and sentenced to pay 60 speciedaler. The complaint had been filed by County Governor Frederik Wilhelm Wedel Jarlsberg.
Curiously, Feilberg"s defender Pavels Hielm was convicted of inappropriate prodecure, having recited a poem during the final speech of the defence, and sentenced to pay 10 speciedaler. Skiens Ugeblad was liquidated in 1839, but in January 1840 he started another newspaper Bratsberg Amtstidende.
In 1843 the name was changed to Bratsberg Amtstidende og Correspondent, and Herman Bagger was brought in as co-editors
In 1844, however, Bagger left to form his own newspaper Bratsberg Amts Correspondent, later changed to Correspondenten. At the same time, Feilberg"s newspaper reverted its name to Bratsberg Amtstidende. Bratsberg Amtstidende and Correspondenten would from now on become fierce rivals.
In 1837, when local government was introduced in Norway, Feilberg was elected to the first city council of Skien.
He then served as mayor of Skien from 1838 to 1840. This was partially possible due to that County Governor Wedel Jarlsberg, who had sued him for libel earlier, had stepped down the same year.