He was the defense minister of Estonia from November 2004 to 10 October 2005. He was appointed to that position in November 2004 in Juhan Parts"s government. When Parts resigned in April 2005, Jaak Jõerüüt was one of the few ministers retained in Andrus Ansip"s government.
Jõerüüt was also acting foreign minister of Estonia for a few days in February 2005.
In September 2005, Jõerüüt resigned voluntarily over the so-called "T-Shirt Affair". Jõerüüt was succeeded by Jürgen Ligi.
Jõerüüt previously served as Estonia"s ambassador to the United Nations in 2004. He served as ambassador to Italy and Malta from 1998 through 2002, and ambassador to Cyprus from 1999 through 2004.
From 1993 to 1997, Jõerüüt was ambassador to Finland and from 2006 to 2010 to Latvia and now to Sweden from 2011.