He studied at the Hugo Treffner Gymnasium and at the University of Tartu, from which he graduated in 1956, specializing in Finno-Ugric languages.
In 1989–2001, Kaalep was the editor-in-chief of the journal Akadeemia. In 2002 he held a one-year professorship of Liberal Arts at the University of Tartu. He has published mainly poetry collections.
In addition, he has translated into Estonian poetry and prose works from German (Johannes Robert Becher, Bertolt Brecht, Heimito von Doderer, Günter Eich, Max Frisch, Albert Paris Gütersloh, Hermann Hesse, Hugo von Hofmannsthal, Ödön von Horváth, Hans Henry Jahnn, Gotthold Ephraim Lessing, Heinrich Mann, Georg Maurer, Hans Erich Nossack, Benno Pludra, Friedrich Schiller), Spanish (Vicente Aleixandre, Federico García Lorca, Lope Félix de Vega Carpio, Octavio Paz, José Soler Puig, César Vallejo), French (Charles Baudelaire, Jacques Prévert, To Hoai), Portuguese (Agostinho Neto, Fernando Pessoa), Catalan (Salvador Espriu), Ukrainian (Taras Shevchenko), Polish (Juliusz Słowacki), English, Galician, Provençal, Turkish (Nâzım Hikmet Ran), Tajik, Uzbek, Georgian, Finnish (Arvo Turtiainen), Latin (Ovid), and Ancient Greek (Sophocles, together with Ülo Torpats).
Kaalep was a member of the Congress of Estonia.