He was appointed Kebiishi (Police Commander) but was exiled to Yamaki, Izu Province (Shizuoka Prefecture), for some offense.
During the heyday of the Tairas, he wielded power as deputy governor of Izu, residing at his mansion at Yamaki and calling himself Yamaki Hogan or Izumi Hogan.
Yoritomo Minamoto, who received Prince Mochihito's command to defeat the Tairas (1180) resolved to kill Kanetaka. A secretary of Yoritomo spied on the defense of his Yamaki Mansion. As result he was assaulted and killed by Moritsuna Sasaki, Kagekado Kato and other retainers of Yoritomo during the annual festival of the Mishima Shrine on August 17, 1180.