There are number of islands south east of the Izu peninsula.  Although they are quite close to the peninsula they are not part of Shizuoka prefecture and are administered by the Tokyo municipal government. The chain of islands runs roughly north to south in the following order: Oshima, Toshima, Niijima, Kouzuhima, Miyake, Mikurajima, Hachijo, Aogashima.

There are different ferry services between the islands but only some of them can be reached directly from the Izu peninsula.  The number of trips may change in the future as COVID winds down.  The ferry company has an English website and there is a complete timetable and fare PDF available here.  There is a port in Atami, Ito, and Shimoda.