The talks were agreed upon on the sidelines of the G7 summit in the UK seaside resort of Carbis Bay, which ended on Sunday.
"From the beginning, our side, with an open mind, expected Japan to respond," the official told Yonhap, adding that "it is regrettable that the Japanese side did not respond to the pull-aside plan, which the two sides had agreed on at a working level, due to the annual drills to safeguard the East Sea territory".
The row over the disputed islands escalated when Dokdo appeared on the map of the Tokyo Olympic torch relay route.
According to Japanese news agency Kyodo, the meeting between the Japanese and South Korean leaders remains off the table until the dispute is resolved.
The Liancourt Rocks — called Dokdo Islands by Seoul and Takeshima Islands by Tokyo — lie almost equidistant from Japan and South Korea, which maintains a small police force there. The islets have for decades been a bone of contention, with both sides stating that they have long-standing historical ties to the island group rich in natural resources.