Way back in May, I climbed up the volcano Rangitoto with my Dad who was up from Arrowtown. After a 25 minute ferry ride, you are dropped off on the island to which you can climb and explore to your hearts desire. (There is also next to no facilities so if you are sans snacks or water, it'd be pretty miserable.)
We climbed to the top, looked at some caves and toddled around quite happily.
There are a collection of old holiday homes, most with closed up curtains but we managed to sneak a peak through one of them. We also climbed through a precarious cave and ate our preprepared sushi while watching the water.
If you miss the 3.30pm ferry, you just have to wait until the next day. I wonder what the statistics are of ferry-missers?