If you're into music you should check out Nerds Record Store in Shibuya or the Shimokitazawa area.

If you're into geeky stuff (which is highly likely) there's always Akihabara in Tokyo and Nipponbashi in Osaka.

Also, one thing I want to do when I get back is to visit the Book Off and Hard Off thrift stores to pick up cheap-ish gadgets and games.