I would suggest carving with one of the suggested tools, but I use sharpened brass or SS tubes, and take "core samples" at shallow angle to remove the bulk materal, Shove one in from one direction at about a 5 degree angle to the surface, pull it out, come from the other side, and merg it with the other cut, or break off the "plug". This removes the wood very quickly with minimal mess.
The other alternative, maximally messy, is the Dremel sanding drum. This is very safe and easy to control (particularly compared the burrs Matt showed above), and also does the work very quickly. It generates massive snow drifts of balsa dust, but that's what they make vacuum cleaners for.
When you get close, coarse sandpaper, loose, curled up and slide longways, smooths out any divots, and I end up with a small round bit of sandpaper (1/2" or so), maybe 180 or 220, glued to my finger with Hot Stuff.
Brett