The right punch helps. the link below is for the Starrett punch with a cupped tip to stay on the pin and is the perfect size for starting those little buggers out. A couple good wacks with a 16 oz hammer does the trick everytime.
http://www.brownells.com/gunsmith-tools-supplies/rifle-tools/pin-tools/taper-pin-starter-prod26485.aspx?ttver=2
http://www.brownells.com/gunsmith-tools-supplies/rifle-tools/pin-tools/taper-pin-starter-prod26485.aspx?ttver=2