Camshaft

Having problems getting a pilot bearing into my new motor. My new 2jz was an auto, so it didn't have one. Used high temp grease during installation, and got it in a little. Problem is, I know that you are not supposed to use a steel hammer to put it in. Brass? Don't want to warp the bearing so I have to wait for days just to get another one. Hints?