Yeah it sounds like the bendix gear is sticking when you start the jeep. When you turn your key there is a spring loaded gear that springs out and engages with the flywheel and once the jeep is started the gear is subposed to retract back to where it was. Sounds like yours is sticking in the out position. You will probably have to get a new starter, if its a cj they are not that expensive, I think about 26.00 at local auto stores.