How can you have a nazir who runs after women?

The Ralbag has a lot of fascinating perushim on this week's haftarah, about the birth of Shimshon. One rather troublesome issue is the nature of Shimshon. He is supposed to be so holy -- he is the chosen of Hashem, gets Divine assistance and a gift of strength, and yet his personal conduct leaves much to be desired. He frumly does not drink wine or cut his hair, but he goes after women. Depending on whether you consider his actions pretexts for actions against Plishtim or personal reactions to slights, some of his actions seem downright churlish. What is going on here?

Ralbag actually relates the two. It was because Hashem knew of this base nature that He reigned him in somewhat by making him a nazir. Thus:

"And since it was revealed and known before He Who Spoke and the world came into being, that Shimshon would in the future chase after women and be punished via them, Hashem planned that from birth, and from the womb, and from conception, he should be kept away from this; and therefore He wished that he would be a nazir of Hashem, for holding back from drinking wine helps a lot for this matter, as they {=Chazal} said "whoever sees a sotah in her disgrace should swear of {as a nazir} wine." And for this reason, his mother held off that the time she became pregnant with him, from drinking wine or strong drink, and warned her from eating anything impure, to inform about the holiness of the lad who would be born, for this is in order that they would come to raise him in holiness. And this too was the reason to guide him such that he would be separate from the vanities of the world, from the bodily desires. And for this reason as well, he commanded that he not pass a razor upon his head, to inform as to his holiness, for all this is part of distancing from his the failing {?} in the matter of intercourse {??} according to what was possible."

It is a neat theory. I would think, though, that if he was willing to refrain from all of this, he might also be willing to go along with a direct command to be celibate.