The Torah tells us that God rewarded Pinchas killing and Israelite man and a Midyanite woman who were copulating in public. Numbers 25:8:
an Israelite man brought to his companions a Midianite woman right before the eyes of Moses and the whole assembly of Israel while they were weeping at the entrance to the Tent of Meeting. When Pinchas son of Eleazar, the son of Aaron, the priest, saw this, he left the assembly, took a spear in his hand and followed the Israelite into the tent. He drove the spear through both of them—through the Israelite and into the woman's body.
The Torah was written at a time when less eyebrows would be raised at this act of violence - but today, the question of whether Pinchas was right to kill the two lovers is impossible to ignore. Some interesting attempts at answering the problem come from the Midrash, from psychology and from the Ishbetzer Rebbe. The Midrashic approach sees Pinchas' spear as a phallic symbol and the psychological approach sees violence as the opposite action to sex, both here. For the Ishbitzer, the verses describe a tragic love story, here.
The question remains a strong one, however.
In attempting to understand the justification that the Torah gives Pinchas' actions, I think it should be noted the verses continue immediately to describe end of the devastation that was being caused:
Then the plague against the Israelites was stopped; Those who died in the plague numbered 24,000.
Thus the Torah actually puts Pinchas' action in a specific context for us - it was an action that saved lives. In Numbers 25:11 and 12, it is this saving of lives that is explicitly implied as being pleasing to God, and what makes Pinchas worthy of reward:
The LORD said to Moses, "Pinchas the son of Eleazar, the son of Aaron, the priest, has turned my anger away from the Israelites because he was as zealous as I am for my honor among them, so that in my zeal I did not put an end to them. Therefore tell him I am making my covenant of peace with him. He and his descendants will have a covenant of a lasting priesthood, because he was zealous for the honor of his God and made atonement for the Israelites."
Thanks to Avram Piha for pointing that out.