Of course, but if the goalie has the ball and, from what I know, you can not even touch him or it is a foul, then how can it be a penalty or be in play...
You can not stop him or bother him, as you say, it's a foul. But this is a rule that pretends to help the flow of the game. It doesn't mean that the game is not on play. If you bother him while he wants to pass the ball, it's a foul. But it doesn't mean that the game is not on play. So if the goalie has the ball in his hands and a defender punch a striker inside the box 5 mts away from the goalie, it's a penalty... stupid, but a penalty.
I remember a Betis-Barcelona some years ago. Valdés and Dani (Betis' striker) clashed, although the goalkeeper got the ball. Even though it was a clash, Dani had left the boot up to make harm, but the referee didn't say anything, because Valdés got the ball anyway. When Dani was getting up from the floor, Valdés, who had the ball in his hands, pushed Dani... and the ref gave Betis the penalty. I wanted to kill Valdés for his huge stupidity, he had the ball in his hands and he pushed Dani anyway. It has been one of the most stupid actions I've ever seen. Too bad I didn't find the video in youtube.