A linguistic irritant XIV

I've always found the expression, "Pardon my French," when someone is about to swear or has just sworn, mildly irritating.  Why should French be a euphemism for profane language?  Why not just say, "Pardon my profanity?"