I've had my KX3 fer 8 years and haven't noticed your issue, I have lots of volume into headphones...I would definitely pose your question to the Elecraft forum and I'm sure some wizard will have some suggestions that may help....the forum is well run. My first thought is the headphone jack needs cleaning or replacing? I'm a big fan of Detoxit for cleaning connections.
Let us know what you found to be the problem for future reference.
GL/73