That's actually not uncommon, especially when the antenna is small,
or not properly tuned.
When the coax shell is not connected to the radio, the whole outside
of the coax often acts as the antenna. It's like sticking one end of a
random wire into the antenna jack - it can pick up lots of signals.
When you connect the coax shell, then that mode should go away,
and you have only the signal from the antenna itself. In some cases,
the antenna may pick up much less RF than the "long wire" using
the coax. (And, of course, some antennas radiate more RF from
the coax than from the "antenna" itself.) That might be because
the antenna is physically small, or not tuned to the operating
frequency.
The result can vary widely, depending on the coax, the antenna,
the use of an internal antenna tuner, how the coax is grounded,
etc. By itself, I wouldn't consider it a symptom that the antenna
isn't working properly.
But if you can't hear any signals on the antenna, when you should
reasonably expect to, then the combination of antenna and feedline
need to be evaluated to find the problem. A good first step is to
measure the SWR at the end of the coax in the shack, without using
an antenna tuner, and compare that to the expected value for your
antenna.
There are possible quirks that can be confusing. I have an old SWR
meter where the center contact on one of the connectors has gotten
bent out slightly (probably by PL-259s with too much solder on the
tips over the years). Usually the center pin makes contact deeper
in the socket. But some crimped plugs taper the center pin for the
crimp connection, and those gave only intermittent contact.
In that case, the cable would work if I plugged it directly into the
transmitter, but not if I plugged it through the SWR meter. Other
meters worked fine. So be methodical when testing your coax
cables - it really might be true that they work when plugged into
one radio but not into another!