Jim, et al,
Not to put too fine a point on things, so even dealing in some approx numbers, there are certainly quite a few circumstances where a switch or relay having 60 - 70db of isolation on HF freqs, would still allow enough signal feed-thru to still be copy-able on another receiver or copy-able on the same radio with the antenna switch in the wrong position.
IF the Un-used portS are DC grounded to the chassis of the switchbox, I woulda thought the isolation would be through the roof. Like, just how are you going to hear a signal, any signal, when the center conductor of the UN-used port is DC grounded ?
That's like putting a shorted PL-259 plug into the back of xcvr #2...with ant connected to the SO-239 of xcvr #1. You won't hear anything on xcvr #2.
Another possible option is to try using 2 x ports that are NOT adjacent to each other.... like say ports #1 and #3...... or ports #1 and #4.
If you shorted-out AND shielded that PL-259 plugged into #2, and your antenna feedline was aways away distant (like a few feet), and securely plugged into #1, then you're correct it's unlikely you'd hear anything on #2!
BUT...
But with coax switches (or internal rig switches / relays), we have a different situation.

The RF is flowing just a fraction of an inch away, and since the attenuation / isolation is effecting the signals and noise equally....things are different.
Here's just one example (one that I have experience with):
On 80m SSB, I regularly speak with friends within the state of Florida, most typically within ~ 200 miles, some as close as 50 miles...
And, signals are quite strong, at S-9 +30 to S-9 +40 with most using 1/2-wave dipoles at ~ 50' (at my temp QTH, mine is only ~11' - 15' high....and, I'm still hanging with them)..
(some have pretty accurate S-meters in their ANAN's, and while my old TR-7 needs an alignment, I'm good at ~ 6db/S-unit....but above S-9+20, it gets pretty stingy...)
So, again using approx numbers, with average signal levels of -40dbm (S-9 +33), and a switch or relay with isolation of 60db - 70db (typical of good quality ham switches / relays) we'd still have -100db to -110dbm signals on the other switch ports...
And, that would still be an approx 20 - 30db S/N....approx S-3 to S-4 (with no noise).
{Now my Bird 74 with ~ 90db+ of isolation would put the S-9+33db SSB signal into / near the noise of the receiver at that bandwidth, and might not be copy-able...but if it was CW and we used a narrow rec bandwidth could still be a usable CW signal...}
Remember we are attenuating the noise along with the signal thru the isolation of the switch/relay, and that our HF receivers are very sensitive (in CW bandwidths, with a 3db S+N/N, most have MDS levels of -130dbm to -135dbm.....and in SSB bandwidths, with a 10db S+N/N, most have specs at ~ -120dbm)}So, if everyone was using a Bird 72, or similar from other manufactures, and signals were more "average" (S-9 to S-9 +10, at best), then it's unlikely that you'd hear anything from the other ports...
But, for the average ham, even using a decent switch, there is going to be some signal available on the other ports...
Have a look at some sweeps of my Daiwa CS-201's here:
Re: Share 2 Antennas on 3 Radios?
https://www.eham.net/community/smf/index.php/topic,132604.msg1217846.html#msg1217846Re: Easy way to keep an antenna analyzer in your system.
https://www.eham.net/community/smf/index.php/topic,136421.msg1264701.html#msg1264701Re: Antenna Switch Isolation
https://www.eham.net/community/smf/index.php/topic,136487.msg1265278.html#msg1265278It might surprise some at how good they are (~ 80db+ isolation on HF)....and, while some may wish to tout other brands (yes, I do use a Bird 74 and a 72, as well, for more critical situations), fact is the Daiwa's and Alpha-Delta's are pretty darn good switches for ham radio, and are more than good enough for most hams / ham shacks...

I hope this helps some?
73,
John, KA4WJA