What, What?

you said>>>I also found that have the shield of the coax connected lowered the sensitivity of the receiver. Meaning that if I unscrewed the outside of the PL259 connector and left just the center conductor connected I got better sensitivity. Any thoughts on why that would be?
This is similar to the conversation below from the movie the SANDLOT:
Squints:
Where did your old man get that ball?
Smalls:
I don't know. Some lady gave it to him. She even signed her name on it... Ruth. Baby Ruth.
All:
BABE RUTH?!
(Everyone screams in horror and rushes to the fence)
Smalls:
I was gonna put the ball back.
Squints:
But it was signed by Babe Ruth!
Smalls:
Yeah, you keep telling me that! Who is she?
Ham Porter:
WHAT?! WHAT?!
Kenny:
The sultan of swat!
Bertram:
The king of crash!
Timmy:
The colossus of clout!
Tommy:
The colossus of clout!
All:
BABE RUTH!
Ham Porter:
[emphatically] THE GREAT BAMBINO!
Smalls:
[in shock and horror] Oh, my God! You mean that's the same guy?!
You are killing me Smalls,
You have serious cable issues not to mention the lack of other pieces of test equipment needed to test your system.
Once you get your interconnect cable issues resolved you then need to perform an 'Effective Sensitivity Test' with everything connected up as follows.
To properly check effective sensitivity you need either a COMM test set that can run in duplex mode which injects a RX signal while simultaneously measuring TX output power.
In lack of this you can use a seperate RF generator coupled into a directional coupler (DC) that is connected at the duplexer output port in line with the coax going to the antenna.
If you go the DC route then you also need to either measure or accept the coupler Mfg.s coupling value in dB, let's just say the Mfg spec's his DC at -30 db for instance on the port that the RF sig will be injected into the repeaters duplexers antenna port.
Also be aware that the TX power out of the duplexer will appear at the RF gens output port minus the directional couplers (DC) coupling in dB.
In our example the full TX power will be 30 db lower than the true tx level so make sure that your Rf gens output port can take this TX level.
For instance if you are putting out +40 dBm at the duplexers ant port then the -30 db DC coupling number will let +10 dbm of tx power appear at the RF gens output port, if your RF gen cannot accept +10 dBm then you need added external attenuation on the DC port coupled in to the duplexers input port until the Rf gen level is safe.
You need to then set your RF gen to the repeaters receive frequency and set the RF gens AUDIO for 3 KHz FM dev with a 1KHz test tone, if your repeaters RX uses CTCSS then you will need a second audio gen set to the repeaters RX CTCSS tone with a Dev between 500 and 750 hz.
Connect the RF gen to the DC so that the sig is coupled into the duplexer antenna port and not towards the antenna.
Next begin with a RF gen level around -80 dBm.
Next increase or decrease the the RF gen level until you have a nice 12 db sinad reading coming out of the repeaters receiver.
Take the RF gen level above and add it to the directional couplers (DC) port loss, -30dB in our example and you have your effective sensitivity number.
IE for example if your Rf gen level is -70 dBm for ~12 db sinad for the above test and your DC has -30 dB on the port coupled into the duplexers antenna port then your effective sens is (-70) + (-30) or -100dBm.
A COMM test set such as an HP8920 set up in full duplex mode takes care of all of the above and makes RPTR eff sens tests fast and easy, your little spectrum analyzer with the built in tracking gen isn't made for this type of test as I don't think that it has the proper audio generators and such.