That's often how signal generators are designed - a stable oscillator
at a fixed level, with a switchable attenuator on the output. You
need very good shielding of the unit to get good results at the
sub-microvolt level.
I've used older HP 8656, HP 608, and Wavetek signal generators
for testing receive sensitivity, with good results, although I'm sure
there are better ones on the market now. I've also used an
external switched attenuator, although, as others have noted,
coax shielding becomes important.
Some ham-grade switched attenuators may have marginal
shielding.
You may find 75 ohm attenuators: they will work, and you can
calculate the corrections for 50 ohm loads if it is critical. But
make sure the attenuator is designed (and shielded) for RF:
I've come across some audio attenuators that are not suitable.
If you are going to build a switched attenuator, I'd suggest a
maximum step size of 20 dB, with internal shields both between
adjacent sections and across the middle of each.