Is there a rule-of-thumb how far below self resonant frequency the tank output coil should be operated at?

There is a certain attraction to Rules of Thumb (RoT), but beware that they aren't just ROT.

If you were to make an air cored solenoid inductor, you could represent it as some inductance (in H) in series with some resistance which represents the loss in the non ideal conductor used to make the inductor. Note that the effective resistance at RF varies with frequency, and a reasonable first approximation is that is proportional to the square root of frequency due to skin effect.

But, this equivalent circuit fall down at higher frequencies because it does not account for the distributed capacitance between turns. So, a better approximation is to shunt the series L and R with a small value of C.

If you were to wind a solenoid that has an inductance of say 10µH, a series resistance of say 0.3 ohms at 1MHz, and an equivalent shunt capacitance of say 2pF, you could easily calculate and plot a two terminal equivalent R and X from 1 to 100MHz.

Firstly, X will not simply double for every octave of frequency, and it will not be a straight line. Secondly, R is not constant, it is not proportional to sqrt(f).

Work back from the two terminal equivalent at say 10MHz and calculate L... how can our 10µH inductor appear to have a higher inductance? What does this say about measuring inductance?

What if you could change the inductor so that it had the same L but less equivalent shunt C, what would that mean for the characteristics discussed above.

Welcome to the real world inductor... well a first approximation of it anyway.

Some of the effects described above are attributed to some to ferrite, but no, they occur with an air cored inductor.

Add a lossy ferrite core, and now the series L varies with frequency, and series R varies with frequency, but not as simply as described above.

Owen