I think the only mode on the FT-857D that will accept modulation audio from the back panel audio connector when transmit is activated in software is the "DIG" mode.
Are you using the back panel and are you using DIG as the mode?
"DIG" mode works fine for SSB transmit audio if you set Menu #038 "DIG MODE" to "USER-U" or "USER-L" for USB or LSB respectively.
It's an annoying issue: you can't change "DIG MODE" remotely nor can you change the filter bandwidth remotely, so using the FT-857D with back panel audio over the internet involves remembering to set everything right while sitting at the radio, and you're stuck on one sideband.
I don't remember if there's a way to work around this aside from using the actual mic jack. It may be that Menu #040 DIG VOX has some effect on this. I do remember setting up a voice keyer for a VHF contest once and I think I had it so that I could send audio from N1MM logger even though I was in USB mode... though I might have just used DIG, and I don't remember.
HRD does give me a live TX button but when I activate it nothing seems to be going out the radio and tuner
So you get the relay to click over and the red light comes on on the radio?
If so try DIG and it should work.
I think if I get serious about internet remote base before I get a new rig, I'll probably just use the mic jack to get rid of some of these issues. But for casual playing around, remembering to set "DIG MODE" to USER-<the desired sideband> and setting the IF filter to a SSB one on DIG mode has worked OK.