OK, good start! If the PTT light on your signalink USB comes on, but the "transmit" light on your 897 doesn't come on, it might be that your software package is trying to use your PC's regular soundcard, not the "USB audio CODEC" of the signalink--I've been caught by that more than a few times.
In FLDIGI, that's under "configure / sound card." Depending on the interface cable from your computer to the Signalink, you may also need to be sure that the right COM port is selected for the cable--that's caught me a few times, too.
If the PTT on the Signalink turns red AND transmit light on the 897 is coming on, that's what it should be doing--so we need to make sure when that happens, you're actually sending RF down the pipe to the antenna--try transmitting with the meter set to show power out, or watching an external wattmeter to doublecheck that.
I haven't used my own 897 for digital in a year or two, but I used to run RTTY and PSK with an 897D, signalink USB, and FLDIGI, so we know it OUGHT to work--just a question of figuring out where the holdup is.