Ignacy,
I have observed a similar latency delay in my SDR Play, which I use as a panadapter for my Elecraft K3. I just started experimenting with using SDR Play and HDSDR as a sub receiver for the K3. I was hoping that I could do that and avoid having to purchase the Elecraft sub receiver. I have it working remotely using RemAud for the audio transport. My "normal" latency is a little less than .1 second, but I have not actually measured it. When I try to also run the "sub receiver" audio, the extra audio channel overloads my DSL Internet connection, slowly down my rig control. I don't have another option for higher speed Internet, so I put the SDR Play sub receiver on the shelf until sometime in the future when hopefully I can get faster Internet.
73,
Don
K9AQ