I looking for a better performer than my 40 foot high flat top dipole for 80 meter. I do some research and try severel design in EZNEC and it's look like the Vertical delta loop feeded 1/4 lamba on the side can perform very well for low angle...
I had this exact
antenna and successfully used it for several years (150+ DXCC). The (important) difference was that my mounting point was at 85' - this kept the bottom section high enuf (~35') above the ground so that I wasn't being effected by excessive ground loss. Additionally, the antenna was out in the open, over 100' away from any house. Finally, the earth soil condx were good to very good.
Another important consideration about your testing is that your tests are being conducted when the Solar Flux is around 140 with noise levels listed on the QRZ page riding around the S1 or S2 level - not necessarily the best condx for the testing lowband DX.
... can someone give some tricks or experience...
This is my experience. As for 'tricks' - well, we often expect too much out of our new antennas; sometimes later surprised by their performance like when we work the VK9X on 80M. One has to do the best they can with what their condx are. Height above ground and far field ground condx (because it is a vertically polarized signal) are perhaps the most influential parameters on antenna systems that we're talking about. ON4UN's book LowBand DX'ing covers this antenna and reviewing the material may lead you to a better performing 80M delta loop antenna.
GL, 73, Rich, K3VAT