... whats the best way to fix this? Shorten the elements??
Fold back the ends of the wires. You might try just adjusting the leg with the coax
running under it. Or revert to your old piece of coax.
A balun reducing the common mode currents due to the shield being connected to one
side of the dipole, but doesn't help prevent coupling between two wires running
parallel to each other. For example, with a 2-element yagi there is enough coupling
between the elements to create the pattern, and they are much further apart (in
wavelengths) than your coax is from one leg of your dipole.
This is why it is recommended to run the coax off at right angles to the antenna
elements for a quarter wavelength or so, to minimize the coupling between them.
But we don't always live in a perfect world, and we sometimes have to improvise to
fit our situation rather than making the situation match the text books. Just retune
your antenna and get on with making contacts.