...But the sinker slipped out of the slingshot webbing in first few attempts...
I suspect the problem is that the leather "pouch" on the slingshot was cut from a flat sheet
of leather. If the weight isn't exactly centered on the point of force, or if there is any
sideways pull from the string, then the force is applied off-center and the weight doesn't
fly straight.
One solution would be to bind the ends of the pouch with heavy string right by the holes
where it is tied to the rubber. That pulls it into more of an actual pocket shape that tends
to keep the weight centered. Three wraps of string pulled tight and tied is probably sufficient.