YOu can just use a transistor or solid state relay. HOwever, what you get in switching speed you trade for by giving up the ideal open and closed switch, which is what you get with a relay. If you cannot tolerate the leakage currents, then perhaps connect a LARGE decoupling capacitor to supply power to your circuit while the relay is turning on and to supress the effects of the bouncing of the contact when it closes. (YOu should have an extra large capacitor anyways, even with a semiconductor switch).