Assuming that the EMI test you failed was for conducted emissions, the classic filter is a balanced pi network optimized for common mode inductance, with additional common mode capacitors to earth ground. Look at the online catalogs and datasheets for Corcom, Schurter, Filter Concepts, etc. Their internal schematics and attenuation tables will show you how the big boys do it. For low power switchers, Power Integrations datasheets, design kits, and app notes have filter design.
You don't say the frequencies at which the tests failed, or which standards your device was being tested to (this planet has over 100 of them).
Note - radiated and conducted emissions are very dependent on component placement and pc board layout techniques. Also, even though the switching frequency is not very high, there could be parasitic oscillations in the hundreds of megahertz caused by such things and the particular type of output rectifier diode. Without detailed test results, everything is guesswork.
ak