You don't want an opamp, you want a power amp - but you don't want a bridged amp either, that would be useless to you. It's not at all 'tricky', it's dead simple and trivial.
An LM386 is perfectly fine (I used LM380's back in the 70's for this).
You could simply use the LM386 circuit as above, and in to 32 ohms this will give you a quarter of the power than it would give to 8 ohms. But, to prevent it been loud enough to damage your ears, you should add a series resistor between the coupling capacitor and the headphones. The value depends how loud you want it to go, the larger the resistor, the quieter it will be - try 100 ohms, and adjust from there.