Since you're simulating anyway, replace them all. It's not really the delay that's a problem, but how it affects the rate of turn on and turn off. What is the rise/fall time on the leading edge and trailing edge? This is what needs to be improved.
Use some very low gate charge mosfets for drivers. They will have the fastest switching rates.
I just did this recently in my buck sim and it more than doubled my leading edge response time. I think it will be very necessary if you're going to try and reach 5Mhz.