Here is a hint:
If you use the limit switch as a "If limit A, then disallow direction A" and "If limit B then disallow direction B"
So, without drawing the schematic, you basically insert a diode in the power line for each limit switch.
The two relays that make up this decision, will either be a wire or a diode for each case. Like C would be in series with the power, NC and C would have a diode between them. When the relay closes (hit a limit) then the diode is inserted. When at a no limit, the diode is shorted out.
The limit relays are effectively activated by the NO contacts of your limit switches.