The LM393 IS a comparator, so making a "comparator window" simply uses two comparators with different voltage thresholds.
In the attached image D1 and D2 are only there for simulation purposes to simulate the open collector nature of the LM393.
R1,R2, and R3 form a reference voltage divider for the comparator window. The reference voltages are close to 3V and 4V using standard resistor values.
*R4 and R5 serve as pullup resistors for the open collector comparator output.
*R4, R6, R7, R8, C1, C2, Q1, Q2, D3, and D4 form a classic multivibrator (oscillator)
Note: *R4 serves a s a pullup resistor for the open collector comparator output as well as one of the resistor legs in the multivibrator.
Note: Circuit blinks the LED at about 2 flashes per second
The multivribrator is configured or "gated" in a way to adhere to the desired input logic requirements ....
1) less than 3V the LED is off
2) 3V to 4V the LED blinks
3) greater than 4V the LED is steady