You don't want to improve its response. You filter is working exactly as designed. It is a 2.12KHz low pass filter and is rolling off the high frequency edges of the 10kHz square-wave, giving an output that looks more or less triangular.
The purpose of the filter is to exactly compensate for the 75µs high-pass increase in the recorded signal (the purpose of this increase in high frequency signal and subsequent roll-off during playback is to minimizes the effect of high frequency record noise). Thus a recorded square-wave would be reproduced correctly since the low-pass filter roll-off matches this high-pass recorded gain increase, giving an overall flat-response at the output.