R33 =18.7K and R34=1k ,votage divider gives a 0.25v(at pin 3) and voltage applied at J2 is 6db (1.262Vpp,sine wave)
I am measuring the LVDS signals(each) with respect to GND. for 1Mhz output is 0 to 3.3V square wave.
The Osilloscope has 100MHz band width.
Any techniques are there to measure the signal or any circuit or procedure to follow how to measure the high frequency signals.
My requirement is to apply a 6db, 30MHZ sinewave and to get 3.3V(with respect to GND,not the differential) at the output.
That additional information above is noted. However, you did not include the value of the source resistor (Rs or R35), which I suspect was far too high given your initial problem of millivolt spikes above 10Mhz. Also left out was the type of construction of your project and the value of Vcc to the comparator. Using the information you did supply I was able to derive the value of Vcc at 5V. Further, I suspect the project is NOT on a purpose designed PCB. Please correct me if my conclusions are in error.
Taking the information provided above, I used the spice model in the data sheet and LTSpice to simulate your comparator circuit, with only a minor adjustment to the subckt pin/spice order to comply with the proper pinout of the device in the SOIC package.
I found, rather quickly, that the circuit configuration had serious limitations. First, if Rs is below the recommended minimum of 300Ω one encounters common mode problems at certain input levels. Second, the input level and/or type can cause wide shifts in the duty cycle at the output; some is to be expected for type of signal owing to delta in rise times IF the circuit is configured correctly. Third, if Rs is too high, the input RC induced time delay causes further limitations of duty cycle to the point of functional failure of output. Fourth, the AD spice model using a Vcc of 5V indicated a high output ranged from 4.64V unloaded down to an excessive load of 500Ω at 3.44V demonstrating that the high output with a Vcc of 5V would likely exceed the input limits of the SN65XXXX line driver operating at 3.3V. Fifth, a budding engineer should never take for granted bits and pieces of information contained in a datasheet...one must learn to read between the lines. By way of example, did you read any data or see any graphics on the AD sheet pertaining to 3.3V operation OR WHY the source resistance HAD to be at least 300Ω?
Pay close attention to the plot I included on the right. There are clues there to lead you to the proper design configuration if you just give it some serious thought without trying to take shortcuts. You will likely be forced to use the 5V option on the comparator so here is one last clue...think open collector buffer.