Please help, precision 1kHz sine filter

[Mr RB]

MrAl, I owe you and The Electrician an apology I was wrong on the second harmonic issue (as I'm sure you both knew). I'm still a bit thrown as to why the voltages created in the RC seem to be inverse to the 2nd harmonic but that's irrelevent to finding as solution which you have been doing while I've been wasting time and annoying people.

I've just been reading some white papers on PWM and distortion in high fidelity and at least that has raised an option of generating a compensated sine table.

What the white paper said is basically if the sine table entries are generated from "natural sampling" there is no harmonic distortion. "Uniform sampling" as I used to generate the table makes a sample at a uniform periof (20uS). Natural sampling is to generate the PWM width by a ramp at the point where the ramp coincides with the signal. This is reasonably easy done in hardware but in a sine table it needs the sine interpolated between each two samples, which is not really hard just a little bit more effort. It should still be do-able in Excel.

Other non-distortion options are centre-aligned PWM (as The Electrician originally suggested) or the option I believe you are suggesting of inverting the lower half of the sine table using the same 25 entries as the first half of the table.

The problem I have with both of those options is that it is hard to do on the minimum level PIC which only has a simple PWM module!

So both would require manual PWM generation, and in that case there are some better options like a 2pin push-pull system that gives double the amplitude as well (as you said very early on), or a high freq manual PWM that uses many HI LO cycles per 20uS PWM "step". Actually both of those could be high frequency systems as it's as just easy to generate manual 2pin or 1pin PWM.

I kind of feel like this is going DejaVu as I did a PIC manual-generated high-freq sine PWM back in dec 2009...

Before i jump into generating code for messy manual high-freq PWM, or makign a compensated sine table, do you have a suggestion for balancing the PWM generated by the simple PIC pwm module?

 

[Mr RB]

I used the technique in the white paper to generate a compensated sine table. The results are below.

The first diagram shows the technique, called "natural sampling" which according to the boffins gives a recovered signal that has no harmonic distortion unlike the "uniform sampling" used in my original table.

Sine 50 table, compensated for 2nd harmonic caused by left-aligned PWM.
Sine amplitude goes from 14% to 86% (total amplitude 73%).

52
57
62
66
70
74
77
80
82
84
85
86
86
86
85
83
81
78
75
72
69
65
61
56
52
48
44
39
35
31
28
25
22
19
17
15
14
14
14
15
16
18
20
23
26
30
34
38
43
48

The sine compensated table is shown in the other diagram. If the boffins are right then this sine table should produce left-aligned PWM (like the PIC PWM module) which will filter to be a sine with zero harmonic error, or as close to it as this sample rate allows.

 

[MrAl]

Hi again MrRB,


No waste of time. Interesting to look at all these issues anyway.

Oh so you dont want to use manual pulse generation? It's been a while since i looked at the PIC built in PWM peripheral so i'd have to review that before i can comment. For manual generation i dont think it would be that hard with a lookup table. The other benefit would be the ability to generate possibly sine and cosine waves simultaneously.

In your new table i assume you are doing it the same way now? That is, for pulse 1 you go high at t=0 and stay high for 52 counts, then go low and stay low for 48 counts, so again it's always high for N counts and low for 100-N counts right? Have you done this using the PIC built in PWM too?

Attached is a technique first introduced by Ravas around 1967. Im not sure how good this works though because i've never tried it. My own patterns i generate based on the average signal level centered at the timing intervals, not two different signal levels.

 

[ericgibbs]

Attached is a technique first introduced by Ravas around 1967. Im not sure how good this works though because i've never tried it. My own patterns i generate based on the average signal level centered at the timing intervals, not two different signal levels.

hi Al,
For reference only.
This is the method I used in my earlier simulations.
ie: Sine wave period and Ramp switching Rate into a comparator.

Eric

 

[MrAl]

Hi Eric and MrRB,

hi Al,
For reference only.
This is the method I used in my earlier simulations.
ie: Sine wave period and Ramp switching Rate into a comparator.

Eric

Eric:
Oh ok sounds good. I guess that method works pretty decently then, which is nice to know.

MrRB:
You're newer pattern looks very good. It's not void of all even harmonics, but at least the pesky ones are gone now so the THD after the filter looks pretty good. Here are the results of a purely mathematical analysis (rise and fall times are always zero). The THD up to the 157th is less than 0.1 percent (yes that is less than one tenth of one percent, not 10 percent and not 1 percent).

Relative harmonic content of raw pattern:
  1: +1.00000
  2: +0.00045     3: +0.00068     4: +0.00213     5: +0.00329
  6: +0.00469     7: +0.00012     8: +0.00365     9: +0.00428
 10: +0.00247    11: +0.00272    12: +0.00050    13: +0.00272
 14: +0.00165    15: +0.00130    16: +0.00148    17: +0.00290
 18: +0.00075    19: +0.00095    20: +0.00077    21: +0.00113
 22: +0.00140    23: +0.00058    24: +0.00321    25: +0.00059
 26: +0.00136    27: +0.00342    28: +0.00098    29: +0.00116
 30: +0.00102    31: +0.00058    32: +0.00239    33: +0.00262
 34: +0.00006    35: +0.00273    36: +0.00111    37: +0.00031
 38: +0.00002    39: +0.00120    40: +0.00124    41: +0.00168
 42: +0.00287    43: +0.00486    44: +0.00679    45: +0.01370
 46: +0.04278    47: +0.15417    48: +0.36025    49: +0.48139
 50: +0.94769    51: +0.48140    52: +0.35892    53: +0.15541
 54: +0.04627    55: +0.00777    56: +0.00256    57: +0.00438
 58: +0.00391    59: +0.00121    60: +0.00190    61: +0.00512
 62: +0.00095    63: +0.00407    64: +0.00041    65: +0.00248
 66: +0.00096    67: +0.00016    68: +0.00062    69: +0.00123
 70: +0.00212    71: +0.00013    72: +0.00216    73: +0.00186
 74: +0.00071    75: +0.00177    76: +0.00036    77: +0.00048
 78: +0.00407    79: +0.00079    80: +0.00233    81: +0.00210
 82: +0.00211    83: +0.00029    84: +0.00149    85: +0.00170
 86: +0.00147    87: +0.00104    88: +0.00011    89: +0.00019
 90: +0.00062    91: +0.00011    92: +0.00541    93: +0.01386
 94: +0.04308    95: +0.08099    96: +0.15709    97: +0.18864
 98: +0.08922    99: +0.10514   100: +0.57989   101: +0.10446
102: +0.09064   103: +0.18487   104: +0.15737   105: +0.08864
106: +0.03630   107: +0.01046   108: +0.00707   109: +0.00358
110: +0.00485   111: +0.00217   112: +0.00109   113: +0.00419
114: +0.00194   115: +0.00286   116: +0.00249   117: +0.00063
118: +0.00013   119: +0.00031   120: +0.00020   121: +0.00347
122: +0.00054   123: +0.00157   124: +0.00101   125: +0.00137
126: +0.00192   127: +0.00171   128: +0.00197   129: +0.00008
130: +0.00453   131: +0.00059   132: +0.00133   133: +0.00053
134: +0.00110   135: +0.00123   136: +0.00016   137: +0.00115
138: +0.00210   139: +0.00120   140: +0.00581   141: +0.01532
142: +0.03429   143: +0.06141   144: +0.09410   145: +0.10492
146: +0.06739   147: +0.04026   148: +0.09124   149: +0.01875
150: +0.37819   151: +0.02072   152: +0.09021   153: +0.03550
154: +0.05976   155: +0.10649   156: +0.09940   157: +0.06011

Total harmonic distortion (up to Nth harmonic only):
1.551013016

Total Harmonic Distortion of Filtered Output:
0.0008551115269

 

[The Electrician]

Attached is a image showing the frequency response of 3 filter configurations.

The blue curve is Mr RB's 3 stage filter without an inductor.

The green curve is the 3 stage filter plus a 2.3 mH inductor.

The red curve is for the 3 stage filter plus an 82 mH inductor. The inductor I used here came from Digikey, quite easy to obtain. It has a DC resistance of about 50 ohms, which was subtracted from the 330 ohm resistor.

The red curve has nearly double the attenuation at 2000 Hz compared to the original filter. It also has nearly 10 times the attenuation at most higher harmonics. With this filter in place, the number of intervals in the PWM waveform can be reduced and still achieve the same low distortion as with the original filter and a larger number of intervals. This will give more freedom to generate a PWM waveform with optimum characteristics.

 

[Mr RB]

...
Oh so you dont want to use manual pulse generation? It's been a while since i looked at the PIC built in PWM peripheral so i'd have to review that before i can comment. For manual generation i dont think it would be that hard with a lookup table. The other benefit would be the ability to generate possibly sine and cosine waves simultaneously.

In your new table i assume you are doing it the same way now? That is, for pulse 1 you go high at t=0 and stay high for 52 counts, then go low and stay low for 48 counts, so again it's always high for N counts and low for 100-N counts right? Have you done this using the PIC built in PWM too?
...

I can do the manual pulse generation but it is a bit of work and there is a chance people might corrupt it if they are changing values in some way (like to generate 2kHz sine).

The PIC PWM module (CCP1) is very simple. You load PR2 with (100-1) and this gives a perfect auto PWM period of 100 ticks. Then load each table entry into CCPR1L, one entry loaded per PWM cycle of course. This is bulletproof, as it must do one PWM cycle every 100 ticks. And if people want a different frequency they can change the table values and PR2 values and it's all automatically handled.

With manual bit-banged PWM it has to be totally re-written and timing checked for any change. But the benefit of manual is that it can toggle the PIC pin at 500kHz or higher, which eases the filter load. But then again, even my poor filter has no problem removing the 50kHz...

Yes the PIC PWM does left-aligned PWM HI-LO, this is the most standard PWM mode.

...
Attached is a technique first introduced by Ravas around 1967. Im not sure how good this works though because i've never tried it. My own patterns i generate based on the average signal level centered at the timing intervals, not two different signal levels.

Yep, that's it, that's what the boffins called "natural sampled" PWM and it is the same as I used in the new sine table. It's extremely easy to do in hardware but harder to do in Excel.

...
For reference only.
This is the method I used in my earlier simulations.
ie: Sine wave period and Ramp switching Rate into a comparator.

NOW we find out! That explains why your filtered PWM exactly matches the sine and had none of the 2nd harmonic we were arguing about.

...
The red curve is for the 3 stage filter plus an 82 mH inductor. The inductor I used here came from Digikey, quite easy to obtain. It has a DC resistance of about 50 ohms, which was subtracted from the 330 ohm resistor.
...

Interesting. I would have expected the 2.3mH inductor to look better than that at 50kHz as it has XL=720 ohms at 50kHz and should have 1/3 the 50kHz component at the stage1 of the first RC filter. At least it did on the scope so that mathed the XL calc. I guess by the end of the filter stage3 there is plenty of overall 50kHz attenuation the first stage 50kHz attenuation is less critical.

I'm relucant to specify something like a 82mH inductor for the hobby guys but I will include your recommendation on the web page for the people that want to go the extra distance.

MrAl, your new table of THD values, I assume the initial 1.5% THD is caused by the granularity of only 50 table entries AND the integer rounding? If that is the case, the final 0.08% THD is practically all filter dependent?

"Fudged" method for compensating the sine table.
Itf anyone is wondering, I used a simple stripped down method to calc the compensated sine PWM values, which was plenty accurate enough to be MUCH less than the integer rounding used for the table steps;

Assuming 2 steps on the sine table; A71% B75%
entry for A = A + (((A+B)/2)/100) * (B-A))

So it averages the difference between A and B, then interpolates that % value along the vector between A and B.

Sine and Cosine?
Is that useful? It would be easy enough on any of the 28pin PICs that have 2 PWM modules, but it would require a sine table divisible by 4.
You could use a 16MHz xtal and a sine table with 40 entries, at a slight increase in distortion.

 

[MrAl]

Mr RB,

You didnt comment on post #85, the harmonic analysis of your new pattern and filtered pattern. Are you satisfied now?

 

[Mr RB]

I did comment! And i am very much appreciative!

"MrAl, your new table of THD values, I assume the initial 1.5% THD is caused by the granularity of only 50 table entries AND the integer rounding? If that is the case, the final 0.08% THD is practically all filter dependent?"

I'm happy the final THD looks so good now, for sure. And I'm a lot more confident the figures are accurate after everyone's continued work on all the figures and analysis of why the figures do what they do!

As I asked above, I'm assuming the THD now is because of the coarse 50:1 sample rate and integer rounding errors, which means IF that is so then all it really needs is more filtering to get a very low THD.

My filtering was always a "minimalist" way to get 2v amplitude out of a junkbox passive filter. And that will do fine for my own prototype.

Maybe if people wanted to suggest the next step up, ie; a common hobby-available opamp filter that is easy to attach to the PIC PWM and not much to go wrong I can include that at the end of the web page for people that want really low THD figures and/or better output drive? I've already included people's comments on making the RC and RLC filter more aggressive.

 

[ericgibbs]

hi Roman
The circuit in post #48, shows a MCP6002 [ dual OPA rail 2 rail] connected to the filter.
The MCP6002 and MCP6004 [quad] are not expensive.

 

[Mr RB]

Hi Eric, I just looked in a couple of hobby level catalogues and could not see that part. Could a common type TL072 or LM358 dual opamp be substituted?

I have just put up the first draft of the web page;
https://www.romanblack.com/onesec/Sine1kHz.htm

It includes some of the suggestions and a coupel of graphics people have made in this thread. But currently it does not contain a THD data table (MrAl) in case anything is changed with my PWM table or filters etc.

Please let me know if anything needs to be added or removed etc!

**broken link removed**

 

[ericgibbs]

hi,
The LM358 output swing is only ~0V thru Vs-1.5V [with a 5Vs thats at best 3.5Vo max ]

Approx 5mV to 20mV low , when the output load is => 10K.

All the data for a TL072 is quoted for a dual supply.

 

[The Electrician]

Interesting. I would have expected the 2.3mH inductor to look better than that at 50kHz as it has XL=720 ohms at 50kHz and should have 1/3 the 50kHz component at the stage1 of the first RC filter. At least it did on the scope so that mathed the XL calc. I guess by the end of the filter stage3 there is plenty of overall 50kHz attenuation the first stage 50kHz attenuation is less critical.

I'm relucant to specify something like a 82mH inductor for the hobby guys but I will include your recommendation on the web page for the people that want to go the extra distance.

The XL of 720 ohms doesn't directly add to the 330 ohms of the resistor to give 1050 ohms, but rather gives an impedance of 792 ohms for the series combination of the inductor and resistor. The attenuation of 50 kHz in the first stage is 1/2.378. When estimating the attenuation don't forget that the vertical axis of the graph is logarithmic.

The attenuation at 50 kHz for the filter with the 82 mH inductor is 78 times greater than the 3 stage filter without any inductor.

MrAl, your new table of THD values, I assume the initial 1.5% THD is caused by the granularity of only 50 table entries AND the integer rounding? If that is the case, the final 0.08% THD is practically all filter dependent?

The initial (unfiltered) distortion is 155%, not 1.5%

 

[The Electrician]

A lot or work on the techniques of generating sine wave approximations from "digital" waveforms has been done by the power folks.

See for example: https://www.electro-tech-online.com/custompdfs/2011/06/msintro1.pdf

and: https://www.electro-tech-online.com/custompdfs/2011/06/trans_pe_mar_2004_complete.pdf

 

[MrAl]

I did comment! And i am very much appreciative!

"MrAl, your new table of THD values, I assume the initial 1.5% THD is caused by the granularity of only 50 table entries AND the integer rounding? If that is the case, the final 0.08% THD is practically all filter dependent?"

I'm happy the final THD looks so good now, for sure. And I'm a lot more confident the figures are accurate after everyone's continued work on all the figures and analysis of why the figures do what they do!

As I asked above, I'm assuming the THD now is because of the coarse 50:1 sample rate and integer rounding errors, which means IF that is so then all it really needs is more filtering to get a very low THD.

My filtering was always a "minimalist" way to get 2v amplitude out of a junkbox passive filter. And that will do fine for my own prototype.

Maybe if people wanted to suggest the next step up, ie; a common hobby-available opamp filter that is easy to attach to the PIC PWM and not much to go wrong I can include that at the end of the web page for people that want really low THD figures and/or better output drive? I've already included people's comments on making the RC and RLC filter more aggressive.

Hi MrRB,

The THD figures are for the raw and filtered pattern, the first is 150 percent and the second is 0.08 percent.

Are you saying you want better than 0.08 percent now? That makes me have to ask what you want to use this for?

As i was saying back earlier in this thread, a bandpass filter works best but you do have to rely on the linearity of the inductor.

I thought that you might also like to know that your original filter with the inductor means a filtered THD of 0.085 percent, but with the inductor shorted out the THD only rises to 0.096 percent, so that's just a little less than one tenth of one percent without the inductor. You may want to mention that for people who dont want to worry about finding an inductor.

 

[Mr RB]

Thanks for the information Ericgibbs the TheElectrician.

...
Are you saying you want better than 0.08 percent now? That makes me have to ask what you want to use this for?
...

Actually I was happy enough with the earlier sine result even with the 1.2% 2nd harmonic THD. But since we had spent so much time on that and as it was such an easy fix in the table and did not require hardware complexity it was well worth adding, so thank you everyone!

This was only supposed to be a "quicky" open-source project while I am doing some more complex stuff with mt GPS-tuned xtal oven and digital frequency meter and a high res digital capacitance meter, I've been making some precision test equipment now I can calibrate using GPS. I have only really devoted a half hour here and there to the sine project between other work. But it took me a little by surprise when people invested so much time here in the thread!

Considering the original design goal of an easy to make 1kHz sine generator that PIC hobby people can make with no real test gear and should perform very well, I think it is done. Possibly overdone!

...
I thought that you might also like to know that your original filter with the inductor means a filtered THD of 0.085 percent, but with the inductor shorted out the THD only rises to 0.096 percent, so that's just a little less than one tenth of one percent without the inductor. You may want to mention that for people who dont want to worry about finding an inductor.

Thanks for the addition suggestion, i will add that, along with TheElectrician's comment on the inductor and the level of the 50kHz component.

 

[MrAl]

Hi MrRB,

So we're done now then?

 

[Mr RB]

Were as done as you want to be MrAl!

If there's no other additions, subtractions or alterations people want on that page than I will make those last couple of small additions and it's all over.

Basically I said the web page was "first draft" in case people wanted changes to the text, since I loosely paraphrased people's suggestions and used a couple of images people had made (from this thread). There's also a link there so people can come directly back to this thread to read the full filter discussion.

I think you might like the next project "High-resolution capacitance meter" (since we discussed cap measuring in another thread). It has been done for a few days but it needs the web writeup. It does 0.01pF resolution, auto-range 0pF to 50uF, and an auto-zero mode that zeros down to 0.00pF when there is no cap connected (so it's constantly trimming out thermal changes etc between cap tests).

 

[The Electrician]

Thanks for the addition suggestion, i will add that, along with TheElectrician's comment on the inductor and the level of the 50kHz component.

I added an 82 mH inductor and checked to see if the non-linearity of the inductor added significant distortion. The inductor (the one I used, anyway) added some detectable 2nd and 3rd harmonic distortion as seen at the distortion analyzer residual output with a low distortion sine wave of 1 kHz applied to the input of the filter. The two additional harmonics amounted to about .1%. So, even with an inductor adding some distortion, it should be possible to get well under 1% THD with a sine table that doesn't introduce 2nd harmonic.

 

[MrAl]

Hi MrRB,

That's interesting that you say that because i was thinking of looking for an optimization routine that could get the best pattern for a given hardware requirement. I havent done that before i just winged it with the averaged algorithm and that usually worked ok, but in this thread it seems we were looking for optimization as well, even if it didnt make a huge huge difference in the real world circuit. Which BTW also brings up the subject of inductor non linearity, which we havent looked into yet either. That wont be apparent from the numbers we all posted because we dont know if the inductor is non linear and to what degree.

High res cap meter sounds cool too. What is the bottom cap range targeted to be, 10pf, 20pf, etc.? And what are you thinking about so far for the basic hardware?