Hardware RNG (measures AC mains chaos to make random numbers)

[MrAl]

Hi again,

MrRB:
The intended idea was to get a string of pseudo random numbers that dont repeat ever, within the space given. That's what it was intended to do, that's what it does. Another one would be x=(1+57*x) mod 128 with no other code for a quick example.
I even pointed out myself that it repeats.
I also noticed you didnt comment on the PHRNG i suggested.

So you say they can get 1000 digits or messages? Then there's no algorithm that is deterministic that will work given enough time to solve it with a computer assist unless the number of bits are increased, and then even though it is still a matter of time it might take too long to solve given a supercomputer of today. Tomorrow's computer may break any and all if all goes according to plan. That's why i assumed you were looking at hardware based algorithms/processes.

Oblivion:
Yes if you have enough time you can break any of these (geeze just guess the next number has to be 0 to 9, only 10 guesses) unless you can generate a huge database of digits. Then you can outpace any attacks, but that would take a lot of computer time.
I also noticed that you too did not comment on the PHRNG i described. Any theory out there on that technique yet?

All:
So far in my searches there's only one way to keep a cracker out...that's to reduce the time frame that they have to experiment. If they know the algorithm and they intercept messages, then i dont see any way to prevent it using PRNG's of any kind. In theory pi would work, but it could take too long to generate the data needed.
But my interest here has not only been on hardware implemented generators, but also algorithms that generate good random data that has some predictability without being not random. So i am interested in HPRNG's as well as PRNG's, probably more toward PRNG's with a little kick from the operating system just to keep it real
So if any of you have seen any theory on the date and time of day PHRNG, now would be a good time to mention it

As a side note, did anyone yet look at using smoke detector guts inside a chamber of smoke particles blowing around inside? Might not be easy to build though.
How about sand being dumped through an optical orifice and converted to analog with a photo detector and light source? Not the smallest machine but interesting
A mixture of sand and oil being stirred sampled by a light source and photo detector.
These are interesting physical phenomena but yes not just electrical.

The ultimate: The time between phone calls from telephone sales people. I get enough calls where i could generate one new PR number every 10 minutes here

More seriously, the Windows op sys (and probably others) has a built in performance counter. No two computers would have the same count over time because they would be slightly different in frequency. Pick a time frame using a regular PRNG, using that count the cycles, use that to seed the PRNG. Given enough of a time span like 10ms or maybe 100ms it would be hard for two computers to come up with the same RN.
Not only that, even the same computer with the same time frame probably wont generate the same seed every time.

As you may have noticed, i lean more toward looking for code solutions that dont required external hardware, but im still open to the ramp generator technique if i can find a really good reason for doing that but it would have to beat any code based technique (with the PC's existing hardware) for me to really be motivated to build one.
BTW sampling anything other than a ramp waveform could result in biased data. Biased because other waveforms have biased distributions over the time frame. I was going to use a resistor with cap being charged, but it has to have a long time constant to meet the requirements of a (pseudo) ramp, otherwise we'll see a trend.

 

[()blivion]

You may not need [a geiger-muller tube]. Elektor magazine did a radiation detector using just a semiconductor device, I think it was a photodiode. They got some usable results.

I though about that. The problem with that idea is that Americium 241 is primarily an Alpha particle emitter. This means that the radiation can be stopped by the smallest of barriers. Skin proves impassable, as well as a few layers of aluminium foil, 4 < inches of air, and most encapsulation for silicon parts are enough to kill the small amounts of radiation being emitted, which is most likely a good thing. If it weren't this way, I wouldn't be playing with it in my hands... probably.

I tried putting it against the back of a small solar cell. It didn't change the noise levels at all. I did it through the back because it was one of those cells where they take a piece of glass and just plate it with silicon. The silicon side was the back side, so I figured that would be the best way to get the most unobstructed exposure. I think it didn't work because the actual active layer is very very thin. Like only a few tens/hundreds of microns thick or something.

So, to do it in solid state, I think I'm going to need something that has a thick cross section and no encapsulation. I was considering finding something that fit the former criterion, and just removing the encapsulation with chemicals or something. But that's a lot of work... and I'm pretty freaking lazy. Even going back to the idea of a geiger tube It would need to to be the thin mica window type, So much $$$. With this in mind, other methods for RNGing are better IMO.

MrAl PHRNG:
Sorry I didn't give any input on this, I just missed it is all.

There is nothing wrong with your idea, or the idea of wanting to make random numbers with just software and the resources available to all PC's in general. It's very useful honestly. Any way to get input into a program, form outside of the system, that can not be easily predicted is a good source of random data and becomes very useful for cryptology, because it is more "truly random".

I know I'm repeating myself again, but the point of a "TRUE" random number generator is simply to avoid the pit falls of PRNGs. Mostly that they are 100% deterministic, just like Pi is. Or in other words, if I have all your starting information, minus your plain text and password, I can easily recreate your "random number" stream. Therefor it isn't really random and your security is significantly weakened.

You should read all of this probably... https://en.wikipedia.org/wiki/Hardware_random_number_generator Particularly the part about "Using observed events". I think that is about what you want to know, and it validates some of your points.



My Chaoscillator
Here is a preliminary schematic of my version of an analog feedback noise generator. Could also be called an analog LFSR...

View attachment 67147

And *HERE* is a link to the simulation for those who have Java enabled. (You will need to click the button a few of times to start it.)

The bar on the bottom with the xmas colored vertical lines is the delay line element. It could be made of any analog electronic system that has a signal delay and the possibility for adding a tap into it at an arbitrary place. For example, by wrapping several hundred turns of very fine magnet wire around a grounded iron former. Such a structure would have a propagation delay because it would be electrically long with much stray inductance, resistance, and capacitance, and could be tapped any number of times, at any turn.

The simulator is of course idealized, and can only do finite state machine, because it is running inside one. So it's not possible for the simulation to produce real analog. This is also why there is a button on the top right, it is to start oscillation because the simulated system has no noise. But in reality, in a real device, it would be 100% analog in nature and contain a lot of non-idealized parts and background noise. So it would be self starting and very non deterministic. Also, the influence of background noise is not so strong that having control over it allows you to bias the system. Nor are you able to record parameters and deduce the effects. For the sake of observing a 1-0 number stream, I'm currently converting to digital with a basic logic gate. In a real device you could do this many other ways.

I have generally only arranged things in a way that satisfies the simulator, a real device would need to pay better attention to real life parameters. The left graph is the raw output of the the delay line element, at the logic gate input. The right graph is the digitized output. Finally, there is no reason that this device has to be built in this particular way. In fact, I was able to get significant chaos with several different configurations.

How it works, why it's random.
Basically, it works very much like an LFSR. The major difference is that it works purely in the analog realm, and thus has many non-ideal factors that come into play. This make it's output entirely unpredictable in theory.

First off, a proper liner feedback serial shift register is quantized by nature. That is, any "memory cell" can only be a 1 or 0. This device however can have any analog voltage on it, at any point in the "analog memory", and it has an infinite number of "memory cells". This satisfies the first requirement for creating true random numbers because it can effectively "go to infinity". Both in space, and time.

Secondly, because the transmission line is not terminated with a matched impedance, there is an abundance of reflections. These reflections cause constructive and destructive interference with the main signal, in a very chaotic and unpredictable way. And since the long run data of the system as a whole is affected by very small disturbances in it's surroundings (butterfly effect/Chaos theory), it shouldn't be possible for the system to ever repeat a sequence of data again, no matter how long it is run. This satisfies the second requirement for random number generation, no global repeat.

Thirdly, If two clone devices were built perfectly identical, down to the atom. Both devices would produce different random number streams, even if activated at the exact same exact time. This is based on the above chaos theory, and the fact that they both could not occupy the exact same space when activated. So they would not have the exact same externally influenced "micro seeding" or "butterfly seeding" that creates the long term data. This prevents short term repeats.

Finally, the delay line is electrically separated into parts that have no common resonant factor between them. This can be done a number of ways. The right half of the delay need only fit into the left half in a mathematically irrational way. Prime numbers, square of 2, 3. Pi, golden ratio, Et cetera. This is only to prevent the system from finding and locking into a frequency that the delay line element will resonate on.

Conclusion.
I have no way to do real statistical testing on the simulated output. But just form observing by eye, it appears to be quite random. The only thing I can see in the simulator that is not random is if you don't have some source of small randomness to kick start the process, it will simply recirculate the signal in some irregular but repeating pattern. I am almost certain that this can not happen in real life because any entropy entered into either input will be amplified and returned to the system as more entropy. And there will always be thermal entropy in any system. And a closed loop with gain and phase shift cannot remain stable, it must oscillate.

Edit: On closer inspection, (actually farther, I had to move away from the screen to see it.) my circuit appears to have a significant problem with banding over time. This should be a problem with just my specific implementation and not a problem with the theory in general. In theory the system can mimic a LFSR, so it should have all the random property's of that + also being analog instead of digital. Also keep in mind that this banding could be caused by being run in a simulator, and not real hardware.

 

[ronv]

How about a photo cell pointing at the sun?

 

[()blivion]

How about a photo cell pointing at the sun?

Was thinking of this too. Light *IS* radiation after all. Any silicon junction exposed to any source of photons should emit randomness. This is prolly why commercial HRNGs use diode junctions as the noise source. There will always be heat, so heat = infrared light = radiation = random noise.

 

[Mr RB]

(regarding circular oscillators) I too would imagine that increases the entropy; but I have a nagging suspicion there could also be a tendency to pull the oscillators into sync, which would reduce the entropy.

That's a good point! Free running oscillators (like your design) generate a relatively simple beat frequency pattern over the short term, but will drift in sync giving entropy in the longer term. Oscillators that circularly affect each other will likely produce good entropy in the short term but have longer term sync effects (like a digital LFSR which will eventually sync and repeat). Maybe some blend of parallel free running oscillators and circular feedback would give the best of both worlds? The PIC software can remove long term trending in software whitening, so my preference would probably be for better short term entropy, ie a more circular approach.

...The intended idea was to get a string of pseudo random numbers that dont repeat ever, within the space given. That's what it was intended to do, that's what it does. Another one would be x=(1+57*x) mod 128 with no other code for a quick example.
I even pointed out myself that it repeats.

Sorry MrAl, my mistake. I assumed the goal was to produce random data over time, and missed the point that it was to generate only a single set of 64 shuffled bytes. It definitely does that!

...I also noticed you didnt comment on the PHRNG i suggested.

Yeah I had just posted two long winded posts in a row and thought it was overdue time for me to shut up!

It's not new to generate random data from the PC's internal clock, or more commonly to use the clock to seed a software PRNG process. When I was a games programmer on dos and early Win games back in late 80's to early 90's in the age of 386 and 486 PCs, we commonly used a fractal; n = (n * a) + b which generates a rough random data and then hashed or seeded that against the low res bits from the PC's PIT (Programmable Interval Timer) IC which times down to the microsecond. And in dos or Win3x/95 the PIT registers were accessible from hardware addresses exactly like a microcontroller timer is now. The result was some pretty good RNG data as there was a unknown relationship between processing time of the software tasks and the fixed xtal PIT chip time.

These days with Pentiums there is no PIT IC, or no easy accessing. All timer accessing is done through system calls under the Windows kernel, so is subject to Windows and timer resolution is usually limited to about 18 counts per second (system ticks used to generate clock seconds).

It's still easy enough to use the PC clock timer to get some entropy, it's just not as good as the old days. Recently I tried a system with my software PRNG free running, capturing its state on each clock timer event (18 times a second) giving some data that could not be predicted, as there was no real way to determine the internal state of the PRNG at the instant the clock tick was detected. Only having 18 clock events per second limits the total amount of entropy that can be generated.

That is partly why I am looking for a fast hardware RNG, that can generate a few kb per second of really good natural entropy, that can seed the software RNG to produce hundreds of kb per second.

...The problem with that idea is that Americium 241 is primarily an Alpha particle emitter. This means that the radiation can be stopped by the smallest of barriers. ...

Well doesn't the smoke detector that you got the source out of have some type of detector? Isn't that how it works; to detect the radiation and detect a change when smoke gets between the source and detector? Mayb the ideal semiconductor sensor is still inside your smoke detector.

...My Chaoscillator
Here is a preliminary schematic of my version of an analog feedback noise generator. Could also be called an analog LFSR...

Very nice! Your theory looks to have produced an incredibly simple working simulation. Congrats! I have literally hundreds of analogue delay lines in my junkbox pulled from the colour circuits of PAL TVs and VCRs, maybe they could be used? I'm not sure of how many taps they have but I love the simplicity of your circuit with one opamp integrator, one resistor and one delay line.

... The only thing I can see in the simulator that is not random is if you don't have some source of small randomness to kick start the process, it will simply recirculate the signal in some irregular but repeating pattern. I am almost certain that this can not happen in real life because any entropy entered into either input will be amplified and returned to the system as more entropy. And there will always be thermal entropy in any system. And a closed loop with gain and phase shift cannot remain stable, it must oscillate. ...

Agreed, the simulator is digital and has a finite resolution so would likely settle into some long pattern, but a real world system with that amplification should have a lot more entropy than the sim. Also, the brute simplicity of your circuit would be easy to add more chaos by a second opamp or some reactive device like a cap or inductor in the loop.

How about a photo cell pointing at the sun?

That would be simple for sure, but not a 24 hour device and also it comes back to amplifying a tiny noise signal (like other diode junction noise detectors) so there are still the problems of swamping and issues caused by the amplification of a tiny entropy signal. I'd still prefer a large reactive process that should be less easily influenced by RF noise and PSU noise etc, or at least influenced in a very complex fashion by PSU noise not in a simple amplitude fashion.

 

[()blivion]

Well doesn't the smoke detector that you got the source out of have some type of detector? Isn't that how it works; to detect the radiation and detect a change when smoke gets between the source and detector? Mayb the ideal semiconductor sensor is still inside your smoke detector.

It works almost like a Geiger-Muller tube, but in reverse. They set up a charge on what is basically two plates of a capacitor. The alpha particles ionize the air. The charge on the plates cause the ions to mobilize to the plates, causing a very minute current. If smoke enters the area, the smoke particles dramatically reduce the alpha particles ability to ionize the air. this causes a drop in the current and triggers the alarm.

The problem with this arrangement is it does not work on counts. This is because it doesn't cause "avalanche discharges". A constant trickle of current is set up. As far as I can tell, the only way to set up avalanche discharges is if the gas is under enough vacuum to be a great insulator, and the plates are charged to very high voltages. So no matter what, to make a random number generator out of a radioactive "seed", it needs a tricky to get/make tube and excessive voltages.

There are other ways.

Very nice! Your theory looks to have produced an incredibly simple working simulation. Congrats! I have literally hundreds of analogue delay lines in my junkbox pulled from the colour circuits of PAL TVs and VCRs, maybe they could be used? I'm not sure of how many taps they have but I love the simplicity of your circuit with one opamp integrator, one resistor and one delay line.

Thanks, it does appear that it is workable. It's a little awkward because it relies on a device that has been all but abandon in the last 20 years. If you have a bunch may want to cut some open and see what and where the leads attach to. I don't have a formula or anything for where the three taps need to be located. But I have got what looks like excellent chaos out of all sorts of odd delay lengths.

One thing that was disappointing is my attempt to make some kind of idealized version of the circuit failed. I tried to make the the longest half a twin prime in length, and the short half two in length. But with any configuration like this I was not able to get good chaos. This would have been a great accomplishment if it had worked because all the lengths of the delay elements would have been prime, and thus would not have had ANY frequency's in common at all by my calculations. As they would be divisible by nothing.

Odd numbers that are in a ratio of 2:1 seem to be where the money is at. I still have nor made one that only needs one pulse to get started. In theory if you add ANY signal to the circuit, it should rebound on it's self, multiplying the signals, then colliding them together, creating new signals. But it doesn't appear to do this.

I'm still playing with it though. I may build a real one if I feel like it's worth the investment.

(PS, you don't strictly need the resistor I found out

 

[MrAl]

Hi again,


MrRB:
Im not sure what you mean by only 18 clock tics per second. The performance counter (assuming your hardware has one) should be able to do much better than that. That's not the date and time, but it's a pseudo random number in the lower digits. Not only that, but you can use a regular PRNG to generate pseudo random time intervals where the counter is read which gives a fairly decent spread to the numbers.
But what i really meant for this kind of thing was for the seed, not necessarily the entire random numb generator. This would seed the generator and then perhaps thousands of numbers generated with the thing re-seeded after some pseudo random time interval. If you want to re-seed for every run, then you'd be using the performance counter every 100us perhaps. So in effect you'd be getting new seeds 10000 times per second. Faster with a faster CPU. With mine i can get a new one approximately every 2us unless i add more random complexity into the timing interval, and then it gets slower.
For example, sampling every pseudo 200us i get a count difference of 403594 on one pass and 374509 on the next pass. If i sample 10 times faster i'll get roughly 40359 and 37450 for example. Maybe not the best but it doesnt cost anything

I was also thinking about using a Jpg algorithm in reverse. Havent looked into this in detail yet.

 

[Mr RB]

(re delay line Chaoscillator) ...Thanks, it does appear that it is workable. It's a little awkward because it relies on a device that has been all but abandon in the last 20 years. If you have a bunch may want to cut some open and see what and where the leads attach to. I don't have a formula or anything for where the three taps need to be located. But I have got what looks like excellent chaos out of all sorts of odd delay lengths.
...

OK, I looked up the PAL delay lines I have, here is a photo from Wiki;

**broken link removed**

It's ultrasonic piezeo device with one input and only one output, each has 2 wires. The total delay is 64uS. I'm not sure if other electrodes could be attached to the "damping pads" which might be conductive paint or copper deposited.

What I was thinking though is that it does not strictly have to be a mechanical delay line? What about two or three inductors in series, would provide a similar "tapped delay line" effect and be small and easy to find parts for. Because it can be driven with a digital pulse 0-5v it does not matter if the inductors are lossy as the subsequent amplification and digitising provides plenty of gain.

That also leads to a hybrid design of analogue/digital/algorithm where the tapped outputs from the delay feed back into software, hashed in software and fed back into the delay line. Originally I would have preferred hardware only but if some software can be added into the mix to add further complexity or entropy with minimal parts (ie add quality for little to no cost) then it might still match the overall goal of a high quality "hardware" RNG.

...
One thing that was disappointing is my attempt to make some kind of idealized version of the circuit failed. I tried to make the the longest half a twin prime in length, and the short half two in length. But with any configuration like this I was not able to get good chaos. This would have been a great accomplishment if it had worked because all the lengths of the delay elements would have been prime, and thus would not have had ANY frequency's in common at all by my calculations. As they would be divisible by nothing.

Odd numbers that are in a ratio of 2:1 seem to be where the money is at.

I'm not really surprised given the extreme simplicity of your one integrator circuit. It's probably working at some really complex beat frequency of the difference in harmonics of wave/2 and wave/3. By heading away from the strong factors of 2 and 3 towards a long prime it's probably damping all the strong harmonics and killing oscillation. More integrator stages and possibly more gain might be the solution?

...Im not sure what you mean by only 18 clock tics per second.
...

That was in line with using the system clock that is avalable on all PCs, as your earlier suggestion was on using the real time system clock.

...
The performance counter (assuming your hardware has one) should be able to do much better than that. That's not the date and time, but it's a pseudo random number in the lower digits.
...

Sorry my fault, I was focussing on the system clock. Re the performance counter, I know absolutely nothing about it and doubt my old Pentiums even have one? Also some Intel chipsets have dedicated hardware RNG built in, if we are talking about modern PC specifics.

I'm still mainly interested in simple hardware RNGs here, that can be used on any computer or even by a microcontroller.

...
If you want to re-seed for every run, then you'd be using the performance counter every 100us perhaps. So in effect you'd be getting new seeds 10000 times per second. Faster with a faster CPU. With mine i can get a new one approximately every 2us unless i add more random complexity into the timing interval, and then it gets slower.

That's pretty fast! Have you checked the linux "entropy pool" system where they use things like that (system performance) along with things like user keypress timing, to dump entropy into a "pool" and then there is always data that can be extracted from the pool when any program needs random numbers? It looks like a good system unless you try to extract too much data over too little time, and then it needs to wait for more user keypresses etc.

...
I was also thinking about using a Jpg algorithm in reverse. Havent looked into this in detail yet.

I'd love to see what you come up with!

 

[()blivion]

It's ultrasonic piezeo device with one input and only one output, each has 2 wires. The total delay is 64uS. I'm not sure if other electrodes could be attached to the "damping pads" which might be conductive paint or copper deposited.

If I remember right, they power those with a differential signal attached at two places. So to add more electrodes you would need to add two more to any edge that the acoustic wave reflects off of. The problem with this is that the electrodes are chemo/electro plated onto the piezo element. Doing that is going to be difficult, if not impossible for the average DIYer I'm thinking.

What I was thinking though is that it does not strictly have to be a mechanical delay line? What about two or three inductors in series, would provide a similar "tapped delay line" effect and be small and easy to find parts for. Because it can be driven with a digital pulse 0-5v it does not matter if the inductors are lossy as the subsequent amplification and digitising provides plenty of gain.

Yeah, that's what I was thinking too. In general, I don't think it matters what is used so long as it is some device that has a propagation delay with a time period that is useful to us and multiple taps.

At the bottom of the Wikipedia entry for delay lines, it shows a device made out of one long wire on a grounded former. This makes a continuous inductor/capacitor combo device that could be taped at any point easily. This inductor/capacitor combo device concept could be extended farther by wrapping the wire around a non-conductive ferromagnetic core with a grounded conductive cover. The cover would/should be cut down the length to prevent losses caused by secondary induced currents. This structure would greatly increase the stray inductance over the Wikipedia version and still maintain the same stray capacitance. For practical DIY parts I would use a large ferrite rod (like what you would find in an AM/FM radio), wrapped in tin foil, and keep the overlapping foil separated with tape or something. Or one could simply accept the loss caused by eddy currents and just use a grounded smooth iron bar. As you said, there is ample gain.

I had made a model for generic delay line out of a long chain of inductors in series with grounded capacitors at each interconnect... Just before LTspice decided to fill the appdata/temp directory of my computer with 20+ GB of useless data in less than an hour, prompting me to uninstall the program. I think that stands well as a physically accurate model for a delay line, but it slays the simulator. I also think the same thing can be achieved by simply summing all the inductors and capacitors together for each division of the delay line, but I have not tested this and I don't know the maths behind it. Also... I ran into another problem...

In LTspice simulation, no matter what configuration I used, the device would bring it's self out of "noisy oscillation" and begin to resonate with some odd waveforms. The waveforms it locks onto were complex to be sure, but they were visible patterns to be sure as well. I tried many many configurations and different "tap points" into my delay line model and the result appears to be the same. After some arbitrary length of good usable noise, the system finds some waveform that it agrees on and locks onto it. Even after adding two op-amps/comps is still wouldn't work.

I think this has a lot to do with conservation of energy. If the system is just the delay line and one comparator (or op-amp) with it's three connections, then only one connection is low impedance. If that one low impedance connection is constantly adding energy to the delay device, then it is constantly gaining energy. The only way for this to stop is (1) if there is a net energy loss in the system somewhere that the system reaches some form of equilibrium. Or (2) if the waveform is such that it exactly cancels out the input so that energy is no longer being added. I believe that the latter is what's happening in simulation. Though strictly speaking I don't believe the waveform has to be any particular pattern or structure to be able to cancel out incoming energy. I think it only needs to cancel the energy on average over time. And a perfectly balanced white noise like system will have on average a 50:50 1 and 0 ratio.

Anyway, Though designing a chaoscillator would be neat. I'm wondering if it's worth it to make this work. Any noise source should be just as good. And the HRNGs out there that use things like diode/transistor noise are well founded working true random number sources. Unless I'm mistaken and you happen to know of some kind of significant flaws in these devices?

 

[Mr RB]

...At the bottom of the Wikipedia entry for delay lines, it shows a device made out of one long wire on a grounded former. This makes a continuous inductor/capacitor combo device that could be taped at any point easily.
...

Yeah I have some wire wound delays lines too, the problem is they are very fast, in the order of nanoseconds delays.

...
This inductor/capacitor combo device concept could be extended farther by wrapping the wire around a non-conductive ferromagnetic core with a grounded conductive cover. The cover would/should be cut down the length to prevent losses caused by secondary induced currents.
...

Maybe it is overcomplicating it by making a long multitapped inductor? I was thinking more like 3 small standard inductors in series, and the tap points have a resistor to ground. It's just a basic 3* LR phase delay line based on the inductor values. For that matter it could probably be done just as easily with 3* RC?

...
I had made a model for generic delay line out of a long chain of inductors in series with grounded capacitors at each interconnect... Just before LTspice decided to fill the appdata/temp directory of my computer with 20+ GB of useless data in less than an hour, prompting me to uninstall the program. I think that stands well as a physically accurate model for a delay line, but it slays the simulator.
...

Yeouch! You broke the Spice!

...
In LTspice simulation, no matter what configuration I used, the device would bring it's self out of "noisy oscillation" and begin to resonate with some odd waveforms. The waveforms it locks onto were complex to be sure, but they were visible patterns to be sure as well. I tried many many configurations and different "tap points" into my delay line model and the result appears to be the same. After some arbitrary length of good usable noise, the system finds some waveform that it agrees on and locks onto it. Even after adding two op-amps/comps is still wouldn't work.
...

Yeah I think staying in pure analogue encourages it to settle into harmonic patterns. The solution for really good entropy will probably be with using some digital in the feedback system, as this is a clipping effect so would stop the energy buildup you mentioned, and also provides some entropy from the transitions of the digital triggering setpoints.

...
Anyway, Though designing a chaoscillator would be neat. I'm wondering if it's worth it to make this work. Any noise source should be just as good.
...

Yeah your suggestion of a delay line type device is brilliant, and I appreciate all your testing and thoughts so far. I still think going pure analogue is flawed, and the best solution would be analogue/digital and maybe with some algorithm mixed in too. It might end up that a really good solution will be as simple as one resistor and one cap, installed in the feeedback loop of a PIC random number generator. Either way I still want to play with some reactive type hardware RNGs and see what might be possible from say one IC and a few discretes.

...
And the HRNGs out there that use things like diode/transistor noise are well founded working true random number sources. Unless I'm mistaken and you happen to know of some kind of significant flaws in these devices?

These are not a great design. There are known issues from amplifying a very small (uV) amplitude noise signal (ie easy to swamp or influence with external energy or PSU energy), and known problems with silicon ageing causing a change in the random data output. There's also the known problem where the noise source could "break" but the PSU and amplifier etc still produce some noise so it still produces data but of a compromised quality and nobody knows. I think all three of those issues could be largely solved with a Chaoscillator RNG.

From what I have seen the real good hardware RNGs use quantum photon detection comparing photons that do or don't make it through a 50% reflective mirror, or use multiple large chains of propagating inverters in silicon (like in the Intel chips and some private FPGA/CPLD devices), hashing together all their outputs with an algorithm. The photon system is still very liable to break and noise so they must use some powerful testing of data before the data is finally output.

I still like the idea of a really simple oscillator type device where you can just sample its output with the PIC at any time and get a good real-world "random bit".

 

[alec_t]

I still like the idea of a really simple oscillator type device where you can just sample its output with the PIC at any time and get a good real-world "random bit".

IMO that will work, providing the sampling interval isn't harmonically related to the oscillator period. Methinks (again, no maths to back this up) it's entropy would be similar to that of a system which periodically samples the output of a fast counter, the counter clock rate not being harmonically related to the sampling interval. The advantage of using a counter is that you get a random byte rather than a random bit, so the RNs are generated faster.

 

[()blivion]

OK, so I simulated a circuit that used the same op-amp/comp feedback setup as before, but I replaced the delayline element with various simplified reactive elements... The results were not great.

Long story short, no matter how convoluted, complex, tricky, and odd you try and make the circuit, and no matter what numbers, types, and configurations of reactive elements you use, all the resonant factors combine together and the system will resonate at what ever frequency it agrees on. For lack of knowing the exact technical term or mechanisms behind it, I'm just going to call it a "resonant heterodyne" effect.

I think this problem happens, because unlike a honest propagation delay medium, single L/C resonant elements will have a particular frequency that they prefer over all others (the Q). This causes the system to trend toward system wide resonance by filtering out all frequency's other than the one that the resonant elements agree on. And although a genuine propagation delay medium will also have resonant properties, based directly off multiples and fractions of it's delay period, in theory such a device wouldn't necessarily filter signals out. This lets any orphaned noise or frequency survive long enough to be amplified and returned to the system. Also, a delay line is a sort of memory. So at any given point along it there exists a state of information that was generated in previous stages. This information/entropy feeds back into the system, and via chaos theory, causes long term instability. With the above L and C delay elements, there can't really be a memory or state as all the information simply sums together and is lost. Such "negative entropy" keeps the system from generating and conserving chaos.

But I don't pretend to know exactly what I'm talking about. This is just my current crude theory based on observations I made this morning when I simulated it and it failed miserably. It could be doing something completely different. It was a good thought, but tests are showing that simple arbitrary L-C nodes attached to an op-amp/comp feed back loop do NOT make a good chaoscillator so far. Here is an image of said system depicting it's failure...

View attachment 67212

 

[alec_t]

I'm guessing the problem there might be that a closed-loop system will inevitably lead to a single oscillation frequency after a period of instability. Perhaps try open-loop sections with combined outputs?

 

[()blivion]

By open-loop, You mean... have a reactive element just hanging off one of the ends, dangling in the wind? Or did you mean something else? Like, terminate it to ground, (-), (+)... ?¿

 

[Mr RB]

OK, so I simulated a circuit that used the same op-amp/comp feedback setup as before, but I replaced the delayline element with various simplified reactive elements... The results were not great.
...
I think this problem happens, because unlike a honest propagation delay medium, single L/C resonant elements will have a particular frequency that they prefer over all others (the Q).
...

I think you are bang on the money about the LC, and thank you too for testing that! Personally I would have tried a "delay line" of LR or RC (not using LC).

I'm guessing the problem there might be that a closed-loop system will inevitably lead to a single oscillation frequency after a period of instability. Perhaps try open-loop sections with combined outputs?

For sure using some open loop oscillators in the system will feed energy into the feedback loop that is not synced to the feedback loop so that would not settle out as easily. Like pushing a child on a swing, if you pushed at times NOT synced to the swing movement the result would be chaotic.

I doubt that a closed loop system will always lead to a single oscillation frequency, because using delay taps it can reintroduce signals of different wavelengths and phase. Similar to the digital LFSR RNG which feeds different bits with a polynomial relationship back into the start bit, the result is an extremely long chaotic pattern with no single frequency. I think what is proven to work in digital could be done in analogue, or partly in analogue with some digital clipping.

On that "Synth chaos circuits" that 4pyros linked to;
**broken link removed**
he has mainly circuits that accept a separate non-synced oscillator input, I guess that makes it easy to tune certain effects to make the desired sound but might also be a necessary part of the Chaos producing system.

He also has a couple examples of closed loop chaos systems like his "Cyclic System" (which seems to make some good chaos) and this simple example in his "Overview" page;
**broken link removed**

However I completely agree that having one or more open loop processes feeding energy into the closed loop system is a good idea, that too is done in digital RNG processes, for the same reasons (smashing resonant patterns etc).

I've been playing with a simple software feedback process using a single RC or dual RC (RCRC) as part of the feedback loop. For speed, it uses one ADC only (on the chain OUTPUT) and multiple taps INTO the chain, so it is like a LFSR in reverse (which works just as well according to Wiki ).

The multiple taps in are easy, each is just resistor(s) from a PIC digital output into the cap.

The problem I have at the moment is a math one, I need a very long random sequence to drive the start of the chain, whcih is easily done with a LFSR and I have an 80bit LFSR coded which takes billions of years to repeat. However an 80bit LFSR can produce strings of 1's or 0's quite long, up to 79 bits long. I want to keep the very long repeat sequence of the 80bit LFSR but make sure no string of 1's or 0's will ever be longer than 1111 or 0000. That is to ensure the average cap voltage remains near 0.5Vdd and does not clip.

Basically I need a PRNG with a very long repeat sequence but only makes very short strings. Maybe MisterT has a suggestion? My ideas so far are to "fast forward" the LFSR and just remove bits of strings but that becomes non synchronous and I would liek the process to be synchronous, ie; 1 event = 1 bit generated.

 

[alec_t]

@()blivion

Or did you mean something else?

open-loop = output of amp 1 not fed back to input of amp 1 but instead fed to input of amp 2.

 

[()blivion]

Oh... OK. Sorry LOL. In my head there are many many configurations that could be called "open loop". Maybe it's an american (ignorance) thing?

Did I post this circuit here yet? It's my model of a delay line using inductors and capacitors. Look at the output at the bottom, quite chaotic.

View attachment 67224

 

[Mr RB]

Man, I don't want to build that!!

Let me guess, that was the circuit that broke the simulator's back?

 

[4pyros]

Oh... OK. Sorry LOL. In my head there are many many configurations that could be called "open loop". Maybe it's an american (ignorance) thing?

Did I post this circuit here yet? It's my model of a delay line using inductors and capacitors. Look at the output at the bottom, quite chaotic.

View attachment 67224

Thats cool but a ***** to build

 

[WTP Pepper]

I think you are bang on the money about the LC, and thank you too for testing that! Personally I would have tried a "delay line" of LR or RC (not using LC).



For sure using some open loop oscillators in the system will feed energy into the feedback loop that is not synced to the feedback loop so that would not settle out as easily. Like pushing a child on a swing, if you pushed at times NOT synced to the swing movement the result would be chaotic.

I doubt that a closed loop system will always lead to a single oscillation frequency, because using delay taps it can reintroduce signals of different wavelengths and phase. Similar to the digital LFSR RNG which feeds different bits with a polynomial relationship back into the start bit, the result is an extremely long chaotic pattern with no single frequency. I think what is proven to work in digital could be done in analogue, or partly in analogue with some digital clipping.

On that "Synth chaos circuits" that 4pyros linked to;
**broken link removed**
he has mainly circuits that accept a separate non-synced oscillator input, I guess that makes it easy to tune certain effects to make the desired sound but might also be a necessary part of the Chaos producing system.

He also has a couple examples of closed loop chaos systems like his "Cyclic System" (which seems to make some good chaos) and this simple example in his "Overview" page;
**broken link removed**

However I completely agree that having one or more open loop processes feeding energy into the closed loop system is a good idea, that too is done in digital RNG processes, for the same reasons (smashing resonant patterns etc).

I've been playing with a simple software feedback process using a single RC or dual RC (RCRC) as part of the feedback loop. For speed, it uses one ADC only (on the chain OUTPUT) and multiple taps INTO the chain, so it is like a LFSR in reverse (which works just as well according to Wiki ).

The multiple taps in are easy, each is just resistor(s) from a PIC digital output into the cap.

The problem I have at the moment is a math one, I need a very long random sequence to drive the start of the chain, whcih is easily done with a LFSR and I have an 80bit LFSR coded which takes billions of years to repeat. However an 80bit LFSR can produce strings of 1's or 0's quite long, up to 79 bits long. I want to keep the very long repeat sequence of the 80bit LFSR but make sure no string of 1's or 0's will ever be longer than 1111 or 0000. That is to ensure the average cap voltage remains near 0.5Vdd and does not clip.

Basically I need a PRNG with a very long repeat sequence but only makes very short strings. Maybe MisterT has a suggestion? My ideas so far are to "fast forward" the LFSR and just remove bits of strings but that becomes non synchronous and I would liek the process to be synchronous, ie; 1 event = 1 bit generated.

The capacitors connect to the output but then go to an unamed pin. Any ideas what pin. Pretty fundametal for a proof.