My BFF Tony Aguilar posted on our long running Dice Etiquette thread asking if I had tested the randomness of any computer random number generators.
As they say in the beer commercial Brilliant!
The one that claims to have this whole random thing really down pat, is Random.org.
Computers have a fairly poor reputation for being able to generate randomness. It’s a bit like someone trying to write a program for a computer’s animated human face; the warm smile is near impossible to pull off. I guess the warm part is what trips them up 😉
Nevertheless Random.org says they have the magic bullet when they say this from their website:
What’s this fuss about true randomness?
Perhaps you have wondered how predictable machines like computers can generate randomness. In reality, most random numbers used in computer programs are pseudo-random, which means they are a generated in a predictable fashion using a mathematical formula. This is fine for many purposes, but it may not be random in the way you expect if you’re used to dice rolls and lottery drawings.
RANDOM.ORG offers true random numbers to anyone on the Internet. The randomness comes from atmospheric noise, which for many purposes is better than the pseudo-random number algorithms typically used in computer programs. People use RANDOM.ORG for holding drawings, lotteries and sweepstakes, to drive games and gambling sites, for scientific applications and for art and music. The service has existed since 1998 and was built and is being operated by Mads Haahr of <!– who is a Lecturer in –>the School of Computer Science and Statistics at Trinity College, Dublin in Ireland.
Okie Dokie! Let’s put them to the test!
Computer programs do not face any of the concerns which us non-virtual world dice throwers are subject to. The material the die is made affects its action on the rolling surface. There is no gravity to contend with. There is absolutely no concern needed regarding the precision of the machine that made the die. There is no fear the six-sided cube is truly square. There are no pips to drill or not drill, to fill or not fill. The issue of the one pip weighing less is gone. This, and a million other dice concerns for DBA are covered at this threat at Fanaticus.org: Dice Etiquette.
As an aside the Japanese use dice with a larger one pip shape to offset any bias the die might exhibit due to the lower weight of only the one dot, such as shown here:
Since Random.org doesn’t have any of the physical limitations of actual dice I was intrigued with the idea. I duplicated my test methodology I used for the dice from my previous post. That is, I tested by having Random.org’s computer program “roll” 30 times, and then 60 times.
If you don’t remember from the previous post, a perfect average for 30 rolls would be 5 for each number and 10 for each number with 60 rolls. The ideal decimal expression of that for any number of rolls is .1667. Anything under .1667 and there were fewer than expected, and more than .1667 were more than expected. A “P” value of less than 0.050 would indicate it is out of the acceptable random generation range.
There is an automatic built-in 5% error rate with the P Value. I have no idea of the algorithm is using so no clue on what their error rate might be (plus or minus). The possible error rate, again, only applies to the P Value, not the distribution. The distribution is what it is; me simply recording how many of each type appeared. From a DBA standpoint the distribution is more important than the actual PASS/FAIL test the P Value represents.
I reloaded the website between 30 and 60 roll tests and cleared my internet cache just in case.
Here are the results with the actual breakdown of the distributions also given:
Random.org 30 Rolls Distr. 60 Rolls Distr.
1’s 5 11
2’s 4 9
3’s 7 11
4’s 2 7
5’s 5 8
6’s 7 14
30X Result 60X Result 30X P 60X
1’s – .1667 .1833 (0.608) (0.669)
2’s – .1333 .1500
3’s – .2333 .1833
4’s – .0667 .1167
5’s – .1667 .1333
6’s – .2333 .2333
So, if you stare at the numbers long enough (do not do this at home without a parent’s permission), you will find that Random.org produces…………………….Drum Roll please…………………..
A die which absolutely HATES fours consistently, and LOVES sixes consistently. The rest of the distribution looks pretty darn random. The P Value is within the accepted range for both.
Now, I made it very clear last time I posted, but I will do so again. 30 rolls is the MINIMUM you need to make any kind of observation about the variance in a die’s performance. 60 is frankly, only marginally better from a sample size standpoint. 1000 rolls would be my idea of the minimum really needed to make a really solid argument for that die’s performance. I chose 30 because it’s the minimum which almost anyone can do quickly with their own dice, and 60 simply because it is double and easy to see a pattern, if one truly exists, at that small sample size.
I am under no delusions that this tiny test on ANY of the dice is conclusive. But it is a fun starting point for conversation about how dice are not dead on .1667 that we would like to think. Statistically, after one million throws maybe the math will equalize to those numbers. I will have to be satisfied with the number 1000.
Yes, soldier, I am planning on rolling some dice 1,000 times! I will shortly perform this same test for Random.org with 120 and 240 “rolls.” I can do this relatively quickly because there is no actual die involved. I will post here when that is completed and add it as an EDIT at the bottom of this posting.
I will probably look like this after I’m done:
I am going to wait so I can order some GamesScience 6 siders and some GameStation precision backgammon dice before I begin that insanity! Those are the two dice manufacturers that the average human can purchase and feel they have done all they could to come up with “unbiased dice.” Casino dice direct from the factory are not easy to acquire but I will look into that at some point as well. Dice cups are equally half of the equation and I strongly recommend them. Precision Dice without using a Dice Cup is Peas without it’s pod.
I recommend using Dice Cups to further your efforts to properly randomized your die rolls! But if you are looking for something really unique and different, how about this Meyer Dice Tube?
Check back or subscribe to my blog if you want to see the Random.org test numbers when I’ve run 120 and 240 versions of the numbers!
MORE TESTING DONE!
I have completed more testing on Random.org’s die generator, as promised. I “rolled” their virtual dice 500 times!
I’m very happy to report that the die numbers did indeed equalize which was what a good performing randomizer should have done with a larger sample than 60 rolls. 500 still isn’t 1000 and while there is some difference between 1’s and 6’s, since the trend from the first two tests did NOT continue and the difference of both one and six is not terribly great, I believe more rolling up to 1000 would probably narrow it even further.
The one thing I did notice was a lot of “clumping”. There were many times one number was repeated more than two times in a row. The exact number was 11 times a number was repeated 3 times in a row and 1 time it was repeated 4 times in a row. Need to find a stat guru to find out if that is excessive or typical in a roll of 500 times. That was the only thing that jumped out at me. It could be pure dumb random luck or maybe how the program keeps itself closer to the .1667 median. From a DBA perspective, if you got on a string of 1’s..you’d be sunk. Although that doesn’t mean it can’t happen randomly obviously.
So, it seems clear the program will come out near the ideal .1667 or close enough to it to not care, if a large number of rolls are involved. The problem would be if the program accomplishes this either directly or as some byproduct by having clumps of numbers appear. If that is the case, the program is only accurate over the long haul. IF, and I only mean if, the clumping is a real phenomenon then that would be a poor way to determine DBA rolls. The DBA game might only involved ten die rolls, or 20 or 30 maybe in a long game. We don’t roll enough to overcome possible built in clumping!
Still, this was just for giggles. It’s a test of a computer algorithm, it says nothing about DICE…since there are no dice involved. I thought you all would enjoy the information though.
Here are the numbers
Random.org 500X P Value =0.48
1’s 97 (1.940)
2’s 79 (.1580)
3’s 78 (.1800)
4’s 80 (.1620)
5’s 72 (.1460)