My Dice Results
This is a follow up to my previous blog on Dedicated DBA Dice. I have now acquired just enough information about Dice Testing to be truly dangerous!
This blog is to assist you if you’ve ever wanted to find out if the dice you’re using are truly random or even more interestingly, if they tend to roll one number more than another statistically.
Since I last blogged, I sat down and worked out Pearson’s chi-square test based on the help of several websites. The following information supposes you are rolling a six-sided die 30 times, using an accepted error rate of 5% which is unavoidable with this small number of rolls.
Now I’m not math major and this whole equation thing was giving me no small trouble. I did finally deduce the terms involved. You add up the number of times each number is rolled. This is the “Distribution.” On a six-sided die, each number should come up evenly throughout no matter how often you roll total. Of course the more you roll, the finer the distinction becomes due to the larger sample size.
When you know how many of each number has been rolled in your sample size, you then subtract 5. The number that results is the “Error”. Then you square the error, which gives you the “Squared Error.” When you add up all of the squared errors, you wind up with the “Sum of the Squared Errors,” or SSE.
What? Your eyes have glazed over already? Have no fear, mine did the same! Actually to conduct your own dice roll testing this explanation isn’t sufficient. Have no fear, soldier, there is an answer!
Thankfully, you DO NOT have to worry about that equation above. It kicked my butt too!
I found a nice gent who has a blog called, Giant Battling Robots-The mathematics and statistics of games, which has a spreadsheet he created that you can download which does ALL the work for you. All you need to do is change the distribution numbers in the first green box on that spreadsheet. You can roll a minimum of 30 times, or knock yourself out and go for the holy grail of 1000 rolls. 1000 rolls isn’t any harbinger of specific accuracy but it is a nice round number that if you were to roll 1000 you could probably say with some certainly what your likely die behavior is.
So let the rolling begin!
The dice I chose to test around my house are not chosen for any other reason than they were easy to find. I did purposefully chose dice from a couple of board games since those dice were no doubt manufactured originally from a larger concern. Also these are vintage games. Later, I may buy some brand new dice and test them as well to compare to the “old school” dice I have lying around.
First the Statistical Beauty Contests should be introduced.
First of all Kudo’s to GDW….see they have themed the die they included in the game–it’s Blue! I purchased this game new in 1983 and have had it ever since. The die then therefore dates at that time.
I purchased this now classic game new in 1981 and have had it ever since. Therefore again, the dice are from that era. I chose one of the brown dice to test.
Just a loose blue die I had sitting around. It’s believe it’s about fifteen years old. This die had the most rounded corners of all of the dice.
Just a loose white die I had sitting around. It’s believe it’s about fifteen years old. This die had the most square corners of all of the dice.
Before I offer up the results, I want to point out that the original blog that set me on this path Delta’s D & D Hotspot, did a follow-up to the original article that I read where he too actually tested dice sitting around his house. As he is math guy he saw no reason to actually test his dice as his point was the math equation portion of it and the theory does not need to be tested further. He did test some dice he had anyway, just for giggles. His results are in his Feb 2009 blog post but the short of it is that, in his sample size at least, the dice with the sharpest corners were the most consistently randomly distributed in their rolls.
He took that information to mean that the stuff he had been hearing from Lou Zocchi was true in his experience as well. I believe Lou recently got out of his business GameScience, as now a company called GameStation is his website forwarding link to purchase stuff.
In short Lou espoused a product with unfinished sharp edges. No one seems to have questioned his rationale statistically until one small rolling flaw was found in a 100 sided die. He subsequently fixed that. A pretty impressive record of achievement! A real gaming pioneer!
Unfortunately I do not own any GameScience dice to add to my testing for this blog post. I will order some soon though.
I conducted two rounds of testing, one doing the math figures for 30 rolls per die contestant and the other using 60 rolls. I did not just add 30 more rolls onto the first 30, I rolled each die, a fresh 60 times. I doubled the numbers to see if any pattern developed, but any testing using this many rolls will still result in not a very refined conclusion. Nonetheless it should provide an adequate conversation starter. When I get some GameScience dice I may test it by rolling 200 times and compare to another die rolling 200 times to give a slightly better sample rate.
Again, if you want to do this testing yourself, just pull up that spreadsheet from Battle Robots. Just let me know if you can’t figure out how to make the spreadsheet work.
On the spreadsheet, for a 30 roll sample, each number would be expected to show up 5 times. This is the Expected Count on the chart. When you roll each die 60 times, each number should show up 10 times , to be evenly distributed randomly. Utilizing the spreadsheet, the expected probability will always be 0.1667 due the six-sided die we’re using. The figure in front of that is the observed probability percentage. This figure is calculated for you when you type into the green box the number of times you threw the number one on the die, the number two, the number three and so on. You roll each die and just write down the number that appears. You track this on paper and at the end, just count the frequency that each number appears. This is what you are typing into the first field. It is ALL you need to do. The program does the rest for you! Make sure you click off of the last field you enter and make sure the math adds up to 30 or 60 or however many rolls you have decided to document.
The last columns show the P value (just a statistical way of tracking) of the whole thing. For a six-sided die any number UNDER .050 would be considered not properly randomized. IMPORTANT: You cannot COMPARE P value from one die to the other! The random nature of the test precludes this from having any meaning. A P value of .65 if not better or worse necessarily than a P value of .89. Something below .05 would be a problem however, assuming the 5% error threshold has not afflicted the rolls.
What you can compare is one observed probability from one die to the other. When you do your testing, you will probably find that it does fall into the above .50 range, meaning the die is properly randomized. What you will see however, is that within the distribution, some dice will show more sixes or more ones or what have you. These individual die number results can be compared with meaning. So you can say, at least in this limited statistical example that this die should roll more sixes than another die, comparing their observed probability percentage for the number of sixes rolled by each die. In that regard it should be predictive!
That last sentence is why I went to the trouble to roll each die 60 times to see if the numbers that came up the most for any given die were repeated when you doubled the sample size. A perfectly random 6 sided die would have an expected probability percentage of .1667, each and every time. In a perfect world anyway 😉 Consistency can allow you to draw mostly positive conclusions about a die. At least you know what you’re getting with consistency, even in an uneven distribution. Mainly to draw a really solid conclusion you would simply have to roll more. The holy grail of 1000 again, springs to mind 🙂
All of the rolls were on a wooden dining table using my left hand to attempt a similar throw. The dice were not allowed to contact anything but the table from start to finish of the roll. Again the dice are shooting for a rate of .1667. A number larger than that and the die is throwing MORE of that number and lower number is throwing LESS of that number than the expected distribution of 5 each for 30 rolls and 10 each for 60 rolls.
So, Finally here are the results:
BLUE MAX DIE 30X 60X
P Value= .85 .73
Die Roll of One .1000 .1167
Die Roll of Two .1333 .1333
Die Roll of Three .2333 .2000
Die Roll of Four .1667 .2167
Die Roll of Five .2000 .1500
Die Roll of Six .1667 .1500
Circus Maximus DIE 30X 60X
P Value= .94 .55
Die Roll of One .2333 .2500
Die Roll of Two .1333 .1167
Die Roll of Three .1333 .1833
Die Roll of Four .1667 .1500
Die Roll of Five .1667 .1333
Die Roll of Six .1667 .1667
Blue Die 30X 60X
P Value= .73 .64
Die Roll of One .0677 .1833
Die Roll of Two .1667 .1000
Die Roll of Three .2000 .1667
Die Roll of Four .1000 .1667
Die Roll of Five .2333 .2333
Die Roll of Six .2333 .1500
White Die 30X 60X
P Value= .49 .82
Die Roll of One .0667 .1500
Die Roll of Two .1667 .1333
Die Roll of Three .2000 .1667
Die Roll of Four .1000 .1667
Die Roll of Five .2333 .1500
Die Roll of Six .1667 .1500
This is a long way from being my field, but based on these number, I would say it shows that if I wanted the chance to roll the greatest number of five’s and sixes, I should choose the Loose Blue Die and the Loose White Die. They are exactly tied for that. Both of these dice threw a low number of “one’s” intially but even when they threw out more one’s ….. it was at the expense of the 2’s, 3’s, and 4’s….not the 5’s and 6’s which stayed exactly the same!
The Circus Maximus Die wins the ignominy of rolling the most 1’s consistently. It is relatively average otherwise. No army with elephants, Artillery or War Wagons should pick up a Circus Maximus Brown Die lol.
The Blue Max Die has a fat middle bell curve, with low 1’s and 2’s relatively. A good hedge your bet kind of die for DBA 😉
It should be noted that all of my dice are fairly standard decent looking manufacturer. None of them would be considered promotional dice, or themed dice. I am now highly suspect of any dice with custom graphics on it. If sharp edge dice do offer the closest .1667 consistency as I suspect, than all the fun I was going to have coming up with themed dice would likely come at the expense of a fair and equitable game!
Do I think tournaments should all start using GameScience dice only? Probably. Good luck implementing that! 😉
Will I still look into using my Phokia Me and Phokia You dice for friendly non-competition games? You bet!