A method to compute an appropriate sample size of a two-level test for the NIST test suite

Abstract

Statistical testing of pseudorandom number generators (PRNGs) is indispensable for their evaluation. A common difficulty among statistical tests is how we consider the resulting probability values (p-values). When a suspicious p-value, such as 10-3, is observed, it is unclear whether it is due to a defect of the PRNG or merely by chance. In order to avoid such a difficulty, testing the uniformity of p-values provided by a certain statistical test is widely used. This procedure is called a two-level test. The sample size at the second level requires a careful choice because too large sample leads to the erroneous rejection, but this choice is usually done through experiments. In this paper, we propose a criterion of an appropriate sample size when we use the Frequency test, the Binary Matrix Rank test and the Runs test at the first level in the NIST test suite. This criterion is based on χ2-discrepancy, which measures the differences between the expected distribution of p-values and the exact distribution of those. For example, when we use the Frequency test with the sample size 106 as the first level test, an upper bound on the sample size at the second level derived by our criterion is 125, 000.