Properties of Sparse Random Matrices over Finite
Fields
Roberto C. Alamino and David Saad
Neural Computing Research Group, Aston University, Birmingham B4 7ET, UK
PACS numbers: 02.10.Yn, 02.70.-c, 05.10.-a
Abstract. Typical properties of sparse random matrices over finite (Galois) fields are
studied, in the limit of large matrices, using techniques from the physics of disordered
systems. We present results for the average kernel dimension, the number of matrices
for a given distribution of entries, and average dimension of the eigenvector spaces and
the distribution of the eigenvalues. The significance of these results to error-correcting
codes and random graphs is also discussed.

Properties of Sparse Random Matrices over Finite Fields 2
1. Introduction
Random matrices are ubiquitous in many branches of the natural sciences and
mathematics ranging from biology to computer science, nuclear physics and quantum
chaos. Activity in the area has been recently boosted by the application of techniques
originated in the statistical mechanics of disordered systems [1, 2, 3, 4, 5, 6]. These
techniques have been used to analyse ensemble properties, including variational
techniques and the replica method, have proved to be valuable tools in several of these
approaches. Most of the research concentrates on real matrices, while the restriction to
matrices over GF (q) considered here makes the solution of the problem more involved.
Random matrices over GF (q) are specially important in coding theory [7] where, for
instance, linear codes are defined by the kernel of the so-called parity-check matrix:
each kernel vector defines a codeword to which the original message vector is mapped
by a linear operation, in the form of a product with a generator matrix. Well known
examples include the Hadamard codes, where properties of the kernel and rank play
an important role [8], and low-density parity-check codes (LDPC), which provide the
best performance to date in many noise regimes; parity-check matrices are constructed
using a special case of the sparse matrices studied here. Although the most studied
and applied case of LDPC codes is of binary codes over GF (2) there is a significant
body of work, of both practical and theoretical nature [9], on codes over more general
finite fields showing an improvement in performance with respect to the binary version.
A statistical physics based analysis of LDPC codes over GF (q) has been reported, for
instance, in [10].
In addition to being an interesting applied problem, properties of these matrices
are of great interest from the pure mathematical point of view and a number of papers
have already addressed related questions, in different instances, with a mathematical
rigorous approach [11, 12, 13].
Here, we analyse key typical properties of sparse random matrices over GF (q),
namely the average dimension of their kernel, the average dimension of the eigenvector
spaces, the eigenvalue distributions and the number of matrices for a given connectivity
distribution, all in the case of large matrices. When the M × N matrices are large,
keeping N → ∞ with η = M/N constant, the problem can be mapped into a
system of interacting “spins” by a very useful group homomorphism, and the powerful
methodology developed for the study of disordered spin lattices in condensed matter
physics can then be used, under some assumptions, to obtain the required properties.
In section 2, we generalise a usual statistical physics mapping of systems over the
binary field GF (2) to spin system by means of a group homeomorphism in such a way
that it can be efficiently applied to any GF (q) allowing for the calculation in sections 3
and 5 of kernel and eigenspace properties, respectively, to be carried out. Making use of
these techniques, the number of matrices for a given distribution of non-zero elements is
then obtained for various connectivity profiles, in section 6. In section 4 we discuss the
importance of the results to LDPC error-correcting codes and in section 7 to general

Properties of Sparse Random Matrices over Finite Fields 3
random graphs. We present a final discussion of the results in section 8.
2. Generalised Mapping of GF (q) onto Spin Systems
A Galois field GF (q) is a finite field with q elements, i.e., a set of q elements
{0, ..., q − 1}, which we symbolise by integers for convenience, which is a commutative
group under addition ⊕ : GF (q) → GF (q), defined as integer addition mod q,
and with a monoid structure with respect to a commutative multiplication operation
⊗ : GF (q) → GF (q). The field also includes the zero element ’0’ and the multiplicative
identity ’1’. The additional requirement that both multiplication and addition have the
algebraic distributive property restricts the number of elements to q = pn, where p is a
prime number and n an integer.
In statistical physics applications to information theory, it is usually convenient to
map binary vectors with entries from the set {0, 1}, into what we call “spin” vectors
with entries from the set {±1}. Given an entry v ∈ {0, 1}, this mapping is usually
accomplished either by the transformation σ(v) = 2v−1 or by σ(v) = (−1)v. Note that
these two transformations are different in the sense that in the former, 0 is mapped to
-1 and 1 is mapped to 1, while in the latter the order is reversed. The second mapping
is much more convenient as it represents a homeomorphism between representations
of GF (2). A further advantage, which we show below, is that this map can also
be generalised to any GF (q) allowing the straightforward use of statistical physics
techniques to a wide range of problems over Galois fields.
The main fact to be noted is that, under the operation of addition ⊕, GF (q) is
homeomorphic to the cyclic group of order q and therefore has a representation as the
complex q-th roots of unity with the group homeomorphism σ : GF (q) → C given by
σ(v) = exp
(2πi
q v
)
, (1)
such that for every v1, v2 ∈ GF (q)
σ(v1 ⊕ v2) = exp
[2πi
q (v1 ⊕ v2)
]
= exp
[2πi
q (v1 + v2)
]
= σ(v1)σ(v2).
(2)
This mapping has a clear geometric interpretation where 2πv/q is an angle in the
unit circle, such that each element of the Galois field is being mapped onto a spin variable
“pointing” in one of q possible directions. Let us denote the inverse of an element v
under addition by −v such that v ⊕ (−v) = 0. Then we also have
σ(v ⊕ (−v)) = σ(0) = 1 ⇒ σ(−v) = [σ(v)]−1, (3)
σ(v ⊕ (−u)) = σ(v)σ(−u) = σ(v)[σ(u)]−1. (4)

Properties of Sparse Random Matrices over Finite Fields 4
In several applications, including the ones studied in this paper, it turns out to be
necessary to use constraints in the form of Kroenecker deltas like
δ(v, u) = δ(v ⊕ (−u), 0). (5)
Using the above mapping, one can write any constraint of this type in a generalised
way as
δ(v, u) = 1q
q−1
∏
m=1
[
1− exp
(
−2πiq m
)
exp
(2πi
q v
)
exp
(
−2πiq u
)]
. (6)
Also based on this representation, we can now define the k-point functions of the
spins by
mk ≡
1
N
N
∑
j=1
σj1 · · ·σjk, (7)
where the lower indices represent k different spin configurations σ1, ...,σk. Note that we
are now working with the spin variables already mapped onto the complex field C and
therefore the operations of multiplication and addition correspond to the usual ones on
C. The first two k-point functions, namely m1 and m2, correspond to the magnetisation
of the spin system and its overlap between two configurations, respectively.
As will become evident in what follows, this kind of representation allows a
factorisation of terms that simplifies the equations and makes replica calculations
simpler.
3. Kernel and Rank
Entries in matrices over GF (q) take values of elements of that finite field, where the usual
additions and multiplications involved in their algebra are defined by the corresponding
operations over the Galois field.
The kernel of an M × N matrix A over GF (q) is a linear space with dimension
dA(0) = logq ΩA(0) where
ΩA(0) =
∑
v
δ(Av, 0), (8)
is the number of vectors in the kernel, δ is the Kroenecker delta and v ∈ GF (q)N .
We term Ts(0) the average kernel dimension density, in the limit of large matrices,
where we also define the entropy density
s(0) ≡ 1T limN→∞
〈dA(0)〉A
N = limN→∞
1
N 〈lnΩA(0)〉A, (9)
with 1/T = ln q and with the ratio η = M/N being a finite positive constant and 〈·〉A
denotes an average over the corresponding ensemble of random matrices. Using the
replica trick we can write the entropy density as
s(0) = lim
N→∞
[ ∂
∂n ln 〈Ω
n
A(0)〉A
]
n=0
. (10)

Properties of Sparse Random Matrices over Finite Fields 5
We focus our analysis on the class of randomly chosen sparse matrices A having
exactly Ki non-zero elements in the i-th row with probability P(K), K ≡ (K1, ..., KM ),
and Cj elements in the j-th column with probability P(C), C ≡ (C1, ..., CN), obeying
the constraint Λ ≡ ∑i Ki =
∑
j Cj, where Λ is the total number of non-zero elements
of the matrix. We also consider that the elements of A are sampled from the finite field
GF (q) with independent equal probabilities P(Aij).
Using the mapping of GF (q) into spin variables defined in the previous section, the
replicated averaged partition function Zn(0) ≡ 〈ΩnA(0)〉A can be written as
Zn(0) =
〈
1
N
∑
{Aij}
[
∏
i,j
P(Aij)
][ M
∏
i=1
δ
( N
∑
j=1
χ(Aij), Ki
)][ N
∏
j=1
δ
( M
∑
i=1
χ(Aij), Cj
)]
×
n
∏
a=1
[
∑
va
δ(Ava, 0)
]〉
K,C,Λ
,
(11)
where the average is over the probability distribution P(K,C,Λ) with χ(Aij) = 0
if Aij = 0 and 1 otherwise, and the normalisation N gives the number of matrices
which obey the constraints averaged over the distributions of the entries. The replica
calculations to obtain the entropy (9) have already been carried out in a previous
paper [14] and we will only discuss the important results. One important difference
of this approach in comparison to previous similar calculations [15, 16] is that, instead
of defining connectivity tensors and summing over them, here and in [14] we sum directly
over the matrices entries, which makes possible to include general constraints over the
class of random matrices.
Using the rank-nullity theorem, we can define the free energy density f(0) as the
average rank density in the form
f(0) ≡ 〈r(A)〉AN = 1− Ts(0), (12)
which implies that the internal energy density of the associated statistical mechanical
model should be constrained to be u = 1. Defining the inverse temperature β = 1/T ,
equation (12) can be written as
βf(0) = − 1N
〈
ln
∑
v
e−βH(v)
〉
A
, (13)
where a formal Hamiltonian can be written as
H(v) ≡ N − ln δ(Av, 0). (14)
The solution given by the replica calculations [14] has the striking property of
being completely independent of the specific distribution of the individual elements of
the matrix P(Aij), depending only on the distribution of K and C (and, obviously, that

Properties of Sparse Random Matrices over Finite Fields 6
0.0 0.5 1.0
λ
0.0
0.5
1.0
Kernel Dimension
Rank
Figure 1. Average kernel dimension density (continuous lines) and average rank
density (dashed lines) calculated as solutions to the replica symmetric saddle point
equations. The plot shows the thermodynamically favoured analytical solution which
is paramagnetic for 0 ≤ η ≤ 1 and ferromagnetic for η > 1.
of Λ). Analytical solutions cannot be obtained in general and we must rely on numerical
methods to obtain them. However, there exists two straightforward analytical solutions,
the paramagnetic and ferromagnetic states. The paramagnetic solution gives the average
kernel density as Ts(0) = 1 − η = 1 −M/N independently of the order q of the finite
field used, and the ferromagnetic solution gives simply Ts(0) = 0. Figure 1 shows
a plot of the thermodynamically dominant solutions for varying η. We see that the
paramagnetic solution dominates until η = 1, after which it becomes unphysical and
the correct solution is given by the ferromagnetic one.
In [14], numerical solutions for three different distributions of K, C and Λ were
thoroughly studied and, apart from small variations resulting from particular properties
of the distributions, the obtained curves seem to reproduce figure 1.
4. Error Correcting Codes
Low-density parity-check codes are linear codes where the codewords are defined by the
kernel of a random sparse matrix, called the parity-check matrix. In most studies of
LDPC codes, it is assumed that a parity-check matrix with M rows (parity-checks) and
N columns exactly defines a code of rate R = 1−M/N = 1− η, which is equivalent to
the assertion that the number of codewords (vector in the kernel) is exactly qNR.
The results of the previous section give us a method to rigorously test this assertion.
Note that when M ≥ N , the code rate, in the case of unbiased messages, would be
negative, which is unphysical. However, our calculations give a clear interpretation for
this case as they show that if η > 1, the dominant solution is the ferromagnetic one with
Ts = 0, implying that the matrix is full rank. As the kernel would be only given by

Properties of Sparse Random Matrices over Finite Fields 7
the zero message, this incidentally means that such matrices cannot be used to define a
parity-check code due to the lack of redundancy.
However, when η < 1, the dominant solution is paramagnetic and the typical
parity-check matrix defines a code of rate exactly (N −M)/N . This is assumed for any
parity-check matrix in most calculations in the literature and is confirmed by our results
to be true on average; however, it is important to point out that the result is true in
the limit of large matrices and is likely to have finite size corrections which may affect
practical applications.
The application to error correcting codes also gives support to our conjecture that
the dominant solution for η < 1 is the paramagnetic one and for η ≥ 1 it is the
ferromagnetic solution for any distribution. The argument is that the solution of the
kernel dimension is mathematically equivalent to the solution of LDPC in channels with
infinite noise, which then leads to this behaviour. As already mentioned, the numerical
results of [14] seem to support this conjecture, although more careful calculations,
varying all the parameters involved must be carried out to confirm this hypothesis
more generally.
5. Eigenspaces and Eigenvalues
In the case of square M × M matrices, which means that η = 1, we can generalise
equation (8) to count the number of vectors in an eigenspace with eigenvalue λ ∈
{1, ..., q − 1} as
ΩA(λ) =
∑
v
δ(Av, λv), (15)
with obvious analogous generalisations for the kernel dimension density dA(λ), its
average density Ts(λ) and all other quantities from section 3 that depend on λ, the
free energy now being the dimension of the complementary space to the λ-eigenspace.
The replica calculations are very similar to the particular case of the kernel [14].
However, as there are key differences and in order to keep this paper as self-contained as
possible, we reproduce the replica calculations including the appropriate changes to the
eigenspace case in Appendix A. The replica symmetric solution, exact for this problem
as shown in Appendix B, is therefore (for λ 6= 0)
s(λ) = − ln q − αM 〈ln [1 + (q − 1)xxˆ]〉x,xˆ
+ 1Mǫ(α)
∑
i
〈
αΛ
Λ!
〈
ln
{[
1 + (q − 1)
Ki
∏
l=1
xl
] Ci
∏
l=1
[1 + (q − 1)xˆl]
+ (q − 1)
(
1−
Ki
∏
l=1
xl
) Cj
∏
l=1
(1− xˆl)



〉
x,xˆ
〉
K,C,Λ
,
(16)
where now the distributions of the auxiliary fields x and xˆ are slightly more complicated

Properties of Sparse Random Matrices over Finite Fields 8
than in the kernel case and are given by the saddle point equations
πˆ(xˆ) = 1αǫ(α)
M
∑
i=1
〈αΛ
Λ!Ki
× δ
(
xˆ−
[Ki−1
∏
l=1
xl
]
Xˆ+(Ci)− Xˆ−(Ci)
Xˆ+(Ci) + (q − 1)Xˆ−(Ci)
)〉
xˆ,K,C,Λ
, (17)
π(x) = 1αǫ(α)
M
∑
i=1
〈αΛ
Λ!Ci
× δ
(
x− X
+(Ki)Xˆ+(Ci − 1)−X−(Ki)Xˆ−(Ci − 1)
X+(Ki)Xˆ+(Ci − 1) + (q − 1)X−(Ki)Xˆ−(Ci − 1)
)〉
xˆ,K,C,Λ
, (18)
0 =
〈αΛ
Λ!
(
1− Λα
)〉
K,C,Λ
, (19)
with the following definitions for simplicity
ǫ(α) ≡
〈αΛ
Λ!
〉
K,C,Λ
, (20)
Xˆ+(C) ≡
C
∏
l=1
[1 + (q − 1)xˆl], Xˆ−(C) ≡
C
∏
l=1
(1− xˆl), (21)
X+(K) ≡ 1 + (q − 1)
K
∏
l=1
xl, X−(K) ≡ 1−
K
∏
l=1
xl. (22)
The paramagnetic solution
πˆ(xˆ) = δ(xˆ), π(x) = δ(x), (23)
and the ferromagnetic solution
πˆ(xˆ) = δ(xˆ− 1), π(x) = δ(x− 1). (24)
in terms of the auxiliary fields are the same as for the kernel. Substituting these solutions
into equation (16), we see that the paramagnetic solution gives the result s(λ) = 0 for
every value of λ, which must be discarded as the kernel calculation shows that s(0) = 0
in the square matrix case. The matrix is therefore diagonalisable and the dimension
of the eigenspaces cannot be zero. On the other hand the ferromagnetic solution gives
s(0) = 0 and s(λ) = Λ/M for λ 6= 0, which means that all the non-zero eigenvalues have
eigenspaces with the same dimension given by the average value of non-zero entries per
row or per column (both must be equal). As Λ scales with M ; it gives a finite average
dimension density as expected. This result also implies that the eigenvalue distribution
is just given by
P(λ) = (q − 1)−1, (25)
for λ 6= 0, i.e., all eigenvalues have the same probability.

Properties of Sparse Random Matrices over Finite Fields 9
6. Number of Matrices
The number ofGF (q) matrices given a connectivity profile is of significant interest within
the discrete mathematics community. Exact results have been obtained for the case of
finite binary matrices [17] in the form of a formula that facilitates the calculation of their
precise number. Using the same techniques from the previous sections, this number and
its average value can be obtained for large GF (q) matrices. For a given number of non-
zero elements per row K = (K1, ..., KM) and per column C = (C1, ..., CN), the number
of matrices is
NA =
∑
{Aij}
[ M
∏
i=1
δ
( N
∑
j=1
χ(Aij), Ki
)][ N
∏
j=1
δ
( M
∑
i=1
χ(Aij), Cj
)]
, (26)
where again we are summing directly over the entries of the matrix instead of over a
connectivity tensor. The detailed calculations where presented in [14] resulting in
NA = (q − 1)Λ
Λ!
∏
i Ki!
∏
j Cj!
, (27)
which is just the number of binary matrices with the given non-zero elements profile
times a factor (q − 1)Λ which is the multiplicity of the non-zero entries.
The average number of matrices is then
N¯A =
〈
(q − 1)Λ Λ!∏
i Ki!
∏
j Cj!
〉
K,C,Λ
=
∑
K
∑
C
P(K|C)P(C)(q − 1)
P
j Cj
(
∑
j Cj
)
!
∏
i Ki!
∏
j Cj!
,
(28)
where the distribution P(K|C) includes the constraint δ
(
∑
i Ki,
∑
j Cj
)
. For a regular
matrix, it is easy to see that this number scales as NCN and a more appropriate quantity
would be the quenched entropy
Ξ ≡
〈 1
N lnNA
〉
= 1N
∑
K
∑
C
P(K|C)P(C) ln

(q − 1)
P
j Cj
(
∑
j Cj
)
!
∏
i Ki!
∏
j Cj!

, (29)
scaling as lnN .
A detailed numerical analysis of these formulas for different distributions is also
given in [14]. These studies indicate that if we keep the number of columns constant
and increase the ratio η by adding rows, whenever the number of rows is much larger
than the number of columns, the average number of matrices becomes independent of
both the ratio and number of rows and also suggest that the average number of matrices
in these cases is basically defined by the average value of the C distributions.

Properties of Sparse Random Matrices over Finite Fields 10
Figure 2. Examples of adjacency matrices. From top to bottom, undirected graphs,
directed graphs and directed weighted graphs.
7. Random Graphs
The results of the previous section can be used to obtain the number and average number
of generalised sparse random graphs. For usual random graphs, directed or undirected,
the adjacency matrix can be written as a binary matrix with a 1 in entry Aij if there
is a link from vertex i to vertex j. If in addition each link is given an integer weight,
the matrix can be viewed as a GF (q) matrix for the sake of counting the graphs. In
figure 2, we give an example of a directed, an undirected and an undirected weighted
graph.
Graphs like the ones in figure 2 can be used in statistical physics to describe systems
with 2-body interactions, like an Ising model, for instance. However, there exists systems
with many-body interactions and we would like to be able to describe them by graphs
and matrices as well. With this in mind, we can generalise random graphs in the
following way. Instead of only links that connect two vertices, representing an interaction
between them, we allow for the existence of polygons connecting p ≥ 3 vertices with a
p-body interaction. This graph construction can then be codified by a binary matrix in
which each line represent a group of interacting vertices (spins, for instance) and each

Properties of Sparse Random Matrices over Finite Fields 11
Figure 3. Example of a graph with p-spin interactions and its matrix representation.
row carries the label of the vertices. This is the type of representation frequently used in
most studies of codes [18]. An example of such a graph is given in figure 3. In fact this
is a matrix representation of the tensor whose indices identify the interacting variables.
In this way, we can represent any interacting system as a random GF (q) matrix,
which allows us to use the results of the former section to calculate their number or
their average number according to our needs.
8. Conclusions
The main object of this work was to study some typical properties of sparse random
matrices over Galois fields using techniques originated in the statistical physics of
disordered systems.
In order to carry on our analysis, we introduced a mapping from Galois matrices
to spin systems based on the group homeomorphism between GF (q) under addition
mod q (denoted by ⊕) and the complex q-th roots of unity. This mapping allows
for a direct analogy between the matrices and statistical mechanical systems in the
canonical ensemble, making it possible to associate properties like rank and nullity with
thermodynamical characteristic functions like the free-energy and entropy.
The key point, which is the calculation of the quenched entropy, was carried out in
a set up that allows for further generalisations of the constraints enforced in defining the
classes of matrices over which the averaging is done by characterising their connectivity
profiles. This quenched average is carried out using the replica approach and we are able
to obtain, as a result, the average dimension of the kernel for a general distribution of
non-zero entries. Solving the resulting equations numerically, we find that the average
kernel density is 1−M/N in all cases studied. We conjecture that this result is always
valid. The replica symmetric ansatz was assumed in the calculation and later on proved
to be exact in this case with the help of the mapping between matrices and spin systems;
this sheds light on the meaning of properties like magnetisation and spin correlation.
The above results have practical relevance in a number of areas, including coding
and network modelling. With respect to LDPC codes, the average kernel density result

Properties of Sparse Random Matrices over Finite Fields 12
implies that randomly generated LDPC codes typically define codes of rate exactly
1−M/N , an assumption which is generally made but lacks rigorous derivation.
A generalisation of the result for the kernel and rank allowed us to obtain the
dimension of each eigenspace and its corresponding eigenvalue distribution.
By using the same mathematical techniques, we were also able to find the total
number of large matrices for fixed K and C, row and column connectivities, respectively,
and their average number. We showed then how the matrices could be used to represent
not only usual graphs, but also the connectivity profile of graphs representing p-
spin interactions. The results obtained for the average number of matrices provide
therefore a principled approach to determine the average number of possible graphs
with given connectivity distributions of a more general nature than the connectivity
profiles examined in this paper.
Extensions and generalisations of the presented framework are under study. A most
desired generalisation would be to continuous matrices. The first step in this direction
is to study matrices with entries in continuous groups instead of the elements of a finite
group, more specifically, U(1) which is a straightforward continuous limit of the cyclic
groups.
Acknowledgements
Support from EPSRC grant EP/E049516/1 is gratefully acknowledged. R.C.A. would
also like to thank Juan P. Neirotti and Jack Raymond for useful discussions.
References
[1] Mezard, M., Parisi, G., and Zee, A. Nucl. Phys. B 559, 689–701 (1999).
[2] Kanzieper, E. Nucl. Phys. B 596, 548–566 (2001).
[3] Nagao, T. and Tanaka, T. J. Phys. A 40, 4973–4987 (2007).
[4] Biroli, G., Bouchaud, J.-P., and Potters, M. Europhys. Lett. 78, 10001 (2007).
[5] Biroli, G., Bouchaud, J.-P., and Potters, M. J of Stat. Mech. , P07019 (2007).
[6] Bohigas, O., de Carvalho, J. X., and Pato, M. P. Phys. Rev. E 77, 011122 (2008).
[7] McEliece, R. Theory of Information & Coding. Cambridge University Press, Cambridge, MA,
(2002 2nd edition).
[8] Phelps, K. T., Rifa`, J., and Villanueva, M. IEEE Trans. Inf. Theory 51, 3931–3937 (2005).
[9] Davey, M. and MacKay, D. IEEE Communications Letters 2, 165 – 167 (1998).
[10] Nakamura, K., Kabashima, Y., and Saad, D. Eurphys. Lett. 56, 610–616 (2001).
[11] Cooper, C. Random Structures and Algorithms 16, 209–232 (2000).
[12] Blo¨mer, J., Karp, R., and Weiz, E. Random Structures and Algorithms 10, 407–419 (1998).
[13] Feng, X. and Zhang, Z. App. Math. and Comp. 185, 689–694 (2007).
[14] Alamino, R. C. and Saad, D. Phys. Rev. E 77, 061123 (2008).
[15] Alamino, R. C. and Saad, D. J. Phys A: Math. Theor. 40, 12259–1279 (2007).
[16] Yano, T., T[]Tanaka, and Saad, D. J Phys A 41(32), 324022 (15pp).
[17] Wang, B.-Y. and Zhang, F. Discrete Mathematics 187, 211–220 (1998).
[18] Gallager, R. Low-Density Parity-Check Codes, Research monograph series. Number 21. MIT
Press, Cambridge,MA, (1963).

Properties of Sparse Random Matrices over Finite Fields 13
[19] Nishimori, H. Statistical Physics of Spin Glasses and Information Processing. Oxford University
Press, Oxford, UK, (2001).
Appendix A. Replica Calculations
For the case square matrices (η = 1) and λ 6= 0, we can use the integral representation
of the Kroenecker delta function in the complex plane such that the averaged replicated
kernel size defined in equation (11) is generalised to the size the λ-eigenspace as
Zn(λ) =
〈
1
N
∑
{va}
∮
DWDZ
∑
{Aij}
[
∏
i,j
P(Aij)(WiZj)χ(Aij)
]
×
M
∏
i=1
∏
a
δ
[ M
⊕
j=1
(
Aij ⊗ vja
)
, λ⊗ via
]〉
K,C,Λ
,
(A.1)
where ⊗ and ⊕ indicate respectively multiplication and summation on GF (q) and
DWDZ =
[ M
∏
i=1
dWi
WKi+1i
][ M
∏
j=1
dZj
ZCj+1j
]
. (A.2)
Using the representation of the Kroenecker delta given in equation (6), the product
over replica indices of the delta function can be written as
∏
a
δ
[ M
⊕
j=1
(
Aij ⊗ vja
)
, λ⊗ via
]
=
∏
a
1
q
q−1
∏
m=1
{
1− exp
(
−2πiq m
)
exp
[
−2πiq
(
λ⊗ via
)
] M
∏
j=1
exp
[2πi
q
(
Aij ⊗ vja
)
]
}
= 1qn
∏
a
[
1 +
q−1
∑
s=1
Fi(s, a)G(s)
]
= 1qn
n
∑
r=0
∑
〈a1···ar〉
∑
s1,...,sr
G(s1) · · ·G(sr)Fi(s1, a1) · · ·Fi(sr, ar),
(A.3)
with
G(s) ≡
∑
〈m1···ms〉
(−1)s exp
(
−2πiq m1
)
· · · exp
(
−2πiq ms
)
, (A.4)
and
Fi(s, a) = exp
[
−2πiq s
(
λ⊗ via
)
] M
∏
j=1
γj(s, a, Aij), (A.5)
where we defined, for simplicity,
γj(s, a, Aij) ≡ exp
[2πi
q s
(
Aij ⊗ vja
)
]
. (A.6)

Properties of Sparse Random Matrices over Finite Fields 14
The partition function becomes
Zn(λ) =
〈
1
N
∑
{va}
∮
DZ
M
∏
i=1
1
qn
n
∑
r=0
∑
〈a1···ar〉
∑
s1,...,sr
G(s1) · · ·G(sr)
∮ dWi
2πi
1
WKi+1i
Γi
〉
K,C,Λ
,
(A.7)
where
Γi = exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
]
×
∏
j
∑
Aij
P(Aij)(WiZj)χ(Aij)γj(s1, a1, Aij) · · · γj(sr, ar, Aij)
= pM exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
]
×
∏
j
[
1 + 1p
q−1
∑
h=1
P(Aij = h)WiZjγj(s1, a1, h) · · · γj(sr, ar, h)
]
,
(A.8)
where we define, for convenience, p ≡ P(Aij = 0). Let us define a probability
distribution over the values of h as
P(h) = P(Aij = h)1− p , (A.9)
in such a way that h varies from 1 to q−1 and the probability over this range is correctly
normalised. Then
Γi = pM exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
]
×
∏
j
[
1 +
(1− p
p
)
WiZj〈γj(s1, a1, h) · · · γj(sr, ar, h)〉h
]
= pM exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
] N
∑
l=0
∑
〈j1···jl〉
(1− p
p
)l
×W liZj1 · · ·Zjl〈γj1(s1, a1, h) · · · γj1(sr, ar, h)〉h · · · 〈γjl(s1, a1, h) · · · γjl(sr, ar, h)〉h.
(A.10)
The integrals over the Wi’s, acting on the Γi’s, select the power of Wi to be Ki and

Properties of Sparse Random Matrices over Finite Fields 15
we therefore obtain
Zn =
〈
κ
∑
{va}
∮
DZ
M
∏
i=1



n
∑
r=0
∑
〈a1···ar〉
∑
s1,...,sr
G(s1) · · ·G(sr)
× exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
]
∑
〈j1···jKi〉
Zj1 · · ·ZjKi
× 〈γj1(s1, a1, h) · · · γj1(sr, ar, h)〉h · · ·
〈
γjKi (s1, a1, h) · · · γjKi (sr, ar, h)
〉
h
}〉
K,C,Λ
≈
〈
κ
∑
{va}
∮
DZ
M
∏
i=1



n
∑
r=0
∑
〈a1···ar〉
∑
s1,...,sr
G(s1) · · ·G(sr)
× exp
[
−2πiq s1
(
λ⊗ via1
)
]
· · · exp
[
−2πiq sr
(
λ⊗ viar
)
]
× N
Ki
Ki!
[
1
N
N
∑
j=1
Zj〈γj(s1, a1, h) · · · γj(sr, ar, h)〉h
]Ki



〉
K,C,Λ
(A.11)
where
κ = pM2
(1− p
p
)
P
i Ki
N−1q−nM . (A.12)
The calculation of N is similar to the calculation of the number of matrices [14]
and we end up with
κ = 1
qnMN (2)A
, (A.13)
where N (2)A is exactly the number of binary matrices (q = 2). Introducing the replica
overlaps
Qs1,...,sr〈a1···ar〉 ≡
1
M
M
∑
j=1
Zj〈γj(s1, a1, h) · · · γj(sr, ar, h)〉h, (A.14)
and the corresponding auxiliary variables Qˆs1,...,sr〈a1···ar〉 by means of Dirac delta functions,
and noting that we are now working with square matrices such that the indices i and j

Properties of Sparse Random Matrices over Finite Fields 16
run through the same set, we can express the partition function as
Zn =
∫
DQDQˆ exp
(
−M
∑
Qs1,...,sr〈a1···ar〉Qˆ
s1,...,sr
〈a1···ar〉
)
×
〈
κM
P
i Ki
∏
i Ki!
∏
i
∑
{va}
{
∑
G(s1) · · ·G(sr)
(
Qs1,...,sr〈a1···ar〉
)Ki
× exp
[
−2πiq s1(λ⊗ va1)
]
· · · exp
[
−2πiq sr(λ⊗ var)
]}
×
∮
DZi exp
[
Zi
∑
Qˆs1,...,sr〈a1···ar〉〈γi(s1, a1, h) · · · γi(sr, ar, h)〉h
]
〉
K,C,Λ
=
∫
DQDQˆ exp
(
−M
∑
Qs1,...,sr〈a1···ar〉Qˆ
s1,...,sr
〈a1···ar〉
)
×
〈
q−nM M
P
i Ki
(∑i Ki)!
∏
i
∑
{va}
{
∑
G(s1) · · ·G(sr)
(
Qs1,...,sr〈a1···ar〉
)Ki
× exp
[
−2πiq s1(λ⊗ va1)
]
· · · exp
[
−2πiq sr(λ⊗ var)
]}
×
[
∑
Qˆs1,...,sr〈a1···ar〉〈γi(s1, a1, h) · · · γi(sr, ar, h)〉h
]Ci
〉
K,C,Λ
(A.15)
where
DQDQˆ ≡
(
∏ dQdQˆ
2πi/M
)
, (A.16)
and the summations run over all the allowed values of r, 〈a1 · · · ar〉 and s1, . . . sr.
Under the assumption of replica symmetry in the form
Qs1,...,sr〈a1···ar〉 = Q0〈x
r〉x, (A.17)
Qˆs1,...,sr〈a1···ar〉 = Qˆ0〈xˆ
r〉xˆ, (A.18)
where the averages over x and xˆ are taken with respect to the field distributions π(x)
and πˆ(xˆ) respectively, we can show by straightforward algebraic manipulations that
∑
Qs1,...,sr〈a1···ar〉Qˆ
s1,...,sr
〈a1···ar〉 = Q0Qˆ0〈[1 + (q − 1)xxˆ]
n〉x,xˆ, (A.19)
∑
G(s1) · · ·G(sr)
(
Qs1,...,sr〈a1···ar〉
)Ki
exp
[
−2πiq s1(λ⊗ va1)
]
· · · exp
[
−2πiq sr(λ⊗ var)
]
=
QKi0
〈 n
∏
a=1
[
1 + ω˜(va)
Ki
∏
l=1
xl
]〉
x
,
(A.20)
with
ω˜(v) ≡
q−1
∑
s=1
G(s) exp
[
−i2πsq (λ⊗ v)
]
, (A.21)

Properties of Sparse Random Matrices over Finite Fields 17
and
[
∑
Qˆs1,...,sr〈a1···ar〉〈γi(s1, a1, h) · · · γi(sr, ar, h)〉h
]Ci
=
QˆCi0
〈 n
∏
a=1
Ci
∏
l=1
[1 + ω(va, hl)xˆl]
〉
xˆ,h
,
(A.22)
with
ω(v, hl) ≡
q−1
∑
s=1
exp
[
i2πsq (hl ⊗ v)
]
=
{
q − 1, ifhl ⊗ v = 0,
−1, otherwise. (A.23)
Therefore, we have, for the sum over {va},
∑
{va}
{
QKi0
〈 n
∏
a=1
[
1 + ω˜(va)
Ki
∏
l=1
xl
]〉
x
}



QˆCi0
〈 n
∏
a=1
Ci
∏
l=1
[1 + ω(va, hl)xˆl]
〉
xˆ,h



= QKi0 QˆCi0
〈{
∑
v
[
1 + ω˜(v)
Ki
∏
l=1
xl
] Ci
∏
l=1
[1 + ω(v, hl)xˆl]
}n〉
x,xˆ,h
= QKi0 QˆCi0
〈{[
1 + (q − 1)
Ki
∏
l=1
xl
] Ci
∏
l=1
[1 + (q − 1)xˆl]
+
{
(q − 1) +
[ q−1
∑
v=1
ω˜(v)
] Ki
∏
l=1
xl
}[ Ci
∏
l=1
(1− xˆl)
]}n〉
x,xˆ
,
(A.24)
where with some amount of algebraic calculations to show that
q−1
∑
v=1
ω˜(v) = −(q − 1), (A.25)
when λ 6= 0, we can write
Zn =
∫
DQDQˆ eMs˜, (A.26)
with
s˜ = − 1M lnN
(2)
A − n ln q −Q0Qˆ0〈[1 + (q − 1)xxˆ]n〉x,xˆ +
1
M lnΦ, (A.27)
where
Φ =
〈
MΛ
Λ! Q
Λ
0 QˆΛ0
∏
i
〈{[
1 + (q − 1)
Ki
∏
l=1
xl
] Ci
∏
l=1
[1 + (q − 1)xˆl]
+ (q − 1)
(
1−
Ki
∏
l=1
xl
) Cj
∏
l=1
(1− xˆl)



n
〉
x,xˆ
〉
K,C,Λ
(A.28)

Properties of Sparse Random Matrices over Finite Fields 18
Let us define α ≡ MQ0Qˆ0. For n ≪ 1, we can consider only the leading
contributions in the number of replicas, which gives
lnΦ = ln ǫ(α) + nǫ(α)
∑
i
〈αΛ
Λ! 〈ln {· · · }〉x,xˆ
〉
K,C,Λ
, (A.29)
with
ǫ(α) =
〈αΛ
Λ!
〉
K,C,Λ
, (A.30)
and where {· · · } stands for the expression inside curly brackets in equation (A.28) above.
Substituting the above formulas in s˜ for n → 0, the extremization with respect to
Q0, Qˆ0, π(x) and πˆ(xˆ) leads to the saddle point equations (17), (18) and (19).
Appendix B. Proof of Replica Symmetry
Let us consider the probability distribution inside a specific eigenspace with λ 6= 0.
Then
P(v|λ) =
[
∑
v
δ(Av, λv)
]−1
= q−dA(λ). (B.1)
The distribution of the k-point functions is
P(mk) =
〈
δ
(
mk −
1
N
N
∑
j=1
σj1 · · ·σjk
)〉
σ1,...,σk
= q−kdA(λ)
∑
v1,...,vk
δ(Av1, λv1) · · · δ(Avk, λvk)
× δ
[
mk −
1
N
N
∑
j=1
exp
(2πi
q
(
vj1 + · · ·+ vjk
)
)
]
.
(B.2)
Let us call
g(v1, ...,vk) ≡ δ
[
mk −
1
N
N
∑
j=1
exp
(2πi
q
(
vj1 + · · ·+ vjk
)
)
]
, (B.3)

Properties of Sparse Random Matrices over Finite Fields 19
and note that g(v1,v2, ...,vk) = g(v1,v2, ...,vk−1 ⊕ vk, 0). Therefore we can write
P(mk) = q−kdA(λ)
∑
v1,...,vk
δ(Av1, λv1) · · · δ(Avk, λvk)g(v1,v2, ...,vk−1 ⊕ vk, 0)
= q−kdA(λ)
∑
v1,...,vk
δ(Av1, λv1) · · · δ(Avk, λvk)
∑
u
δ(u,vk−1 ⊕ vk)g(v1,v2, ...,u, 0)
= q−kdA(λ)
∑
v1,...,vk−2,u
g(v1,v2, ...,u, 0)
×
∑
vk−1
δ(Avk−1, λvk−1)
∑
vk
δ(Avk, λvk)δ(u,vk−1 ⊕ vk)
= q−kdA(λ)
∑
v1,...,vk−2,u
g(v1,v2, ...,u, 0)
×
∑
vk−1
δ(Avk−1, λvk−1)δ(A(u⊕ (−vk−1)), 0)
= q−(k−1)dA(λ)
∑
v1,...,vk−2,u
δ(Au, λu)g(v1,v2, ...,u, 0)
=
〈
δ
(
mk −
1
N
N
∑
j=1
σj1 · · ·σjk−1
)〉
σ1,...,σk−1
.
(B.4)
Therefore, the distribution of the k-point functions is the same as the (k − 1)-
point functions for any k > 2. Therefore, if the magnetisation is zero, all other higher
order correlations are also zero and, therefore, there is neither a spin-glass phase nor
more complex types of phases but the paramagnetic one. This implies that there is
no replica symmetry breaking in the system [19]. The fact that the solution is replica
symmetric means that the temperature T = 1/ ln q can be associated with the Nishimori
temperature of the system.