IOP PUBLISHING JOURNAL OF PHYSICS A: MATHEMATICAL AND THEORETICAL
J. Phys. A: Math. Theor. 40 (2007) 12259–12279 doi:10.1088/1751-8113/40/41/001
Statistical mechanics analysis of LDPC coding in
MIMO Gaussian channels
Roberto C Alamino and David Saad
Neural Computing Research Group, Aston University, Birmingham B4 7ET, UK
Received 11 May 2007, in final form 5 September 2007
Published 25 September 2007
Online at stacks.iop.org/JPhysA/40/12259
Abstract
Using analytical methods of statistical mechanics, we analyse the typical
behaviour of a multiple-input multiple-output (MIMO) Gaussian channel with
binary inputs under low-density parity-check (LDPC) network coding and joint
decoding. The saddle point equations for the replica symmetric solution are
found in particular realizations of this channel, including a small and large
number of transmitters and receivers. In particular, we examine the cases of a
single transmitter, a single receiver and symmetric and asymmetric interference.
Both dynamical and thermodynamical transitions from the ferromagnetic
solution of perfect decoding to a non-ferromagnetic solution are identified for
the cases considered, marking the practical and theoretical limits of the system
under the current coding scheme. Numerical results are provided, showing the
typical level of improvement/deterioration achieved with respect to the single
transmitter/receiver result, for the various cases.
PACS numbers: 02.50.−r, 02.70.−c, 89.20.−a
1. Introduction
The statistical physics of disordered systems has been systematically developed over the past
few decades to analyse systems of interacting components under different interaction regimes
[1, 2]. It enables one to derive typical macroscopic properties of systems comprising a
large number of units under conditions of quenched disorder, which correspond to different
randomly sampled instances of the problem.
While their origin lies in the study of spin glasses [3–5], methods of statistical mechanics
have been successfully employed to study a broad range of interdisciplinary subjects, from
thermodynamics of fluids to biological and even sociological problems. In these studies, the
problems were mapped onto known statistical physics models, such as Ising spin systems, and
analysed using established methods and techniques.
In particular, these methods have been successfully employed recently to investigate hard
computational problems [6, 7] as well as problems in information theory [8, 9] and multi-user
1751-8113/07/4112259+21$30.00 © 2007 IOP Publishing Ltd Printed in the UK 12259

12260 R C Alamino and D Saad
communication [10]. They proved to be highly useful for gaining insight into the properties of
the problems studied and in providing exact typical case results that complement the rigourous
bounds reported in the theoretical computer science and information theory literature.
In the current study, we employ the powerful analytical methods of statistical mechanics
to examine the typical properties of multiple-input multiple-output (MIMO) communication
channels where messages are encoded using state of the art low-density parity-check (LDPC)
error correcting codes [11–14].
MIMO channels are becoming increasingly more relevant in modern communication
networks that rely on adaptive and ad hoc configurations. Sensor networks, for instance, may
rely on simultaneous transmission of information from a large number of transmitters that give
rise to high levels of interference, while multiple access, at various levels, is exercised daily
by millions of mobile phone users.
Communication over a MIMO channel is particularly amenable to a statistical physics
analysis (see for example [15]) for the following reasons: firstly, previous studies in the
areas of LDPC error-correcting codes [8] and code division multiple access (CDMA) [10, 16]
paved the way for the study of MIMO systems, and secondly, the framework of multi-user
communication channels is difficult to analyse using traditional methods of information theory
[17], but can be readily accommodated within the statistical physics framework, particularly
in the case of a large number of users. Previous results, derived via information theoretic
approaches can be found for CDMA in [18–21] and for the MIMO channel in [22].
The paper is organized as follows. In section 2 we introduce the model to be analysed,
followed by statistical physics framework in section 3. We then study several communication
channels: a single transmitter and multiple receivers in section 4, multiple access in 5 and
channels with symmetric and asymmetric interference in section 6. In each of the sections
we will consider both cases of a small and large number of users. We conclude with general
insights and future directions.
2. The model
LDPC codes, introduced by Gallager [11], are used to encode N-dimensional message vectors
s into M-dimensional codewords t. They are defined by a binary matrix A = [C1 | C2], called
parity-checkmatrix, concatenating two very sparse matrices known to both sender and receiver,
with C2 (of dimensionality (M − N) × (M − N)) being invertible and C1 of dimensionality
(M −N)×N . The matrix A can be either random or structured, characterized by the number
of non-zero elements per row/column. Irregular codes show superior performance compared
to regular codes [12, 23] if constructed carefully. However, to simplify the presentation, we
focus on regular constructions; a generalization to irregular constructions is straightforward
[24, 25].
Encoding refers to the mapping of N-dimensional binary original messages s ∈ {0, 1}N
to M-dimensional codewords t ∈ {0, 1}M (M > N) by the linear product
t = Gs (mod 2), (1)
with summations performed in the field {0, 1} being indicated by (mod 2). The generator
matrix has the form
G =
[
I
C−12 C1
]
(mod 2), (2)
where I is the N × N identity matrix. By construction AG = 0 (mod 2) and the first N bits of
t correspond to the original message s.

Statistical mechanics of MIMO channels 12261
Decoding is carried out by estimating the most probable transmitted vector from the
received corrupted codeword [8, 24].
Here we analyse MIMO Gaussian channels with L senders and O receivers in which each
of L binary messages si ∈ {0, 1}N, i = 1, . . . , L, is encoded with an LDPC error-correcting
code with an independently chosen parity-check matrix Ai into a binary codeword ti ∈ {0, 1}M .
Messages si and codewords ti are both vectors with two different indices, the bit index µ and
the sender/receiver index i. Boldface notation denotes the sets in the sender/receiver indices,
and the bit index is explicitly denoted. We concentrate on regular Gallager codes, with K
non-zero elements per row and C non-zero elements per column in the parity-check matrix,
obeying C = (1 − R)K , where R = N/M is the code rate. The codewords are transmitted in
discrete units of time.
We use, for mathematical convenience, the map x → (−1)x from the Boolean variables
ti ∈ {0, 1}M onto spin variables ti ∈ {1,−1}M . Although different, we denote both with the
same letter ti ; the appropriate use of each one being clear from the context. At each discrete
time step µ, the (already mapped) vector tµ, µ = 1, . . . ,M is transmitted and corrupted by
additive white Gaussian noise (AWGN) obeying
rµ = Stµ + νµ, (3)
where S is the O × L interference matrix, which plays an essential role in the current
analysis as it crosses messages between senders and receivers and is responsible for important
interference effects. The time-independent AWGN is given by the vector νµ =
(
νµ1 , . . . , ν
µ
O
)
with νµj ∼ N
(
0, σ 2j
)
, j = 1, . . . , O,∀µ, i.e., zero mean and variance σ 2j .
3. Replica analysis
The statistical mechanics-based analysis focuses on the decoding process as it is directly linked
to the Hamiltonian within the physics framework [26].
Decoding is carried out as in LDPC error-correcting codes; the estimate of the first N
bits of the codeword, which contain the original uncoded message, is made by introducing
L dynamical variable values τi ∈ {±1}M , representing candidate vectors for each of the
transmitted codewords and give rise to its corresponding estimates tˆi , i = 1, . . . , L, by the O
receivers, each one having access to all received messages.
In the statistical analysis, we are interested in the behaviour averaged over the system’s
disorder, given by the quenched variables r, all possible encodings (equivalently, all parity-
check matrices Ai) and all transmitted codewords ti .
Allowing some degree of error in the decoding, in the form of a prior error probability,
the bit error probability is minimized by the marginal posterior maximizer (MPM) for each
dynamical variable in τ = (τ1, . . . , τL) [25, 27]:
tˆi
µ
= sgn
〈
τµi
〉
P(τ |r). (4)
The expected overlap between the estimated and the transmitted codewords serves as a
measure for the error correction performance
di =
1
M
M
∑
µ=1
〈
tµi sgn
〈
τµi
〉
P(τ |r)
〉
A1,...,AL,r,t
, (5)
where the average is taken over the joint probability distributionP(A1, . . . , AL, r, t). Its value
also indicates the dynamical transition from the ferromagnetic solution of perfect decoding to
a non-ferromagnetic solution.

12262 R C Alamino and D Saad
The free energy in the thermodynamic limit M → ∞ is given by
f = − lim
M→∞
1
βML
〈ln Z〉A1,...,AL,r,t, (6)
where Z is the partition function
Z =
∑
τ
exp


−β
O
∑
j=1
Hj (τ |r)

,
with the Hamiltonian component for each receiver j ,
Hj (τ |r) =
1
2σ 2j
M
∑
µ=1
(
rµj −
L
∑
i=1
Sjiτ
µ
i
)2
. (7)
The Hamiltonian gives rise to a likelihood term for the agreement between the received
aggregated vector and the candidate codewords. The decoding temperature β is considered
the same for every receiver and each τi obeys the parity-check constraint, which for the spin
variables is defined by
M
∏
µ=1
(
τµi
)(Ai)νµ
= 1, ν = 1, . . . ,M − N. (8)
The decoding process is aimed at maximizing the probability
P(τ |r) =
1
Z
exp


−β
O
∑
j=1
Hj (τ |r)

. (9)
To calculate f in the thermodynamic limit M,N → ∞, keeping the code rate R = N/M
constant, we use the replica method [1, 2] which relies on the identity
〈ln Z〉 = lim
n→0
∂ ln〈Zn〉
∂n
, (10)
and employs an analytical continuation of integer values of n to a real value that approaches
zero. The calculations follow the same guidelines of [25] (see the appendix for further details).
The partition function is given by
Z =
∑
τ
[
L
∏
i=1
χ(Ai, τi)
]
exp


−β
O
∑
j=1
M
∑
µ=1
1
2σ 2j
(
rµj −
L
∑
i=1
Sjiτ
µ
i
)2
, (11)
with χ(Ai, τi) = 0 if τi does not obey the parity checks in Ai and 1 otherwise. All Ai are
chosen from the same ensemble, meaning that all code rates are equal, Ri = R.
From information theoretical considerations, the capacity region is given by αR < C,
where α ≡ L/O is a characteristic constant of the system called its load and C, the capacity with
joint decoding for an arbitrary distribution of inputs, is obtained by conventional information
theoretical methods [17] as
C = 12 log2 det
(
IO + SS
T C−1ν
)
, (12)
where T indicates transposition, IO is the O-dimensional unit matrix and Cν is an O-
dimensional square diagonal noise matrix given by (Cν)jk = σ 2j δjk . This result will be
used as a benchmark and an upper bound for our results. In the following sections we compare
the replica symmetric (RS) results with known information theoretical limits.

Statistical mechanics of MIMO channels 12263
4. Single transmitter
The case L = O = 1 is easily seen to recover the usual results for a simple Gaussian channel
as obtained in [25]. In the particular case of one sender and an arbitrary number of receivers,
the channel matrix is an O-dimensional column vector. The replica symmetric (RS) saddle
point equations are (see appendix A)
πˆ(xˆ) =
〈
δ
(
xˆ −
K−1
∏
l=1
xl
)〉
x
, (13)
π(x) =
〈
δ
(
x − tanh
[
C−1
∑
l=1
atanh xˆl + β
O
∑
j=1
rjSj
σ 2j
])〉
xˆ,r
, (14)
where P(r) =
∏O
j=1 N
(
Sj , σ 2j
)
, and the averages 〈·〉x and 〈·〉xˆ are taken with respect to π(x)
and πˆ(xˆ), respectively.
The overlap is given by
d = 〈sgn(ρ)〉ρ, (15)
with
P(ρ) =
〈
δ
(
ρ − tanh
[
C
∑
l=1
atanh xˆl + β
O
∑
j=1
rjSj
σ 2j
])〉
xˆ,r
. (16)
The free energy is
βf =
C
K
ln 2 + C〈ln(1 + xxˆ)〉x,xˆ −
C
K
〈
ln
(
1 +
K
∏
m=1
xm
)〉
x
−
〈
ln
{
∑
τ=±1
exp
[
−
O
∑
j=1
β
2σ 2j
(
rj − Sjτ
)2
]
C
∏
l=1
(
1 + τ xˆl
)
}〉
xˆ,r
. (17)
The ferromagnetic solution,
πˆ(xˆ) = δ(xˆ − 1), and π(x) = δ(x − 1), (18)
represents perfect decoding; it exists for all noise levels and has free energy f = O/2. The
internal energy and the entropy can be derived by the well-known relations
u =
∂
∂β
(βf ), s = β(u − f ). (19)
Let us study the symmetric case where all transmitters emit with the same unit power, all
entries of S are equal to 1 and all receivers experience the same noise level σ 2. When equated
to R, the capacity, derived from equation (12), gives the threshold noise σ 2t corresponding to
the Shannon limit of perfect decoding
C =
1
2
log2
(
1 +
O
σ 2
)
= αR ⇒ σ 2t =
O
22R/O − 1
. (20)
To obtain numerical solutions for the various cases we iterated the saddle-point
equations (13) using population dynamics and then calculated the quantities of interest such
as the overlap d, the free energy f and the entropy s of equations (15)–(19).
Figure 1 shows the overlap for L = 1 (one sender), O = 2 (two receivers), σ 2j = σ 2
(equal noise level for all receivers) and R = 1/4 (with K = 4 and C = 3) at Nishimori’s

Statistical mechanics of MIMO channels 12265
Table 1. Comparison between the Shannon limit for a simple Gaussian channel and the MIMO
channel, the dynamical transition point and the thermodynamical transition for the single-sender
case (L = 1).
Shannon’s Limit Shannon’s limit Dynamical Thermodynamical
O (Gaussian channel) (MIMO channel) transition transition
1 2.41 2.41 1.59 2.24
2 5.28 10.57 3.28 4.59
3 8.17 24.50 4.90 6.68
here (second column) with the theoretical limit for the MIMO channel given by equation (20)
(third column) and the points of the dynamical (fourth column) and the thermodynamical (fifth
column) transitions obtained by numerical integration of the RS equations for O = 1, 2, 3
receivers. For O > 3, the numerical instabilities grow larger with O and a precise evaluation
of the points is increasingly more difficult. The dynamical and thermodynamical transitions
clearly occur always before Shannon’s limit. As expected, the more are the receivers, the higher
the noise level tolerated by the system. However, the differences between the dynamical and
the thermodynamical transition values, and between the latter and Shannon’s limit, increase.
Both are related to the fact that, in adding more receivers, the number of metastable states
increases; they emerge earlier and contribute to a higher entropy.
Comparing the theoretical limit for sending a message O times by a simple Gaussian
channel and for the MIMO channel with one sender and O receivers, we see that the latter
is just O times the former. This can be understood noting that the information sent in the
MIMO channel is the same as in the O-replicated Gaussian channel, but with O times the
power, while in the MIMO channel, the O bits are sent with power 1 at each time step. We can
see by the results that the transition points are even below the theoretical limit for the simple
Gaussian channel and significantly below the MIMO limit. This clearly shows that in this
type of communication channel, even with joint decoding, the available information is being
poorly used. It makes a strong case for the use of network coding, i.e., to encode jointly the
vectors tµ prior to transmission. Network coding, for instance using fountain codes [30, 31],
is likely to make a better use of the resource by generating codewords that are more suited for
better extraction of information under joint decoding.
Another case of interest is the infinite number of receivers, O → ∞. The average over
r’s in equation (13) can be substituted by an average over the Gaussian variable
v ≡
O
∑
j=1
rjSj
σ 2j
, (21)
which, for equal noise and Sj = 1, has zero mean and variance O/σ 2 reflecting the signal-to-
noise ratio appearing in the capacity expression (20).
5. Multiple access channel
The multiple access channel (MAC) is a particular case where O = 1 and S is an L-dimensional
row matrix. Let us consider once more the symmetric case where Sji = 1 and σ 2j = σ 2. Again,
we find the threshold noise σ 2t by equating the capacity to the code rate
C =
1
2
log2
(
1 +
L
σ 2
)
= αR ⇒ σ 2t =
L
22LR − 1
. (22)

12266 R C Alamino and D Saad
Here, due to the interference effect in the received message, one must guarantee that the
interference term has the correct order in L. Taking into account that the received messages
are independent, we normalize their sum by the factor 1/
√
L.
The simplest case is L = 2 and the RS saddle point equations for user 1 are
πˆ1(xˆ1) =
〈
δ
(
xˆ −
K−1
∏
l=1
xl1
)〉
x
, (23)
π1(x1) =
〈
δ
(
x − tanh
{
C−1
∑
l=1
atanh xˆl1 +
βr
σ 2
√
2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
(
∑C
l=1 atanh xˆl2 +
βr
σ 2
√
2
)
1 + tanh
( β
2σ 2
)
tanh
(
∑C
l=1 atanh xˆl2 +
βr
σ 2
√
2
)
]})〉
xˆ,r
, (24)
and the overlap is
d1 = 〈sgn (ρ)〉ρ, (25)
P(ρ) =
〈
δ
(
ρ − tanh
{
C
∑
l=1
atanh xˆl1 +
βr
σ 2
√
2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
(
∑C
l=1 atanh xˆl2 +
βr
σ 2
√
2
)
1 + tanh
( β
2σ 2
)
tanh
(
∑C
l=1 atanh xˆl2 +
βr
σ 2
√
2
)
]})〉
xˆ,r
, (26)
where P(r) = N (
√
2, σ 2).
The corresponding set of equations for user 2 is identical to (23)–(26) except for
interchanging indices 1 and 2. The free energy is given by
βf =
C
K
ln 2 +
C
2
2
∑
i=1
〈ln(1 + xi xˆi)〉x,xˆ −
C
2K
2
∑
i=1
〈
ln
(
1 +
K
∏
m=1
xmi
)〉
x
−
1
2
〈
ln
{
∑
τ1,τ2
exp
[
−
β
2σ 2
(
r −
τ1 + τ2
√
2
)2
] 2
∏
i=1
C
∏
l=1
(
1 + τi xˆli
)
}〉
xˆ,r
. (27)
For the ferromagnetic solution (18), f = 0.25. Indeed, for the MIMO Gaussian channel
studied here, we always have that the ferromagnetic free energy given by f = 1/2α.
By iteratively solving the saddle point equations we obtain the quantities of interest.
The free and internal energies, for L = 2, R = 1/4 (K = 4, C = 3) and Nishimori’s
temperature, are represented in figure 2 by the solid and dashed lines, respectively; Shannon’s
limit is given by σ 2 = 2 (dot-dashed line). The point, where the free energy differs from the
internal energy and the overlap becomes less than 1, marks the dynamical transition point.
The thermodynamical transition point is identified by the crossing of the two energies and is
denoted by the dotted line. The entropy function, shown in the inset plotted against the noise
level, also helps to identify the dynamical and thermodynamical transitions (where the entropy
becomes negative and where it crosses back the coordinate axis, respectively). Both points
are below Shannon’s limit.

12268 R C Alamino and D Saad
t1
2t
r1
r2
t1 r1
r22t
Symmetric Interference Asymmetric Interference
Figure 3. Diagram representing channels with symmetric (left) and asymmetric (right)
interference. The first and second transmitters and receivers are denoted by t1, t2 and r1, r2,
respectively. Arrows represent the transmitted messages and the double line between the receivers
indicates joint decoding.
6. Channels with interference
This section deals with a scenario where several transmitters send data simultaneously to an
equal number of receivers; the transmission from a given transmitter to the corresponding
receiver is corrupted by (small) deterministic interference from all other transmitters. The
receivers can then communicate with each other to optimally extract the original messages.
Some sensor networks are among the most well-known examples of systems that could be
modelled by this kind of channel.
In the following, we study two basic types of interference, the symmetric and the
asymmetric cases. For simplicity, we limit the number of transmitters and receivers to
L = O = 2, making the interpretation of the results easier and more transparent. Both
channels are depicted in figure 3. In the symmetric case, the transmitters send messages to
both receivers (left picture) while in the asymmetric case only the first transmitter sends a
message to the first receiver, while the second transmitter sends to both.
Although the term interference channel is conventionally used when there is no
cooperation between the receivers, the actual definition of this channel, as given in [17] for
instance, does not explicitly exclude some information exchange between receivers. Therefore,
the channels studied here can be viewed as instances of the interference channel where receivers
cooperate to decode their messages by exploiting information gathered by other receivers to
better estimate their own message.
Nevertheless, to avoid confusion and ambiguity we would like to point out that the
channels with interference investigated here do not correspond to the conventional use of the
term interference channel.
6.1. Symmetric interference
We first study the case L = O = 2 with a symmetric interference matrix
S =
(
1 
 1
)
, (28)
where 0 < 
1. The capacity can be derived using equation (12) to obtain
C =
1
2
log2
[
1 +
2(1 + 2)
σ 2
+
(1 − 2)2
σ 4
]
. (29)

Statistical mechanics of MIMO channels 12269
The RS saddle point equations are given by
πˆ1(xˆ1) =
〈
δ
(
xˆ1 −
K−1
∏
l=1
xl1
)〉
x
, (30)
π1(x1) =
〈
δ
(
x1 − tanh
{
C−1
∑
l=1
atanh xˆl1 +
β
σ 2
√
2
(r1 + r2)
+
1
2
ln
[
1 − tanh
( β
σ 2
)
tanh
( β(r1+r2)
σ 2
√
2
+
∑C
l=1 atanh xˆl2
)
1 + tanh
( β
σ 2
)
tanh
( β(r1+r2)
σ 2
√
2
+
∑C
l=1 atanh xˆl2
)
]})〉
xˆ,r
, (31)
where P(ri) = N ((1 + )/
√
2, σ 2), i = 1, 2. The corresponding equations for πˆ2 and π2 are
similar to those of πˆ1 and π1 and are obtained by interchanging indices 1 and 2.
Here, the same scaling as in the MAC case is necessary due to the interference. However,
for  = 0, this scaling should be omitted as the interference vanishes, leaving two separate
Gaussian channels.
The overlaps are given by
di = 〈sgn(ρ)〉ρ, (32)
P(ρ) =
〈
δ
(
ρ − tanh
{
C
∑
l=1
atanh xˆl1 +
β
σ 2
√
2
(r1 + r2)
+
1
2
ln
[
1 − tanh
( β
σ 2
)
tanh
( β(r1+r2)
σ 2
√
2
+
∑C
l=1 atanh xˆl2
)
1 + tanh
( β
σ 2
)
tanh
( β(r1+r2)
σ 2
√
2
+
∑C
l=1 atanh xˆl2
)
]})〉
xˆ,r
. (33)
The free energy f is
βf =
C
K
ln 2 +
C
2
2
∑
i=1
〈ln(1 + xi xˆi)〉x,xˆ −
C
2K
2
∑
i=1
〈
ln
(
1 +
K
∏
m=1
xmi
)〉
x
−
1
2
〈
ln
{
∑
τ1,τ2
exp
[
−
β
2σ 2
(
r1 −
τ1 + τ2
√
2
)2
−
β
2σ 2
(
r2 −
τ1 + τ2
√
2
)2
]
×
2
∏
i=1
C
∏
l=1
(
1 + τi xˆli
)
}〉
xˆ,r
. (34)
Accordingly, the free energy of the ferromagnetic solution (18) is f = 0.5, as for the
simple Gaussian channel.
We solved numerically the saddle point equations (30) and calculated quantities of
relevance in this case. The graphs for the overlap, entropy and energy are qualitatively
the same as in the other two cases, with a similar behaviour with the appearance of both
dynamical and thermodynamical transition points before Shannon’s limit.
Figures 4 and 5 show the field distributions π(x) and πˆ(xˆ), respectively, for four different
values of the noise level in the RS ansatz, with  = 1.0, β = 1 and R = 1/4 (K = 4,
C = 3). It should be noted that, before the dynamical transition point, these distributions are
delta functions centred at 1, corresponding to the ferromagnetic solution (18). The plotted
distributions are histograms with 500 bins for 40 000 fields. In figure 4, we see how the π

Statistical mechanics of MIMO channels 12271
0.0 0.2 0.4 0.6 0.8 1.0
ε
0.0
1.0
2.0
3.0
4.0
5.0
6.0
σ
2
3.33
2.70
2.09
1.65
1.16
2.63
2.14
1.72
1.41
1.09
Shannon’s Limit
Thermodynamical Transition
Dynamical Transition
Figure 6. Transition points and theoretical limits as a function of the interference level . The solid
line represents the theoretical limit obtained from information theoretical methods; the dashed-
dotted and dashed lines correspond to the thermodynamical and dynamical transition points,
respectively.
the graphs, it is visible how the scales increase very fast as the noise level attains higher
values.
If one keeps a constant code rate R = 1/4 but allows  to vary, one obtains the dependence
of the threshold noise as a function of , depicted in figure 6 (for β = 1,K = 4 and C = 3).
Both dynamical (dashed line) and thermodynamical transition values (dashed-dotted line)
are upper bounded by the theoretical limit. Although this may seem counterintuitive, the
communication resilience against noise increases with the interference level. This can be
understood in the case of joint detection by noting that the increased interference provides
more information about the other transmitters, such that higher levels of noise can be tolerated
by joint decoding.
For large O with L ∼ O(1) or large L with O ∼ O(1), the results should approach
those obtained for a large number of users in the single transmitter and in the MAC case,
respectively. The behaviour must be dictated by the value of the system load α. In this case,
we expect the results to cross from a behaviour similar to the one of a MAC channel for α > 1
to one that resembles the single transmitter case for α < 1. We are currently working on the
analytical and computational aspects of this last case as well as on the case of large O and L
values while keeping the ratio L/O ∼ O(1) finite.
6.2. Asymmetric interference
A variant of the channel discussed in section 6.1, for the case of L = O = 2, is the case
with asymmetric interference. This realization is highly relevant to cases where receivers
are distributed at random and experience different noise levels, for instance, in the case of
sensor networks. The interference matrix is asymmetric in this case and takes the form (for
L = O = 2)
S =
(
1 
0 1
)
, (35)
with 0 < 
1. The corresponding capacity is now (again by (12))
C =
1
2
log2
[
1 +
(2 + 2)
σ 2
+
1
σ 4
]
. (36)

12272 R C Alamino and D Saad
The RS saddle point equations are given by
πˆi(xˆi) =
〈
δ
(
xˆi −
K−1
∏
l=1
xli
)〉
x
, i = 1, 2, (37)
π1(x1) =
〈
δ
(
x1 − tanh
{
C−1
∑
l=1
atanh xˆl1 +
βr1
σ 2
√
2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+ βr2σ 2 +
∑C
l=1 atanh xˆl2
)
1 + tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+ βr2σ 2 +
∑C
l=1 atanh xˆl2
)
]})〉
xˆ,r
, (38)
π2(x2) =
〈
δ
(
x2 − tanh
{
C−1
∑
l=1
atanh xˆl2 +
βr1
σ 2
√
2
+
βr2
σ 2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+
∑C
l=1 atanh xˆl1
)
1 + tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+
∑C
l=1 atanh xˆl1
)
]})〉
xˆ,r
, (39)
where P(r1) = N ((1 + )/
√
2, σ 2) and P(r2) = N (1, σ 2).
In this case, the scaling 1/
√
L (although here L = 2, the treatment can be extended to
include a general number of sources) appears only in the first receiver, as it is being affected
by the interference.
The overlaps are given by
di = 〈sgn(ρi)〉ρi , i = 1, 2, (40)
P(ρ1) =
〈
δ
(
ρ1 − tanh
{
C
∑
l=1
atanh xˆl1 +
βr1
σ 2
√
2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+ βr2σ 2 +
∑C
l=1 atanh xˆl2
)
1 + tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+ βr2σ 2 +
∑C
l=1 atanh xˆl2
)
]})〉
xˆ,r
, (41)
P(ρ2) =
〈
δ
(
ρ2 − tanh
{
C
∑
l=1
atanh xˆl2 +
βr1
σ 2
√
2
+
βr2
σ 2
+
1
2
ln
[
1 − tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+
∑C
l=1 atanh xˆl1
)
1 + tanh
( β
2σ 2
)
tanh
( βr1
σ 2
√
2
+
∑C
l=1 atanh xˆl1
)
]})〉
xˆ,r
. (42)
The free energy f is obtained from
βf =
C
K
ln 2 +
C
2
2
∑
i=1
〈ln(1 + xi xˆi)〉x,xˆ −
C
2K
2
∑
i=1
〈
ln
(
1 +
K
∏
m=1
xmi
)〉
x
−
1
2
〈
ln
{
∑
τ1,τ2
exp
[
−
β
2σ 2
(
r1 −
τ1 + τ2
√
2
)2
−
β
2σ 2
(r2 − τ2)
2
]
×
2
∏
i=1
C
∏
l=1
(
1 + τi xˆli
)
}〉
xˆ,r
, (43)
with the free energy of the ferromagnetic solution f = 0.5.

12274 R C Alamino and D Saad
codewords of size M = 2000. Again, as in the case of the single transmitter (section 4),
the BP solution is clearly in accord with the replica symmetric calculation. The small
disagreements are due to noise and to finite size effects; they tend to disappear as the system
size increases and the average is taken over a large number of realizations.
The result shows that information can be distributed among the receivers in a highly
non-trivial way and also that for systems with many users, the thermodynamical transition is
determined mostly by the weakest node (which experiences the highest levels of interference)
and may lead to practical limits very far from Shannon’s bound.
7. Conclusions
We investigated the properties of coded Gaussian MIMO channels using methods of statistical
mechanics. The problems investigated relate to the cases of a single transmitter, multiple access
and interference in the case of multiple receivers and transmitters. In all cases, transmissions
are coded using LDPC error-correcting codes.
The method used in the analysis, the replica approach, enables one to obtain typical
case results that complements the theoretical bounds reported in the information theory
literature. The numerical results obtained for particular MIMO channels and parameter values
are presented and contrasted with information theoretical results.
MIMO channels are characterized by an interference matrix S which mixes inputs from
the various transmitters to provide the messages at the receiving end. We examine cases where
the interference matrix is deterministic. This requires the introduction of a non-trivial scaling
in order to obtain meaningful results.
The results obtained provide characteristic, typical case, results in all cases. For the
single transmitter and MAC cases, we show both dynamical and thermodynamical transitions
as functions of the number of receivers and transmitters, respectively. We see that the gaps
between the practical and theoretical thresholds (dynamical and thermodynamical transitions,
respectively), and the gap between them and Shannon’s limit, increase with the number of
receivers for the single transmitter and decrease with the number of transmitters in the MAC
case.
For a single transmitter, this results from the increase in the number of variables
and consequentially also in the number of metastable states. The point where metastable
solutions emerge determines the dynamical transition (practical threshold), while the number of
metastable states affects the thermodynamic transition. The increasing number of transmitters
in the MAC case enables one to effectively reduce the noise level by averaging over a higher
number of random-independent noise sources.
The comparison with theoretical limits for the single transmitter case reveals an important
feature of multi-user channels as to how the available information is used. The huge gap
between the transition points and Shannon’s limit is indicative of a poor use of resource,
and suggests network coding as a measure to achieve a better use of them; without it,
the system’s efficiency remains below the achievable theoretical limit for sending the same
message repeatedly via a simple Gaussian channel. One possible solution that we are currently
investigating is the use of fountain codes [30, 31] for making a more efficient use of the available
resource.
The main result for the case with symmetric interference is the increase in both dynamical
and thermodynamical transition points as a function of the interference parameter . Results for
low  values are similar to the case of separate Gaussian channels; as  increases, both values
come closer to Shannon’s limit with the thermodynamical transition point showing a stronger
increase. This could be explained by the increase of (mixed) information in comparison to

Statistical mechanics of MIMO channels 12275
the noise level which can be decoded jointly, with an effectively lower noise level. The more
moderate increase in the practical threshold (dynamical transition) is due to the difficulty in
jointly decoding the various sources in practice due to the emergence of metastable states.
In the asymmetric case, we found a striking different behaviour of the system. The new
feature observed is the second transition suffered by the system as a whole. We also detected
a surprising behaviour of the receiver which experiences interference; in spite of the joint
decoding, the information available to it is suppressed by the second receiver. Only when the
second receiver stops decoding perfectly, the performance of the first receiver improves.
An interesting extension, of significant practical relevance, would be to extend the LDPC
coding framework to complex MIMO channels, where circular noise is considered [32].
Another possible extension is the case of a large number of senders and receivers where the
ratio between them remains finite. The study of these and other related problems is underway.
Acknowledgments
Support from EVERGROW, IP no. 1935 in FP6 of the EU is gratefully acknowledged.
Appendix. Replica symmetric calculations
From the partition function (11), we can write the averaged replicated partition function
Zn ≡ 〈Zn〉A1,...,AL,r,t as
Zn =
λM
2NL
∑
{τa}
∫
dr exp
[
−
O
∑
j=1
M
∑
µ=1
1
2σ 2j
(
rµj −
L
∑
i=1
Sjiτ
µ
i0
)2 ]
× exp
[
−
n
∑
a=1
O
∑
j=1
M
∑
µ=1
β
2σ 2j
(
rµj −
L
∑
i=1
Sjiτ
µ
ia
)2 ][ L
∏
i=1
i({τia})
]
, (A.1)
λ ≡
O
∏
j=1
(
2πσ 2j
)
−1/2
, (A.2)
where the multiplicative constants come from the normalization of the probability distributions
in the outside average and we defined τi0 ≡ ti . Following [16], we have
i({τia}) ≡
〈
n
∏
a=0
χ(Ai, τia)
〉
Ai
=
1
NA
∮
DZi
[
∑
ωi
(
1
M
∑
µ
Zµi τ
µ
iai1
· · · τµiaimi
)K]M−N
, (A.3)
where
DZi ≡
(
1
2M−N
)n+1 M
∏
µ=1
dZµi
2π i
1
(
Zµi
)C+1 , ωi ≡
〈
aimi
〉
, (A.4)
and the variables mi assume all integer values for the index i from 0 to n + 1.
Defining
qωi ≡
1
M
∑
µ
Zµi τ
µ
iai1
· · · τµiaimi
, (A.5)

12276 R C Alamino and D Saad
and using integral representations for the delta functions, we can write
Zn = 2−NL
∫
(
L
∏
i=1
∏
ωi
dqωi dqˆωi
2π i/M
)[
L
∏
i=1
∑
ωi
(
qωi
)K
]M−N
×
L
∏
i=1
exp
(
−M
∑
ωi
qωi qˆωi
)
×
∑
{τa}
L
∏
i=1
[
∮
DZi exp
(
∑
ωi
qˆωi
∑
µ
Zµi τ
µ
iai1
· · · τµiaimi
)]
× λM
∫
dr exp
[
−
O
∑
j=1
M
∑
µ=1
1
2σ 2j
(
rµj −
L
∑
i=1
Sjiτ
µ
i0
)2 ]
× exp
[
−
n
∑
a=1
O
∑
j=1
M
∑
µ=1
β
2σ 2j
(
rµj −
L
∑
i=1
Sjiτ
µ
ia
)2 ]
. (A.6)
Defining
L
∏
i=1
∏
ωi
dqωi dqˆωi
2π i/M
≡ DqDqˆ, and γ ≡
2−(M−N)(n+1)2−N
NA
, (A.7)
and integrating over the variables Zµi , the µ indices factorize and we obtain
Zn =
∫
DqDqˆ exp[ML ˜f (q, qˆ)], (A.8)
with
˜f (q, qˆ) ≡
1
M
ln γ +
(1 − R)
L
L
∑
i=1
ln
[
∑
ωi
(
qωi
)K
]
−
1
L
L
∑
i=1
∑
ωi
qωi qˆωi +
1
L
ln , (A.9)
and
 ≡ λ
∫
dLr
∑
{τa}
[
L
∏
i=1
1
C!
(
∑
ωi
qˆωi τiai1 · · · τiaimi
)C]
× exp
[
−
O
∑
j=1
1
2σ 2j
(
rj −
L
∑
i=1
Sjiτi0
)2]
× exp
[
−
n
∑
a=1
O
∑
j=1
β
2σ 2j
(
rj −
L
∑
i=1
Sjiτia
)2]
. (A.10)
Using the replica symmetric (RS) ansatz
qωi = q
i
0
〈
(xi)
mi−i
〉
xi
, xi ∼ πi(xi),
qˆωi = qˆ
i
0
〈
(xˆi)
mi−i
〉
xˆi
, xˆi ∼ πˆi(xˆi),
(A.11)

Statistical mechanics of MIMO channels 12277
where
i =
{
1, 0 ∈
{
ai1, . . . , a
i
mi
}
0, otherwise.
(A.12)
For small n,
ln
[
∑
ωi
(
qωi
)K
]
= ln
[
2
(
qi0
)K
]
+ n
〈
ln
(
1 +
K
∏
m=1
xmi
)〉
x
, (A.13)
where 〈·〉x indicates the average over all variables xmi and
∑
ωi
qωi qˆωi = 2qi0qˆ
i
0
[
1 + n〈ln(1 + xi xˆi)〉xi ,xˆi
]
, (A.14)
∑
ωi
qˆωi τ
µ
iai1
· · · τµiaimi
= qˆi0(1 + τi0)
〈
n
∏
a=1
(1 + τiaxˆi)
〉
xˆ
. (A.15)
Inserting the result in  and summing over the zeroth replicas, we have
 =
(2L ˆQ0)C
(C!)L
〈
∑
{τa}
C
∏
l=1
n
∏
a=1
L
∏
i=1
(
1 + τiaxˆli
)
× exp
[
−
n
∑
a=1
O
∑
j=1
β
2σ 2j
(
rj −
L
∑
i=1
Sjiτia
)2]〉
r,xˆ
, (A.16)
where ˆQ0 ≡
∏
i qˆ
i
0 and P(r) =
∏O
j=1 N
(
∑L
i=1 Sji, σ
2
j
)
.
The sum over the n replicas factorizes to
 =
(2L ˆQ0)C
(C!)L
〈{
∑
τ1,...,τL
C
∏
l=1
L
∏
i=1
(
1 + τi xˆli
)
× exp
[
−
O
∑
j=1
β
2σ 2j
(
rj −
L
∑
i=1
Sjiτi
)2]}n〉
r,xˆ
. (A.17)
A.1. Single transmitter
Let us consider L = 1. Then, for small n,
ln  = ln
(2qˆ0)C
C!
+ n
〈
ln
{
∑
τ
C
∏
l=1
(1 + τ xˆl) exp
[
−
O
∑
j=1
β
2σ 2j
(rj − Sj τ )
2
]}〉
r,xˆ
. (A.18)
Derivations with respect to q0 and qˆ0 give 2q0qˆ0 = C and functional derivatives with
respect to π(x) and πˆ(xˆ) give equations (13) of section 4.
A.2. MAC
In this case, O = 1,
ln  = ln
(2L ˆQ0)C
(C!)L
+ n
〈
ln
{
∑
{τi }
C
∏
l=1
L
∏
i=1
(
1 + τi xˆli
)
exp
[
−
β
2σ 2
(
r −
L
∑
i=1
Siτi
)2]}〉
r,xˆ
,
(A.19)
and the corresponding extremization, including the necessary normalization, gives
equations (23) of section 5.

12278 R C Alamino and D Saad
A.3. Interference channel
The case with L = O = 2 can be viewed as an interference Gaussian channel where the
receivers cooperate to decode the received message. In this case
 =
(4 ˆQ0)C
C!
〈{
∑
τ1,τ2
C
∏
l=1
[(
1 + τ1xˆl1
)(
1 + τ2xˆl2
)]
× e−
β
2σ2
(r1−S11τ1−S12τ2)2 e−
β
2σ2
(r2−S21τ1−S22τ2)2
}n〉
r,xˆ
. (A.20)
Extremization with respect to πi, i = 1, 2 results in
πˆi(xˆi) =
〈
δ
(
xˆi −
K−1
∏
l=1
xli
)〉
x
. (A.21)
Equating the functional derivative with respect to πˆ1 to zero, we obtain
π1(x1) = 〈δ(x1 − h1(r, xˆ))〉r,xˆ, (A.22)
where
h1(r, xˆ) ≡
∑
τ1,τ2
τ1P τ1τ2
∏C−1
l=1
(
1 + τ1xˆl1
)
∏C
l=1
(
1 + τ2xˆl2
)
∑
τ1,τ2
P τ1τ2
∏C−1
l=1
(
1 + τ1xˆl1
)
∏C
l=1
(
1 + τ2xˆl2
)
, (A.23)
and
P τ1τ2 ≡ e−
β
2σ2
(r1−S11τ1−S12τ2)2 e−
β
2σ2
(r2−S21τ1−S22τ2)2 . (A.24)
The final equations with the interference normalization are already given in section 6 for
both symmetric (subsection 6.1) and asymmetric (subsection 6.2) cases. These equations can
be easily generalized for any number of L and O values. In numerical calculations, however,
the numerical errors occurring due to the introduction of additional fields in this direct form
are difficult to control. Clever algebraic manipulations are necessary to keep these errors under
control in order to obtain accurate results.
References
[1] Nishimori H 2001 Statistical Physics of Spin Glasses and Information Processing (Oxford, UK: Oxford
University Press)
[2] Me´zard M, Parisi G and Virasoro M 1987 Spin Glass Theory and Beyond (Singapore: World Scientific)
[3] Edwards S F and Anderson P W 1975 J. Phys. F: Metal Phys. 5 965–74
[4] Sherrington D and Kirkpatrick S 1975 Phys. Rev. Lett. 35 1792–6
[5] Kirkpatrick S and Sherrington D 1978 Phys. Rev. B 17 4384–403
[6] Monasson R and Zecchina R 1996 Phys. Rev. Lett. 76 3881–5
[7] Hartmann A K and Weigt M 2005 Phase Transitions in Combinatorial Optimization Problems: Basics,
Algorithms and Statistical Mechanics (New York: Wiley)
[8] Kabashima Y and Saad D 2004 J. Phys. A: Math. Gen. 37 R1–R43
[9] Murayama T 2002 J. Phys. A: Math. Gen. 35 L95–L100
[10] Tanaka T 2002 IEEE Trans. Inf. Theory 11 2888–910
[11] Gallager R 1962 IRE Trans. Inf. Theory IT-8 21–8
[12] Richardson T, Shokrollahi A and Urbanke R 2001 IEEE Trans. Inf. Theory 47 619–37
[13] Richardson T and Urbanke R 2001 IEEE Trans. Inf. Theory 47 599–618
[14] MacKay D 2003 Information Theory, Inference and Learning Algorithms (Cambridge, MA: Cambridge
University Press) http://wol.ra.phy.cam.ac.uk/mackay/
[15] de Miguel R, Shental O, Muller R R and Kanter I 2007 J. Phys. A: Math. Theor. 40 5241–60
[16] Tanaka T and Saad D 2003 Technical report unpublished

Statistical mechanics of MIMO channels 12279
[17] Cover T and Thomas J 1991 Elements of Information Theory (New York: Wiley)
[18] Yeap B L, Guo F, Kuan E L, Wei H and Hanzo L 2004 Veh. Technol. Conf. 3 1376–80
[19] Chung J, Guo F, Ng S X and Hanzo L 2003 Veh. Technol. Conf. 3 1487–91
[20] Yue G and Wang X 2004 IEEE Trans. Wirel. Commun. 3 1734–45
[21] Hoiu J, Yi Y and Lee M H 2004 EURASIP J. Wirel. Commun. Netw. 1 141–8
[22] Sanderovich A, Peleg M and Shamai S 2005 IEEE Trans. Inf. Theory 51 1437–50
[23] Kanter I and Saad D 1999 Phys. Rev. Lett. 83 2660–3
[24] Vicente R, Saad D and Kabashima Y 2002 Advances in Imaging and Electron Physics vol 125 ed P Hawkes
(New York: Academic) pp 232–353
[25] Vicente R, Saad D and Kabashima Y 2000 J. Phys. A: Math. Gen. 33 6527–42
[26] Sourlas N 1994 Europhys. Lett. 25 159–64
[27] Iba Y 1999 J. Phys. A: Math. Gen. 32 3875–88
[28] Franz S, Leone M, Montanari A and Ricci-Tersenghi F 2002 Phys. Rev. E 66 046120
[29] Migliorini G and Saad D 2006 Phys. Rev. E 73 026122
[30] Luby M 2002 Proc. of 43rd Annual IEEE Symposium on Foundation of Computer Science pp 271–82
[31] Luby M G, Mitzenmacher M, Shokrollahi M A, Spielman D A and Stemann V 1997 Proc. of 29th Annual ACM
Symposium on Theory of Computing pp 150–9
[32] Moustakas A L, Simon S H and Sengupta A M 2003 IEEE Trans. Inf. Theory 49 2545–61