502 JOURNAL OF LIGHTWAVE TECHNOLOGY Fiber Impairment Compensation Using Coherent Detection and Digital Signal Processing Ezra M. Ip and Joseph M. Kahn, Fellow, IEEE (Invited Paper) Abstract���Next-generation optical fiber systems will employ coherent detection to improve power and spectral efficiency, and to facilitate flexible impairment compensation using digital signal processors (DSPs). In a fully digital coherent system, the electric fields at the input and the output of the channel are available to DSPs at the transmitter and the receiver, enabling the use of arbitrary impairment precompensation and postcompensation algorithms. Linear time-invariant (LTI) impairments such as chromatic dispersion and polarization-mode dispersion can be compensated by adaptive linear equalizers. Non-LTI impairments, such as laser phase noise and Kerr nonlinearity, can be compen- sated by channel inversion. All existing impairment compensation techniques ultimately approximate channel inversion for a subset of the channel effects. We provide a unified multiblock nonlinear model for the joint compensation of the impairments in fiber transmission. We show that commonly used techniques for over- coming different impairments, despite their different appearance, are often based on the same principles such as feedback and feedforward control, and time-versus-frequency-domain repre- sentations. We highlight equivalences between techniques, and show that the choice of algorithm depends on making tradeoffs. Index Terms���Adaptive signal processing, optical fiber commu- nication. I. INTRODUCTION C OHERENT detection has emerged as one of the key tech- nologies in the development of high-speed, high-spectral- efficiency, dynamically reconfigurable optical networks suitable for long-distance transmission. By recovering the electric field in the two fiber polarizations, a coherent receiver allows infor- mation symbols to be encoded in all the degrees of freedom available in a fiber, leading to improved power and spectral ef- ficiency. Interest in coherent optical communications began in the late 1980s. Carrier synchronization was an early challenge due to the high carrier-linewidth-to-symbol-rate ratio compared to wire- less and digital subscriber line (DSL) systems. Using narrow Manuscript received June 08, 2009 revised July 02, 2009. First published July 24, 2009 current version published February 03, 2010. E. M. Ip is with NEC Labs America, Princeton, NJ 08540 USA (e-mail: ezra. ip@nec-labs.com). J. M. Kahn is with the Department of Electrical Engineering, Stanford Uni- versity, Stanford, CA 94305-9515 USA (e-mail: jmk@ee.stanford.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/JLT.2009.2028245 linewidth lasers and optimized phase-locked loops (PLLs), co- herent 4-Gb/s binary phase shift keying (BPSK) was demon- strated in [1], while 310-Mb/s quadriphase shift keying (QPSK) was demonstrated in [2]. Development in coherent optical sys- tems was stalled in the 1990s by the invention of the erbium- doped fiber amplifier (EDFA), which enabled repeaterless trans- mission over long-haul distances. The combined spectra of the C-band (1530���1570 nm) and L-band (1570���1610 nm) offer a bandwidth of 10 THz. But with the rapid growth of Internet traffic, driven by new applications, such as video and music sharing, this bandwidth is rapidly becoming fully utilized. Thus, there is renewed interest in high-spectral-efficiency transmis- sion. Another factor contributing to the resurgence of coherent systems is the recent advance in very large scale integration (VLSI), which has made digital compensation of fiber impair- ments at giaghertz baud rates feasible. Digital compensation can be done either at the transmitter prior to upconversion onto an optical carrier, or at the receiver after the optical signal has been downconverted to the electronic domain. In both cases, provided the baseband signal in the electronic domain is sampled above the Nyquist rate, the digitized signal has the full information of the analog electric field, enabling digital signal processing (DSP) compensation to have no loss in performance compared to analog impairment compensation performed in either the optical or electronic domain. DSP has the advantage that signals can be delayed, split, amplified, and manipulated in other manner without degradation in signal quality. DSP-based receivers are already ubiquitous in wireless and DSL systems operating at comparatively low data rates. In recent years, research on coherent systems has progressed at a rapid rate. Real-time coherent detection of polarization mul- tiplexed 40-Gb/s QPSK (PM-QPSK) was demonstrated in [3], while 100-Gb/s systems and beyond are currently being devel- oped. As the baud rate, constellation size, and transmission dis- tances are increased, DSP algorithms will become increasingly complex to overcome a diversity of optical phenomena that cur- rently limit achievable capacity. A key difference between optical fiber transmission and other media, such as wireless or DSL, is that signal propagation is nonlinear, due to the tight confinement of light leading to high electric field intensity. Although nonlinearity also arises in radio-frequency (RF) transmission due to amplifier saturation, this nonlinearity is localized and instantaneous, and can be pre- compensated effectively. In optical fiber, nonlinearity imposes 0733-8724/$26.00 �� 2010 IEEE