Download the Full Ebook and Access More Features - ebooknice.

com

(Ebook) Error Correction Coding: Mathematical

Methods and Algorithms by Todd K. Moon ISBN
9781119567479, 1119567475

https://ebooknice.com/product/error-correction-coding-
mathematical-methods-and-algorithms-23269180

OR CLICK HERE

DOWLOAD EBOOK

Download more ebook instantly today at https://ebooknice.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) Error Correction Coding : Mathematical Methods and Algorithms by Todd K.

Moon ISBN 9780471648000, 9780471739142, 0471648000, 0471739146

https://ebooknice.com/product/error-correction-coding-mathematical-methods-and-
algorithms-1760676

ebooknice.com

(Ebook) Foundations of Coding: Compression, Encryption, Error Correction by Jean-

Guillaume Dumas, Jean-Louis Roch, Éric Tannier, Sébastien Varrette ISBN
9781118881446, 1118881443

https://ebooknice.com/product/foundations-of-coding-compression-encryption-
error-correction-4990816

ebooknice.com

(Ebook) Error-Correction Coding and Decoding: Bounds, Codes, Decoders, Analysis and
Applications by Martin Tomlinson, Cen Jung Tjhai, Marcel A. Ambroze, Mohammed Ahmed,
Mubarak Jibril (auth.) ISBN 9783319511023, 9783319511030, 3319511025, 3319511033

https://ebooknice.com/product/error-correction-coding-and-decoding-bounds-codes-
decoders-analysis-and-applications-5839528

ebooknice.com

(Ebook) Quantum Error Correction and Fault Tolerant Quantum Computing by Frank
Gaitan ISBN 9780849371998, 9781420006681, 0849371996, 1420006681

https://ebooknice.com/product/quantum-error-correction-and-fault-tolerant-
quantum-computing-4767458

ebooknice.com
(Ebook) Trellis and Turbo Coding: Iterative and Graph-Based Error Control Coding by
Christian B. Schlegel, Lance C. Perez ISBN 9781118083161, 1118083164

https://ebooknice.com/product/trellis-and-turbo-coding-iterative-and-graph-
based-error-control-coding-6675174

ebooknice.com

(Ebook) Cryptology and Error Correction: An Algebraic Introduction and Real-World

Applications by Lindsay N. Childs ISBN 9783030154516, 9783030154530, 3030154513,
303015453X

https://ebooknice.com/product/cryptology-and-error-correction-an-algebraic-
introduction-and-real-world-applications-10486798

ebooknice.com

(Ebook) Iterative Error Correction: Turbo, Low-Density Parity-Check and Repeat-

Accumulate Codes by Sarah J. Johnson ISBN 9780521871488, 0521871484

https://ebooknice.com/product/iterative-error-correction-turbo-low-density-
parity-check-and-repeat-accumulate-codes-4572954

ebooknice.com

(Ebook) Quantum Information Processing, Quantum Computing, and Quantum Error

Correction: An Engineering Approach by Ivan B. Djordjevic ISBN 9780128219829,
0128219823

https://ebooknice.com/product/quantum-information-processing-quantum-computing-
and-quantum-error-correction-an-engineering-approach-23549368

ebooknice.com

(Ebook) Curve and surface reconstruction: Algorithms with mathematical analysis by

Tamal K. Dey ISBN 9780511261336, 9780521863704, 0511261330, 0521863708

https://ebooknice.com/product/curve-and-surface-reconstruction-algorithms-with-
mathematical-analysis-874772

ebooknice.com
Error Correction Coding
Error Correction Coding
Mathematical Methods and Algorithms

Second Edition

Todd K. Moon
Utah State University, Utah, USA
This second edition first published 2021
© 2021 by John Wiley and Sons, Inc.
Edition History
John Wiley and Sons, Inc. (1e, 2005)
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means,
electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from
this title is available at http://www.wiley.com/go/permissions.
The right of Todd K. Moon to be identified as the author of this work has been asserted in accordance with law.
Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
Editorial Office
111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this
book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or
exercises in this book. This work’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by
The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software.

While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the
accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or
promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source
of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may
provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional
services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate.
Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it
is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
Library of Congress Cataloging-in-Publication Data

Names: Moon, Todd K., author.

Title: Error correction coding : mathematical methods and algorithms /
Todd K. Moon.
Description: Second edition. | Hoboken, NJ : Wiley, 2021. | Includes
bibliographical references and index.
Identifiers: LCCN 2020027697 (print) | LCCN 2020027698 (ebook) | ISBN
9781119567479 (cloth) | ISBN 9781119567486 (adobe pdf) | ISBN
9781119567493 (epub)
Subjects: LCSH: Engineering mathematics. | Error-correcting codes
(Information theory)
Classification: LCC TA331 .M66 2021 (print) | LCC TA331 (ebook) | DDC
621.3820285/572–dc23
LC record available at https://lccn.loc.gov/2020027697
LC ebook record available at https://lccn.loc.gov/2020027698
Cover Design: Wiley
Cover Image: © Putt Sakdhnagool/Getty Images
Set in 10/12pt NimbusRomNo9L by SPi Global, Chennai, India

10 9 8 7 6 5 4 3 2 1
Contents

Preface xvii

List of Program Files xxiii

List of Laboratory Exercises xxix

List of Algorithms xxxi

List of Figures xxxiii

List of Tables xli

List of Boxes xliii

About the Companion Website xlv

Part I Introduction and Foundations 1

1 A Context for Error Correction Coding 3
1.1 Purpose of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Introduction: Where Are Codes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The Communications System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Basic Digital Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4.1 Binary Phase-Shift Keying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4.2 More General Digital Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Signal Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5.1 The Gaussian Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5.2 MAP and ML Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5.3 Special Case: Binary Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5.4 Probability of Error for Binary Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.5.5 Bounds on Performance: The Union Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.5.6 The Binary Symmetric Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.5.7 The BSC and the Gaussian Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.6 Memoryless Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.7 Simulation and Energy Considerations for Coded Signals . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.8 Some Important Definitions and a Trivial Code: Repetition Coding . . . . . . . . . . . . . . . . 27
1.8.1 Detection of Repetition Codes Over a BSC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.8.2 Soft-Decision Decoding of Repetition Codes Over the AWGN . . . . . . . . . . . . 31
1.8.3 Simulation of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.9 Hamming Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.9.1 Hard-Input Decoding Hamming Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.9.2 Other Representations of the Hamming Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.10 The Basic Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.11 Historical Milestones of Coding Theory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
1.12 A Bit of Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
1.12.1 Information- Theoretic Definitions for Discrete Random Variables . . . . . . . . . 39
1.12.2 Data Processing Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
1.12.3 Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
1.12.4 Channel Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
1.12.5 Information Theoretic Definitions for Continuous Random Variables . . . . . . . 47
1.12.6 The Channel Coding Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
vi Contents

1.12.7 “Proof” of the Channel Coding Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

1.12.8 Capacity for the Continuous-Time AWGN Channel . . . . . . . . . . . . . . . . . . . . . . 53
1.12.9 Transmission at Capacity with Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
1.12.10 The Implication of the Channel Coding Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 55
1.12.11 Non-Asymptotic Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Lab 1 Simulating a Communications Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Use of Coding in Conjunction with the BSC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Resources and Implementation Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
1.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
1.14 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Part II Block Codes 69

2 Groups and Vector Spaces 71
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.2 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.2.1 Subgroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.2.2 Cyclic Groups and the Order of an Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.2.3 Cosets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.2.4 Lagrange’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.2.5 Induced Operations; Isomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.2.6 Homomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2.3 Fields: A Prelude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.4 Review of Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
2.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3 Linear Block Codes 93

3.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2 The Generator Matrix Description of Linear Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.1 Rudimentary Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3 The Parity Check Matrix and Dual Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.1 Some Simple Bounds on Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.4 Error Detection and Correction Over Hard-Output Channels . . . . . . . . . . . . . . . . . . . . . . 99
3.4.1 Error Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.4.2 Error Correction: The Standard Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.5 Weight Distributions of Codes and Their Duals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.6 Binary Hamming Codes and Their Duals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.7 Performance of Linear Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.7.1 Error Detection Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.7.2 Error Correction Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.7.3 Performance for Soft-Decision Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.8 Erasure Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.8.1 Binary Erasure Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.9 Modifications to Linear Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.10 Best-Known Linear Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Contents vii

4 Cyclic Codes, Rings, and Polynomials 123

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.2 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.3 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.3.1 Rings of Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4 Quotient Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.5 Ideals in Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.6 Algebraic Description of Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.7 Nonsystematic Encoding and Parity Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.8 Systematic Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.9 Some Hardware Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.9.1 Computational Building Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.9.2 Sequences and Power Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.9.3 Polynomial Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.9.4 Polynomial Division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.9.5 Simultaneous Polynomial Division and Multiplication . . . . . . . . . . . . . . . . . . . . 141
4.10 Cyclic Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.11 Syndrome Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.12 Shortened Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.12.1 Method 1: Simulating the Extra Clock Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.12.2 Method 2: Changing the Error Pattern Detection Circuit . . . . . . . . . . . . . . . . . . 155
4.13 Binary CRC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.13.1 Byte-Oriented Encoding and Decoding Algorithms . . . . . . . . . . . . . . . . . . . . . . . 157
4.13.2 CRC Protecting Data Files or Data Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Appendix 4.A Linear Feedback Shift Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Appendix 4.A.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Appendix 4.A.2 Connection With Polynomial Division . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Appendix 4.A.3 Some Algebraic Properties of Shift Sequences . . . . . . . . . . . . . . . . . . . . 167
Lab 2 Polynomial Division and Linear Feedback Shift Registers . . . . . . . . . . . . . 168
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Preliminary Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Programming Part: BinLFSR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Resources and Implementation Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Programming Part: BinPolyDiv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Follow-On Ideas and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Lab 3 CRC Encoding and Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Resources and Implementation Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.14 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.15 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

5 Rudiments of Number Theory and Algebra 179

5.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
5.2 Number Theoretic Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.2.1 Divisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.2.2 The Euclidean Algorithm and Euclidean Domains . . . . . . . . . . . . . . . . . . . . . . . . 184
5.2.3 An Application of the Euclidean Algorithm: The Sugiyama Algorithm . . . . . 189
5.2.4 Congruence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
5.2.5 The 𝜙 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
5.2.6 Some Cryptographic Payoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
viii Contents

5.3 The Chinese Remainder Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

5.3.1 The CRT and Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
5.4 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
5.4.1 An Examination of ℝ and ℂ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
5.4.2 Galois Field Construction: An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
5.4.3 Connection with Linear Feedback Shift Registers . . . . . . . . . . . . . . . . . . . . . . . . 207
5.5 Galois Fields: Mathematical Facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.6 Implementing Galois Field Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.6.1 Zech Logarithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.6.2 Hardware Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
5.7 Subfields of Galois Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.8 Irreducible and Primitive Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.9 Conjugate Elements and Minimal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
5.9.1 Minimal Polynomials. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.10 Factoring xn − 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
5.11 Cyclotomic Cosets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Appendix 5.A How Many Irreducible Polynomials Are There? . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Appendix 5.A.1 Solving for Im Explicitly: The Moebius Function . . . . . . . . . . . . . . . . . 227
Lab 4 Programming the Euclidean Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Preliminary Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Lab 5 Programming Galois Field Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Preliminary Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
5.12 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5.13 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

6 BCH and Reed–Solomon Codes: Designer Cyclic Codes 241

6.1 BCH Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
6.1.1 Designing BCH Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
6.1.2 The BCH Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.1.3 Weight Distributions for Some Binary BCH Codes . . . . . . . . . . . . . . . . . . . . . . . 246
6.1.4 Asymptotic Results for BCH Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
6.2 Reed–Solomon Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.2.1 Reed–Solomon Construction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.2.2 Reed–Solomon Construction 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6.2.3 Encoding Reed–Solomon Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.2.4 MDS Codes and Weight Distributions for RS Codes . . . . . . . . . . . . . . . . . . . . . . 252
6.3 Decoding BCH and RS Codes: The General Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6.3.1 Computation of the Syndrome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6.3.2 The Error Locator Polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.3.3 Chien Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.4 Finding the Error Locator Polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6.4.1 Simplifications for Narrow-Sense Binary Codes; Peterson’s Algorithm . . . . . 258
6.4.2 Berlekamp–Massey Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
6.4.3 Characterization of LFSR Length in Massey’s Algorithm . . . . . . . . . . . . . . . . . 262
6.4.4 Simplifications for Binary Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.5 Nonbinary BCH and RS Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.5.1 Forney’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Visit https://ebooknice.com to
discover a wide range of
eBooks across various genres.
Enjoy exclusive deals and
discounts to enhance your
reading experience. Start your
digital reading journey today!
Contents ix

6.6 Euclidean Algorithm for the Error Locator Polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

6.7 Erasure Decoding for Nonbinary BCH or RS Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
6.8 Galois Field Fourier Transform Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
6.8.1 Equivalence of the Two Reed–Solomon Code Constructions . . . . . . . . . . . . . . 281
6.8.2 Frequency-Domain Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
6.9 Variations and Extensions of Reed–Solomon Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.9.1 Simple Modifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.9.2 Generalized Reed–Solomon Codes and Alternant Codes . . . . . . . . . . . . . . . . . . 283
6.9.3 Goppa Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
6.9.4 Decoding Alternant Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.9.5 Cryptographic Connections: The McEliece Public Key Cryptosystem . . . . . . 286
Lab 6 Programming the Berlekamp–Massey Algorithm . . . . . . . . . . . . . . . . . . . . . . . 287
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Preliminary Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Resources and Implementation Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Lab 7 Programming the BCH Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Preliminary Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Resources and Implementation Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Follow-On Ideas and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Lab 8 Reed–Solomon Encoding and Decoding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Appendix 6.A Proof of Newton’s Identities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
6.11 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
6.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

7 Alternate Decoding Algorithms for Reed–Solomon Codes 299

7.1 Introduction: Workload for Reed–Solomon Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
7.2 Derivations of Welch–Berlekamp Key Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
7.2.1 The Welch–Berlekamp Derivation of the WB Key Equation . . . . . . . . . . . . . . . 300
7.2.2 Derivation from the Conventional Key Equation . . . . . . . . . . . . . . . . . . . . . . . . . 304
7.3 Finding the Error Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
7.4 Methods of Solving the WB Key Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
7.4.1 Background: Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
7.4.2 The Welch–Berlekamp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
7.4.3 A Modular Approach to the Solution of the WB Key Equation . . . . . . . . . . . . . 315
7.5 Erasure Decoding with the WB Key Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.6 The Guruswami–Sudan Decoding Algorithm and Soft RS Decoding . . . . . . . . . . . . . . . 326
7.6.1 Bounded Distance, ML, and List Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.6.2 Error Correction by Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
7.6.3 Polynomials in Two Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
7.6.4 The GS Decoder: The Main Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
7.6.5 Algorithms for Computing the Interpolation Step . . . . . . . . . . . . . . . . . . . . . . . . . 340
7.6.6 A Special Case: m = 1 and L = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
7.6.7 An Algorithm for the Factorization Step: The Roth–Ruckenstein Algorithm . 353
7.6.8 Soft-Decision Decoding of Reed–Solomon Codes . . . . . . . . . . . . . . . . . . . . . . . . 358
7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
7.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
x Contents

8 Other Important Block Codes 371

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
8.2 Hadamard Matrices, Codes, and Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
8.2.1 Introduction to Hadamard Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
8.2.2 The Paley Construction of Hadamard Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
8.2.3 Hadamard Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
8.3 Reed–Muller Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
8.3.1 Boolean Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
8.3.2 Definition of the Reed–Muller Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
8.3.3 Encoding and Decoding Algorithms for First-Order RM Codes . . . . . . . . . . . . 380
8.3.4 The Reed Decoding Algorithm for RM(r, m) Codes, r ≥ 1 . . . . . . . . . . . . . . . . . 385
8.3.5 Other Constructions of Reed–Muller Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
8.4 Building Long Codes from Short Codes: The Squaring Construction . . . . . . . . . . . . . . . 393
8.5 Quadratic Residue Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
8.6 Golay Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
8.6.1 Decoding the Golay Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
8.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
8.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

9 Bounds on Codes 407

9.1 The Gilbert–Varshamov Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
9.2 The Plotkin Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
9.3 The Griesmer Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
9.4 The Linear Programming and Related Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
9.4.1 Krawtchouk Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
9.4.2 Character . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
9.4.3 Krawtchouk Polynomials and Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
9.5 The McEliece–Rodemich–Rumsey–Welch Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
9.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
9.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

10 Bursty Channels, Interleavers, and Concatenation 425

10.1 Introduction to Bursty Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
10.2 Interleavers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
10.3 An Application of Interleaved RS Codes: Compact Discs . . . . . . . . . . . . . . . . . . . . . . . . . 428
10.4 Product Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
10.5 Reed–Solomon Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
10.6 Concatenated Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
10.7 Fire Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
10.7.1 Fire Code Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
10.7.2 Decoding Fire Codes: Error Trapping Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . 434
10.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
10.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437

11 Soft-Decision Decoding Algorithms 439

11.1 Introduction and General Notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
11.2 Generalized Minimum Distance Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
11.2.1 Distance Measures and Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
11.3 The Chase Decoding Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
11.4 Halting the Search: An Optimality Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
11.5 Ordered Statistic Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
11.6 Soft Decoding Using the Dual Code: The Hartmann Rudolph Algorithm . . . . . . . . . . . 449
Contents xi

11.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

11.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

Part III Codes on Graphs 453

12 Convolutional Codes 455
12.1 Introduction and Basic Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
12.1.1 The State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
12.2 Definition of Codes and Equivalent Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
12.2.1 Catastrophic Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
12.2.2 Polynomial and Rational Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
12.2.3 Constraint Length and Minimal Encoders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
12.2.4 Systematic Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
12.3 Decoding Convolutional Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
12.3.1 Introduction and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
12.3.2 The Viterbi Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
12.3.3 Some Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
12.4 Some Performance Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
12.5 Error Analysis for Convolutional Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
12.5.1 Enumerating Paths Through the Trellis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
12.5.2 Characterizing Node Error Probability Pe and Bit Error Rate Pb . . . . . . . . . . . . 499
12.5.3 A Bound on Pd for Discrete Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501
12.5.4 A Bound on Pd for BPSK Signaling Over the AWGN Channel . . . . . . . . . . . . 503
12.5.5 Asymptotic Coding Gain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
12.6 Tables of Good Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
12.7 Puncturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
12.7.1 Puncturing to Achieve Variable Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508
12.8 Suboptimal Decoding Algorithms for Convolutional Codes . . . . . . . . . . . . . . . . . . . . . . . 509
12.8.1 Tree Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
12.8.2 The Fano Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
12.8.3 The Stack Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
12.8.4 The Fano Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
12.8.5 Other Issues for Sequential Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
12.8.6 A Variation on the Viterbi Algorithm: The M Algorithm . . . . . . . . . . . . . . . . . . 524
12.9 Convolutional Codes as Block Codes and Tailbiting Codes . . . . . . . . . . . . . . . . . . . . . . . 524
12.10 A Modified Expression for the Path Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
12.11 Soft Output Viterbi Algorithm (SOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
12.12 Trellis Representations of Block and Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
12.12.1 Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
12.12.2 Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
12.12.3 Trellis Decoding of Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
Lab 9 Programming Convolutional Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
Lab 10 Convolutional Decoders: The Viterbi Algorithm . . . . . . . . . . . . . . . . . . . . . . . 537
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
xii Contents

12.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538

12.14 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543

13 Trellis-Coded Modulation 545

13.1 Adding Redundancy by Adding Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
13.2 Background on Signal Constellations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
13.3 TCM Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
13.3.1 The General Ungerboeck Coding Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553
13.3.2 The Set Partitioning Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553
13.4 Some Error Analysis for TCM Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
13.4.1 General Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
13.4.2 A Description of the Error Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
13.4.3 Known Good TCM Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560
13.5 Decoding TCM Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
13.6 Rotational Invariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
13.6.1 Differential Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
13.6.2 Constellation Labels and Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
13.7 Multidimensional TCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
13.7.1 Some Advantages of Multidimensional TCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
13.7.2 Lattices and Sublattices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
13.8 Multidimensional TCM Example: The V.34 Modem Standard. . . . . . . . . . . . . . . . . . . . . 578
13.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
Lab 11 Trellis-Coded Modulation Encoding and Decoding. . . . . . . . . . . . . . . . . . . . 586
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
13.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586

Part IV Iteratively Decoded Codes 589

14 Turbo Codes 591
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
14.2 Encoding Parallel Concatenated Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
14.3 Turbo Decoding Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
14.3.1 The MAP Decoding Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
14.3.2 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
14.3.3 Posterior Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598
14.3.4 Computing 𝛼t and 𝛽t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 600
14.3.5 Computing 𝛾t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
14.3.6 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602
14.3.7 Summary of the BCJR Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
14.3.8 A Matrix/Vector Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
14.3.9 Comparison of the Viterbi Algorithm and the BCJR Algorithm . . . . . . . . . . . . 605
14.3.10 The BCJR Algorithm for Systematic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606
14.3.11 Turbo Decoding Using the BCJR Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
14.3.12 Likelihood Ratio Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
14.3.13 Statement of the Turbo Decoding Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612
14.3.14 Turbo Decoding Stopping Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612
14.3.15 Modifications of the MAP Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
14.3.16 Corrections to the Max-Log-MAP Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
14.3.17 The Soft-Output Viterbi Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
Contents xiii

14.4 On the Error Floor and Weight Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616

14.4.1 The Error Floor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
14.4.2 Spectral Thinning and Random Permuters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
14.4.3 On Permuters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621
14.5 EXIT Chart Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622
14.5.1 The EXIT Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
14.6 Block Turbo Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
14.7 Turbo Equalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628
14.7.1 Introduction to Turbo Equalization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628
14.7.2 The Framework for Turbo Equalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629
Lab 12 Turbo Code Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
Programming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
14.8 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
14.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634

15 Low-Density Parity-Check Codes: Introduction, Decoding, and Analysis 637

15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637
15.2 LDPC Codes: Construction and Notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637
15.3 Tanner Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641
15.4 Decoding LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642
15.4.1 Decoding Using Log-Likelihood Ratios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642
15.4.2 Decoding Using Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652
15.4.3 Variations on Decoding Algorithms: The Min-Sum Decoder . . . . . . . . . . . . . . 659
15.4.4 Variations on Decoding Algorithms: Min-Sum with Correction . . . . . . . . . . . . 662
15.4.5 Hard-Decision Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
15.4.6 Divide and Concur Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
15.4.7 Difference Map Belief Propagation Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
15.4.8 Linear Programming Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682
15.4.9 Decoding on the Binary Erasure Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 691
15.4.10 BEC Channels and Stopping Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 692
15.5 Why Low-Density Parity-Check Codes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
15.6 The Iterative Decoder on General Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695
15.7 Density Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696
15.8 EXIT Charts for LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699
15.9 Irregular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 702
15.9.1 Degree Distribution Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703
15.9.2 Density Evolution for Irregular Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
15.9.3 Computation and Optimization of Density Evolution . . . . . . . . . . . . . . . . . . . . . 706
15.9.4 Using Irregular Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
15.10 More on LDPC Code Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
15.11 Encoding LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
15.12 A Variation: Low-Density Generator Matrix Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710
Lab 13 Programming an LDPC Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710
Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710
Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
Numerical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
15.13 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712
15.14 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715
xiv Contents

16 Low-Density Parity-Check Codes: Designs and Variations 717

16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
16.2 Repeat-Accumulate Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
16.2.1 Decoding RA Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 720
16.2.2 Irregular RA Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 721
16.2.3 RA Codes with Multiple Accumulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722
16.3 LDPC Convolutional Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723
16.4 Quasi-Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725
16.4.1 QC Generator Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725
16.4.2 Constructing QC Generator Matrices from QC Parity Check Matrices . . . . . . 727
16.5 Construction of LDPC Codes Based on Finite Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 730
16.5.1 I. Construction of QC-LDPC Codes Based on the Minimum-Weight
Codewords of a Reed–Solomon Code with Two Information Symbols . . . . . . 731
16.5.2 II. Construction of QC-LDPC Codes Based on a Special Subclass of
RS Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
16.5.3 III. Construction of QC-LDPC Codes Based on Subgroups of a Finite Field . 734
16.5.4 IV. Construction of QC-LDPC Codes Based on Subgroups of the
Multiplicative Group of a Finite Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735
16.5.5 Construction of QC-LDPC Codes Based on Primitive Elements of a Field . . 736
16.6 Code Design Based on Finite Geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736
16.6.1 Rudiments of Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736
16.6.2 A Family of Cyclic EG-LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
16.6.3 Construction of LDPC Codes Based on Parallel Bundles of Lines . . . . . . . . . . 741
16.6.4 Constructions Based on Other Combinatoric Objects . . . . . . . . . . . . . . . . . . . . . 742
16.7 Ensembles of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
16.7.1 Regular ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743
16.7.2 Irregular Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743
16.7.3 Multi-edge-type Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745
16.8 Constructing LDPC Codes by Progressive Edge Growth (PEG) . . . . . . . . . . . . . . . . . . . 746
16.9 Protograph and Multi-Edge-Type LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 749
16.10 Error Floors and Trapping Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 750
16.11 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
16.11.1 Importance Sampling: General Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752
16.11.2 Importance Sampling for Estimating Error Probability . . . . . . . . . . . . . . . . . . . . 754
Conventional Sampling (MC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755
Importance Sampling (IS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756
16.11.3 Importance Sampling for Tanner Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765
16.12 Fountain Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771
16.12.1 Conventional Erasure Correction Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772
16.12.2 Tornado Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772
16.12.3 Luby Transform Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774
16.12.4 Raptor Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774
16.13 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774

Part V Polar Codes 777

17 Polar Codes 779
17.1 Introduction and Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779
17.2 Notation and Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 781
17.3 Channel Polarization, N = 2 Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782
17.3.1 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782
17.3.2 Synthetic Channels and Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783
Contents xv

17.3.3 Synthetic Channel Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786

17.3.4 An Example with N = 2 Using the Binary Erasure Channel . . . . . . . . . . . . . . . 786
17.3.5 Successive Cancellation Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787
17.3.6 Tree Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792
17.3.7 The Polar Coding Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793
17.4 Channel Polarization, N > 2 Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793
17.4.1 Channel Combining: Extension from N ∕ 2 to N channels. . . . . . . . . . . . . . . . . . 793
17.4.2 Pseudocode for Encoder for Arbitrary N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 797
17.4.3 Transition Probabilities and Channel Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . 798
17.4.4 Channel Polarization Demonstrated: An Example Using the Binary Erasure
Channel for N > 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 802
17.4.5 Polar Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805
17.4.6 Tree Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807
17.4.7 Successive Cancellation Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807
17.4.8 SC Decoding from a Message Passing Point of View on the Tree . . . . . . . . . . 810
17.4.9 A Decoding Example with N = 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 811
17.4.10 A Decoding Example with N = 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813
17.4.11 Pseudo-Code Description of Successive Cancellation Algorithm . . . . . . . . . . . 823
17.5 Some Theorems of Polar Coding Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826
17.5.1 I(W) and Z(W) for general B-DMCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826
17.5.2 Channel Polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 827
17.5.3 The Polarization Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 830
17.5.4 A Few Facts About Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 830
17.5.5 Proof of the Polarization Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 831
17.5.6 Another Polarization Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832
17.5.7 Rate of Polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
17.5.8 Probability of Error Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
17.6 Designing Polar Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 837
17.6.1 Code Design by Battacharyya Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 837
17.6.2 Monte Carlo Estimation of Z(WN(i) ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838
17.7 Perspective: The Channel Coding Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839
17.8 Systematic Encoding of Polar Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839
17.8.1 Polar Systematic Encoding via the Encoder Graph . . . . . . . . . . . . . . . . . . . . . . . . 840
17.8.2 Polar Systematic Encoding via Arıkan’s Method . . . . . . . . . . . . . . . . . . . . . . . . . 842
17.8.3 Systematic Encoding: The Bit Reverse Permutation . . . . . . . . . . . . . . . . . . . . . . 843
17.8.4 Decoding Systematically Encoded Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
17.8.5 Flexible Systematic Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
17.8.6 Involutions and Domination Contiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846
17.8.7 Polar Codes and Domination Contiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 847
17.8.8 Modifications for Polar Codes with Bit-Reverse Permutation . . . . . . . . . . . . . . 850
17.9 List Decoding of Polar Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 850
17.9.1 The Likelihood Data Structure P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 851
17.9.2 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853
17.9.3 Code to Compute P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854
17.9.4 The Bits Data Structure C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855
17.9.5 Code to Compute C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856
17.9.6 Supporting Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 857
17.9.7 Code for Polar List Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 857
17.9.8 An Example of List Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 862
17.9.9 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864
17.9.10 Modified Polar Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865
17.10 LLR-Based Successive Cancellation List Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866
17.10.1 Implementation Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867
xvi Contents

17.11 Simplified Successive Cancellation Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867

17.11.1 Modified SC Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 870
17.12 Relations with Reed–Muller Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 871
17.13 Hardware and Throughput Performance Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 871
17.14 Generalizations, Extensions, and Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 871
Appendix 17.A BN is a Bit-Reverse Permutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 872
Appendix 17.B The Relationship of the Battacharyya Parameter to Channel Capacity . . . . . . 873
Appendix 17.B.1 Error Probability for Two Codewords . . . . . . . . . . . . . . . . . . . . . . . . . . 873
Appendix 17.B.2 Proof of Inequality (17.59) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874
Appendix 17.B.3 Proof of Inequality (17.60) [16] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 879
Appendix 17.C Proof of Theorem 17.12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 880
17.15 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 882
17.16 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 883

Part VI Applications 885

18 Some Applications of Error Correction in Modern Communication Systems 887
18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 887
18.2 Digital Video Broadcast T2 (DVB-T2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 887
18.2.1 BCH Outer Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 887
18.2.2 LDPC Inner Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 888
18.3 Digital Cable Television . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892
18.4 E-UTRA and Long-Term Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
18.4.1 LTE Rate 1/3 Convolutional Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897
18.4.2 LTE Turbo Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897
18.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 898

Part VII Space-Time Coding 899

19 Fading Channels and Space-Time Codes 901
19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 901
19.2 Fading Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 901
19.2.1 Rayleigh Fading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904
19.3 Diversity Transmission and Reception: The MIMO Channel . . . . . . . . . . . . . . . . . . . . . . 905
19.3.1 The Narrowband MIMO Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 907
19.3.2 Diversity Performance with Maximal-Ratio Combining . . . . . . . . . . . . . . . . . . . 908
19.4 Space-Time Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 909
19.4.1 The Alamouti Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 909
19.4.2 A More General Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 911
19.4.3 Performance Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 911
19.4.4 Complex Orthogonal Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916
19.5 Space-Time Trellis Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917
19.5.1 Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 919
19.6 How Many Antennas? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 919
19.7 Estimating Channel Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 921
19.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 922
19.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 923

References 925

Index 939
Preface
The goals of this second edition are much the same as the first edition: to provide a comprehensive
introduction to error correction coding suitable for the engineering practitioner and to provide a solid
foundation leading to more advanced treatments or research. Since the first edition, the content has
been modernized to include substantially more on LDPC code design and decoder algorithms, as well
as substantial coverage of polar codes. The thorough introduction to finite fields and algebraic codes
of the first edition has been retained, with some additions in finite geometries used for LDPC code
design. As observed in the first edition, the sophistication of the mathematical tools used increases
over time. While keeping with the sense that this is the first time most readers will have seen these
tools, a somewhat higher degree of sophistication is needed in some places.
The presentation is intended to provide a background useful both to engineers, who need to
understand algorithmic aspects for the deployment and implementation of error correction coding, and
to researchers, who need sufficient background to prepare them to read, understand, and ultimately
contribute to the research literature. The practical algorithmic aspects are built upon a firm foundation
of mathematics, which are carefully motivated and developed.

Pedagogical Features
Since its inception, coding theory has drawn from richly interacting variety of mathematical areas,
including detection theory, information theory, linear algebra, finite geometries, combinatorics,
optimization, system theory, probability, algebraic geometry, graph theory, statistical designs, Boolean
functions, number theory, and modern algebra. The level of sophistication has increased over
time: algebra has progressed from vector spaces to modules; practice has moved from polynomial
interpolation to rational interpolation; Viterbi makes way for SOVA and BCJR. This richness can
be bewildering to students, particularly engineering students who may be unaccustomed to posing
problems and thinking abstractly. It is important, therefore, to motivate the mathematics carefully.
Some of the major pedagogical features of the book are as follows.

• While most engineering-oriented error-correction-coding textbooks clump the major mathemat-

ical concepts into a single chapter, in this book the concepts are developed over several chapters
so they can be put to more immediate use. I have attempted to present the mathematics “just in
time,” when they are needed and well-motivated. Groups and linear algebra suffice to describe
linear block codes. Cyclic codes motivate polynomial rings. The design of cyclic codes moti-
vates finite fields and associated number-theoretical tools. By interspersing the mathematical
concepts with applications, a deeper and broader understanding is possible.
• For most engineering students, finite fields, the Berlekamp–Massey algorithm, the Viterbi
algorithm, BCJR, and other aspects of coding theory are abstract and subtle. Software imple-
mentations of the algorithms bring these abstractions closer to a meaningful reality, bringing
deeper understanding than is possible by simply working homework problems and taking tests.
Even when students grasp the concepts well enough to do homework on paper, these programs
provide a further emphasis, as well as tools to help with the homework. The understanding
becomes experiential, more than merely conceptual.
Understanding of any subject typically improves when the student him- or herself has the chance
to teach the material to someone (or something) else. A student must develop an especially clear
understanding of a concept in order to “teach” it to something as dim-witted and literal-minded
as a computer. In this process the computer can provide feedback to the student through
debugging and program testing that reinforces understanding.
In the coding courses I teach, students implement a variety of encoders and decoders, including
Reed–Solomon encoders and decoders, convolutional encoders, turbo code decoders, and
LDPC decoders. As a result of these programming activities, students move beyond an on-paper
understanding, gaining a perspective of what coding theory can do and how to put it to work. A
colleague of mine observed that many students emerge from a first course in coding theory more
xviii Preface

α α2 α3 α4 α5 α6 α7 α8 α9
confused than informed. My experience with these programming exercises is that my students
+
are, if anything, overconfident, and feel ready to take on a variety of challenges.
01010 00000
In this book, programming exercises are presented in a series of 13 Laboratory Exercises. These
α10α11α12α13α14α15α16
are supported with code providing most of the software “infrastructure,” allowing students to
LabIntro.pdf
focus on the particular algorithm they are implementing.
These labs also help with the coverage of the course material. In my course I am able to offload
classroom instruction of some topics for students to read, with the assurance that the students
will learn it solidly on their own as they implement it. (The Euclidean algorithm is one of these
topics in my course.)
Research in error control coding can benefit from having a flexible library of tools for the
computations, particularly since analytical results are frequently not available and simulations
are required. The laboratory assignments presented here can form the foundation for a research
library, with the added benefit that having written major components, the researcher can easily
modify and extend them.
It is in light of these pedagogic features that this book bears the subtitle Mathematical Methods
and Algorithms.

There is sufficient material in this book for a one- or two-semester course based on the book, even
for instructors who prefer to focus less on implementational aspects and the laboratories.
Over 200 programs, functions and data files are associated with the text. The programs are written
in MATLAB,1 C, or C++. Some of these include complete executables which provide “tables” of
primitive polynomials (over any prime field), cyclotomic cosets and minimal polynomials, and BCH
codes (not just narrow sense), avoiding the need to tabulate this material. Other functions include those
used to make plots and compute results in the book. These provide example of how the theory is put
into practice. Other functions include those used for the laboratory exercises. The files are highlighted
in the book by the icon
α α2 α3 α4 α5 α6 α7 α8 α9
+

01010 00000

α10 α11 α12 α13 α14 α15 α16

as in the marginal note above. The files are available at https://github.com/tkmoon/eccbook.

Other aspects of the book include the following:

• Many recent advances in coding have resulted from returning to the perspective of coding as
a detection problem. Accordingly, the book starts off with a digital communication framework
with a discussion of detection theory.
• Recent codes such as polar codes or LDPC codes are capable of achieving capacity, or nearly so.
It is important, therefore, to understand what capacity is and what it means to transmit at capacity.
Chapter 1 also summarizes information theory, to put coding into its historical and modern
context. Added in this second edition is an introduction to nonasymptotic information theory,
which will be increasingly important in developing communication systems. The information
theory informs EXIT chart analysis of turbo and LDPC codes.

1 MATLAB is a registered trademark of The Mathworks, Inc.

Visit https://ebooknice.com to
discover a wide range of
eBooks across various genres.
Enjoy exclusive deals and
discounts to enhance your
reading experience. Start your
digital reading journey today!
Preface xix

• Pedagogically, Hamming codes are used to set the stage for the book by using them to
demonstrate block codes, cyclic codes, trellises and Tanner graphs.
• Homework exercises are drawn from a variety of sources and are at a variety of levels. Some
are numerical, testing basic understanding of concepts. Others provide the opportunity to prove
or extend results from the text. Others extend concepts or provide new results. Because of the
programming laboratories, exercises requiring decoding by hand of given bit sequences are few,
since I am of the opinion that is better to know how to tell the computer than to do it by hand.
I have drawn these exercises from a variety of sources, including problems that I faced as a
student and those which I have given to students on homework and exams over the years.
• Number theoretic concepts such as divisibility, congruence, and the Chinese remainder theorem
are developed at a point in the development where students can appreciate it.
• Occasionally connections between the coding theoretic concepts and related topics are pointed
out, such as public key cryptography and shift register sequences. These add spice and
motivate students with the understanding that the tools they are learning have broad applicability.
• There has been considerable recent progress made in decoding Reed–Solomon codes by
re-examining their original definition. Accordingly, Reed–Solomon codes are defined both
in this primordial way (as the image of a polynomial function) and also using a generator
polynomial having roots that are consecutive powers of a primitive element. This sets the
stage for several decoding algorithms for Reed–Solomon codes, including frequency-domain
algorithms, Welch–Berlekamp algorithm and the soft-input Guruswami–Sudan algorithm.
• Turbo codes, including EXIT chart analysis, are presented, with both BCJR and SOVA
decoding algorithms. Both probabilistic and likelihood decoding viewpoints are presented.
• LDPC codes are presented with an emphasis on the decoding algorithm. Density evolution
analysis is also presented.
• Polar Codes are a family of codes developed since the first edition. They offer low complexity
encode and decode with soft inputs and without iterative decoding. They also provably achieve
channel capacity (as the code length goes to infinity). This book offers a deep introduction to
polar codes, including careful description of encoding and decoding algorithms, with operational
C++ code.
• Several Applications involving state-of-the-art systems illustrate how the concepts can be
applied.
• Space-time codes, used for multi-antenna systems in fading channels, are presented.

Courses of Study
A variety of courses of study are possible. In the one-semester course I teach, I move quickly through
principal topics of block, trellis, and iteratively–decoded codes. Here is an outline of one possible
one-semester course:

Chapter 1: Major topics only.

Chapter 2: All.
Chapter 3: Major topics.
Chapter 4: Most. Leave CRC codes and LFSR to labs.
Chapter 5: Most. Touch on RSA.
Chapter 6: Most. Light on GFFT and variations.
Chapter 12: Most. Skip puncturing and stack-oriented algorithms.
xx Preface

Chapter 13: Most. Skip the V.34 material.

Chapter 14: Basic definition and the BCJR algorithm.
Chapter 15: Basic definition of LDPC codes and the sum-product decoder.
Chapter 17: Introduce the idea of

A guide in selecting material for this course is: follow the labs. To get through all 13 labs, selectivity
is necessary.
An alternative two-semester course could be a semester devoted to block codes followed by a
semester on trellis and iteratively decoded codes. A two semester sequence could move straight through
the book, with possible supplements from the research literature on topics of particular interest to
the instructor. A different two-semester approach would follow the outline above for one semester,
followed by more advanced coverage of LDPC and polar codes.
Theorems, lemmas, corollaries, examples, and definitions are all numbered sequentially using the
same counter in a chapter, which should make identifying these environments straightforward. Figures,
tables, and equations each have their own counters. Definitions, examples, and proofs are terminated
by the symbol ◽.

Use of Computers
α α2 α3 α4 α5 α6 α7 α8 α9
+
The computer-based labs provide a means of working out some of the computational details that
01010 00000
otherwise might require drudgery. These are primarily (with the exception of the first lab) to be done
in C++, where the ability to overload operators and run at compiled speed make the language very
α10α11α12α13α14α15α16
well suited.
comptut.pdf
It may be helpful to have a “desk calculator” for homework and exploring ideas. Many tools exist
now for this. The brief tutorial comptut.pdf provides an introduction to gap and magma, both of
which can be helpful to students doing homework or research in this area. The sage package built on
Python also provides considerable relevant capability.

Why This Book?

In my mind, the purposes of a textbook are these:

1. To provide a topographical map into the field of study, showing the peaks and the important
roads to them. (However, in an area as rich as coding theory, it is impossible to be exhaustive.)
2. To provide specific details about important techniques.
3. To present challenging exercises that will strengthen students’ understanding of the material
and present new perspectives.
4. To have some reference value, so that practitioners can continue to use it.
5. To provide references to literature that will lead readers to make their own discoveries. With
a rapidly-changing field, the references can only provide highlights; web-based searches have
changed the nature of the game. Nevertheless, having a starting point in the literature is still
important.
6. To present a breadth of ideas which will motivate students to innovate on their own.

A significant difficulty I have faced is selection. The terrain is so richly textured that it cannot
be mapped out in a single book. Every communication or information theory conference and every
issue of the IEEE Transactions on Information Theory yields new and significant results. Publishing
restrictions and practicality limit this book from being encyclopedic. My role as author has been merely
to select what parts of the map to include and to present them in a pedagogically useful way. In so
Preface xxi

doing, I have aimed to choose tools for the general practitioner and student (and of interest to me).
Other than that selective role, no claim of creation is intended; I hope I have given credit as appropriate
where it is due.
This book is a result of teaching a course in error correction coding at Utah State University for
nearly three decades. Over that time, I have taught out of the books [45], [489], and [275], and my
debt to these books is clear. Parts of some chapters grew out of lecture notes based on these books
and the connections will be obvious. I have felt compelled to include many of the exercises from
the first coding course I took out of [275]. These books have defined for me the sine qua non of
error-correction coding texts. I am also indebted to [296] for its rich theoretical treatment, [401] for
presentation of trellis coding material, [462] for discussion of bounds, [190] for exhaustive treatment
of turbo coding methods, and to the many great researchers and outstanding expositors whose works
have contributed to my understanding. More recently, the extensive coverage of LDPC codes in [391]
has been extremely helpful, and motivated my own implementation of almost all of the decoding
algorithms.

Acknowledgments
I am grateful for the supportive environment at Utah State University that has made it possible to
undertake and to complete this task. Students in coding classes over the years have contributed to this
material.
I have benefitted tremendously from feedback from careful readers and translators who have
provided their own error correction from the first edition. Their negative acknowledgment protocols
have improved the delivery of this packet of information. I thank you all!
To my six wonderful children — Leslie, Kyra, Kaylie, Jennie, Kiana, and Spencer — and presently
twelve grandchildren, and my wife Barbara, who have seen me slip away too often and too long to
write, I express my deep gratitude for their trust and patience. In the end, all I do is for them.

T.K.M
Logan, UT, October 2020
List of Program Files
comptut.pdf Introduction to gap and magma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
bpskprobplot.m Set up to plot prob. of error for BPSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
bpskprob.m Plot probability of error for BPSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
repcodeprob.m Error prob. for (n, 1) repetition codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
testrepcode.cc Test the repetition code performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
repcodes.m Plot results for repetition code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
mindist.m Find minimum distance using exhaustive search . . . . . . . . . . . . . . . . . . . . 33
hamcode74pe.m Probability
( of) error for (7, 4) Hamming code . . . . . . . . . . . . . . . . . . . . . . . 35
n
nchoosektest.m Compute k
and test for k < 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
plotcapcmp.m Capacity of the AWGN and BAWGNC channels . . . . . . . . . . . . . . . . . . . 49
cawgnc2.m Compute capacity of AWGN channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
cbawgnc2.m Compute the capacity of the BAWGN channel . . . . . . . . . . . . . . . . . . . . . 49
h2.m Compute the binary entropy function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
plotcbawn2.m Plot capacity for AWGN and BAWGN channels . . . . . . . . . . . . . . . . . . . 54
cbawgnc.m Compute capacity for BAWGN channel . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
cawgnc.m Compute capacity for AWGN channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
philog.m Log 𝜙 function for the BAWGNC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
phifun.m 𝜙 function for the BAWGNC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
gaussj2 Gaussian elimination over GF(2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Hammsphere Number of points in a Hamming sphere . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
genstdarray.c Generate a standard array for a code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
progdetH15.m Error detection prob. for (15, 11) Hamming code . . . . . . . . . . . . . . . . . . . 105
progdet.m Error detection prob. for (31, 21) Hamming code . . . . . . . . . . . . . . . . . . . 105
polyadd.m Add polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polysub.m Subtract polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polymult.m Multiply polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polydiv.m Polynomial quotient and remainder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polyaddm.m Add polynomials modulo a number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polysubm.m Subtract polynomials modulo a number . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
polymultm.m Multiply polynomials modulo a number . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
primitive.txt Table of primitive polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
primfind.c Program to find primitive polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
BinLFSR.h (lab, complete) Binary LFSR class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
BinLFSR.cc (lab, complete) Binary LFSR class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
testBinLFSR.cc (lab, complete) Binary LFSR class tester . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
MakeLFSR (lab, complete) Makefile for tester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
BinPolyDiv.h (lab, complete) Binary polynomial division . . . . . . . . . . . . . . . . . . . . . . . . 170
BinPolyDiv.cc (lab, incomplete) Binary polynomial division . . . . . . . . . . . . . . . . . . . . . . 170
testBinPolyDiv.cc (lab, complete) Binary polynomial division test . . . . . . . . . . . . . . . . . . . . . 170
gcd.c A simple example of the Euclidean algorithm . . . . . . . . . . . . . . . . . . . . . . 177
crtgamma.m Compute the gammas for the CRT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
fromcrt.m Convert from CRT representation to integer . . . . . . . . . . . . . . . . . . . . . . . . 197
tocrt.m Compute the CRT representation of an integer . . . . . . . . . . . . . . . . . . . . . 197
testcrt.m An example of CRT calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
testcrp.m Test a polynomial CRT calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
tocrtpoly.m CRT representation of a polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
fromcrtpolym. Polynomial CRT representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
crtgammapoly.m Gammas for the CRT representation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
primfind Find primitive polynomials in GF(p)[x] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
cyclomin Cyclotomic cosets and minimal polynomials . . . . . . . . . . . . . . . . . . . . . . . 223
ModArnew.h Templatized Modulo arithmetic class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
testmodarnew1.cc Templatized Modulo arithmetic class tester . . . . . . . . . . . . . . . . . . . . . . . . 228
xxiv List of Program Files

polynomialT.h (lab, complete) Templatized polynomial class . . . . . . . . . . . . . . . . . . . . . . . . 229

polynomialT.cc (lab, complete) Templatized polynomial class . . . . . . . . . . . . . . . . . . . . . . . . 229
testpoly1.cc Demonstrate templatized polynomial class . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
testgcdpoly.cc (lab, complete) Test polynomial GCD function . . . . . . . . . . . . . . . . . . . . . . . 229
gcdpoly.cc Polynomial GCD function (incomplete) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
GF2.h (lab, complete) GF(2) class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
GFNUM2m.h (lab, complete) Galois field GF(2m ) class . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
GFNUM2m.cc (lab, incomplete) Galois field GF(2m ) class . . . . . . . . . . . . . . . . . . . . . . . . . . 230
testgfnum.cc (lab, complete) Test Galois field class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
bchweight.m Weight dist. of BCH code from weight of dual . . . . . . . . . . . . . . . . . . . . . . . 247
bchdesigner Design a t-error correcting binary BCH code . . . . . . . . . . . . . . . . . . . . . . . . . 248
reedsolwt.m Weight distribution for an (n, k) RS code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
masseymodM.m Return the shortest LFSR for data sequence . . . . . . . . . . . . . . . . . . . . . . . . . . 265
testRSerasure.cc RS with erasures using Euclidean alg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
erase.mag Erasure decoding example in magma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
testBM.cc Test the Berlekamp–Massey algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Chiensearch.h (lab, complete) Chien Search class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Chiensearch.cc (lab, incomplete) Chien Search class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
testChien.cc (lab, complete) Test the Chien Search class . . . . . . . . . . . . . . . . . . . . . . . . . . 289
BCHdec.h (lab, complete) BCHdec decoder class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
BCHdec.cc (lab, incomplete) BCHdec decoder class . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
testBCH.cc (lab, complete) BCHdec decoder class tester . . . . . . . . . . . . . . . . . . . . . . . . . 289
RSdec.h (lab, complete) RS decoder class header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
RSdec.cc (lab, incomplete) RS decoder class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
testRS.cc (lab, complete) Test RS decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
rsencode.cc (lab, complete) Encode a file using RS encoder . . . . . . . . . . . . . . . . . . . . . . . 291
rsdecode.cc (lab, complete) Decode a file using RS decoder . . . . . . . . . . . . . . . . . . . . . . 291
bsc Simulate a binary symmetric channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
testpxy.cc Concepts relating to 2-variable polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . 328
computekm.m Compute Km for the Guruswami–Sudan decoder . . . . . . . . . . . . . . . . . . . . . 336
computeLm.cc Maximum list length for GS(m) decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
computeLm.m Maximum list length for GS(m) decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
testft.m Test the Feng–Tzeng algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
fengtzeng.m Poly. s.t. the first l + 1 columns are lin. dep. . . . . . . . . . . . . . . . . . . . . . . . . . 344
invmodp.m Compute the inverse of a number modulo p . . . . . . . . . . . . . . . . . . . . . . . . . . 344
testGS1.cc Test the GS decoder (Kotter part) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
kotter.cc Kotter interpolation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
testGS3.cc Test the GS decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
testGS5.cc Test the GS decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
kotter1.cc Kotter algorithm with m = 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
testGS2.cc Test the Roth–Ruckenstein algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
rothruck.cc Roth–Ruckenstein algorithm (find y-roots) . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
rothruck.h Roth–Ruckenstein algorithm (find y-roots) . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
Lbarex.m Average performance of a GS(m) decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
computetm.m Tm error correction capability for GS decoder . . . . . . . . . . . . . . . . . . . . . . . . 358
computeLbar.m Avg. no. of CWs around random pt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
computeLm.m Compute max. length of list of decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
pi2m1 Koetter–Vardy map reliability to multiplicity . . . . . . . . . . . . . . . . . . . . . . . . 363
genrm.cc Create a Reed–Muller generator matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
List of Program Files xxv

rmdecex.m RM(1, 3) decoding example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

hadex.m Computation of H8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
testfht.cc Test the fast Hadamard transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
fht.cc Fast Hadamard transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
fht.m Fast Hadamard transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
rmdecex2.m RM(2, 4) decoding example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
testQR.cc Example of arithmetic for QR code decoding . . . . . . . . . . . . . . . . . . . . . . . 398
golaysimp.m Derive equations for algebraic Golay decoder . . . . . . . . . . . . . . . . . . . . . . . 401
testGolay.cc Test the algebraic Golay decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
golayrith.m Arithmetic Golay decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
plotbds.m Plot bounds for binary codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
simplex1.m Linear program in standard form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
pivottableau.m Main function in simplex1.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
reducefree.m Auxiliary linear programming function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
restorefree.m Auxiliary linear programming function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
krawtchouk.m Compute Krawtchouk polynomials recursively . . . . . . . . . . . . . . . . . . . . . . 415
lpboundex.m Solve the linear programming for the LP bound . . . . . . . . . . . . . . . . . . . . . 418
utiltkm.cc Sort and random functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
utiltkm.h Sort and random functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
concodequant.m Quantization of the Euclidean metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
chernoff1.m Chernoff bounds for conv. performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502
plotconprob.m Performance bounds for a conv. code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504
finddfree Executable: Find dfree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
teststack.m Test the stack algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
stackalg.m The stack algorithm for convolutional decoding . . . . . . . . . . . . . . . . . . . . . 514
fanomet.m Fano metric for convolutionally coded data . . . . . . . . . . . . . . . . . . . . . . . . . 514
fanoalg.m The Fano algorithm for convolutional decoding . . . . . . . . . . . . . . . . . . . . . 516
BinConv.h (lab, complete) Base class for bin. conv. enc. . . . . . . . . . . . . . . . . . . . . . . . . 531
BinConvFIR.h (lab, complete) Bin. feedforward conv. enc. . . . . . . . . . . . . . . . . . . . . . . . . . 536
BinConvFIR.cc (incomplete) Bin. feedforward conv. encoder . . . . . . . . . . . . . . . . . . . . . . . 536
BinConvIIR.h (lab, complete) Binary recursive conv. enc. . . . . . . . . . . . . . . . . . . . . . . . . . 536
BinConvIIR.cc (lab, incomplete) Binary recursive conv. enc. . . . . . . . . . . . . . . . . . . . . . . . 536
testconvenc (lab, complete) Test convolutional encoders . . . . . . . . . . . . . . . . . . . . . . . . 536
Convdec.h (lab, complete) Convolutional decoder class . . . . . . . . . . . . . . . . . . . . . . . . 533
Convdec.cc (lab, incomplete) Convolutional decoder class . . . . . . . . . . . . . . . . . . . . . . 533
BinConvdec01.h (lab, complete) Binary conv. decoder, BSC . . . . . . . . . . . . . . . . . . . . . . . . . 533
BinConvdecBPSK.h (lab, complete) Bin. conv. dec., BPSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
BinConvdecBPSK.cc (lab, complete) Bin. conv. dec., BPSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
testconvdec.cc (lab, complete) Test the convolutional decoder . . . . . . . . . . . . . . . . . . . . . . 538
makeB.m Make the B matrix for an example code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
tcmt1.cc Test constellation for rotationally invariant code . . . . . . . . . . . . . . . . . . . . . 557
tcmrot2.cc Test constellation for rotationally invariant code . . . . . . . . . . . . . . . . . . . . . 565
lattstuff.m Gen. matrices for A2 , D4 , E6 , E8 , Λ16 , Λ24 . . . . . . . . . . . . . . . . . . . . . . . . . . 570
voln.m Volume of n-dimensional unit-radius sphere . . . . . . . . . . . . . . . . . . . . . . . . 570
latta2.m Plot A2 lattice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576
lattz2m Plot Z2 lattice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576
BCJR.h (lab, complete) BCJR algorithm class header . . . . . . . . . . . . . . . . . . . . . . . . 631
BCJR.cc (lab, incomplete) BCJR algorithm class . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
testbcjr.cc (lab, complete) Test BCJR algorithm class . . . . . . . . . . . . . . . . . . . . . . . . . . 631
testturbodec2.cc (lab, complete) Test the turbo decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
makegenfromA.m Systematic gen. matrix from parity check matrix . . . . . . . . . . . . . . . . . . . . 638
xxvi List of Program Files

gaussj2.m Gauss–Jordan elimination over GF(2) . . . . . . . . . . . . . . . . . . . . . 638

Agall.m A parity check matrix for an LDPC code . . . . . . . . . . . . . . . . . . . 640
Agall.txt Sparse representation of the matrix . . . . . . . . . . . . . . . . . . . . . . . . 640
writesparse.m Write a matrix into a file in sparse format . . . . . . . . . . . . . . . . . . 640
ldpcdecoder.cc Many different LDPC decoding algorithms . . . . . . . . . . . . . . . . 642
ldpctest2.cc ldpcdecoder runner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642
decode() Decoding in class LDPCDECODER . . . . . . . . . . . . . . . . . . . . . . 649
lldecode() LL BP decoding in class LDPCDECODER . . . . . . . . . . . . . . . . 649
probdecode() Probability BP decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655
ldpc.m Demonstrate LDPC decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
ldpcdecode.m LDPC decoder (nonsparse representation) . . . . . . . . . . . . . . . . . 656
ldpclogdecode.m LDPC decoder (log likelihood) . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
phidecode() Belief propagation decoding using 𝜙 function . . . . . . . . . . . . . . 660
qphidecode() Quantized 𝜙 decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660
minsumdecode() Min sum decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
minsumcorrectdecode() Corrected min sum decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659
aminstartdecode() Approximation min∗ decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 662
rcbpdecode() Reduce complexity box-plus decoder . . . . . . . . . . . . . . . . . . . . . . 665
showchktable.m Compare RCBP quantization and ⊞ . . . . . . . . . . . . . . . . . . . . . . . 667
bitflip1decode() Bit Flipping LDPC decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
galAbitflipdecode() Gallager Algorithm A Bit Flipping Decoder . . . . . . . . . . . . . . . . 671
weightedbitflip decode() Weighted bit flipping decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673
modwtedbitflipdecode() Modified weighted bit flipping decoder . . . . . . . . . . . . . . . . . . . . 673
grad1bitflipdecode() Gradient descent bit flipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673
grad2bitflipdecode() Multigradient bit flip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673
DC1decode() LDPC decoding using divide and concur . . . . . . . . . . . . . . . . . . . 674
DMBPdecode() DMBP decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682
LPdecode() Linear Programming LDPC Decoding . . . . . . . . . . . . . . . . . . . . . 682
ldpctest1 LDPC err. prob. & iteration plots . . . . . . . . . . . . . . . . . . . . . . . . . 695
psifunc.m Plot the Ψ function used in density evolution . . . . . . . . . . . . . . . 697
densev1.m An example of density evolution . . . . . . . . . . . . . . . . . . . . . . . . . . 698
densevtest.m Plot density evolution results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 698
Psi.m Plot the Ψ function used in density evolution . . . . . . . . . . . . . . . 698
Psiinv.m Compute Ψ−1 used in density evolution . . . . . . . . . . . . . . . . . . . . 698
threshtab.m Convert threshold table to Eb ∕N0 . . . . . . . . . . . . . . . . . . . . . . . . . 698
ldpcsim.mat LDPC decoder simulation results . . . . . . . . . . . . . . . . . . . . . . . . . 700
exit1.m Plot histograms of LDPC decoder outputs . . . . . . . . . . . . . . . . . . 700
loghist.m Find histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
exit3.m Plot mutual information as a function of iteration . . . . . . . . . . . 700
dotrajectory.m Mutual information as a function of iteration . . . . . . . . . . . . . . . 700
exit2.m Plot EXIT chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
doexitchart.m Take mutual information to EXIT chart . . . . . . . . . . . . . . . . . . . . 700
getinf.m Convert data to mutual information . . . . . . . . . . . . . . . . . . . . . . . 700
getinfs.m Convert multiple data to mutual information . . . . . . . . . . . . . . . 700
sparseHno4.m Make sparse check matrix without girth 4 . . . . . . . . . . . . . . . . . . 708
Asmall.txt H matrix example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
ldpcdec.h (lab, complete) LDPC decoder class header . . . . . . . . . . . . . . . . 711
ldpcdec.cc (lab, incomplete) LDPC decoder class . . . . . . . . . . . . . . . . . . . . . 711
ldpctest.cc (lab, complete) LDPC decoder class tester . . . . . . . . . . . . . . . . . 711
ldpc.m (lab, complete) Demonstrate LDPC decoding . . . . . . . . . . . . . . 711
ldpcdecode.m LDPC decoder (not sparse) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
List of Program Files xxvii

ldpctest1.cc Prob. of err. plots for LDPC code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711

A1-2.txt Rate 1/2 parity check matrix, sparse format . . . . . . . . . . . . . . . . . . . . . . . . . .
A1-3.txt Rate 1/3 parity check matrix, sparse format . . . . . . . . . . . . . . . . . . . . . . . . . . 711
gf257ex.m LDPC check matrix min. wt. RS codewords . . . . . . . . . . . . . . . . . . . . . . . . . 732
ldpcencode.m Encode LDPC codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
ldpccodesetup1.m Make parity check matrix for DVB-T2 IRA code . . . . . . . . . . . . . . . . . . . . . 732
fadeplot.m Realizations of the amplitude of a fading channel . . . . . . . . . . . . . . . . . . . . . 903
jakes.m Jakes method for amplitude of a fading channel . . . . . . . . . . . . . . . . . . . . . . 903
fadepbplot.m BPSK performance over Rayleigh fading channel . . . . . . . . . . . . . . . . . . . . 904
Another Random Scribd Document
with Unrelated Content
 CHAPTER VII.
THE WILES OF A SCHEMER.

Jode Lenning was alone in the tent, which had been erected for
his use, when Mingo, a Mexican distance runner, who belonged to
the G. H. A. C., thrust his head through the flap and announced that
Colonel Hawtrey had arrived in camp.
Lenning, at the moment, had his back to the opening and was
wrapping a long, flat package in his handkerchief.
“What?” he gasped, throwing a startled look over his shoulder at
Mingo.
The other repeated his announcement.
“The devil!” gulped Lenning, in a flurry. “He’s found out what
happened at the house, and put for here on the jump. Now for
merry blazes, and a little slick work by yours truly.”
His hand shook a little as he crowded the handkerchief-wrapped
package into the breast of his Norfolk jacket; then, getting up, he
hurried out of the tent and ran to meet the tall man with the gray
mustache.
“Ah, my boy!” exclaimed Colonel Hawtrey, making no effort to
conceal the pleasure the meeting gave him. “You’re looking fit, I
must say, so there’s not much use asking how you feel.”
“Fine as silk, uncle,” said Lenning, clasping the colonel’s hand.
“How did you find everything at the mines?”
“The mines are all right,” was the answer, “but it was something I
discovered after I got home this morning that has rather shaken me.
Take me to a place where we can be by ourselves and talk.”
“My tent will fill the bill.” They walked together in the direction of
Lenning’s headquarters. “Was that Hawkins I saw leading away the
horses?” Lenning asked.
“Yes, that was Hawkins.” That there was a load of some sort on
the colonel’s mind was evidenced by his tone and manner. “It’s
possible,” he added, “that I am going to need Hawkins in—er—an
official capacity.”
“This sounds pretty warlike!” exclaimed Lenning.
“I suppose so,” and the old soldier stiffened a little. “I have made
some discoveries, Jode, which will astonish you. They nearly carried
me off my feet. By the way, what started you on this camping trip?”
“I thought it would be a good thing for our eleven,” Lenning
explained. “This Merriwell chap took the Ophir team out into the
hills, and I reckoned we’d follow suit. And, say! We bumped into the
Ophir outfit right here at Tinaja Wells. How’s that for a coincidence?”
“Queer, to say the least,” answered the colonel. “I hope all you
fellows will remember that you are true sportsmen, which is only
another term for gentlemen, and avoid any unpleasantness.”
“You can depend upon us to prove a credit to you, colonel!” said
Lenning, with a fine show of admiration for the erect, soldierly old
fellow beside him. “I have a lease from Struthers, and Merriwell has
one from Packard. Now,” and Lenning laughed, “which of us has the
right of it?”
“That’s hard to tell, my boy, until the lawsuit is decided. What sort
of a character is young Merriwell? Anything like his father?”
“I don’t know much about his father, sir; but young Merriwell
seems to be trying to make himself the whole thing. Of course,”
Lenning added, “I tried to smooth matters over, and it looks as
though I had succeeded. As you see, we’re both camped on the
same ground.”
“I’ll have a talk with Merriwell myself, and see what I can do with
him. All that, however, must wait on the important business that
brings me here. I have never had anything make such an impression
on me. Is this your tent, Jode?”
“Yes, uncle. Walk inside and make yourself comfortable.”
When Colonel Hawtrey had seated himself comfortably on a camp
stool, and Lenning had dropped down facing him on a pile of
blankets, the colonel lighted a cigar—possibly to soothe or cover his
nervousness—and began.
“You remember, Jode,” said he, “that I drew a thousand dollars
from the bank on the forenoon of the day I left town, expecting to
pay it out to Judson for an interest in that promising claim of his.”
Lenning nodded.
“You drew the money,” said he, “and Judson didn’t show up; then
you were called from town in a hurry, and locked up the money in
your safe. I remember all that very distinctly.”
“You knew the combination, and were to give Judson the money if
he called for it.”
“Yes, sir; but he didn’t call.”
“I know that. I had scarcely reached town when I saw him, and
he said he’d be around this afternoon to get the thousand. Then I
went home—and found that I had been robbed!”
“Robbed!” gasped Lenning, starting up.
“Yes, my boy, robbed! Of course, a thousand dollars isn’t very
much to me, but it’s losing the money in such a way as that that
gets under my skin. The safe in my study was open, the window had
been unlocked, and the thousand was gone!”
 “Had the safe been blown open?”
“No. Some one had worked the combination and——”
“Uncle!” exclaimed Lenning, in consternation. “You and I are the
only ones who know the combination. You were away from home,
and I—I——”
The colonel leaned forward and dropped an affectionate hand on
his nephew’s shoulder.
“Tut, tut!” said he brusquely. “You know I trust you as I would
myself. There is some one else who knows the combination, and
who at one time had as free access to that safe as you or I. I refer
to—to your half brother, Darrel.”
“But Ellis perished in that train wreck!”
“Supposed to, but I have always had a feeling that there might be
some mistake. That graceless young scamp wasn’t born to shuffle
off in any such way as that. What I should have done, I suppose,
was to have the combination changed. But I did not. This is the
result.”
“I wouldn’t be in too much of a hurry to judge Ellis, Uncle Al,”
pleaded Lenning. “You’re only working on a theory, you know, and
——”
There was sorrow in the fine old face of the colonel, but over all
was the sternness of an iron will.
“I have evidence,” he interrupted; “much as it grieves me to tell it,
Jode, yet I have evidence which cannot be denied. It is like you, boy,
to plead for the rascal who has disgraced our blood; but, as for me,
I shall not be victimized a second time without making him pay the
penalty. I—— You are pale!” exclaimed the colonel, leaning forward
to stare into his nephew’s face; “and you are trembling, too! What
ails you, Jode? Brace up; don’t take this too much to heart.”
 “I have something to tell you, uncle,” answered Lenning; “but,
first, let me hear your evidence.”
The colonel took a knife from his pocket and handed it to Lenning.
“You recognize that, don’t you?” he asked harshly.
“Why,” murmured Lenning, “it’s the knife you gave Ellis years ago.”
“It is,” was the grim rejoinder, “and I found it under the unlocked
window in my study.”
Lenning seemed stunned and incapable of words.
“But that isn’t all,” preceded the colonel. “I hunted up Hawkins,
who happened to be in town, and together we learned that a fellow
answering Darrel’s description had been in Gold Hill the night before
I got home. He had called on Haff, our club secretary, and asked for
me, and about you. Haff told him that you were camping, with some
of our lads, at Tinaja Wells. Supposing that Darrel had come here,
Hawkins and I secured a couple of mounts and made a quick trip
down the cañon. Have you seen anything of Darrel?”
“Then it’s true, it’s true!” Lenning was muttering, as though to
himself.
“What is true?” demanded his uncle. “Don’t try to shield the
fellow, Jode. Your first duty is to me, not to him.”
“There is a fellow here—Merriwell seems to be looking after him—
who says he is Ellis Darrel.” Lenning spoke with apparent reluctance.
“I believed him to be an imposter. How could I think anything else
after the report we had of that Colorado wreck? The fellow seemed
bent on proving that he was really my half brother, and challenged
me to run a race with him. You see——”
“What folly!” cut in the colonel.
“I’m pretty fast in a sprint, uncle, but El was a shade faster. And
you know he had a queer way about him when he was running. I
think he is counting on that race to make his identity known to me
and the rest of the Gold Hill fellows.”
“We don’t need any proof of his identity, Jode! We can take his
word, and then confront him with this damning evidence of his
rascality!”
Lenning put out his hand and rested it on his uncle’s arm.
“Colonel,” said he, his voice shaking, “let us have this race to-
morrow afternoon. Don’t interfere. There’s a chance that, after all,
the fellow is not Darrel.”
“There’s not a shadow of a doubt, not a shadow!”
“But you needn’t hurry about arresting him, need you? Let’s find
out how far Merriwell will go in trying to shield him. Wait until after
the race; and then—well,” and Lenning drew a long, regretful sigh,
“do what you think you have to—what you think you must.”
“If Darrel knows I am here with Hawkins he may suspect
something, and clear out,” demurred the colonel. “It isn’t well, my
boy, to dally too much with an affair of this kind.”
“Have Hawkins watch him,” suggested Lenning.
“True,” said the colonel, “I could probably do that. It’s impossible,
though, that Young Merriwell is mixed up, in any way, with Darrel’s
wrongdoing. He has been deceived in the fellow. I know of the elder
Merriwell, and a straighter man or a better all-round athlete the
world never produced.”
“I hope young Merriwell is square, and a real chip of the old block,
as I understand his friends mean to suggest when they call him
‘Chip’—but, well, I don’t like the way he has been acting. To-morrow
afternoon, uncle, we may know a lot more about him and about
Darrel, too.”
 “Very well,” said the colonel, though reluctantly, “we’ll leave the
matter, Jode, as you desire.”
“Thank you, sir,” said Lenning gratefully.
Why was Lenning so anxious to have his uncle defer action
against Darrel? Had the packet, wrapped in his handkerchief and
stowed in the breast pocket of his Norfolk jacket, anything to do
with his wish to delay proceedings? In view of what happened later,
this seemed like the logical explanation.
 CHAPTER VIII.
A JOKE—WITH RESULTS.

Hawkins, the deputy sheriff, had not much to say to Merriwell

during their walk from the mesa back to the camp. Hawkins was an
admirer, and in many ways had shown himself a true friend, of
Frank’s; and, out of the kindness of his heart and, without divulging
any secrets, he strove to warn him against Darrel.
“They’re talkin’ a heap, down in the camp,” said Hawkins, “of what
a big hit this Darrel person has made with you. Don’t cotton to him
too strong, Merriwell. He isn’t wuth it.”
“What do you mean?” Frank demanded.
“Between ourselves—the thing not to go any further, you
understand—this Darrel’s nothin’ more than a plain thief.”
“You’re mistaken, Hawkins,” said Frank, with spirit. “I can’t believe
it.”
“Well, son, you’ll have the proof before you’re many hours older.”
“Then I’ll wait for the proof, Hawkins; and it will have to be
copper-riveted before I turn against Ellis Darrel.”
“Jest a warnin’ I’m handing you, Merriwell,” grinned Hawkins. “And
you’re to keep what I said to yourself, mind.”
“Of course, Hawkins. I’m obliged to you for taking all this trouble,
but you’re mistaken, and will find it out. It’s the colonel’s business,
isn’t it?”
“Now, I’m not sayin’ another word,” answered the deputy, “and
maybe I’ve let out more’n I ought to, as it is.”
 That ended the brief conversation, and, while it did not shake
Merriwell’s confidence in Ellis Darrel, nevertheless it left him with
vague forebodings of fresh disaster hanging over the head of the
“boy from Nowhere.”
The members of the rival athletic clubs were carefully avoiding
each other. There was no display of ill feeling, perhaps because the
bad blood had no chance to show itself, or because the presence of
the colonel in the Gold Hill camp was a restraining influence. Be that
as it may, yet the topic of conversation in both camps was the
hundred-yard dash to be run on the following afternoon. The object
of the race, unique in the annals of sport, lent the event a
fascination which nothing else could have done. Until ten o’clock the
affair was discussed by the Ophir fellows, and then, agreeable to
schedule, lights went out and the Ophir lads sought their blankets.
By an arrangement, enforced from the very first night that Frank
and his companions went into camp, a watch of three was posted to
look after the live stock and other property during the night. A trio of
lads went on sentry-go from seven to eleven; when their duty was
finished, they aroused three others to do guard duty from eleven to
three; and these, in turn, awoke three more for the morning watch
from three to seven. On this night, the first to be passed on the flat
with the Gold Hillers, Ballard was one of the three who had the
midwatch of four hours around midnight. Ballard’s post was in the
cañon, just below the flat, where the saddle and pack stock had
been gathered.
He had a lonely vigil for an hour. Somewhere in the neighboring
hills the coyotes were howling—a noise, by the way, not calculated
to soothe a person’s nerves. While Ballard was listening to the
coyotes, and thinking more or less about the next day’s race, he
heard a sound as of some one sliding down the slope from the flat.
Alert on the instant, Ballard started up and peered into the gloom
and listened. Some one was breathing heavily and floundering and
stumbling through bushes and over stones.
 “Can’t be a prowler,” murmured Ballard, “for he’s making too much
noise. I’ll just lay hands on the fellow and make him give an account
of himself.”
Creeping forward, and screening himself as well as he could in the
shadows, Ballard was able to rise up suddenly and seize the
wabbling figure.
“Himmelblitzen!” wheezed a voice. “Oof you peen vone oof der
Inchun shpooks, den I bet you I faint fits righdt on der shpot!
Whoosh!” and the voice died away with a suggestion of chattering
teeth.
“Carrots!” laughed Ballard. “Say, you crazy chump, what are you
fooling around the gulch for at this time of night?”
“Oh, Pallard!” puffed Fritz, in great relief. “Vell, vell, vat a
habbiness! Dere vas t’ings vich ve don’d know till ve findt dem oudt,
hey? I vas looking for you, Pallard, yah, so helup me!”
“Looking for me?” echoed Ballard; “what for?”
“Meppyso I gif you haluf oof dot dreasure oof you go along und
hellup me get him.”
“Oh, blazes!” chuckled Ballard. “I thought you’d got over that
treasure notion, Carrots.”
“Lisden, vonce, und I told you someding.” Fritz dropped his voice
to an explosive whisper. “Vat you dink? Py shiminy, so sure as
nodding I findt me dot shtone mit der gross on. Yah, you bed my
life! It vas so blain as I can’t tell, Pallard. Aber ven I roll avay der
shtone und tig mit der shovel, I hear me some voices oof an Inchun
chief. Dot shkared me avay. Haf you got der nerfs to go mit me to
der blace back, Pallard? I peen shaky all ofer, und my shkin geds oop
und valks on me mit coldt feet, yet I bed you I go back, und I findt
der dreasure. You come, und so hellup me I gif you haluf!”
 The excitement at the Wells, incident to the arrival of the Gold
Hillers and following hard upon the rapid return of Fritz and Silva to
the camp, had temporarily closed the fun Merry and his friends had
had in the cañon. More important events had claimed the attention
of the lads who had participated in the joke, and no one had
explained matters to Fritz or the Mexican. So it chanced that the
Dutchman was still laboring under his delusion.
Ballard wondered whether he had better set Fritz right, or keep
the joke going. He finally decided that the stock would not suffer if
he played out the Dutchman a little, and watched his antics in the
supposedly spook-haunted gulch.
“When an Injun goes to the happy hunting grounds, Carrots,”
remarked Ballard gravely, “it’s just as well not to stir him up. I’d hate
to have a red spook get a strangle hold on me—there wouldn’t be
treasure enough in the whole of Arizona to pay a fellow for an
experience of that kind.”
“Haf you no chincher?” demanded Fritz. “Iss it not vort’ a leedle
shcare chust to load oop mit goldt dot vill make you a rich mans for
life, hey? Vell, I bed you! I t’ink him all oudt, und I arrife py der
gonglusion dot a shpook iss nodding more as a shadow in der sun,
oder der moon. Vat a shpook does makes no odds aboudt der
tifference. Ve go, ve ged der goldt, und ve come back. Dot’s all
aboudt it. I got me a shovel in vone handt, und a glub in der odder.
Mit vone, I tig oop der goldt; mit der odder, I knock ofer der
shpooks. Und dere you vas. Ve shall be gompany mit each odder,
Pallard.”
“I don’t see how I can back out, Carrots,” said Ballard, “the way
you put it up to me. You’re an awful persuader. How much gold is
there?”
“I see it in der tream dot dere iss more as ve can carry, yes.”
“Maybe that dream is just fooling you, Carrots.”
 “You say yourselluf dot treams iss somet’ing, Pallard.”
“Did I? Well, maybe they are something. You go first, will you,
Carrots? I’ve got a weak heart, and if I should run onto a spook
without any warning it would knock me stiff.”
“I vill go fairst,” agreed Fritz, generously and valiantly, “und you
precede. I vill vatch aroundt carefully, und oof ve don’d make some
noises, den meppy der shpooks von’t hear, und ve gif dem der slip.”
Fritz waddled off into the darkness, and Ballard, enjoying himself
hugely, trailed after him. Suddenly, without the least warning, Fritz
dropped the shovel and the club, whirled in his tracks, and took
Ballard in a convulsive embrace.
“Ach, du lieber!” he whimpered. “I hear me someding, py shiminy!
Lisden, vonce, Pallard! Vat it iss, hey?”
“Coyotes,” answered Ballard, in a smothered voice. “Brace up,
Carrots. Don’t lose your nerve.”
“Sooch dreasure hundings I don’d like,” mumbled Fritz, slowly
untangling himself from Ballard and cautiously groping for his shovel
and club. “I vish der plame’ coyotes vould go to shleep. Ach, vat a
nervousness I got all droo me. I shake like I hat some agues. Sooch
a pitzness iss vort’ all der dreasures vat ve findt.”
Suddenly Ballard, clapping a hand over Fritz’s mouth, whispered a
hissing warning for him to keep still, and pulled him out of the
narrow trail and in between a couple of huge bowlders.
“V-v-vat iss der drouple!” inquired Fritz feebly. “You see a shpook
yourselluf, Pallard? I bed you——”
Again Ballard clapped a hand to his companion’s mouth.
“Sh-h-h!” he murmured. “There’s some one coming, right behind
us. Not a word, now; not so much as a whisper.”
 Somehow, Ballard got it into his head that the man who was
following them was Silva. The Mexican, he remembered, was also
mixed up, rather vaguely, with Fritz in the treasure hunting. Ballard
had it in mind to give Silva a bit of a scare, and so make the most of
that midnight experience.
Peering out from their dark retreat, Fritz and Ballard saw a dark
figure gliding toward them along the trail. It was impossible for them
to discover who the man was. He was in a hurry, that was evident,
and a peculiar, musical jingling accompanied him as he came on.
The sound was not loud, but more like a tinkling whisper, and barely
distinguishable.
But Silva—if Silva it was—did not pass the two behind the
bowlders. He halted, so close that Ballard could have reached out
and touched him, went down on his knees, and worked at
something in the dark. Even with the fellow so near, the heavy
gloom successfully hid his identity.
Ballard’s desire for fun was lost in a mighty curiosity. The fellow
took something white from his pocket, and, apparently, pushed it
under a stone; then, rising, he sped away in the direction from
whence he had come.
“Vell, vell!” muttered Fritz. “Vat you t’ink iss dot, Pallard?”
“That’s a conundrum, Carrots. How many fellows are looking for
that treasure of yours, eh?”
“No vone but me und you, Pallard.”
“Wait here for a couple of shakes, Fritz. I want to explore.”
Ballard crept to the place where the mysterious figure had been at
work, groped under a stone, and pulled forth a package wrapped in
something white. Lighting a match, he examined his find. Fritz could
hear him muttering excitedly as the match dropped from his fingers.
“Vat it iss, Pallard?” quavered Fritz.
 “I’ve had enough treasure hunting for one night,” answered
Ballard, in a strange voice. “I’m going back to the live stock, Fritz.
Come on!”
Fritz protested, but Ballard stood firm. Fritz would not continue on
without company, and so they returned to the camp—Ballard with
the white packet snugly stowed in his pocket.
 CHAPTER IX.
THE RACE.

Most of the forenoon, every day except Sunday, Merriwell, Clancy,

and Ballard had to give up to the “grind.” Professor Phineas
Borrodaile rigidly insisted on certain hours for study and recitation,
and would not temper his discipline even on the day that notable
race was to be run between Lenning and Darrel.
Following breakfast, each camp continued to flock by itself. The
live stock belonging to each party was picketed in widely separated
grazing grounds, so there was no opportunity for Silva and the other
packer to wind up their disagreements in a final clash. Peace
hovered over the region adjacent to Tinaja Wells, but to Merry it
suggested a calm preceding a storm.
Hawkins buried himself among the Gold Hillers, and seemed very
careful not to overstep the “dead line” which had been drawn
between the two camps. Colonel Hawtrey also appeared content to
remain in seclusion among the members of his own party.
About eleven o’clock in the forenoon, Frank and his chums, and
the professor and Darrel overheard a brief address which the old
soldier was making to the young athletes of the Gold Hill club. Only
scraps of the colonel’s little speech floated to the fellows in the Ophir
tent, but what they overheard made a deep impression on them.
“Sports of the right kind, properly indulged in, are of vastly more
benefit to the upbuilding of character, my young friends, than to
your muscles and bodily endurance. Understand me, I do not say
that physical development is of less importance than mental
development. Both of these should proceed hand in hand; but if,
over all, the moral and manly qualities do not grow as they should,
all your training in the class and on the track and field will have been
in vain. Try, my lads, to develop the faculty of being good losers, and
to admire and applaud in others those abilities, natural or acquired,
which you possess, but not in the same degree.”
As these words, spoken in a deep and earnest voice, wafted
themselves from the rival camp, the professor softly clapped his
hands.
“Noble sentiments most nobly expressed, young gentlemen,” he
murmured. “This Colonel Hawtrey must surely be a man of splendid
character.”
“He is,” said Darrel, in a low voice. “The colonel is one of the finest
men that ever lived.”
“Listen!” whispered the professor.
Again the colonel’s words drifted into the rival camp:
“If an amateur athlete is not a true sportsman, which is but
another term for gentleman, he is not fit to compete with other true
sportsmen. Your real gentleman, if you please, has courage; but,
more than that, he is so imbued with the spirit of fair play and so
completely captain of his own soul, that the stings of honorable
defeat leave him unscathed.”
These were fine words, and well calculated to inspire a spirit of
high emprise.
“I hope Jode is taking that in,” whispered Darrel to Merriwell; “but,
I’ll gamble my spurs, he’s going to beat the pistol, just the same.”
Ballard, all that morning, had been preoccupied to an extent that
had drawn some criticism from the professor. The interesting events
of the night, which he had not only kept a secret himself but had
likewise warned Fritz to keep in the background, probably had a
good deal to do with his poor showing at the problems put up to him
by Borrodaile.
 At eleven-thirty, when the studious ones were allowed a breathing
spell before dinner, Ballard hooked onto Merriwell and led him to a
secluded place for a talk. Fritz had to call them three times to “grub
pile,” and when the two finally arrived, their faces were flushed with
excitement, and there was an air about them that suggested
mysterious things.
At two-thirty in the afternoon a general movement set in toward
the mesa. Both camps emptied themselves upon the little plateau,
so that nearly forty spectators assembled to watch the race between
Darrel and Lenning.
The course had already been marked off by Brad, Spink, and
Handy. Beman, for Lenning, had looked it over and pronounced it
O. K. On one side of this course the Gold Hill men were grouped,
and on the other side the fellows from Ophir.
Colonel Hawtrey and Hawkins stood together, and Merriwell, for
the first time, got a good look at the colonel. He was much
impressed with his soldierly bearing, but in his face could be read
sternness and determination—and a sadness which did not, in the
least, diminish the more Spartan qualities.
Bleeker, of Gold Hill, crossed the course and stepped up to
Merriwell.
“There ought to be a judge and a starter, I reckon,” said he. “I
don’t see any need of makin’ this event top-heavy with officials. Do
you?”
“Not at all,” Frank answered. “I’d suggest that Colonel Hawtrey act
as judge of the race.”
“He says he won’t have a thing to do with it.”
“Then how about Hawkins, the deputy sheriff?”
“Suits Lenning to a t, y, ty. Lenning would like to have Beman for
starter.”
 Merriwell was expecting this, and yet it came to him with
something like surprise. It pointed to crookedness on the part of
Lenning—and after that fine talk the colonel had given his fellows
that morning, too!
“Let Beman act as starter, then,” assented Frank, keeping to the
plan broached by Darrel.
Bleeker hurried away to inform Hawkins and Beman of the work
laid out for them; and a few minutes later Darrel and Lenning, in
sprinting costumes, came trotting up from the camp.
Merriwell watched Darrel and the colonel. As the old soldier fixed
his eyes on his discredited nephew, a queer play of emotions
showed in his face. In Darrel’s look was a wistfulness and affection
which caused his uncle to turn abruptly and gaze in another
direction.
Beman, a round-shouldered, lanky chap, stepped out back of the
starting line, pistol in hand.
“All ready, you two?” he called.
Darrel and Lenning answered by stepping to the line. Not a sound
of approval or disapproval went up from the gathered throng.
Silence reigned on the mesa.
“This is about as cheerful as a funeral procession, Chip,” muttered
Clancy.
“Everybody’s mightily interested in the race, for all they have
bottled up their feelings,” Merriwell answered.
“Maybe,” was the skeptical response, “but it takes a lot of rooters
to stir up the enthusiasm. This looks about as sporty as the track
event of a deaf-and-dumb school. That Lenning carries himself well.
He walks with a spring that leads you to think he ‘feels his feet.’ But
I don’t like the cut of his jib a little bit.”
 “Nor I. His eyes are shifty, and his face doesn’t inspire much
confidence.”
“The old colonel is about as hilarious as he would be trying to
hunt up a nephew in the morgue. Whoo! I’ll go dippy in a minute if
somebody doesn’t yell. Guess I’ll tear off a whoop myself.”
He suited his action to the word, but it was a melancholy effort.
No one joined in with him, not even Merry or Ballard. From across
the course, the Gold Hillers gave him a startled look of disapproval.
“Once will do, thanks,” muttered Clancy. “I’m frosted so badly I’ve
got chilblains. Why doesn’t that starter set ’em off?”
The words were hardly out of Clancy’s mouth before Beman
shouted: “On your mark!”
Both sprinters dropped in well-nigh perfect style.
“Set!”
With that word, and the tense preparations of the sprinters for the
start, Merry and Brad began watching Lenning keenly. Merry ticked
off the seconds in his mind—one, two, three—and then intuitively he
sensed the forward plunge of Lenning, coming a fraction of a second
before the crack of the pistol. Lenning had not waited to hear the
pistol, and had got away at the explosion.
“He did it, by thunder!” whispered Brad. “Darrel had the skunk
dead to rights. Eh, Chip?”
“No doubt about it, Brad!”
Further talk just then was out of the question. The first stride of
the race had taken Lenning into the lead, and Darrel, waiting
honorably for the signal to start, was rushing to overhaul his
competitor.
“Dig, you kid from Nowhere!” whooped Clancy. “The race isn’t
done till you breast the tape.”
 “Go to it, Darrel!” Merriwell shouted. “You’ll pass him at the
eighty-yard line!”
“Wow!” yelped Ballard; “I’ll bet the boy from Nowhere gets
Somewhere before he’s many seconds older.”
A murmur went up from the Gold Hill side of the course. The
peculiar form in which Darrel was racing was recognized. Various
little mannerisms connected with his sprinting were recalled. They
were all here, in this clean-cut athlete whom Lenning had declared
an impostor! Gold Hill sentiments, it was plain, were undergoing a
change.
Not the least interested observer in the Gold Hill crowd was the
colonel. He leaned forward, the joy of wholesome sport temporarily
brushing aside the sterner proceedings which were to wait upon the
finish of that hundred-yard dash. The object of that race—the “boy
from Nowhere’s” attempt to prove his identity—did not concern
Colonel Hawtrey. He knew Lenning’s competitor was Ellis Darrel, race
or no race. What flamed up in him, as he gazed spellbound, was a
pure love of track athletics, aroused by a contest that was superb.
In about four seconds after the start the Gold Hillers had loosened
up. There were cries of, “Go it, Darrel!” and, “This looks like old
times, Curly!” which proved that Darrel was already winning the
recognition he coveted, no matter whether he won or lost the dash.
At the eighty-yard line, just as Merry had prophesied, Darrel drew
ahead of Lenning. The latter called on his reserve powers for a final
spurt, but Darrel also had speed in reserve. In ten seconds, or a
trifle more or less, Darrel tore away the tape at the finish, a full
stride in the lead.
A roar went up from all sides. The enthusiasm, which had been
held in check, rushed forth like a tidal wave. A rush was made
toward Darrel, but Hawkins, the deputy sheriff, grim and relentless,
waved the throng back. Stepping to the side of the victor, he
dropped an official hand on his shoulder.
 “Youngster,” said he crisply, “I’m sorry a heap to come down hard
on you at a time like this, but you’re under arrest.”
“Arrest?” echoed Darrel, recoiling. “For what?”
“For openin’ your uncle’s safe an’ stealin’ a thousand in cold cash.
Don’t make a fuss, bec’us’ it won’t do you any good.”
Then, amid the dead hush that fell over the mesa, Darrel’s eyes
sought only one face in all the crowd surrounding him. And that face
was Merriwell’s!
 CHAPTER X.
A HELPING HAND.

The explosion of a bomb could not have caused greater

consternation among the throng on the mesa than that official action
of the deputy sheriff. Hawtrey, erect and with a soldierly stride,
passed out of the stunned crowd and placed himself beside Hawkins.
Merriwell, giving Darrel a reassuring look, also advanced. He had a
sweater on his arm, and began pulling it over Darrel’s head and
shoulders.
“You’d better keep out of this, Merriwell,” Hawkins murmured in
Frank’s ear. “I warned you. The kunnel means biz, and no mistake.”
“So do I,” Frank answered, with a flash of his dark eyes. “Keep
your nerve,” he added, in a low tone to Darrel; “we’ve got a few
cards of our own to play.”
“You are Frank Merriwell?” inquired Colonel Hawtrey, leveling his
gaze at Frank.
“Yes, colonel.”
“The son of Frank Merriwell, of Bloomfield, and the T-Bar Ranch,
in Wyoming?”
“Yes.”
“You are also seeking to befriend this misguided young man,
here?”
“I am Darrel’s friend,” said Merry, with spirit, “right from the drop
of the hat.”
 “Then, my lad, your father will some time hear of it with regret.
What Hawkins said is the truth. This fellow opened my safe and took
from it a thousand dollars in cash night before last. I have the
proof.”
“Pardon me, colonel,” returned Frank respectfully, “but inasmuch
as I am Darrel’s friend, will you let me handle this case for him in my
own way?”
“If you mean to defend him,” frowned Hawtrey, “you will have
your trouble for your pains. He has no defense!”
“Will you let me try and see if I cannot make one, and one that
will command your attention and best judgment?”
“Sufferin’ centipedes, Merriwell!” broke in Hawkins. “I never
reckoned you’d be tryin’ to save the scalp of a plain, out-and-out
thief!”
The white ran into Darrel’s face and his hands clenched. Merry laid
a soothing hand on his arm.
“This isn’t a time for any snap judgments, Hawkins,” said Frank.
“First,” and he turned to the Gold Hillers, “I want to ask if this boy
from Nowhere has proved that he is Ellis Darrel, of Gold Hill?”
“Yes!” came a chorus of responses.
Merry partly turned to face Lenning. The latter, a sneering smile
on his dark face, was standing at a little distance, keenly alive to
everything that was said and done.
“How about you, Lenning?” queried Frank.
“He’s my half brother, all right,” was the answer. “I reckon there’s
not a shadow of doubt about that.”
“You agree, too, colonel?”
 “I knew the fellow was Darrel before the race,” answered Hawtrey.
“If he had proved to be an impostor, this accusation of theft might
not have carried. Now it is absolutely proven—ab-so-lutely.”
“Darrel has been accused here, before all his old friends,” Frank
continued, marshaling all his wits to acquit himself creditably of the
task of clearing Darrel, “and it’s only a fair shake that he should be
proven innocent before them. Colonel, will you please tell us of the
robbery, and show your proofs?”
Hawtrey was visibly annoyed. Nevertheless, he was a great stickler
for fair play, and he had to acknowledge that the position taken by
Merry was logical.
“I have been away from Gold Hill for a week,” said he, “visiting
some of my mining properties. Before I went, I drew a thousand
dollars in cash from the bank to pay to a man from whom I was
purchasing an interest in a ‘prospect.’ I was called from town
hurriedly, before the payment was made. The money was locked up
in the safe in my study, at home. Jode, here, who knows the
combination of the safe, was to pay over the money if the man
presented himself during my absence. The man did not come, and
Jode started off on this camping trip, three days ago. When I
reached home yesterday morning, I found the window of my study
unlocked, the safe door swinging open, the thousand dollars gone,
and this knife lying under the window, inside the room. Hand the
knife to Darrel, Merriwell, and see if he recognizes it.”
The colonel seemed averse to having any direct dealings with
Darrel. He gave the pocketknife to Frank, and the latter presented it
to Hawkins’ prisoner.
“It’s mine,” admitted Darrel huskily.
“Haff, an official of our athletic club, told Hawkins and me,” the
colonel proceeded, “that a fellow answering Darrel’s description had
been in town the night before I got home, that he had made
inquiries about me, that he had told the fellow I was away from
home, and that Jode was off on a camping trip, and that Darrel
started down the cañon to join the Gold Hill campers. Hawkins and I
got horses and hurried on to Tinaja Wells. Ask Darrel, Merriwell, if
he denies being in my house night before last?”
“No, colonel,” spoke up Darrel, without waiting for Merriwell to put
the question, “I do not deny it. I was there. I pushed open the sash
lock with this knife, and went in through your study and up to my
old room. I had the key to my room—have had it in my pocket for a
year. All I wanted to get was my running suit. After I had taken that,
I locked up the room and left by the window. Naturally, I could not
relock the window from the outside. That’s all, sir. I did not tamper
with your safe.”
A sneer of incredulity crossed Lenning’s face. It faded into a
sorrowful look, however, as the colonel gave him a swift glance.
“You admit being in the house,” said the colonel harshly, “so why
not admit the rest of it?”
“Because it is not the truth,” Darrel answered, with spirit.
“Did you know the combination of the safe, Darrel?” asked Frank.
“Yes—that is, if it hasn’t been changed during the past year.”
“It hasn’t,” put in the colonel. “That was my fault, I suppose.”
“Then, three of you knew the combination,” went on Frank,
“yourself, colonel, and Darrel and Lenning.”
“That is the way of it.”
The crowd on the mesa was listening with absorbed attention to
the talk which was going forward over the hapless head of the “boy
from Nowhere.” Nearly all, perhaps, felt that Darrel’s admission that
he had gone to the house for his running suit was a trivial excuse to
cover a design on the safe. Dark looks were thrown at Darrel, and
only here and there was anything bordering on sympathy shown for
him.
“Now,” said Frank, keeping the points he wanted to make well in
mind and working toward them with all the skill he could muster,
“you said, colonel, that Lenning and his camping party left Gold Hill
three days ago?”
“Yes.”
“Less than half a day would be required to make the trip from
Gold Hill to Tinaja Wells, for a mounted party with pack animals.
How does it happen, then, that the Gold Hillers only reached the
Wells yesterday afternoon?”
Colonel Hawtrey seemed puzzled. He turned to Lenning.
“Explain that, will you, Jode?” he requested. “Why didn’t you
reach the Wells day before yesterday?”
“Well, sir,” Lenning answered, “we were about halfway between
town and Tinaja Wells when we found out that Merriwell and his
crowd were camped at the place we wanted.”
“Ah! And what did you do then?”
“I had the boys make temporary camp in a side cañon while I—er
—went back to Gold Hill.”
“That,” said Frank, “would bring you in Gold Hill night before last—
the night of the robbery?”
Lenning reddened and looked confused.
“Why,” he faltered, “I reckon it would.”
“What was your business in Gold Hill, Lenning?”
“I don’t know,” snapped Lenning, “that you’ve got any call to
pump me.”
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com