Archive for August 23rd, 2021

August 23, 2021

Information

by Neil Rickert

People use the word “information” in a variety of ways, some of them mutually inconsistent. In this post, I shall describe how I prefer to use the term. My own usage is partly informed by my study of perception. And, of course, as a retired professor of computer science, my usage is partly informed by the use of information in technology.

The word “information” is used a lot in information technology. We talk about information being communicated or copied or transmitted. But we also think of newspapers as being sources of information. Likewise, human speech is normally considered to be information.

Shannon information

Claude Shannon is famous for his theory of information. He actually called it a theory of communication. People use the expression “Shannon information” in somewhat inconsistent ways. I shall try to stay close to what Shannon discussed.

Shannon was concerned with the question of transmitting information over a noisy channel, and with the problem of minimizing the errors that are due to noise in the channel. Shannon considered the use of both analog signals and digital signals. But the bulk of his work has to do with digital signals, and that will be my main emphasis in this discussion.

Shannon did much of his work at Bell Labs. And, of course, a major concern of Bell Labs was with telephone communication. These day, largely due to the work of Shannon and others, most telephone communication is done by encoding voice signals in a stream of binary digits. It turns out that this kind of digital communication can better deal with noise in the channel.

read more »