Presentation measurement of information content approach

Class: 10.

The purpose of the lesson: To teach how to measure the information volume of a message through meaningful approach.

Lesson Objectives:

Educational: teach to measure the information volume of a message through a meaningful approach.
Developmental: development of thinking, speech, fine motor skills, imaginative perception.
Educational: grafting careful attitude to information and technology, personal responsibility for work results, accuracy, perseverance, self-discipline.

Lesson type: Explanation of new material with elements of a workshop.

Textbooks:

“Informatics 10” (basic course), ed. N.V. Makarova, “Peter”, 2003.
Ugrinovich N.D. Computer science. Basic course Grade 10. - M.: Publishing house "BINOM".

Basic concepts:

Half division method;

During the classes

I. Organizational moment

Setting the mood for the work environment.

II. New material

In the last lesson we learned to distinguish informative messages from uninformative ones.

We found out that to determine the amount of information in a message about the occurrence of one event out of more than two equally possible ones, the following formulation is necessary: “A message that reduces uncertainty by 2 times contains 1 bit of information.” We analyzed the problem with tossing a coin: “Before tossing the coin, there were two equally probable outcomes. This determines the uncertainty of the situation. In other words, uncertainty is the number of possible events. After receiving a message about the result, there was only one option left. How much has the uncertainty of the situation decreased?”

Now let's solve the problem of determining the amount of information in a message using the method of halves (dichotomy). So that at each search step exactly half of the options can be discarded. We will organize the work in the form of a game “Guess the answer”.

For example, I think that a book is on some shelf, but I don’t tell you about it. You need to determine which of the 8 shelves the book is on. Questions must be asked in such a way that each answer (“yes” or “no”) reduces the uncertainty by exactly half. Consequently, no matter how many questions are asked, so many bits of information carry the message about the guessed object. Fills in during the game table 2, establishing the relationship between the number of events and the amount of information in the message.

Analyzing the solution to previous problems, we introduce symbols and we deduce the formula of R. Hartley. For example, the chain of reasoning could be as follows:

When guessing the mark, two questions were asked, each of which reduced the uncertainty of the situation by half, and there were four possible options in total. Formalization of the reasoning – 2 · 2 = 4, i.e. 2 2 = 4.
When guessing the location of the book, three questions were asked, each of which reduced the uncertainty of the situation by half, and there were eight possible options in total. Formalization of the reasoning – 2 · 2 · 2 = 8, i.e. 2 3 = 8.
Based on this, we can derive the formula 2 i = N, Where i– amount of information in the message, N– number of options (events).
We use the resulting formula to determine the amount of information when tossing a coin. 2 1 = 2, i= 1 bit.

Number 2 in the formula means reducing the uncertainty by half, in accordance with the definition of the concept of “bit”. Using the formula, we fill out the table of integer powers of two up to 210 = 1024. The table establishes the relationship between the quantities of information in the message ( i) and the number of equally probable events ( N) and is a support for students in solving problems.

Let's create a general diagram:

Let's solve the problem using an example.

Task 1. Classes can take place in one of the rooms, numbered from 1 to 16. How much information does the teacher's message contain that classes will be held in room No. 7?

III. Summarizing

Today we studied:

Half division method;
Measuring the amount of information in a message in two ways: by formula and by the halving method,
Measuring the amount of information in a message over several actions,
Measuring the number of events if the information volume of the message is known.

IV. Homework

Solve the problem: There are 16 red apples in a bag. How much information does the message that you got a red apple contain?

1) a person receives a message about some event; it is known in advance uncertainty of knowledge person about the expected event. Uncertainty of knowledge can be expressed either by the number of possible options for an event, or by the probability of expected options for an event; 2) as a result of receiving the message, the uncertainty of knowledge is removed: from a certain possible number of options, one was chosen; 3) the formula calculates the amount of information in the received message, expressed in bits.

The formula used to calculate the amount of information depends on the situations, of which there can be two:

2. Probabilities ( p) possible options for the event are different and they are known in advance:

(p i ), i = 1.. N. It's still here N- the number of possible options for the event.

1. Everything possible options events are equally probable. Their number is finite and equal N .

Equally probable events. If denoted by the letter i amount of information in the message that one of the events occurred N equally probable events, then the values i And N are related by Hartley's formula:

2 i = N

1 bit is the amount of information in a message about one of two equally probable events .

Hartley's formula is an exponential equation. If i is an unknown quantity, then the solution to this equation will be:

i= log 2 N

These formulas are identical to each other.

Example 1. How much information does the message that a queen of spades was drawn from a deck of cards contain?

Solution: There are 32 cards in the deck. In a shuffled deck, the loss of any card is an equally probable event. If i- the amount of information in the message that a specific card has fallen (for example, the queen of spades), then from Hartley’s equation:

2 i = 32 = 2 5

From here: i= 5 bits.

Example 2. How much information does the message about rolling up a side with the number 3 on a six-sided die contain?

Solution: Considering the loss of any edge to be an equally probable event, we write Hartley’s formula:

2 i = 6.

From here: i= log 2 6 = 2.58496 bits.

Unequally probable events (probabilistic approach). If the probability of some event is p , A i (bit) is the amount of information in the message that this event occurred, then these quantities are related to each other by the formula:

2 i = 1/ p

Solving this exponential equation for i, we get:

i= log 2 (1/ p) Shannon's formula

Qualitative approach

Information is the knowledge people receive from various messages.
Message - This information flow(data stream), which in the process of transmitting information reaches the receiving entity.

Message

Informative message , which replenishes human knowledge, i.e. carries information for him.

Uninformative information is “old”, i.e. the person already knows this, or the content of the message is unclear to the person

Quantitative approach in the equiprobability approximation

Events are equally probable , if none of them has an advantage over the others.

Let's look at an example. “How much information is conveyed by the message about the result of throwing a six-sided die?” From Hartley's equation: 2 i = 6.

Since 2 2 i

Then we determine more exact value(accurate to five decimal places) that i= 2.58496 bits. Note that when this approach the amount of information can be expressed as a fraction.

Probabilistic approach to information measurement

Probability of some event is a quantity that can take values from zero to one.

Probability impossible event is equal to zero

(for example: “tomorrow the Sun will not rise above the horizon”)

Probability reliably th event is equal to unit

(for example: “Tomorrow the sun will rise above the horizon”).

Probability some events are determined through repeated observations (measurements, tests). Such measurements are called statistical. And what large quantity measurements are performed, the more accurately the probability of an event is determined.

Let's look at a few examples:

Example 3. Two bus routes stop at the bus stop: No. 5 and No. 7. The student is given the task: to determine how much information is contained in the message that bus No. 5 has arrived at the stop, and how much information is contained in the message that bus No. 7 has arrived.

Solution: The student did the research. During the entire working day, he estimated that buses approached the stop 100 times. Of these, bus No. 5 arrived 25 times and bus No. 7 approached 75 times. Making the assumption that buses run with the same frequency on other days, the student calculated the probability of bus No. 5 appearing at the stop: p 5 = 25/100 = 1/4, and the probability of bus number 7 appearing: p 7 = 75/100 = 3/4.

Hence, the amount of information in the message about bus No. 5 is equal to: i 5 = log 2 4 = 2 bits. The amount of information in the message about bus number 7 is equal to:

i 7 = log 2 (4/3) = log 2 4 – log 2 3 = 2 – 1.58496 = 0.41504 bits.

Example 4 . Let's consider another version of the bus problem. Buses No. 5 and No. 7 stop at the stop. The message that bus No. 5 has arrived at the stop carries 4 bits of information. The probability of bus No. 7 appearing at the stop is two times less than the probability of bus No. 5 appearing. How many bits of information does the message about bus No. 7 appearing at the stop carry?

Solution: Let us write the problem condition in the following form:

i 5 = 4 bits, p 5 = 2 p 7

Let's remember the connection between probability and amount of information: 2 i = 1/ p

From here: p = 2 – i

Substituting into the equality from the problem statement, we get:

Answer the questions orally:

What is meant by information?
What can you do with the information?
What types of information representation in a computer do you know?
What message encoding techniques were used in ancient times?
What is code and information encoding?
Give examples in various ways encoding information.
List the advantages and disadvantages of coding used in computers.
What is the name of the encoding used to represent characters entered from the keyboard?

Let's think about what can serve as an estimate of the amount of information?
Is it true that it's worn out
book, if not in it
torn pages, carries for
there are exactly the same number of you
information, how much is the same
new?

Let's think about what can serve as an estimate of the amount of information?
A block of stone weighing three tons carries as much information for archaeologists as a good photograph of it in an archaeological magazine.
Is not it?

Let's think about what can serve as an estimate of the amount of information?

Let's think about what can serve as an estimate of the amount of information?
When a Moscow radio studio broadcasts the latest news, both a resident of the Moscow region and a resident of Novosibirsk receive the same information. But the flow of radio wave energy in Novosibirsk is much less than in Moscow.

Consequently, the signal power, as well as the weight of the carrier, cannot in any way serve as an estimate of the amount of information carried by the signal.
How then can we measure the amount of information?

Different approaches to defining and measuring information
Meaningful
(probabilistic) approach:
Amount of information as a measure of uncertainty reduction
knowledge

Watch the video

Let's summarize what has been said

Let us
there is a coin
which we
throw it on the flat
surface.

Possible events

Event that happened

One of the following is equally likely to happen
two possible events - a coin
will end up in one of two positions:
"heads or tails".

Events are equally probable if, with an increasing number of experiments, the numbers of heads and tails gradually become closer.

Before the throw there is uncertainty of our knowledge ( two events are possible), and after the throw there is complete certainty.
The uncertainty of our knowledge is reduced by half, since out of two possible equally probable events, one was realized.

Reducing knowledge uncertainty

When throwing an equilateral tetrahedral pyramid, there are 4 equally probable events.

When throwing a six-sided die, there is
6 equally probable events.

Reducing knowledge uncertainty

A message that reduces the uncertainty of knowledge by half,
carries 1 bit of information.

Bit – minimum unit information measurements.

1 byte = 23 bits = 8 bits
1 KB = 210 bytes = 1024 bytes
1 MB = 210 KB = 1024 KB
1 GB = 210 MB = 1024 MB

kbyte

:1024

:1024

:1024

:1024

*1024

*1024

*1024

*1024

The amount i of information contained in the message that one of N equally probable events has occurred is determined by solving the exponential equation
2i = N

Task: In roulette, the total number of holes is 128. How much information will we receive in a visual message about the ball stopping in one of the holes?

N=128

i - ?

Given:

Solution:

2i = N
2i = 128
27 = 128
i = 7 bits

Answer: i = 7 bits

Number of possible events and amount of information

Task:

Task:
There are 32 pencils in the box, all pencils are different colors. They pulled out a red one at random. How much information was obtained?

Solution.
Since drawing a pencil of any color from the 32 pencils in the box is equally probable, the number of possible events is 32.
N = 32, i = ?
N = 2i, 32 = 25, i = 5 bits.
Answer: 5 bits

№ 1

The book has 512 pages. How much information does a message convey that a bookmark is on a page?

Solving problems in a notebook

№ 2

How much information does the message that on a 4x4 square field contain, one of the cells is shaded?

Solving problems in a notebook

№ 3

How much information does the message about rolling up a side with the number 3 on a six-sided die contain?

Solving problems in a notebook

What is the meaning of a content approach to measuring information?

What is the meaning of a content approach to measuring information?
What formula was studied?
Name in ascending order what units of measurement of information you know.
How are units of measurement of information interrelated?

Fixing the material

1. You approached a traffic light when the light was red. After that the yellow light turned on. How much information did you receive?

1. You approached a traffic light when the light was red. After that the yellow light came on. How much information did you receive?

Solve orally

2. You approached a traffic light when the light was yellow. After that the light turned green. How much information did you receive?

2. You approached a traffic light when the light was yellow. After that the light turned green. How much information did you receive?

Solve orally

3. "Are you getting off at the next stop?" - they asked the man on the bus. “No,” he replied. How much information does the answer contain?

3. "Are you getting off at the next stop?" - they asked the man on the bus. “No,” he replied. How much information does the answer contain?

Solve orally

4. How much information does the message indicate that the program you need is on one of the eight floppy disks?

4. How much information does the message indicate that the program you need is on one of the eight floppy disks?

Solve orally

Homework

1. Analyze the entries in the notebook.
2. Solve 2 individual problems on cards.

3 Content-based approach to measuring information For a person, information is human knowledge. Let's consider the issue from this point of view. Receipt new information leads to increased knowledge. If some message leads to a decrease in the uncertainty of our knowledge, then we can say that such a message contains information. It follows that a message is informative (i.e., contains non-zero information) if it adds to a person’s knowledge. For example, the weather forecast for tomorrow is an informative message, but a message about yesterday's weather is uninformative, because we already know this. It is easy to understand that the information content of the same message may be different for different people. For example: “2x2=4” is informative for a first-grader learning the multiplication table, but uninformative for a high school student.

4 Informative content of the message But in order for the message to be informative, it must also be understandable. To be understandable means to be logically related to a person's previous knowledge. Definition of "meaning" definite integral is equal to the difference between the values of the antiderivative integrand at the upper and lower limits,” most likely will not improve the knowledge of a high school student, because it is not clear to him. In order to understand this definition, you need to finish studying elementary mathematics and know the beginnings of higher mathematics. Obtaining any knowledge should go from simple to complex. And then each new message will at the same time be understandable, which means it will carry information for the person. A message carries information for a person if the information it contains is new and understandable to him.

5 Unit of measurement of information Obviously, distinguishing only two situations: “no information” and “there is information” is not enough to measure information. We need a unit of measurement, then we can determine which message contains more information and which contains less. The unit of measurement of information was defined in a science called information theory. This unit is called “bit”. Its definition is as follows: A message that reduces the uncertainty of knowledge by half carries 1 bit of information. The uncertainty of knowledge about an event is the number of possible outcomes of the event.

6 Example: After taking a test or completing a test, a student is tormented by uncertainty; he does not know what grade he received. “Pass”, “fail”? "2", "3", "4" or "5"? Finally, the teacher announces the results, and he receives one of two information messages: “pass” or “fail”, and after the test, one of four information messages: “2”, “3”, “4” or “5”. An information message about an assessment for a test leads to a halving of knowledge uncertainty, since one of two possible information messages is received. Information message about the assessment for test leads to a fourfold reduction in knowledge uncertainty, since one of four possible information messages is received.

7 Example: There are eight shelves on a bookshelf. The book can be placed on any of them. How much information does the message contain about where the book is? We ask questions: - Is the book located above the fourth shelf? - No. - Is the book below the third shelf? - Yes. - Is the book on the second shelf? - No. - Well, now everything is clear! The book is on the first shelf! Each answer reduced the uncertainty by half. A total of three questions were asked. This means that 3 bits of information have been typed. And if it were immediately said that the book is on the first shelf, then the same 3 bits of information would be transmitted by this message.

8 Formula for calculating the amount of information If we denote the possible number of events, or, in other words, the uncertainty of knowledge N, and the letter I is the amount of information in the message that one of N events occurred, then we can write the formula: 2 I = N Amount of information , contained in the message that one of N equally probable events has occurred, is determined from solving the exponential equation: 2 I = N.

10 Task 1: How much information does the message that a king of spades was taken from a deck of cards contain? Solution: There are 32 cards in the deck. In a shuffled deck, any card falling out is an equally probable event. N = 32. I - ? 2 I = N 2 I = = 32 I = 5 bits
12 12 Task 2: How much information does the message about the number 3 appearing on a six-sided die contain? Solution: N = 6. I - ? 2 I = N 2 I =