STRATA
Skip to study
XLVII

I.10 / GNOSIS

Semantic information and meaning

What must a recipient already know to recover meaning from a transmitted sequence?

A source-led introduction1 source references1 related experiments

Start with the question.

Semantic information concerns what a message means to someone within a language, practice, or decision. Shannon's framework gives a useful boundary: statistical uncertainty alone does not determine reference or truth. This topic therefore follows the additional structures that make a symbol interpretable, including a codebook, a shared context, and the consequences of acting on the message.

Keep the bits fixed while changing the interpretation.

Use the binary string 01000001. Under one agreed character encoding it can represent a letter; under another task it can be an integer or a set of flags. Its eight stored bits have not changed. Compare what each recipient must already know and what action each interpretation permits.

What would you inspect?

  • The encoding rule, shared vocabulary, and receiver's task.
  • Whether a misunderstanding changes behavior despite unchanged transmitted symbols.

Take the next step.

Document one real message with its encoding and context, then identify the extra evidence needed to assess whether its interpretation is accurate.

Put part of the question to work.

These models isolate a mechanism you can vary. Their assumptions define the part of this topic they can illuminate.

Try entropy and compression.

Change one parameter, compare the numerical output, and export the result. The experiment loads when you open it.

Open its full page

Watch the quantities change.

12 seconds · Original model visualization · Silent

Entropy and compression

Compare symbol frequencies with the ordering that a code can exploit.

Read the explanation and method

The first sweep changes the probability of A while drawing each symbol independently. The other three symbols share the remaining probability.

As one symbol dominates, the observed symbol entropy falls. Huffman codes use short words for frequent symbols; the comparison excludes headers and the codebook.

The second sweep gives all four new draws equal probability and increases repetition. Long runs reward run-length coding, even when a symbol histogram cannot describe the dependence.

Method. The histogram gives an order-zero Shannon entropy. Huffman coding repeatedly joins the two least frequent branches to build a prefix code. The run-length format stores a two-bit symbol and an eight-bit count; counts run from 1 to 255, and a longer run uses another packet.

Read the supporting material.

  1. A Mathematical Theory of Communication

    Nokia Bell Labs · Communication entropy, coding, and noise; does not turn statistical information into a theory of semantic truth.

Continue through the versioned study records.