Information Theory Fundamentals - Part I

The meaning of probability
The axiomatic approach to probability
The classical approach to probability
Basics of Set theory
The pervasiveness of information
Information theory and the basic sciences intersected
Requirements
Exposure to probability
Description
In this course, students will see that information is found in the stock market, in Brownian motion of pollen, in neural firing patterns, in the quantum realm of particle creation and annihilation, and also in intersection with many of the basic sciences. Information intersects with quantum physics, to yield quantum information theory, with neuroscience to yield computational neuroscience, and with economics to yield investment theory. This broad overview of information is accompanied by a discussion of the philosophical issues that information touches upon. We present an outline for a complete course on information theory of which the present course is the zeroth offering. In the lecture on probability, we start with a brief historical introduction to probability. We go on to define the meaning of probability via probability axioms, we introduce the classical definition, and the relative frequency definition of probability. We provide examples to illustrate these definitions. These are simple examples not requiring a calculator. Finally, we introduce the student to set theory fundamentals, explaining the notion of belonging, exclusion, disjointedness, intersection, union, complements. We also introduce partitions, De Morgan's laws and the properties of unions and intersections. All of the above makes this a strong foundation on which other course work can build.
Overview
Section 1: Introduction to Information
Lecture 1 Introduction: Pervasiveness of Information
Section 2: Introduction to Probability
Lecture 2 Meaning of Probability
Lecture 3 Probability Space and Terminology
Lecture 4 Axioms of Probability
Lecture 5 Fields and Borel Fields
Advanced learners having mathematical maturity, ready to challenge themselves to learn this very practical theory.


Homepage