date: 2024-09-23
title: Statictical Learning
status: DONE
author:
  - AllenYGY
tags:
  - NOTE
  - MachineLearing
publish: True

Statictical Learning

Probability Distribution

Probability Mass Function (PMF) is the probability that the value of r.v. is :

In a test, event happens with probability , does not happen with probability .
If using r.v. to indicate the number of occurrences of event , then can be 0 or 1. Its distribution is:

In the times Bernoulli distribution, if r.v. represents the number of occurrences of event , the value of is , and its corresponding distribution:

The binomial coefficient represents the total number of combinations of elements taken out of elements regardless of their order.

Probability distribution can be described by the Probability Density Function (PDF) :

The Cumulative Distribution Function (CDF) is the probability that the value of r.v. is less than or equal to :

For a discrete random vector , when is known, the conditional probability of r.v. is:

Sampling: given a probability distribution p(x), generate samples that meet the conditions

Expectation: the average of random variable

For discrete r.v. :

For continuous r.v. :

When the number of samples is large, the sample mean and the real mean (expectation) are fully close.

Given N independently and identically distributed (I.I.D.) samples

Its mean value converges to the expected value: