AI Notes - Data Analysis and Decision-Making
Artificial Intelligence - Data Analysis and Decision-Making
This part is divided into four layers:
- Data collection
- Data cleaning
- Data analysis
- Decision-making
Data Collection
Data collection is a critical step in AI: it is the source of input used to train models.
As a refresher on the basics, data collection is simply sampling the target, and we need scientific methods to make the sample close to the target.
Sampling methods generally fall into three kinds:
- Simple random sampling
- Systematic sampling
- Stratified sampling
Simple random sampling is the easiest to understand: a random number generator draws samples at random from the target space. Systematic sampling usually emphasizes some kind of ordering. However, the real world cannot always be represented in order, so stratified sampling can be used instead. It first divides the target into categories, and within each category other sampling methods such as systematic or simple random sampling can be applied. This kind of sampling often has more reference value for the target.
Although there are three sampling methods, there are many channels for sampling. Common ones include:
- Surveys
- Questionnaires
- Interviews or focus groups
- Transaction tracking
- Online tracking
- Social media monitoring
Data collection must be formal, legal, and ethical.
Data Cleaning
Data cleaning makes the sample data reflect the target more accurately. Cleaning starts with defining the variables, which makes the data easier to work with.
Data Types
Numerical data
- Discrete data
- Continuous data
Categorical data
- Ordinal data
- Nominal data
Discrete data takes the form of counts—definite integers—whereas continuous data can be measured with arbitrary precision and is represented as floating-point numbers. Ordinal data has an inherent ordering among its values, while nominal data does not.
Data Collection Guidelines
-
Avoid bias
Avoiding bias depends on the reliability of the data source: the data must truly reflect reality. Beyond that, the sampling method and measurement approach matter just as much.
-
Data representation
Data is usually represented as a table, for example with columns for the recorded variable values and rows for the attributes of each sampling unit.
-
Data correction
Data correction removes duplicate or misspelled records to maximize data validity as far as possible.
-
Data enrichment
Data enrichment adds more attributes to each sampling unit so that more sample attributes can be collected.
-
Outliers
Outliers are erroneous data points that appear during sampling, and they should be removed.
The most important step in data cleaning is removing outliers.
Data Analysis
Before analyzing data, you need some basic mathematical concepts. They help you understand the distribution of the data and analyze it better.
Basic Statistics
-
Mode
The value that occurs most frequently.
-
Mean
The average of the data in the sample space.
-
Median
The middle value in the sample space. For an odd number of samples it is the middle value; for an even number of samples it is the average of the two middle values.
-
Range
The difference between the largest and smallest values in the sample.
-
Variance
Variance comes from the standard variance formula. The larger the variance, the more spread out the sample data is.
-
Standard deviation
The standard deviation is the positive square root of the variance and has similar properties.
Principles of Probability Theory
Basic properties of probability:
- Let S be the full sample space.
- Let A and B be subsets of the sample space.
- Assume A and B do not intersect.
This gives three axioms:
-
For any event A, its probability is non-negative.
-
For a certain event, the probability is 1.
-
For any number of mutually exclusive events, the probability of their union equals the sum of their individual probabilities.
Probability distributions usually fall into two kinds:
- Discrete distributions, described by a probability mass function (PMF)
- Continuous distributions, described by a probability density function (PDF)
Common Distributions
-
Simple discrete distribution
There is no free parameter, and outcomes are equally likely—for example, the probability of a tossed coin landing heads or tails.
-
Binomial distribution
The binomial distribution is a discrete distribution that typically has these four properties:
- Repeated trials (denoted N)
- Independent outcomes that do not affect each other
- Only two outcomes, which do not overlap
- The probability does not change between trials
The binomial distribution is described by a probability mass function (PMF), and its free parameter is p.
-
Gaussian distribution
The Gaussian distribution, also called the normal distribution, is a common model for continuous distributions.
The Gaussian distribution function is a probability density function (PDF), and its free parameters are the mean μ and the standard deviation σ.
-
Poisson distribution
The Poisson distribution models the probability of events occurring within a unit of time or space. Its defining feature is that events occur independently, meaning earlier and later events do not influence each other. In practice, the Poisson distribution is widely used in service system modeling, queueing theory, telecommunications, and other fields.
The Poisson distribution is described by a probability mass function (PMF), and its free parameter is λ.
Decision-Making
The sections above only introduced various mathematical analysis methods and models for data analysis. For AI, the critical step is making decisions, so the samples we use must accurately represent the target—only then can correct decisions be made.
Here we use mathematical principles to simulate and verify, in order to determine the mathematical model of the sample.
Estimation
This section focuses on maximum likelihood estimation (MLE). The method has five steps:
-
Choose the model and parameters.
A mathematical model can be chosen based on data observation or experience. For example, if the target has only two outcomes, we can assume a binomial distribution, whose free parameter is p.
-
Write the likelihood function.
The probability mass function (PMF) of the binomial distribution is:
Its likelihood function is:
-
Take the log-likelihood.
Then compute the log-likelihood function:
Simplifying (dropping the constant term):
-
Find the extremum with respect to the parameter.
Simplifying gives:
Let:
Solving yields:
-
Compute the free parameter using the likelihood estimate.
The maximum likelihood estimate is:
Statistical Hypothesis Testing
With the method in the previous section, the value of the free parameter can be obtained through likelihood estimation. However, we have not discussed whether the chosen mathematical model is correct, or whether it matches the actual target space.
In fact, the model we choose is often wrong, and we may need to try several models in turn before finding one that fits. This section uses statistical hypothesis testing so that our model accurately matches the target, allowing AI to make correct decisions.
Steps of Statistical Hypothesis Testing
Statistical hypothesis testing is the process of deciding whether to accept or reject a statistical hypothesis. The general steps are:
- Formulate hypotheses: First, set up two hypotheses. The null hypothesis (H0) usually represents the original theory being tested. The alternative hypothesis (H1) is the new theory we want to prove.
- Choose a significance level: The significance level (usually denoted α) is the probability you are willing to accept of making a Type I error (rejecting a true null hypothesis). Common significance levels are 0.05 and 0.01.
- Choose an appropriate statistical test: Pick a test that fits your data type and goal. For example, to compare the means of two groups, you might choose a t-test.
- Determine the critical value or compute the p-value:
- Critical value method: Compare the computed test statistic with the critical value. If the test statistic exceeds the critical value, reject the null hypothesis.
- p-value method: If the p-value is less than the significance level α, reject the null hypothesis.
- Conclusion: If we reject the null hypothesis, we accept the alternative hypothesis and consider our new theory correct. If we fail to reject the null hypothesis, we say there is not enough evidence to support the new theory—but that does not necessarily mean the old theory is correct.