Machine Learning Basics

Machine Learning is a way of building software that learns patterns from examples instead of relying only on rules written by a developer.

In traditional programming, you provide data and explicit rules. In supervised Machine Learning, you provide data together with known results, and a training algorithm uses those examples to create a model. The trained model can then make predictions for new data:

Samples, features, and targets

A dataset contains samples. Each sample represents one observation, such as one computer or one email.

The input values that describe a sample are called features. The value that the model should learn to predict is called the target.

For example, a computer-performance dataset may contain:

Features: MYCT, MMIN, MMAX, CACH, CHMIN, CHMAX
Target:   PRP

One row is one computer. Its hardware properties are the features, while its published relative performance (PRP) is the target.

An email dataset may contain:

Features: num_links, num_words, has_offer, sender_score, all_caps
Target:   is_spam

Here, one row represents one email. The model uses the five features to predict whether that email is spam.

Regression

Regression predicts a numerical value. The result can take many possible values rather than belonging to a fixed category.

The number of input features and the relationship in the data determine which regression model fits the problem. Compare the examples below.

Classification

Classification predicts which category a sample belongs to.

The target may contain two classes, several classes, or categories inferred from image pixels. Compare the common patterns below.

Regression or classification?

The target determines the type of supervised learning problem:

A useful first question for any Machine Learning project is therefore:

What exactly should the model predict?

If the answer is a quantity, the problem is usually regression. If the answer is a category, the problem is classification.

Training, testing, and generalization

A model must learn from one group of samples and be evaluated on different samples. This separation checks whether it learned a reusable pattern instead of memorizing the examples it already saw.

An 80/20 train/test split is a common starting point, although the right proportion depends on dataset size. The important rule is that the test set must not participate in training or in fitting preprocessing values.

When a model performs very well on training data but poorly on unseen data, it is overfitting. When it performs poorly even on training data, it is usually underfitting and may need better features, more training, or a more suitable architecture.

Reading evaluation metrics

Evaluation translates model behavior into measurements. A single metric rarely tells the complete story, so choose metrics that match the task and the cost of different mistakes.

Regression metrics

MSE can be written as:

MSE = sum((actual - predicted)^2) / number of samples

Because the errors are squared, one large error influences MSE more strongly than several small errors.

Classification metrics

A binary classifier produces four possible outcomes:

These counts form a confusion matrix and support several metrics:

accuracy  = (TP + TN) / all predictions
precision = TP / (TP + FP)
recall    = TP / (TP + FN)
F1        = 2 * precision * recall / (precision + recall)

• Accuracy describes overall correctness, but can hide poor behavior when classes are imbalanced.
• Precision matters when a false alarm is expensive. In spam detection, it asks how many emails marked as spam really were spam.
• Recall matters when missing a positive case is expensive. In fraud detection, it asks how much real fraud the model found.
• F1 summarizes the balance between precision and recall.

Always interpret these values in the context of the application rather than choosing a model only because one number is largest.

Key terms

Continue learning

Build the simplest trainable model for predicting a continuous value.

Next lessonLinear Regression→