This glossary defines the main terms used throughout the Deep Netts course.
A · B · C · D · E · F · G · H · I · L · M · N · O · P · R · S · T · U · V · W
A function applied to a neuron's weighted sum. It determines how that value is transformed before being passed to the next layer or returned as an output. Examples include Linear, ReLU, and Sigmoid.
A mechanism that lets a model assign different importance to parts of its input when creating a representation. In text models, attention helps one token use relevant information from other token positions.
The proportion of classification predictions that are correct. Accuracy can be misleading when one class is much more common than another.
The process used to calculate how much each network parameter contributed to the loss. These calculations allow the optimizer to update weights and biases during training.
A sparse text representation with one feature per vocabulary term. Each feature records whether or how often that term occurs, while token order is discarded.
A group of training samples processed before the model parameters are updated. A batch size of 64 means that 64 samples contribute to one update.
A trainable value added to a neuron's weighted inputs. It allows the neuron to shift its output independently of the input values.
A classification problem with two possible classes, such as SPAM and NOT SPAM.
A supervised learning task in which the target is one of a fixed set of categories.
A calculus rule for differentiating nested functions. Backpropagation applies the chain rule from the loss toward earlier layers to calculate a gradient for every trainable parameter.
A classification problem with three or more possible classes. The Iris example predicts SETOSA, VERSICOLOR, or VIRGINICA.
A class representation with one target column per class. The correct class contains 1 and all other class columns contain 0.
A table that counts correct and incorrect classification results for each class. It shows which types of mistakes the model makes.
A loss function commonly used for classification. It penalizes predicted probabilities that disagree with the known target class.
The state in which training updates have reached a stable region and the loss no longer improves substantially. Slow or unstable convergence can indicate an unsuitable learning rate, preprocessing, or architecture.
The accidental use of information during training that would not be available when predicting genuinely unseen data. Fitting preprocessing on the complete dataset before a train/test split is one example.
A collection of samples used for training, evaluation, or prediction. In the course examples, datasets are stored as CSV files.
The boundary that separates regions assigned to different classes. Logistic regression learns a linear decision boundary, while hidden layers can represent nonlinear boundaries.
A generative-model family that learns to create data by reversing a gradual noising process. Diffusion models are commonly used for image, audio, and video generation.
A dense numerical vector representing a token, sentence, document, or another item. Learned embeddings can place similar items near one another in vector space.
One complete pass through the training dataset.
The process of measuring a trained model using suitable metrics, preferably on data that was not used for training.
Additional training that adapts a pretrained model to a narrower task, domain, or behavior using a more focused dataset.
A classification metric that combines precision and recall into one value. It is useful when both false positives and false negatives matter.
An input value that describes a sample. For example, num_links is one feature in the email-spam-detection dataset.
A positive sample incorrectly predicted as negative. For fraud detection, this is a fraudulent transaction predicted as legitimate.
A negative sample incorrectly predicted as positive. For spam detection, this is a legitimate email incorrectly marked as spam.
A neural network in which information flows from the input layer through any hidden layers to the output layer.
The process of sending input values through the network to calculate a prediction.
Machine Learning systems designed to produce new content such as text, images, audio, or code. Generative systems still rely on model parameters, loss functions, backpropagation, and optimization, but use objectives and architectures suited to generation.
A generative architecture in which a generator creates samples and a discriminator learns to distinguish generated samples from real ones. The two models improve through competing objectives.
A model's ability to make useful predictions for samples that were not part of its training data.
A value describing how the loss changes with respect to a model parameter. Backpropagation calculates gradients so the optimizer knows how to adjust weights and biases.
An optimization method that updates parameters in the direction that reduces loss. The learning rate controls the size of each update.
A neural-network layer between the input and output layers. Hidden layers can learn intermediate and non-linear representations of input features.
A configuration value chosen by the developer rather than learned as a model weight. Examples include learning rate, number of epochs, batch size, and layer size.
The first neural-network layer. It receives the feature values and defines the expected number of model inputs.
Using a trained model to produce a prediction for new input data. Inference must apply the same feature order and fitted preprocessing used during training.
A language model trained at large scale to predict and generate token sequences. An external LLM service is not an integrated Deep Netts feature unless an application connects to it explicitly.
A training hyperparameter that controls the size of each parameter update. A value that is too large can make training unstable, while a value that is too small can make learning slow.
An activation that returns its input without restricting its range. It is commonly used in an output layer for regression.
A binary classification model that applies Sigmoid to a weighted linear score. It returns a positive-class probability and learns a linear decision boundary.
A model's raw score before a probability-producing function such as Sigmoid or Softmax is applied. Temperature changes the scale of logits before sampling in a generative model.
A function that measures the difference between a model prediction and the known target. Training adjusts model parameters to reduce this value.
A regression metric that calculates the average absolute difference between predicted and actual values. Lower values are better.
A regression loss or metric that calculates the average squared difference between predicted and actual values. Squaring gives larger errors more influence.
A preprocessing transformation that maps numerical features to a defined range, commonly from 0 to 1.
A subset of training samples used to calculate one parameter update. Mini-batch training balances the noisy updates of one-sample training with the memory cost of processing the complete dataset at once.
A learned mapping from input features to a prediction. In a neural network, the learned information is represented primarily by weights and biases.
Several attention operations performed in parallel with different learned projections. Their results are combined so the model can represent multiple kinds of relationships between token positions.
Linear regression with two or more input features. It learns one weight per feature and a shared bias to predict a continuous target.
Machine Learning and software techniques for analyzing, representing, retrieving, or generating human language. NLP pipelines convert text into numerical representations before a model processes it.
A model composed of connected layers of neurons that transform input features into predictions.
A computational unit that combines weighted inputs and a bias, then usually applies an activation function.
The algorithm that updates weights and biases during training. Stochastic Gradient Descent (SGD) is used in the introductory examples.
A condition in which a model learns its training data too specifically and performs substantially worse on unseen samples.
The final neural-network layer. Its size and activation function are selected to match the prediction task.
A value learned during training, such as a weight or bias.
Of all samples predicted as belonging to a class, the proportion that actually belongs to that class. For spam detection, it answers: of all emails marked as spam, how many were really spam?
The initial training phase in which a model learns broad patterns from a large dataset before any task- or domain-specific fine-tuning.
The output produced by a trained model for new input features.
Transformations or preparation applied to data before model training or prediction. Examples include standardization and Min-Max scaling.
A regression metric describing how much of the target variation is explained by the model. Values closer to 1 generally indicate a stronger fit, although the metric must be interpreted in context.
Of all samples that actually belong to a class, the proportion correctly identified. For spam detection, it answers: of all real spam emails, how many did the model detect?
A commonly used hidden-layer activation function. It returns zero for negative inputs and returns positive inputs unchanged.
A shortcut that adds a block's original input to its transformed output. Residual connections help information and gradients travel through deep architectures such as Transformers.
A supervised learning task in which the target is a continuous numerical value, such as computer performance or house price.
The square root of Mean Squared Error. It expresses prediction error on the same general scale as the target. Lower values are better.
One observation in a dataset. In tabular data, one row usually represents one sample.
Selecting an output from a model's probability distribution instead of always choosing the highest-probability option. Generative systems use sampling controls such as temperature to vary output diversity.
Attention in which queries, keys, and values are derived from the same input sequence. It creates context-aware token representations by comparing positions within that sequence.
An activation function that produces a value between 0 and 1. It is commonly used for the output of a binary classifier.
Linear regression with one input feature and one continuous target. It learns the slope and intercept of a fitted line, represented by \(\hat{y}=wx+b\).
An output activation that converts multiple scores into class probabilities that sum to approximately 1. The class with the largest probability becomes the prediction.
A preprocessing transformation that centers and scales numerical features so their values are comparable. Its parameters should be learned from training data only.
A type of Machine Learning in which the training data contains both input features and known targets.
The value or class that a supervised model learns to predict. PRP and is_spam are targets in the course examples.
An activation function that smoothly maps values into the range from -1 to 1. Its zero-centered output can be useful in hidden layers.
A sampling control that rescales logits before Softmax. Lower values concentrate probability on likely outputs, while higher values produce a flatter distribution and more varied choices.
Term Frequency–Inverse Document Frequency, a sparse text representation that emphasizes terms common in one document but less common across the training corpus.
The subset of data reserved for evaluating a trained model on unseen samples. It must not influence model training or fitted preprocessing parameters.
A unit processed by a language model, such as a word, part of a word, punctuation mark, or other encoded text fragment. Next-token prediction estimates a probability distribution for the token that follows the current context.
The process of converting text into an ordered sequence of tokens. Tokenization rules are part of the input contract and must remain consistent between training and inference.
The process of learning model parameters from examples by repeatedly calculating predictions, measuring loss, and updating weights and biases.
The subset of data used to fit preprocessing and learn model parameters.
A negative sample correctly predicted as negative. For spam detection, this is a legitimate email correctly left outside the spam class.
A positive sample correctly predicted as positive. For fraud detection, this is a fraudulent transaction correctly detected as fraud.
The division of a dataset into separate training and evaluation subsets.
A sequence-model architecture built from attention, learned projections, feed-forward networks, residual connections, normalization, and positional information. The current cookbook explains the concept but does not include a runnable Deep Netts Transformer project.
A condition in which a model has not learned the relevant pattern well enough and performs poorly even on its training data.
A training problem in which gradients become extremely small as they are propagated through many layers, causing early layers to learn very slowly or stop learning.
A generative model that learns a structured probability distribution in a latent space and decodes sampled latent values into new data.
The stable mapping between known text tokens and their identifiers or feature positions. Vocabulary selection and corpus-derived statistics must be fitted on training data only.
A trainable parameter that controls how strongly an input or neuron output influences another neuron.
A method for learning static word embeddings from patterns of nearby words in a text corpus. Words used in similar contexts receive nearby vectors, but each word keeps the same vector across different sentences.
