Reference
Glossary
One entry per concept listed under “Related concepts” across the demos and the fundamentals section. Search by term or filter by content; each term links back to where it appears.
23 terms found
- centroids
Points that represent the center of each cluster in K-Means, computed as the mean of the points assigned to that cluster; recalculated on every iteration.
- choosing k
Deciding how many clusters to look for in K-Means before running the algorithm — unlike KNN, k here has no single correct answer and usually relies on heuristics like the elbow method.
- cosine similarity
A similarity measure between two vectors based on the angle between them, not their magnitude; TF-IDF uses it to compare documents represented as term vectors.
- cross-validation
A technique to estimate how well a model generalizes by splitting the data into several subsets and rotating which one is used as the test set, instead of relying on a single train/test split.
- depth
The number of levels in a decision tree from the root to its farthest leaf. Bounding it is one of the simplest ways to limit a tree's overfitting.
- distance
A numeric measure of how far apart two points are in feature space. KNN classifies based on the distance (Euclidean or Manhattan) to the nearest training points.
- entropy
An alternative to Gini for measuring a node's impurity, based on information theory; both look for the same kind of split, with a different formula.
- feature scaling
Transforming features so they share a comparable scale (for example, standardizing to mean 0 and standard deviation 1) before measuring distances, so no single feature dominates the result just because of its unit.
- Gini
An impurity index that measures how mixed the classes are at a decision tree node; CART picks, at each split, the division that most reduces weighted Gini impurity.
- gradient descent
An iterative algorithm that adjusts a model's parameters by taking steps in the direction that most reduces the loss function, controlled by a learning rate.
- inertia
The sum of squared distances from each point to its assigned centroid in K-Means; a measure of how compact the resulting clusters are, used to compare runs or choose k.
- inference cost
The computational work required to predict on a new point. In brute-force KNN it grows with the size of the training dataset, unlike models that precompute a fixed rule during training.
- initialization
The choice of initial centroid positions in K-Means, before the first iteration. A bad initialization can lead to convergence on a poor local optimum.
- log loss
Binary cross-entropy: a classification loss function that penalizes a confident, wrong prediction more heavily than an unsure, wrong one. Logistic regression minimizes it via gradient descent.
- loss function
A function that measures how far a model's predictions are from the real values; training is, almost always, minimizing it. Linear regression uses mean squared error.
- metrics
Numbers that summarize a classification model's performance from the confusion matrix — accuracy, precision, recall, F1, among others — each sensitive to a different kind of error.
- outliers
Points that sit far from the rest of the dataset. They can distort a fit (like linear regression's) far more than their count would suggest.
- overfitting
When a model learns the noise and quirks of the training set instead of the general pattern, and so generalizes poorly to new data.
- regularization
A penalty added to a loss function to discourage overly complex models and reduce overfitting, at the cost of extra bias.
- sigmoid function
A function that squashes any real number into the (0, 1) interval, shaped like an S: σ(z) = 1 / (1 + e⁻ᶻ). Logistic regression uses it to turn a linear combination of features into a probability.
- TF-IDF
Term Frequency–Inverse Document Frequency: weighs each term by how often it appears in a document (TF) versus how many documents in the collection contain it (IDF), to give more weight to distinctive terms.
- tokenization
Splitting a text into smaller units (tokens, typically words) before processing it. The first step of TF-IDF on any document.
- underfitting
When a model is too simple to capture the real pattern in the data, and so fails on both training and test — the opposite of overfitting.