Dimensionality Reduction

This involves simplifying data while striving to retain as much information as possible.

Supervised Learning Algorithms

This type of learning focuses on making predictions based on labeled data. For example, predicting house prices, where the label is the house price itself.

• K-Nearest Neighbors
• Linear Regression
• Logistic Regression
• Support Vector Machines (SVM)
• Decision Trees
• Random Forests
• Neural Networks

Unsupervised Learning Algorithms

Here, predictions are made without labeled data. For instance, predicting house prices without knowing the actual prices beforehand.

• Clustering (KMeans, DBScan, HCA)
• Anomaly and novelty detection (One-class SVM, Isolation Forest)
• Visualization and dimensionality reduction (PCA, Kernel PCA, LLE)
• Association Rule Learning (Apriori, Eclat)

Association Rule Learning

This concept can be illustrated through the example of arranging products on a store shelf, where items that are frequently purchased together are placed in proximity.

Reinforcement Learning

In this learning approach, an agent learns by interacting with its environment. For example, teaching a robot to walk.

Batch Learning

This method involves the system learning in stages.

Online Learning

In contrast, online learning allows the system to learn from small batches of data. When providing services, it’s critical to monitor system performance closely, as poor incoming data can compromise both the model and service quality. Setting limits and regularly reviewing data is advisable.

Learning Rate

This parameter determines the size of the steps taken during learning.

• High Learning Rate: Quick process, low memory use, sensitive to outliers.
• Low Learning Rate: Slower process, high memory use, less sensitivity to outliers.

Instance-Based Learning

This approach enables the system to learn from past experiences and apply this knowledge to predict new events.

Model-Based Learning

In model-based learning, the system analyzes data, constructs a model to interpret it, and makes predictions based on this model.

Model Parameters

The function of model parameters can be illustrated through examples.

Sampling Noise

When working with limited data, the model's performance may suffer.

Sampling Bias

On the other hand, having a large dataset does not guarantee effective model performance.

Feature Engineering

This involves combining features to identify the optimal parameters for a model, specifically those that correlate strongly with the target variable you wish to predict.

Feature Extraction

When model parameters are interrelated, they can be consolidated. For instance, in real estate data, one might consider the number of rooms per square meter.

Regularization

This process limits model complexity to prevent overfitting.

Hyperparameter

Hyperparameters set the values for model parameters, such as learning rate, guiding the search for an optimal-performing model.

Grid Search

This technique allows for the manual adjustment of hyperparameters.

Stratified Sampling

This method ensures that a sample accurately represents the population. For example, if the population distribution is 54% male and 46% female, the sample should reflect this ratio.

Map-Reduce

In cases of large datasets, dividing the data across multiple servers is referred to as map-reduce.

Cross-Validation

This involves using a smaller validation set to train different models, helping to identify the best-performing model for production use.

Randomized Search

When faced with numerous hyperparameters, this technique randomly selects a limited number of combinations for each hyperparameter, rather than exhaustively evaluating all possibilities.