Professional Documents
Culture Documents
models (1)
models (1)
models (1)
Data Science
Boosting
Stacking
RNN
Reinforcement
Time series
Decomposition Linear
Non-Linear
Matrix Factorization
Manifold Learning
Column4
ALGORITHM
Linear Regression
Polynomial Regression
Ridge Regression
Lasso
Elastic Net
XGBoost
Decision Trees
Random forest
K-Nearest neighbours
Naïve Bayes
Decision Trees
Logistic
Gradient Boost
Random forest
K-Means
Mean shift
Heirarchial clustering
Agglomerative
Apriori
Euclat
FP-Growth
Random forest
AdaBoost
LightBoost
LeNet-5
AlexNet
VGGNet
GoogLeNet
MobileNet
EfficientNet
Heirarchial Network
Bidirectional
Generator
Discriminator
seq2seq
Genetic Algorithm
Deep Q-Network
Q-Learning
ARIMA
SARIMA
Exponential Smoothing
Seasonal Decomposition
Vector Autoregression(VAR)
Guassian processes
Prophet
Neural prophet
Wavelet Transforms
Factor Analysis
Multidimensional Scaling
Isomap
Kernel PCA
Auto Encoders
Difussion maps
Column5
DESCRIPTION
Predicts the target variable as a linear function of a single predictor variable.
Adds a penalty equal to the square of the magnitude of coefficients to the loss function to prevent
overfitting.
Adds a penalty equal to the absolute value of the magnitude of coefficients to the loss function,
leading to sparse models.
Combines L1 and L2 regularization penalties to take advantage of both ridge and lasso regression.
Uses support vector machine principles for regression, capable of fitting complex, non-linear
functions.
Builds an ensemble of trees sequentially, where each tree corrects errors made by the previous
ones. Combines the predictions of weak learners to form a strong prediction.
Splits the data into regions and fits a simple model (usually a constant) within each region.
Prediction is the mean value of the target variable in each leaf node.
An ensemble of decision trees, where each tree is trained on a random subset of the data and
predictions are averaged. Aggregates predictions from multiple decision trees.
Incorporates prior distributions for the model parameters and updates these distributions based on
the data. Bayesian inference is used to derive the posterior distribution of the model parameters.
Classifies a data point based on the majority class among its k-nearest neighbors in the feature
space. Simple and effective for small datasets with clear class boundaries.
Assumes independence among predictors given the class. Simple, fast, and works well with small
datasets and text classification.
Finds the hyperplane that best separates classes in the feature space. Uses kernel functions to
handle non-linear boundaries.
Splits the data into subsets based on feature values, forming a tree-like structure. Can handle both
numerical and categorical data.
Models the probability of a binary outcome. Uses the logistic function to constrain the output
between 0 and 1.
Builds an ensemble of trees sequentially, where each tree corrects errors made by the previous
ones. Includes popular implementations like XGBoost, LightGBM, and CatBoost.
An ensemble of decision trees, where each tree is trained on a random subset of the data.
Combines predictions from multiple trees to improve accuracy and reduce overfitting.
One of the most popular clustering algorithms, K-means partitions the data into K clusters by
iteratively assigning data points to the nearest cluster centroid and updating the centroids based
on the mean of the points assigned to each cluster. It aims to minimize the within-cluster variance.
Mean Shift is a non-parametric clustering algorithm that doesn't require specifying the number of
clusters beforehand. It works by iteratively shifting each data point towards the mode (peak) of the
kernel density estimate of the data until convergence.
This method creates a hierarchy of clusters by either bottom-up (agglomerative) or top-down
(divisive) approaches. In agglomerative clustering, each data point starts in its own cluster, and
pairs of clusters are merged as one moves up the hierarchy. In divisive clustering, all data points
start in one cluster, which is recursively split as one moves down the hierarchy.
DBSCAN groups together points that are closely packed together, based on a specified distance
(epsilon) and minimum number of points (minPts) within that distance. It can discover clusters of
arbitrary shapes and is robust to noise.
This hierarchical clustering algorithm starts with each point as its own cluster and merges the
closest pair of clusters iteratively until only one cluster remains.
Identifies frequent itemsets and generates association rules. Used in market basket analysis.
Uses a vertical database format for finding frequent itemsets. Efficient for datasets with many
frequent itemsets.
More efficient than Apriori by using a prefix tree to store frequent patterns. Avoids generating
candidate sets explicitly.
An extension of bagged decision trees where each tree is trained on a bootstrap sample of the data
and, additionally, a random subset of features is considered for splitting at each node. This
decorrelates the trees and improves performance.
GBM builds an ensemble of decision trees sequentially, where each tree is trained to correct the
errors of the previous trees. It uses gradient descent to minimize the loss function.
Specifically designed to handle categorical features efficiently, CatBoost uses ordered boosting to
reduce prediction shift and support categorical data directly.
A variant of gradient boosting that introduces randomness by subsampling the training data and
features. This can help reduce overfitting.
Combines multiple base models (e.g., decision trees, logistic regression, SVMs) and trains a meta-
model on their predictions. The base models are typically trained on the original data, and the
meta-model is trained on the outputs (predictions) of the base models.
Introduced the concept of residual learning to address the degradation problem in deep networks.
Uses skip connections to allow gradients to flow more easily through the network.
One of the earliest CNN architectures designed by Yann LeCun for handwritten digit recognition
(MNIST dataset).
A deeper architecture that popularized CNNs in large-scale image classification tasks. It introduced
the use of ReLU activation and dropout for regularization.
Characterized by its use of very small (3x3) convolutional filters and a deep architecture with up to
19 layers.
Introduced the Inception module, which allows for more efficient computation by combining
multiple convolutions with different filter sizes.
Designed for mobile and embedded vision applications, it uses depthwise separable convolutions
to reduce the number of parameters.
Uses a compound scaling method to balance network depth, width, and resolution for improved
efficiency.
Uses dense blocks where each layer receives input from all previous layers, improving gradient flow
and feature reuse.
An extension of RNN designed to overcome the vanishing gradient problem. LSTMs have a more
complex architecture that includes gates (input gate, forget gate, and output gate) to control the
flow of information.
A simplified version of LSTM that combines the input and forget gates into a single update gate and
merges the cell state and hidden state. GRUs are computationally more efficient than LSTMs while
still addressing the vanishing gradient problem.
Combines the strengths of CNNs and RNNs by applying convolutional layers to capture spatial
features and recurrent layers to capture temporal features.
Organizes RNN layers hierarchically to capture different levels of abstraction at different temporal
scales.
Consists of two RNNs running in parallel, one in the forward direction and one in the backward
direction. This allows the network to have both past and future context.
The generator algorithm is responsible for creating new data samples. It takes random noise as
input and generates data that should ideally be indistinguishable from real data.
The discriminator algorithm is like a binary classifier. It tries to distinguish between real data
samples and the fake samples generated by the generator. Its goal is to correctly classify real data
as real (label 1) and fake data as fake (label 0).
An encoder and a decoder, widely used for tasks like machine translation and text summarization,
where it maps input sequences to output sequences, effectively capturing contextual information
and generating meaningful responses. It utilizes recurrent neural networks or transformers to
encode source sequences into fixed-length representations and decode them into target
sequences, enabling the modeling of complex relationships between inputs and outputs.
Genetic Algorithms are optimization techniques inspired by natural selection, using selection,
crossover, and mutation to evolve candidate solutions towards optimal outcomes across diverse
problem domains. They efficiently explore large search spaces, providing robust solutions to
complex optimization problems, albeit with sensitivity to parameter settings and computational
demands.
DQN is an extension of Q-Learning that uses deep neural networks to approximate the action-value
function. It employs experience replay and target networks to stabilize training and improve
sample efficiency.
Exponential smoothing methods, such as Simple Exponential Smoothing (SES), Double Exponential
Smoothing (DES), and Triple Exponential Smoothing (Holt-Winters), are simple yet effective
techniques for forecasting time series data. They assign exponentially decreasing weights to past
observations.
STL decomposes time series data into seasonal, trend, and remainder components using a process
of repeated filtering.
VAR models are used for multivariate time series forecasting, where multiple time series variables
are modeled simultaneously as a system of equations.
State space models, including the Kalman Filter and Bayesian Structural Time Series (BSTS) models,
represent time series data as a latent state process evolving over time.
LSTM networks are a type of recurrent neural network (RNN) capable of learning long-term
dependencies in sequential data, making them well-suited for time series forecasting tasks.
Gaussian processes are a flexible non-parametric Bayesian approach for modeling time series data.
They model the distribution over functions and can capture complex patterns in the data.
Prophet is a forecasting tool developed by Facebook that decomposes time series data into trend,
seasonality, and holiday effects and utilizes a piecewise linear or logistic growth curve model.
Wavelet transforms are used for time-frequency analysis of non-stationary time series data,
allowing for both time and frequency localization.
DLMs are Bayesian state space models that can be used for time series analysis and forecasting by
specifying dynamic relationships between observed and unobserved variables.
Projects data onto the directions of maximum variance. Commonly used for feature extraction and
data visualization.
Projects data to a lower-dimensional space by maximizing class separability. Often used for
supervised classification tasks.
Models observed variables and their linear combinations using latent factors. Used for uncovering
hidden relationships in the data.
Decomposes multivariate signals into additive, independent components. Commonly used in signal
processing and for separating mixed signals.
Projects data into a lower-dimensional space while preserving pairwise distances as well as
possible. Used for data visualization.
Reduces dimensions by modeling pairwise similarities and preserving local structure. Primarily used
for visualization of high-dimensional data.
Preserves geodesic distances between all points. Captures the intrinsic geometry of the data
manifold.
Preserves local linear relationships among neighboring points. Good for unwrapping manifolds in
high-dimensional data.
Extends PCA using kernel methods to handle non-linear relationships. Used for capturing non-
linear structures.
Neural network-based method that reduces dimensions while preserving topological properties.
Used for clustering and visualization.
Neural networks trained to compress and reconstruct data. Capture complex non-linear
relationships and are used for unsupervised learning.
Decomposes a matrix into singular vectors and singular values. Used for latent semantic analysis,
noise reduction, and feature extraction.
Factorizes a matrix into non-negative factors. Useful for parts-based representation in image
processing and text mining.
Uses Gaussian processes to learn a latent space representation. Captures uncertainty in the
embedding process.
Uses diffusion processes to model the data geometry. Preserves the global data structure and is
robust to noise.
Column6
Hyperparameters
fit_intercept, nomalize
degree, include_bias
alpha, solver
alpha, max_iter
alpha, l1_ratio
c, kernel, gamma
n_neighbors, weights
c, kernel, gamma
penalty, c, solver
n_estimators, learning_rate, max_depth, min_samples_split, min_samples_leaf, min_weight_fraction_leaf, max_features,
max_leaf_nodes, min_impurity_decrease, min_impurity_split, init, random_state, verbose, warm_start, presort,
validation_fraction, n_iter_no_change, tol, ccp_alpha
n_clusters, max_iter
n_estimators' specifies the number of boosting rounds, or equivalently, the number of trees to be built sequentially. Range(10
'learning_rate' controls the step size at each iteration while moving toward a minimum of the loss function. Range(0.001 - 1)
'max_depth' determines how deeply each tree is allowed to grow during the boosting process. Range(1 - 20 or more)
'min_child_weight' specifies the minimum sum of instance weights (also known as the minimum sum of Hessians) n
'gamma' controls the minimum loss reduction required to make a further partition on a leaf nod
'subsample' controls the fraction of the training data that is randomly sampled to train each tr
'colsample_bytree' controls the fraction of features (columns) to be randomly sam
'colsample_bylevel' controls the fraction of features (columns) to be ran
construction process. Range(0.1 - 1) 'colsample_bynode' control the fraction of features (co
construction process. Range(0.1 - 1) 'reg_alpha' applies L1 regularization to feature we
penalizing large weights. Range(0 - 10) 'reg_lambda' applies L2 regula
overfitting and improve model generalization. Range (0.0 - 10.0) 'scale_po
adjusting the weight of positive examples.
node weight can take during each boosting iteration. Range(0 - 10)
'objective' specifies the learning task and the corresponding loss function that the model aims to optimize. String type range
'booster' specifies the type of boosting algorithm to use during training. String type range
'tree_method' specifies the method used t
'grow_p
range
information printed during the training process. discrete integer values range
'sampling_method' specifies the method used to sample data points or features during the tree construction process.
n_neighbors ' determines the number of neighbors to consider when making predictions for a new data point. Range(positive
'weights ' determines how the contributions of neighboring points are weighted when making predictions for a n
alpha' applied to avoid zero probabilities for features that are not present in the training data. Range(0 to positive infinity)
'fit_prior ' controls whether class prior probabilities are learned from the data during training. Range(Boolea
'class_prior ' allows you to specify the prior probabilities of the classes explicit
c' controls the trade-off between achieving a low error on the training data and minimizing the complexity of the decision bou
'kernel' specifies the type of kernel function to be used in the algorithm. Range(string values)
'gamma' controls the influence of individual training examples. Range(positive float
max_depth ' controls the maximum depth of the tree. It determines the longest path from the root node to a leaf node. Range
'min_samples_split ' determines the minimum number of samples required to split an internal node during the c
'min_samples_leaf ' specifies the minimum number of samples required to be at a leaf node. Range(po
penalty' determines the type of regularization to apply to the model. Range(string values or None)
'c' controls the regularization applied to the model. Range( positive float values)
'solver' determines the algorithm u
values)
n_estimators ' specifies the number of boosting stages (decision trees) to be iteratively built. Range(positive integer values)
'learning_rate ' determines the contribution of each tree (or each iteration) to the overall ensemble. R
'max_depth ' controls the maximum depth of each individual tree in the ensemble. Ra
'min_samples_split ' determines the minimum number of sam
'min_samples_leaf ' spe
node after a split. Range(positive integer and float values) 'm
total input samples required to be in a leaf node. Range(0.0 - 0.5)
features to consider when looking for the best split at each node. Range(integer, float and string)
determines the minimum impurity decrease required for a split to occur at a node. Range(float values)
'min_impurity_split' is used to determine the minimum impurity threshold below which a node will not be split. Range( 0 - 0.5
'init 'specifies the initial estimator that will be used to compute the initial predictions.
'random_state ' serves as a source of randomness that contro
algorithm.Range(non-negative integer value) 'verbose ' controls the verbo
learned information and continuing training from that point onwards with additional iterations. Range(boolean )
faster tree building. Range(boolean )
'validation_fraction ' specifies the fraction of training data to set aside as a validation set for early stopping. Range(0
n_iter_no_change ' controls early stopping based on a lack of improvement in
'tol' determines the stopping criterion for the optim
'ccp_alpha ' controls the com
individual trees during the training process. Range(0 and extends to positive infinity)
n_estimators ' determines the number of decision trees to be used in the ensemble. Range(1 and extends to any positive integ
'criterion' determines the function used to measure the quality of a split when building decision trees within the
'max_depth ' controls the maximum depth of each decision tree in the ensemble. Range(
'min_samples_split' determines the minimum number of samples r
tree in the ensemble. Range(2 to infinity) 'min_samples_leaf ' determines the minimum number of samples requ
value) 'min_weight_fraction_leaf 'specifies
be at a leaf node. Range(0 - 0.5) 'max_features ' d
the best split at each node.Range(integer,float or string)
leaf nodes allowed in each decision tree within the ensemble. Range(2 to infinity)
specifies the minimum impurity decrease required for a split to happen during the construction of each decision tree within th
'min_impurity_split' minimum impurity decrease required for a split to occur during the construction of each decision tree wit
'bootstrap ' controls whether bootstrap samples are used when building trees in the ensemble. R
'oob_score ' controls whether to use out-of-bag samples to estimate the generalization accuracy of the model. Range(Boolean
'n_jobs ' determines the number of parallel jobs to run during the model training process. Rang
'random_state' serves as a seed for the random number
RandomState instance) 'verbose ' con
algorithm will produce output messages during training. Range(integer values or boolean )
previous call to fit() as the initialization for the next call. Range(Boolean)
assign different weights to classes in the dataset.
'ccp_alpha' controls the complexity of the trees in the forest by enforcing pruning based on the Cost-Complexity Pruning (CCP
'max_samples ' determines the maximum number of samples to be drawn from the dataset to train each
n_clusters ' specifies the number of clusters the algorithm should form. Range(1, number of samples in the dataset)
'max_iter ' specifies the maximum number of iterations the algorithm will run for a single run. Range
bandwidth' determines the radius of the window used to define the neighborhood around each point. Range(positive real num
'seeds' clustering specifies the initial points from which the algorithm starts its ite
Range(none,array) 'bin_seeding' used to accelerate the clustering process by initially se
'min_bin_freq' specifies the minimum num
potential seed point during the binning process. Range(1 to 10 or higher) 'c
cluster, even if they are far from any mode (peak) of the data distribution. Range(Boolean)
algorithm.
'max_iter'specifies the maximum number of iterations the algorithm will perform to converge to the cluster centroid
'verbose' controls the verbosity or level of detail of the algorithm's output during training. Range(no
n_clusters ' indicates the number of clusters the algorithm should ultimately produce. Range(1 to the total number of data po
'affinity' specifies the metric used to compute the linkage between clusters. Ranger(distance metrics)
'linkage' specifies the method used to compute the distan
'distance_threshold ' specifies
whether to compute the full tree of hierarchical clusters or only a truncated version of it. Range(Boolean)
'connectivity' specifies the connectivity matrix representing the connectivity constraints
'compute_distances ' specifies whether distances b
'memory
clustering process.
verbosity level of the algorithm's output during the clustering process. Range(integer values)
eps' controls the maximum distance between two samples for them to be considered as neighbors.
'min_samples ' determines the minimum number of points required to form a dense region, which can b
starts at 2)) 'metric ' specifies the distance metric to use when calculating the distance between points in t
metric_params 'allows you to specify ad
'algorithm
the clustering process. Range('auto', 'ball_tree', 'kd_tree', 'brute')
used for efficient nearest neighbor search. Range(integer value)
used in the Minkowski distance metric. Range(positive real numbers)
'n_jobs' specifies the number of parallel jobs to run for neighbor search and other parallelizable tasks. Range(-1 to the numb
n_clusters ' specifies the number of clusters to find. Range(2 to the total number of data points in the dataset )
'affinity ' specifies the metric used to compute the linkage between the clusters. Range(distance metrics)
'linkage' specifies the method used to compute the distance between
'distance_threshold ' specifies the threshold to use
the full tree of hierarchical clusters or only a truncated version of it. Range(Boolean)
'connectivity' specifies the connectivity matrix representing the connectivity constraints of the data.
'compute_distances ' specifies whether distances between points should
'memory' used to specify cach
min_support ' specifies the minimum support threshold for frequent itemsets. Range(0 - 1)
'min_confidence ' represents the minimum confidence threshold for association rules to be considere
'max_length ' specifies the maximum number of items (or f
unique items in the dataset) 'verbose' controls the verbosity level of the algorithm's out
'show_progress ' controls whether progres
Range(boolean)
and other parallelizable tasks. Range(-1 to the number of available CPU cores)
controls the random seed used for generating random numbers. Range( integer value)
min_support ' specifies the minimum support threshold for frequent itemsets. Range(0 - 1)
'max_length ' specifies the maximum number of items (or features) that can appear in an
'verbose' controls the verbosity level of the algorithm's output during th
'show_prog
execution of the algorithm. Range(boolean) 'n_j
other parallelizable tasks. Range(-1 to the number of available CPU cores)
controls the random seed used for generating random numbers. Range( integer value)
min_support ' specifies the minimum support threshold for frequent itemsets. Range(0 - 1)
'support' determines the minimum support threshold for identifying frequent itemsets. Range(0 - 1
'min_confidence ' represents the minimum confidence thresho
'max_length ' specifies the maximum number of items (or f
unique items in the dataset) 'max_itemsets ' determines the maximum number of freque
'n_jobs' specifies the number of parallel jobs to r
number of available CPU cores)
random numbers. Range( integer value)
the algorithm's output during the clustering process. Range(integer values)
n_estimators ' determines the number of decision trees to include in the ensemble. Range(10 - infinity)
'criterion' determines the function used to measure the quality of a split.
'max_depth ' controls the maximum depth of the decision trees in t
'min_samples_split' determines the min
construction of each decision tree in the ensemble. Range( 2 to 20) 'min_samples_leaf'
Range( integer value greater than or equal to 1)
samples required to be at a leaf node. (0 - 0.5)
number of features to consider when looking for the best split. It can be specified as an integer, float, or string. Range(0 - 0.5)
specifies the maximum number of leaf nodes that any individual tree in the ensemble is allowed to have. Range(2 and can go
'min_impurity_decrease ' specifies the minimum reduction in impurity that is required for a split to be considered in a node. R
'min_impurity_split ' is a threshold for early stopping in tree
'bootstr
individual trees. Range(Boolean)
accuracy of the model using out-of-bag samples. Range(Boolean)
controls the number of CPU cores used to perform parallel processing during model training and prediction.
'random_state ' controls the randomness of various aspects of the algorithm, en
'verbose' controls the verbosity le
n_estimators ' determines the number of decision trees to include in the ensemble. Range(10 - infinity)
'learning_rate ' controls the step size at each iteration while moving toward a minimum of the loss function. Range( 0.01 a
'n_estimators ' specifies the number of boosting rounds or trees that the model will build.
'max_depth ' specifies the maximum depth of the in
'min_child_weight
child. Range(1 to infinity)
make a further partition on a leaf node of the tree. Range(0 to positive infinity)
fraction of the training data that is randomly sampled to grow each tree. Range(0.1 - 1.0)
'colsample_bytree ' controls the fraction of features to consider when constructing each tree. Range(0.1 - 1.0)
'colsample_bylevel ' controls the fraction of features to consider when
'colsample_bynode ' con
Range(0.1 -1.0)
controls L1 regularization term on weights. It penalizes large individual weights, encouraging sparsity in the model. Range(0, ∞
'reg_lambda ' controls the L2 regularization term on weights. Range(0, ∞)
'scale_pos_weight ' used to balance the class distribution in binary classification tasks
'max_delta_step ' used to help control the step size during the update of th
'objective 'specifies the loss function to be opti
'
the base learner used in the boosting process. Range(categorical)
used to build decision trees during training.
'grow_policy ' determines how trees are grown during the boosting process. Range (categorical)
'verbosity ' controls the amount of information printed during training. Range(0 - 3)
'sampling_method ' determines the method used for sampling d
Linear
Polynomial
Ridge
Lasso
Regression
Elastic Net
Bayesian
Quantile
Support vector
Logistic
Classification Support vector
KNN