ARTIFICIAL INTELLIGENCE

What is Artificial Intelligence ?

Ans : Artificial Intelligence is composed of two words Artificial and Intelligence,

where Artificial defines "man-made," and intelligence defines "thinking power", hence
AI means "a man-made thinking power."

So, we can define AI as:

"It is a branch of computer science by which we can create intelligent machines which
can behave like a human, think like humans, and able to make decisions."=

What are Goals of AI ?

Ans :

1. Replicate human intelligence

2. Solve Real world complex tasks3-6+
3. Building a machine which can perform tasks that requires human intelligence
such as:
1. Proving a theorem
2. Playing chess
3. Plan some surgical operation
4. Driving a car in traffic

What are the types of AI ?

Ans - Weak AI or Narrow AI: Weak AI is capable of performing some dedicated

tasks with intelligence. Siri is an example of Weak AI.

Strong AI: It is the hypothetical concept that involves the machine that will be better
than humans and will surpass human intelligence.

What is Turing Test ?

Ans :The Turing test is a method to test a machine’s ability to match human-level intelligence. A
machine is used to challenge human intelligence, and when it passes the test it is considered
intelligent.
What is an expert system?

An expert system is an Artificial Intelligence program that has expert-level

knowledge about a specific area and how to utilize its information to react
appropriately.

What is an A* algorithm search method?

A* is a computer algorithm that is extensively used for the purpose of finding

the path or traversing a graph in order to find the most optimal route between
various points called the nodes.

MACHINE LEARNING

What is ML - Machine learning enables a machine to automatically learn from data,

improve performance from experiences, and predict things without being explicitly
programmed. A Machine Learning system learns from historical data, builds the
prediction models, and whenever it receives new data, predicts the output for
it.

What are the types of ML ?

Ans –

Supervised Learning - It is a type of machine learning method in which we provide

sample labelled data to the machine learning system in order to train it, and on that
basis, it predicts the output. The system creates a model using labeled data to
understand the datasets and learn about each data, once the training and processing
are done then we test the model by providing a sample data to check whether it is
predicting the exact output or not. The goal of supervised learning is to map input
data with the output data.

Unsupervised Learning - The training is provided to the machine with the set of data
that has not been labeled, classified, or categorized, and the algorithm needs to act
on that data without any supervision. The goal of unsupervised learning is to
restructure the input data into new features or a group of objects with similar
patterns.
In unsupervised learning, we don't have a predetermined result. The machine tries to
find useful insights from the huge amount of data.

Reinforcement Learning - Reinforcement learning is a feedback-based learning

method, in which a learning agent gets a reward for each right action and gets a
penalty for each wrong action. The agent learns automatically with these feedbacks
and improves its performance. In reinforcement learning, the agent interacts with the
environment and explores it. The goal of an agent is to get the most reward points,
and hence, it improves its performance.

DIFFERENCE BETWEEN AI & ML ?

Artificial Intelligence Machine learning

Artificial intelligence is a superset which Machine learning is a subset of AI which

contains ml & dl.

The goal of AI is to make a smart to solve The goal of ML is to allow machines to

complex problems like humans learn from data so that they can give
accurate output.

Linear Regression : It is a statistical method that is used for predictive analysis. Linear
regression algorithm shows a linear relationship between a dependent (y) and one or
more independent (y) variables, hence called as linear regression. Since linear
regression shows the linear relationship, which means it finds how the value of the
dependent variable is changing according to the value of the independent variable.

The main aim of LR is to find the best fit line so that the actual and predicted points
are close to each other.

What is multicollinearity and how does it affect linear regression analysis?

Multicollinearity refers to a situation in which two or more independent variables in a linear

regression model are highly correlated with each other. This can create problems in the
regression analysis, as it can be difficult to determine the individual effects of each
independent variable on the dependent variable.

What are the common techniques used to improve the accuracy of a linear
regression model?

 Feature selection: selecting the most relevant features for the model to improve its
predictive power.
 Feature scaling: scaling the features to a similar range to prevent bias towards certain
features.
  Cross-validation: dividing the data into multiple partitions and using a different
partition for validation in each iteration to avoid overfitting.
 Ensemble methods: combining multiple models to improve the overall accuracy
and reduce variance

Classification Algorithm : These algorithm tries to classify the data into labels like
Yes/No, 0/1, Spam/Not Spam. Regression give output in value but classification give
output in category.

Confusion Matrix – It is a matrix that gives us the performance of the model in terms
of true and false positive/negative

Log loss – It is generally used for the model whose output is between 0 and 1

1. Logical Regression - It is used for predicting the categorical dependent

variable using a given set of independent variables. Logistic regression
predicts the output of a categorical dependent variable. Therefore the
outcome must be a categorical or discrete value. It can be either Yes or No, 0
or 1, true or False, etc. but instead of giving the exact value as 0 and 1, it gives
the probabilistic values which lie between 0 and 1.

2. K-Nearest Neighbour : It works on similarity method i.e based on the old data
it has stored it sees the similarity between the new and the old data and based
on that it classify the new data . It is also called as lazy-learner coz it spends
less time on learning on the training dataset.The decision is based on
Euclidean or manhatten distance depending on which data points are
classified

3. SVM algorithm: Support Vector machine tries to find the optimal

hyperplane(boundary) which can divide the data points in different classes in
the feature space. The dimension of the hyperplane depends upon the number
of features. If the number of input features is two, then the hyperplane is just
a line. If the number of input features is three, then the hyperplane becomes a
2-D plane.

Support vector : Data points that are close to the hyperplane

Hard Margin : hard margin hyperplane is a hyperplane that properly separates

the data points of different categories without any misclassifications.

Soft Margin: When the data is not perfectly separable or contains outliers,
SVM permits a soft margin technique.
 Regression Algorithm Classification Algorithm

In Regression, the output variable must be of In Classification, the output variable

continuous nature or real value. must be a discrete value.

Decision Tree Algorithm : It is a tree data structure where the nodes represents the
features of the dataset , edges represent the decision rules and the leaf node
represent the output .It starts with a single dataset as a root node , after this the best
attribute through attribute selection measure is been calculated and it is been
divided into its possible values and this process is recursively applied until no further
splitting of node is possible.

Ensemble learning - Ensemble learning is a combination of the results obtained from

multiple machine learning models to increase the accuracy for improved decision-making.
Example: A Random Forest with 100 trees can provide much better results than using just one
decision tree.

Random Forest : It is a type of ensemble learning algorithm which takes the average of all the
output of various number of decision tree and then predicts the output.

K-means clustering : It is an unsupervised machine learning algorithm that is used for

clustering purpose It tries to group the data into different cluster based on their similarity .

BIAS : Difference between the predicted and actual values.

VARIANCE : It shows how much a model function changes by changing the training dataset.

Regularization : Regularization is like adding a gentle constraint to the learning process,

encouraging the model to focus on the most important features and avoid relying too much on noise or
irrelevant details in the data. It is about making the model to learn more and memorize
less.It is used to reduce overfitting.

The main goal of each machine learning model is to generalize well. Here
generalization defines the ability of an ML model to provide a suitable output by
adapting the given set of unknown input.
Overfitting : When the model tries to learn the training data so well that it also
catches the noise and unwanted characteristics . When it is introduced to new
dataset it gives wrong output.It has low bias and high variance.

It can be controlled by :

1. Regularization – It is a basically a constraint on learning process of a model so

that it does not depends only on single feature . It is like saying Yes, you can use
all the features, but don't go crazy with any single one.
2. By reducing the features

Underfitting : When the model fails to learn the hidden relationships among the
features in the training dataset it gives false output.

Feature Engineering : Feature Engineering is the process of extracting and organizing the
important features from raw data in such a way that it fits the purpose of the machine learning
model. It includes selecting relevant features, handling missing data, encoding the data(data
transformation).

Handling of missing values : We can drop the column if its not relevant , or can either imput
mean,median of the column , or can set default values for the columns.

Outliers : They are the datapoints that significantly deviates from the rest of the data points .
It may be due to inconsistent data entry . It can be detected with the help of z-score - It is
used to determine how much a datapoint is away from the mean.In order to deal with outlier
we can either remove them or impute them with median.

Principal Component Analysis : It is a dimensionality reduction techniques which identifies

important relationships , than transform the exiting data based on these relationhips so that
we can keep these features and drop the others.

Accuracy – We cant use this in case of imbalance dataset

Precision : Ratio of Tp/Tp+Fp

Recall : Tp/Tp+Fn

DEEP LEARNING
 Deep Learning is a computer software that mimics the network of
Defn :
neurons in a brain. It is a subset of machine learning based on artificial
neural networks. It is called deep learning because it makes use of deep
neural networks. The basic components present in a deep learning are :

1. Input layer : In this input is been fedded

2. Hidden Layer : In this layer mathematical computation by different nural
networks are been performed.
3. Output Layer : It gives us the output

Application :

 Pattern recognition and natural language processing.

 Classification and Detection of Objects.

Artificial Neural Network ?

Ans : Artificial neural networks are built on the principles of the structure and
operation of human neurons. An artificial neural network’s input layer, which is the
first layer, receives input from external sources and passes it on to the hidden layer,
which is the second layer. Each neuron in the hidden layer gets information from the
neurons in the previous layer, computes the weighted total, and then transfers it to
the neurons in the next layer.

Machine Learning Deep Learning

Apply statistical algorithms to learn Uses artificial neural network architecture

the hidden patterns and relationships to learn the hidden patterns and
in the dataset. relationships in the dataset.

Can work on the smaller amount of Requires the larger volume of dataset
dataset compared to machine learning

Takes less time to train the model. Takes more time to train the model.

Learning Rate : The learning rate determines how quickly or slowly a neural network model
adapts to a given situation and learns. A higher learning rate value indicates that the model only
needs a few training epochs and produces rapid changes, whereas a lower learning rate indicates
that the model may take a long time to converge or may never converge and become stuck on a poor
solution.

Types of Neural Network :

1. Feed Forward Neural Network : This is the most basic type of neural network, in which
flow control starts at the input layer and moves to the output layer. These networks only
have a single layer or a single hidden layer. The input layer of this network receives the sum
of the weights present in the input. These networks are utilised in the computer vision-based
facial recognition method.
CNN is a multi-layered neural network
2. Convolutional Neural Network :
with a unique architecture designed to extract increasingly
complex features of the data at each layer to determine the
output. CNN is mostly used when there is an unstructured data set
(e.g., images) .
Eg : A model has to predict a caption for the image . The image is of cat but for model it’s a
collection of pixels . The hidden layer will try to extract relevant features such as tail of
cat , head of a cat . After this depending on the probability of these features the model will
predict the caption.
3.

Convolutional Neural Network

In human brain there is a cerebral cortex which contains vision cortex that
consist of multiple layers which tries to extract some relevant information from
an images.
What is Convolutional ?
Ans: The term convolution refers to the mathematical combination of two
functions i.e matrix multiplication of image pixel matrix and filter matrix to
produce a third function(matrix). It merges two sets of information. In the case
of a CNN, the convolution is performed on the input data with the use of a filter
by the help of keras. Because the size of the feature map is always smaller than
the input, we have to do something to prevent our feature map from shrinking.
This is where we use padding. An extra row and column in input matrix of
zero-value pixels is added to surround the input with zeros, so that our feature
map will not shrink.
Maxpooling – So there might be an instane where there are multiple objects in
the image to detect , so inorder to detect all the object after the convolutional
layer we add an maxpooling layer which takes the high intensity pixels from the
image and insert it in output.
Forward Propogation – We recieves the input with weights which is then passed
to hidden layers in which first layer will compute the optimized weights and
passed to the second and so on untill we reach the output layer.So the
information flows in forward direction.
Backward Propogation - It is the method of fine-tuning the weights of a
neural network based on the error rate obtained in the previous epoch
(i.e., iteration). Proper tuning of the weights allows you to reduce error
rates and make the model reliable by increasing its generalization.
Backpropagation in neural network is a short form for “backward
propagation of errors.” This method helps calculate the gradient of a
loss function with respect to all the weights in the network.

How to read csv file : pd.read_csv(‘document.csv’)

data.shape : gives no of rows and columns
data.dtype: Give the datatype of each column.
data.describe : Give u the statistical data like count , mean , std, frequency etc.
data.isnull() : Give u the null value present in the data.

Other times we might want to do a quick check to see if we have any

missing values at all.
# Any missing values?
print df.isnull().values.any()Out:
True

We might also want to get a total count of missing values.

# Total number of missing values
print df.isnull().sum().sum()Out:
8
# Replace missing values with a number
df['ST_NUM'].fillna(125, inplace=True)
More likely, you might want to do a location based imputation.
Here’s how you would do that.
# Location based replacement
df.loc[2,'ST_NUM'] = 125

A very common way to replace missing values is using a median.

# Replace using median
median = df['NUM_BEDROOMS'].median()
df['NUM_BEDROOMS'].fillna(median, inplace=True)
What is Imbalanced dataset and how to deal with it ?
Ans : These types of problems occurred generally in classification algorithm
which tries to classify the output into Yes/No etc. When one of the class labels’s
number of observation is significantly lower than other class labels.In simple
terms there is uneven distribution of data. Some times when the records of a
certain class is much more than the other class then the classifier gets biased
towards it.
For balanced dataset we can use the accuracy as an evaluation metric but for
imbalanced dataset we can use F1 score which is a harmonic mean of precision
and recall.It give equal importance to both as imbalance data can lead to more
false negative on minority class.
Precision : Ratio of Tp/Tp+Fp
Recall : Ratio of Tp/Tp+Fn
Resampling : In this we can either delete the rows from majority class and
match to minority this is called as undersampling or we can add rows in
minority to match with majority this is called as oversampling.
What is Type 1 and Type 2 error ?
Ans : Type 1 error is false positive eg : Telling a man that he is pregnant. Type 2
error is false negative eg : Telling a pregnant women that she is not having a
baby.
What is more important model accuracy or model performance ?
Ans : Both accuracy and performance are important , sometimes accuracy is
important as it tell the no of correct predictions made but this parameter fails
due to imbalanced data as the model gets bias towards majority class .
Performance on the other end gives us the insights of the model at different
scenario and hence depending on the problem statement we can think of this.
What is Roc Curve ?
What is Hyperparameter Tuning ?
Ans : Hyperparameters are parameter that decides the model architecture like
its learning rate, complexity of model. These are not model parameters that
can be learned during the training process.
Grid search
With this technique, we simply build a model for each possible
combination of all of the hyperparameter values provided,
evaluating each model, and selecting the architecture which
produces the best results.

What is Machine learning Libraries ?

What Is ‘naive’ in the Naive Bayes Classifier?

The classifier is called ‘naive’ because it makes assumptions that may or may not turn out to
be correct.

The algorithm assumes that the presence of one feature of a class is not related to the
presence of any other feature (absolute independence of features), given the class variable.

For instance, a fruit may be considered to be a cherry if it is red in color and round in shape,
regardless of other features. This assumption may or may not be right (as an apple also
matches the description).

Correlation: Correlation tells us how strongly two random variables are related to each other.
It takes values between -1 to +1.

Matplotlib : It is also an open-source library and plots high-defined figures like pie
charts, histograms, scatterplots, graphs, etc.
Pandas : It is an open-source machine learning library that provides flexible high-
level data structures and a variety of analysis tools. It eases data analysis, data
manipulation, and cleaning of data. Pandas support operations like Sorting,
Concatenation, Conversion of data, Visualizations, Aggregations, etc.
Numpy : It is a popular machine learning library that supports large matrices and
multi-dimensional data. It consists of in-built mathematical functions for easy
computations.
Scipy : The name “SciPy” stands for “Scientific Python”. It is an open-source library
used for high-level scientific computations.
Scikit-learn is an open-source library that supports machine learning. It supports
variously supervised and unsupervised algorithms like linear regression,
classification, clustering, etc.
Tensorflow: It is an end-to-end machine learning library that deals specifically with
deep learning-related task. It provides various methods libraries that can be used on
unstructured data.
Keras: It is a high-level API developed by Google for implementing neural network-
related tasks.
If your dataset is suffering from high variance, how would you handle it?

For datasets with high variance, we could use the bagging algorithm to handle it. Bagging
algorithm splits the data into subgroups with sampling replicated from random data. After the
data is split, random data is used to create rules using a training algorithm. Then we use
polling technique to combine all the predicted outcomes of the model.

Explain the differences between Random Forest and Gradient Boosting

machines.

Random Forests Gradient Boosting

Random forests are a significant

number of decision trees pooled Gradient boosting machines also combine decision trees but at
using averages or majority rules the beginning of the process, unlike Random forests.
at the end.

The random forest creates each Gradient boosting yields better outcomes than random forests if
tree independent of the others parameters are carefully tuned but it’s not a good option if the
while gradient boosting develops data set contains a lot of outliers/anomalies/noise as it can result
one tree at a time. in overfitting of the model.

What is the difference between regularization and normalisation?

Normalisation Regularisation
 Normalisation adjusts the data; . If your data is on very
different scales (especially low to high), you would want Regularisation adjusts the
to normalise the data. Alter each column to have prediction function.
compatible basic statistics. This can be helpful to make Regularization imposes some
sure there is no loss of accuracy. One of the goals of control on this by providing
model training is to identify the signal and ignore the simpler fitting functions over
noise if the model is given free rein to minimize error, complex ones.
there is a possibility of suffering from overfitting.

34. Explain the difference between Normalization and Standardization.

Normalization and Standardization are the two very popular methods used for feature scaling.

Normalisation Standardization

Normalization refers to re-scaling the values to fit

into a range of [0,1]. Standardization refers to re-scaling data to
Normalization is useful when all parameters need have a mean of 0 and a standard deviation
to have an identical positive scale however the of 1 (Unit variance)
outliers from the data set are lost.