Artificial intelligence (Al) refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. The term may also be applied to any machine that exhibits traits associated with a human mind such as learning and problem-solving. The ideal characteristic of artificial intelligence is its ability to rationalize and take actions that have the best chance of achieving a specific goal. A subset of artificial intelligence is machine learning (ML), which refers to the concept that computer programs can automatically learn from and adapt to new data without being assisted by humans. Deep learning techniques enable this automatic learning through the absorption of huge amounts of unstructured data such as text, images, or video.Now you may wonder, how is it different from traditional programming? Well, in traditional programming, we would feed the input data and a well-written and tested program into a machine to generate output. When it comes to machine learning, input data, along with the output, is fed into the machine during the learning phase, and it works out a program for itself. To understand this better, refer to the illustration below: DATA (INPUT) —_»>> erry — rey) ——>> output Program DATA (INPUT) erat —__» , eer] Frogeam OutputWhat is Machine Learning? Machine Learning is defined as the study of computer algorithms for automatically constructing computer software through past experience and training data. It is a branch of Artificial Intelligence and computer science that helps build a model based on training data and make predictions and decisions without being constantly programmed. Machine Learning is used in various applications such as email filtering, speech recognition, computer vision, self-driven cars, Amazon product recommendation, etc.Types of Machine Learning "Linear Regression = Neural Network Reinforcement Learning Decision Making = Naive Bayes Classifier =K-Means Clustering if i" *=Mean-shift "Decision Trees Regression Clustering *QLearni Support Vector = Support Vector *=DBSCAN Clustering ®R Le: rae Machines Regression = Agglomerative aie Random Forest =DecisionTree eee ral mami *K—Nearest Regression Clute or Neighbors pe ea aaa *Ridge Regression1. Poor Quality of Data Data plays a significant role in the machine learning process. One of the significant issues that machine learning professionals face is the absence of good quality data. Unclean and noisy data can make the whole process extremely exhausting. We don’t want our algorithm to make inaccurate or faulty predictions. Hence the quality of data is essential to enhance the output. Therefore, we need to ensure that the process of data preprocessing which includes removing outliers, filtering missing values, and removing unwanted features, is done with the utmost level of perfection.2. Underfitting of Training Data This process occurs when data is unable to establish an accurate relationship between input and output variables. It simply means trying to fit in undersized jeans. It signifies the data is too simple to establish a precise relationship. To overcome this issue: ¢ Maximize the training time « Enhance the complexity of the model ¢ Add more features to the data ¢ Reduce regular parameters ¢ Increasing the training time of model 3. Overfitting of Training Data Overfitting refers to a machine learning model trained with a massive amount of data that negatively affect its performance. It is like trying to fit in Oversized jeans. Unfortunately, this is one of the significant issues faced by machine learning professionals. This means that the algorithm is trained with noisy and biased data, which will affect its overall performance. Let’s understand this withWe can tackle this issue by: + Analyzing the data with the utmost level of perfection « Use data augmentation technique « Remove outliers in the training set e Select a model with lesser features To know more, you can visit here. 4. Machine Learning is a Complex Process The machine learning industry is young and is continuously changing. Rapid hit and trial experiments are being carried on. The process is transforming, and hence there are high chances of error which makes the learning complex. It includes analyzing the data, removing data bias, training data, applying complex mathematical calculations, and a lot more. Hence it is a really complicated process which is another big challenge for Machine learning professionals.6. Slow Implementation This is one of the common issues faced by machine learning professionals. The machine learning models are highly efficient in providing accurate results, but it takes a tremendous amount of time. Slow programs, data overload, and excessive requirements usually take a lot of time to provide accurate results. Further, it requires constant monitoring and maintenance to deliver the best output. 7. Imperfections in the Algorithm When Data Grows So you have found quality data, trained it amazingly, and the predictions are really concise and accurate. Yay, you have learned how to create a machine learning algorithm!! But wait, there is a twist; the model may become useless in the future as data grows. The best model of the present may become inaccurate in the coming Future and require further rearrangement. So you need regular monitoring and maintenance to keep the algorithm working. This is one of the1. Supervised Machine Learning As its name suggests, Supervised machine learning is based on supervision. It means in the supervised learning technique, we train the machines using the "labelled" dataset, and based on the training, the machine predicts the output. Here, the labelled data specifies that some of the inputs are already mapped to the output. More preciously, we can say; first, we train the machine with the input and corresponding output, and then we ask the machine to predict the output using the test dataset. Let's understand supervised learning with an example. Suppose we have an input dataset of cats and dog images. So, first, we will provide the training to the machine to understand the images, such as the shape & size of the tail of cat and dog, Shape of eyes, colour, height (dogs are taller, cats are smaller), etc. After completion of training, we input the picture of a cat and ask the machine to identify the object and predict the output. Now, the machine is well trained. so it will check all the features of theThe main goal of the supervised learning technique is to map the input variable(x) with the output variable(y). Some real-world applications of supervised learning are Risk Assessment, Fraud Detection, Spam filtering, etc. Categories of Supervised Machine Learning Supervised machine learning can be classified into two types of problems, which are given below: © Classification o Regressiona) Classification Classification algorithms are used to solve the classification problems in which the output variable is categorical, such as "Yes" or No, Male or Female, Red or Blue, etc. The classification algorithms predict the categories present in the dataset. Some real-world examples of classification algorithms are Spam Detection, Email filtering, etc. Some popular classification algorithms are given below: ~) 7 Raa cena de a ha Geant ad? 4 Ser Eel ata! o Random Forest Algorithm © Decision Tree Algorithm © Logistic Regression Algorithm o Support Vector Machine Algorithm b) Regressionb) Regression Regression algorithms are used to solve regression problems in which there is a linear relationship between input and output variables. These are used to predict continuous output variables, such as market trends, weather prediction, etc. Lasik Eye Surgery In Hyderabad: Prices In 2023 May Surprise You __ Search Ads mgid > Some popular Regression algorithms are given below: o Simple Linear Regression Algorithm o Multivariate Regression Algorithm o Decision Tree Algorithm o Lasso RegressionAdvantages: © Since supervised learning work with the labelled dataset so we can have an exact idea about the classes of objects. o These algorithms are helpful in predicting the output on the basis of prior experience. Disadvantages: Fiber Laser Cutting Machine fou as hwo om 4 em lt / Laser Cutting Machine > Bodor®Laser © These algorithms are not able to solve complex tasks.o These algorithms are not able to solve complex tasks. o It may predict the wrong output if the test data is different from the training data. o It requires lots of computational time to train the algorithm. Applications of Supervised LearningSome common applications of Supervised Learning are given below: o Image Segmentation: Supervised Learning algorithms are used in image segmentation. In this process, image classification is performed on different image data with pre-defined labels. o Medical Diagnosis: Supervised algorithms are also used in the medical field for diagnosis purposes. It is done by using medical images and past labelled data with labels for disease conditions. With such a process, the machine can identify a disease for the new patients. o Fraud Detection - Supervised Learning classification algorithms are used for identifying fraud transactions, fraud customers, etc. It is done by using historic data to identify the patterns that can lead to possible fraud. o Spam detection - In spam detection &2. o Spam detection - In spam detection & filtering, classification algorithms are used. These algorithms classify an email as spam or not spam. The spam emails are sent to the spam folder. Speech Recognition - Supervised learning algorithms are also used in speech recognition. The algorithm is trained with voice data, and various identifications can be done using the same, such as voice- activated passwords, voice commands, etc. Unsupervised Machine LearningUnsupervised learning is different from the Supervised learning technique; as its name suggests, there is no need for supervision. It means, in unsupervised machine learning, the machine is trained using the unlabeled dataset, and the machine predicts the output without any supervision. In unsupervised learning, the models are trained with the data that is neither classified nor labelled, and the model acts on that data without any supervision.The main aim of the unsupervised learning algorithm is to group or categories the unsorted dataset according to the similarities, patterns, and differences. Machines are instructed to find the hidden patterns from the input dataset. Let's take an example to understand it more preciously; suppose there is a basket of fruit images, and we input it into the machine learning model. The images are totally unknown to the model, and the task of the machine is to find the patterns and categories of the objects. So, now the machine will discover its patterns and differences, such as colour difference, shape difference, and predict the output when it is tested with the test dataset. Categories of Unsupervised Machine Learning Unsupervised Learning can be further classified into two types, which are given below: o Clustering o Association1) Clustering The clustering technique is used when we want to find the inherent groups from the data. It is a way to group the objects into a cluster such that the objects with the most similarities remain in one group and have fewer or no similarities with the objects of other groups. An example of the clustering algorithm is grouping the customers by their purchasing behaviour. Some of the popular clustering algorithms are given below: o K-Means Clustering algorithm o Mean-shift algorithm o DBSCAN Algorithm ° Principal Component Analysis ° Independent Component Analysis 2) Association Association rule learning is an unsupervised learning technique, which finds interesting relations among variables within a large dataset.2) Association Association rule learning is an unsupervised learning technique, which finds interesting relations among variables within a large dataset. The main aim of this learning algorithm is to find the dependency of one data item on another data item and map those variables accordingly so that it can generate maximum profit. This algorithm is mainly applied in Market Basket analysis, Web usage mining, continuous production, etc. Some popular algorithms of Association rule learning are Apriori Algorithm, Eclat, FP-growth algorithm. Advantages and Disadvantages of Unsupervised Learning Algorithm Advantages: o These algorithms can be used for complicated tasks compared to the supervised ones because these algorithms work on the unlabeled dataset.o Unsupervised algorithms are preferable for various tasks as getting the unlabeled dataset is easier as compared to the labelled dataset. Disadvantages: © The output of an unsupervised algorithm can be less accurate as the dataset is not labelled, and algorithms are not trained with the exact output in prior. © Working with Unsupervised learning is more difficult as it works with the unlabelled dataset that does not map with the output. Applications of Unsupervised Learning o Network Analysis: Unsupervised learning is used for identifying plagiarism and copyright in document network analysis of text data for scholarly articles.o Recommendation Systems: Recommendation systems widely use unsupervised learning techniques for building recommendation applications for different web applications and = e- commerce websites. o Anomaly Detection: Anomaly detection is a popular application of unsupervised learning, which can identify unusual data points within the dataset. It is used to discover fraudulent transactions. o Singular Value Decomposition: Singular Value Decomposition or SVD is used to extract particular information from the database. For example, extracting information of each user located at a particular location.4. Reinforcement Learning Reinforcement learning works on a feedback- based process, in which an Al agent (A software component) automatically explore its surrounding by hitting & trail, taking action, learning from experiences, and improving its performance. Agent gets rewarded for each good action and get punished for each bad action; hence the goal of reinforcement learning agent is to maximize the rewards. In reinforcement learning, there is no labelled data like supervised learning, and agents learn from their experiences only. The reinforcement learning process is similar to a human being; for example, a child learns various things by experiences in his day-to-day life. An example of reinforcement learning is to play a game, where the Game is the environment, moves of an agent at each step define states, and the goal of the agent is to get a high score. Agent receives feedback in terms of punishment and rewards.Categories of Reinforcement Learning Reinforcement learning is categorized mainly into two types of methods/algorithms: o Positive Reinforcement Learning: Positive reinforcement learning specifies increasing the tendency that the required behaviour would occur again by adding something. It enhances the strength of the behaviour of the agent and positively impacts it. o Negative Reinforcement Learning: Negative reinforcement learning works exactly opposite to the positive RL. It increases the tendency that the specific behaviour would occur again by avoiding the negative condition.Real-world Use cases of Reinforcement Learning o Video Games: RL algorithms are much popular in gaming applications. It is used to gain super- human performance. Some popular games that use RL algorithms are AlphaGO and AlphaGO Zero. co Resource Management: The "Resource Management with Deep Reinforcement Learning" paper showed that how to use RL in computer to automatically learn and — schedule resources to wait for different jobs in order to minimize average job slowdown. o Robotics: RL is widely being used in Robotics applications. Robots are used in the industrial and manufacturing area, and these robots are made more powerful with reinforcement learning. There are different industries that have their vision of building intelligent robots using Al and Machine° Text Mining Text-mining, one of the great applications of NLP is now being implemented with the help of Reinforcement Learning by Salesforce company. Advantages and Disadvantages of Reinforcement Learning Advantages o It helps in solving complex real-world problems which are difficult to be solved by general techniques. o The learning model of RL is similar to the learning of human beings; hence most accurate results can be found. © Helps in achieving long term results. Disadvantage o RL algorithms are not preferred for simple problems. o RL algorithms require huge data and computations.Bias and _ Variance — in Machine Learning Machine learning is a branch of Artificial Intelligence, which allows machines to perform data analysis and make predictions. However, if the machine learning model is not accurate, it can make predictions errors, and these prediction errors are usually known as Bias and Variance. In machine learning, these errors will always be present as there is always a slight difference between the model predictions and actual predictions. The main aim of ML/data science analysts is to reduce these errors in order to get more accurate results. In this topic,Errors in Machine Learning? In machine learning, an error is a measure of how accurately an algorithm can make predictions for the previously unknown dataset. On the basis of these errors, the machine learning model is selected that can perform best on the particular dataset. There are mainly two types of errors in machine learning, which are: o Reducible errors: These errors can be reduced to improve the model accuracy. Such errors can further be classified into bias and Variance. Tare Tal Ce cog © Irreducible errors: These errors will always be present in the modelBias The bias is known as the difference between the prediction of the values by the ML model and the correct value. Being high in biasing gives a large error in training as well as testing data. Its recommended that an algorithm should always be low biased to avoid the problem of underfitting. By high bias, the data predicted is in a straight line format, thus not fitting accurately in the data in the data set. Such fitting is known as Underfitting of Data. This happens when the hypothesis is too simple or linear in nature. Refer toThe variance would specify the amount of variation in the prediction if the different training data was used. In simple words, variance tells that how much a random variable is different from its expected value. Ideally, a model should not vary too much from one training dataset to another, which means the algorithm should be good in understanding the hidden mapping between inputs and output variables. Variance errors are either of low variance or high variance. Low variance means there is a small variation in the prediction of the target function with changes in the training data set. At the same time, High variance shows a large variation in the prediction of the target function with changes in the training dataset.Bias-Variance Trade-Off While building the machine learning model, it is really important to take care of bias and variance in order to avoid overfitting and underfitting in the model. If the model is very simple with fewer parameters, it may have low variance and high bias. Whereas, if the model has a large number of parameters, it will have high variance and low bias. So, it is required to make a balance between bias and variance errors, and this balance between the bias error and variance error is known as the Bias-Variance trade-off. Variance Error Optimal Model Complexity Model ComplexityFor an accurate prediction of the model, algorithms need a low variance and low bias. But this is not possible because bias and variance are related to each other: o If we decrease the variance, it will increase the bias. o If we decrease the bias, it will increase the variance. Bias-Variance trade-off is a central issue in supervised learning. Ideally, we need a model that accurately captures the regularities in training data and simultaneously generalizes well with the unseen dataset. Unfortunately, doing this is not possible simultaneously. Because a high variance algorithm may perform well with training data, but it may lead to overfitting to noisy data. Whereas, high bias algorithm generates a much simple model that may not even capture important regularities in the data. So, we need to find a sweet spot between bias and variance to make an optimal model.Train and Test datasets in Machine Learning Machine Learning is one of the booming technologies across the world that enables computers/machines to turn a huge amount of data into predictions. However, these predictions highly depend on the quality of the data, and if we are not using the right data for our model, then it will not generate the expected result. In machine learning projects, we generally divide the original dataset into training data and test data. We train our model over a subset of the original dataset, i.e., the training dataset, and then evaluate whether it can generalize well to the new or unseen dataset or test set. Therefore, train and test datasets are the two key concepts of machine learning, where the training dataset is used to fit the model, and the test dataset is used to evaluate the model.Training a model simply means learning (determining) good values for all the weights and the bias from labeled examples. In supervised learning, a machine learning algorithm builds a model by examining many examples and attempting to find a model that minimizes loss; this process is called empirical risk minimization. Loss is the penalty for a bad prediction. That is, loss is a number indicating how bad the model's prediction was on a single example. If the model's prediction is perfect, the loss is zero; otherwise, the loss is greater. The goal of training a model is to find a set of weights and biases that have /ow loss, on average, across all examples. For example, Figure 3 shows a high loss model on the left and a low loss model on the right. Note the following about the figure: ¢ The arrows represent loss. ¢ The blue lines represent predictions.Mean square error (MSE) is the average squared loss per example over the whole dataset. To calculate MSE, sum up all the squared losses for individual examples and then divide by the number of examples: 1 Cis 2 MSE = W Ss (y — prediction(z)) (z,y)ED where: + (x,y) is an example in which « xis the set of features (for example, chirps/minute, age, gender) that the model uses to make predictions. ¢ yis the example's label (for example, temperature). + prediction(«) is a function of the weights and bias in combination with the set of features z. e Dis a data set containing many labeled examples, which are (a, y) pairs. e Nis the number of examples in D.Sampling Distribution Of The Estimator Sampling distribution of the Estimator: In statistics, it is the probability distribution of the given statistic estimated on the basis of a random sample. It provides a generalized way to statistical inference. The estimator is the generalized mathematical parameter to calculate sample statistics. An estimate is the result of the estimation. The sampling distribution of estimator depends on the sample size. The effect of change of the sample size has to be determined. An estimate has a single numerical value and hence they are called point estimates. There are various estimators like sample mean, sample standard deviation, proportion, variance, range etc. Sampling distribution of the mean: It is the population mean from which the samples are drawn. For all the sample sizes, it is likely to be normal if the population distribution is normal. The population mean is equal to the mean of the sampling distribution of the mean. Sampling distribution of mean has the standard deviation, which is as follows: 6<= ueWhere ou, is the standard deviation of the sampling mean, @ is the population standard deviation and n is the sample size. As the size of the sample increases, the spread of the sampling distribution of the mean decreases. But the mean of the distribution remains the same and it is not affected by the sample size. The sampling distribution of the standard deviation is the standard error of the standard deviation. It is defined as: o,=—2 > Bn Here, °s isthe sampling distribution of the standard deviation. It is positively skewed for small n but it approximately becomes normal for sample sizes greater than 30.Learning roves 9 Hop supevued prodit am Pree eo lal Tut datalovEing ot unsuperuistd leernye lp Raw date Algorithry ) (--0-O-L unlabe ~~ onianetied