6 Questions to Ask About Any Machine Learning Algorithm

Friday, November 04, 2016 8:25 AM

6 Questions To Ask About Any Algorithm

There are 6 questions that you can ask to get to the heart of any machine learning algorithm:
1. How do you refer to the technique (e.g. what name)?
2. How do you represent a learned model (e.g. what coefficients)?
3. How to you learn a model (e.g. the optimization process from data to the representation)?
4. How do you make predictions from a learned model (e.g. apply the model to new data)?
5. How do you best prepare your data for the modeling with the technique (e.g. assumptions)?
6. How do you get more information on the technique (e.g. where to look)?
You will note that I have phrased all of these questions as How-To. I did this intentionally to separate
the practical concerns of how from the more theoretical concerns of why. I think knowing why a
technique works is less important than knowing how it works, if you are looking to use it as a tool to
get results. More on this in the next section.
Let’s take a closer look at each of these questions in turn.
1. How Do You Refer To The Technique?
This is obvious but important. You need to know the canonical name of the technique.
You need to be able to recognize the classical name or the name of the method from other fields as
well and know that it is the same thing. This also includes the acronym for the algorithm, because
sometimes they are less than intuitive.
This will help you sort out the base algorithm from extensions and the family tree of where the
algorithm fits and relates to similar algorithms.
2. How Do You Represent a Learned Model?
I really like this nuts and bolts question.
This is question often overlooked in textbooks and papers and is perhaps the first question an
engineer has when thinking about how a model will actually be used and deployed.
The representation is the numbers and data structure that captures the distinct details learned from
data by the learning algorithm to be used by the prediction algorithm. It’s the stuff you save to disk or
the database when you finalize your model. It’s the stuff you update when new training data becomes
available.
Let’s make this concrete with an example. In the case of linear regression, the representation is the
vector of regression coefficients. That’s it. In the case of a decision tree is is the tree itself including
the nodes, how they are connected and the variables and cut-off thresholds chosen.
3. How Do You Learn a Model?
Given some training data, the algorithm needs to create the model or fill in the model representation.
This question is about exactly how that occurs.
Often learning involves estimating parameters from the training data directly in simpler algorithms.
In most other algorithms it involves using the training data as part of a cost or loss function and an
optimization algorithm to minimize the function. Simpler linear techniques may use linear algebra to
achieve this result, whereas others may use a numerical optimization.
Often the way a machine learning algorithm learns a model is synonymous with the algorithm itself.
This is the challenging and often time consuming part of running a machine learning algorithm.
The learning algorithm may be parameterized and it is often a good idea to list common ranges for
parameter values or configuration heuristics that may be used as a starting point.
4. How Do You Make Predictions With A Model?
Once a model is learned, it is intended to be used to make predictions on new data. Note, we re

Machine Learning Page 1

 Once a model is learned, it is intended to be used to make predictions on new data. Note, we re
exclusively talking about predictive modeling machine learning algorithms for classification and
regression problems.
This is often the fast and even trivial part of using a machine learning algorithm. Often it is so trivial
that it is not even mentioned or discussed in the literature.
It may be trivial because prediction may be as simple as filling in the inputs in an equation and
calculating a prediction, or traversing a decision tree to see what leaf-node lights up. In other
algorithms, like k-nearest neighbors the prediction algorithm may be the main show (k-NN has no
training algorithm other than “store the whole training set”).
5. How Do You Best Prepare Data For The Algorithm?
Machine learning algorithms make assumptions.
Even the most relaxed non-parametric methods make assumptions about your training data. It is good
or even critical to review these assumptions. Even better is to translate these assumptions into
specific data preparation operations that you can perform.
This question flushes out transforms that you could use on your data before modeling, or at the very
least gives you pause to think about data transforms to try. What I mean by this is that it is best to
treat algorithm requirements and assumptions as suggestions of things to try to get the most out your
model rather than hard and fast rules that your data must adhere to.
Just like you cannot know which algorithm will be best for your data before hand, you cannot know
the best transforms to apply to your data to get the most from an algorithm. Real data is messy and it
is a good idea to try a number of presentations of your data with a number of different algorithms to
see what warrants deeper investigation. The requirements and assumptions of machine learning
algorithms help to point out presentations of your data to try.
6. How Do You Get More Information About the Algorithm?
Some algorithms will bubble up as generally better than others on your data problem.
When they do, you need to know where to look to get a deeper understanding of the technique. This
can help with further customizing the algorithm for your data and with tuning the parameters of the
learning and prediction algorithms.
It is a good idea to collect and list resources that you can reference if and when you need to dive
deeper. This may include:
• Journal Articles
• Conference Papers
• Books including textbooks and monographs
• Webpages
I also think it is a good idea to know of more practical references like example tutorials and open
source implementations that you can look inside to get a more concrete idea of what is going on.
For more on researching machine learning algorithms, see the post How to Research a Machine
Learning Algorithm.
Summary
In this post you discovered 6 questions that you can ask of a machine learning, that if answered, will
give you a very good and practical idea of how it works and how you can use it effectively.
These questions were focused on machine learning algorithms for predictive modeling problems like
classification and regression.
These questions, phrased simply are:
1. What are the common names of the algorithm?
2. What representation is used by the model?
3. How does the algorithm learn from training data?
4. How can you make predictions from the model on new data?
5. How you can best prepare your data for the algorithm?
6. Where you can you look for more information about the algorithm?
For another post along this theme of defining an algorithm description template see How to Learn a
Machine Learning Algorithm.

Machine Learning Page 2

Machine Learning Algorithm.
Do you like this approach? Let me know in the comments.

From <http://machinelearningmastery.com/question-to-understand-any-machine-learning-algorithm/>

Machine Learning Page 3