Research and Development Guidelines

Contents

1. Purpose
2. Resources needed
3. Definitions
4. Procedure
5. Attachments
6. References
Research and Development Guidelines

1. Purpose
This guideline shall explain in detail how the R&D shall be designing experiments, conducting
experiments, documenting experiments, analyzing results, presenting findings, classifying and
managing confidential information, and so on.
2. Resources needed
 Computer software
Minitab software
 Basic understanding of the following topics:
 Descriptive and inferential statistics
 Type of data
 Basics of design of experiments
 Basic statistical designs
 Regression analysis
3. Definitions
 Research
The term ‘Research’ consists of two words: re and search. ‘Re’ means again and again, and
‘Search’ means to find out something.
 Research methods
Research is a process for collecting, analyzing, and interpreting information to answer
questions.
 Research methodology
The methodology is the study of the methods used and the reasoning behind why those
particular methods were used.
 Type of data

 Controllable variable
Those controlled by a process engineer/production engineer in a production environment.
Research and Development Guidelines

 Uncontrollable variable (or noise variables)

Those which are difficult or expensive to control in actual production environments. Variables
such as ambient temperature fluctuations, humidity fluctuations, raw material variations, etc.
are examples of noise variables.
 Design of experiment (DOE)
It is defined as a branch of applied statistics that deals with planning, conducting, analyzing,
and interpreting controlled tests to evaluate the factors that control the value of a parameter or
group of parameters. DOE is a powerful data collection and analysis tool that can be used in
various experimental situations.
 Replication
Repetition of a complete experimental treatment, including the setup.
 Randomization
Refers to the order in which the trials of an experiment are performed. A
randomized sequence helps eliminate the effects of unknown or
uncontrolled variables.
 Blocking
When randomizing a factor is impossible or too costly, blocking lets you
restrict randomization by carrying out all of the trials with one setting of
the factor and then all the trials with the other setting.
 Factor or variable level
A level is a value that a process variable holds in an experiment.
 Effect size
Effect size tells you how meaningful the relationship
between variables or the difference between groups is, and it indicates
the practical significance of a research outcome.
 Power analysis
A power analysis is a calculation that helps you determine the minimum
sample size for your study.
 Statistical power
the likelihood that a test will detect an effect of a certain size if there is
one, usually set at 80% or higher.
4. Procedure
The following are the phases for designing an experiment:
 planning phase
 designing phase
 conducting phase
 analyzing phase

Phases Steps/procedure Documention needed

4.1 planning 4.1.1 Problem Recognition and Formulation  Copies of each revision to the
phase  Writing a clear and succinct statement of the problem. DOE problem statement.
The statement should include:  Copy of the final DOE
 contain a specific, measurable objective and can yield problem statement with
practical value to the company. appropriate approvals (for
 A multidisciplinary team should be created to have a example, initials or signatures)
shared understanding of the problem, which is critical required prior to proceeding
Research and Development Guidelines

in the planning phase. with the experimental

 The multidisciplinary team should be led by program.
someone with good knowledge of the process (a  Copy of the name and position
DOE specialist), good communication skills, good of the team members.
interpersonal skills, and awareness of team
dynamics
 Some manufacturing problems that can be addressed
using an experimental approach include:
 development of new products; improvement of
existing processes or products;
 improvement of the process/product performance
relative to the needs and demands of customers;
 reduction of existing process spread, which leads to
poor capability.
4.1.2 Selection of Response or Quality Characteristic  A document containing:
 The selection of a suitable response for the experiment  the response variable
(qualitative vs. quantitative)  effect size
 The measurement system should be determined prior to  resources needed
the experiment by:
 Determining:
 what to measure
 where to measure
 who is doing the measurements
 Determining the smallest practically significant
change in the response that the experiment is expected
to detect for the purpose of sample-size calculations.
 Determining human resources, equipment, and
measurement methods
 Ensuring the existence, suitability, accessibility, and
calibration of equipment.
 Evaluating the repeatability, reproducibility, and
uncertainty of the measurements prior to the execution
of industrial experiments
 Ensuring that the measurement system is capable,
stable, robust, and insensitive to environmental
changes is advisable.
4.1.3 Selection of Process Variables or Design  A document containing the
Parameters process variable selected
 Some possible ways to identify potential process variables  Preexperiments needed to
are using engineering knowledge of the process, historical select the significant factors
data, cause-and-effect analysis, and brainstorming.
 Decide whether you want to investigate one-factor
(single factor with different levels) or multiple factors
(factorial experiments)
 In the first phase of any experimental investigation,
conducting a screening experiment (i.e., 2K) to identify
the most important design parameters or process variables
is good.
 Consider relevant historical data or other experiments that
were performed to study the problem.
4.1.4 Classification of Process Variables  List of process variables with
 Having identified the process variables, the next step is to their classification
classify them into controllable and uncontrollable  Randomization and blocking
Research and Development Guidelines

variables. plan
 Uncontrollable variables may have an immense impact  Description of and
on the process variability and, therefore, must be dealt justification for the
with to understand our process better. Such nuisance randomization and blocking
variables can be minimized by effectively applying plan.
DOE principles such as blocking, randomization and  Copy of analysis validating
replication. the randomization plan (for
example, analysis of run
order as the response).
 Copies of the actual data
sheets (with the runs in
random order) to be used
for data collection.
4.1.5 Determining the Levels of Process Variables List of the process variable and
 The number of levels depends on the nature of the process their classification
variable to be studied for the experiment
 For quantitative variable
 Two levels are sufficient for screening purposes
 Three levels or more are required if the non-linear
function is expected for the response variable.
 For qualitative variable
 More than two levels may be required
4.1.6 List All the Interactions of Interest (for List of important interactions
experiments with more than one factor) between process variables
 Interaction among variables is quite common in industrial
experiments. In order to effectively interpret the results of
the experiment, it is highly desirable to have a good
understanding of the interaction between two process
variables
4.2 Selecting the appropriate design  Preliminary (screening)
designing  The choice of design depends upon a number of factors experiments
phase such as:  Summary statement of the
 the number of factors to be studied purpose and results of each
 one way ANOVA for one-factor experiments preliminary experiment.
 Screening factorial experiment (2K)for experiments  Original data records,
with 6-15 factor notes, and analyses from
 Optimization factorial designs for experiments each preliminary
with 2-5 factors experiments experiment.
 the number of levels at which the factors are to be  Notes on any follow-up
explored actions taken as a result of
 the resources and budget allocated for the experiment, findings from preliminary
 the nature of the problem and objectives to be experiments.
achieved, etc.  Main Experiment design
 Hypothesis testing  Final classification of each
 Correlation or association input variable into one of
 Regression analysis the following categories:
experimental variable,
 Design of experiment
variable to be held fixed,
 The principles of randomization, replication, and blocking uncontrolled variable not
should be carefully taken into account depending upon recorded, or uncontrolled
the nature of the problem and the objectives set for the variable recorded.
experiment.  Copies of the variable and
design matrices.
 Copy of the sample-size
Research and Development Guidelines

calculation or other sample-

size justification.
4.3 4.3.1 considerations recommended prior to executing  Copies of any required formal
conducting an experiment authorizations to build the
the  selection of a suitable location for carrying out the experiment with the required
experiment experiment. It is important to ensure that the location is resources.
not affected by any external noise sources (vibration,
humidity, temperature, etc.)
 availability of materials/parts, operators, machines, etc.
required for carrying out the experiment
 An experiment should not be run unless there is
consensus among the key people involved that everything
is ready.
4.3.2 considerations recommended while performing  Copies of all original data
the experiment records.
 The person responsible for the experiment should be  Copies of all notes taken
present throughout the experiment. In order to reduce the during the execution of the
operator-to-operator variability, it is best to use the same experiment.
operator for the entire experiment.
 Monitor the experimental trials. This is to find any
discrepancies while running the experiment. It is
advisable to stop running the experiment if any
discrepancies are found.
 Even after a large experiment is started, you may realize
that some aspect of the process was not considered, and it
may be necessary to suspend the experiment until the
issue is resolved. Sometimes this can be done
immediately after the problem is discovered, but when a
clear and effective solution is not apparent, it is usually
better to walk away, regroup, and come back another day.
 Record the observed response values on the prepared
datasheet or directly into the computer.
 Any experiment deviations and unusual occurrences must
be recorded and analyzed.
4.4  Before performing any analysis of the experimental data,  Copy of the experimental data
Analyzing the accuracy of the recorded data values should be after transcription into the
phase confirmed by checking each recorded value in the electronic worksheet.
worksheet against the original data record. All  Copies of the analyses of the
discrepancies must be resolved before any analysis is full and refined models,
performed. including the residuals
 The experimental data should be analyzed using analysis.
MINITAB or some other suitable statistical software  Copies of any alternative
package. models considered.
 The raw data should be plotted in some meaningful way.  Copies of any special notes or
 The full model, including all relevant main effects, observations from the analysis
interactions, and other terms, should be run, and a concerning unusual
complete set of residuals diagnostic plots should be observations, and so on, and
created, including plots of the residuals versus each of the any related findings from
design variables, residuals versus fitted values, and follow-up activities.
residuals versus the run order. A normal probability plot
of the residuals should also be created.
 Special consideration should be given to outliers or highly
influential observations. It may be necessary to compare
outliers detected in the statistical analysis to records or
Research and Development Guidelines

notes of unusual conditions that occurred during the

experiment. Outliers must not be dropped from the data
set without correlation to a clear special cause.
 A simplified model should be created if the full model is
excessively complex. Both models, the full and simplified
models, should be retained in their complete form because
it is usually necessary to report both.
 The statistical analysis of the data should be done by the
DOE project leader with support from the DOE statistical
specialist if necessary.
4.5  Before interpreting any model, the residuals  Written interpretation of the
interpretatio diagnostic plots should be evaluated for normality, statistical analysis with
n phase homoscedasticity, and independence. Only after references to graphs and
these conditions and any other requirements are tables.
found to be valid should the statistical model be  Explanation of special
interpreted. applications of the final
 The following are the possible objectives to be achieved model, for example,
from this phase: optimization, variable
 Determine the design parameters or process variables settings to achieve a
that affect the mean process performance. specified value of the
 Determine the design parameters or process variables response, and so on.
that influence performance variability.
 Description of any special
 Determine the design parameter levels that yield the
optimum performance.
conditions or observations
 Determine whether further improvement is possible. that might indicate the need
for a follow-up experiment
or influence the direction of
the confirmation
experiment.
 A brief statement about the
process's strengths and
weaknesses, experiment,
and/or analysis is
appropriate.
 Recommendations for the
next step in the
experimental project include
proceeding to a
confirmation experiment,
running more replicates of
the original experiment,
performing another
designed experiment, and so
on.
4.6 Run a  To protect yourself and the organization from  Description of and justification
confirmation irreproducible results, you should always follow up a for the confirmation
experiment designed experiment with a confirmation experiment. experiment.
 The purpose of a confirmation experiment is to  Copies of the original data
demonstrate the model's validity derived from the records, statistical analysis,
designed experiment. and interpretation of the
 The confirmation experiment may be quite small, perhaps confirmation experiment.
consisting of just a single crucial condition, but it should  Summary statement about the
address the full experiment's most important claims or success or failure of the
Research and Development Guidelines

conclusions. It should be run well after the original confirmation experiment and
experiment and under typical operating conditions. implications for the goals of
 If the conclusions from the original experiment were the experimental project.
robust, then the confirmation experiment will successfully
reproduce the desired results. However, if something was
overlooked or changed from the time the original
experiment was performed, then the variable or changes
need to be identified and incorporated into the analysis.
Never report the results from the original experiment until
the confirmation experiment has been successfully run.
 The confirmation experiment is usually designed by the
DOE project leader and is performed by the operators
who run the process. The experiment usually does not
take too long or consume many resources unless its
results are inconsistent with the original experiment. If the
confirmation experiment requires an unusual combination
of variable levels, it may be necessary for the process
engineer to consult or even participate in the confirmation
experiment.
4.7 reporting  The DOE project leader should write the formal report,  Selected information
the results but the report should be reviewed and approved by those should be documented
members of the team who have the technical skills to using an experiment report
understand it.
 The following is the content of the report:
 Findings. An executive summary of the experiment
summarizing the design, the analysis, and the results.
This section should be no longer than just a few
sentences.
 Background. Some technical background on the
process to be studied, a description of the problem,
and a statement of the purpose of the experiment. This
section should be no more than one page long.
 Experiment design. A description of the experiment
design and the randomization and blocking plan that
was used. It may also be necessary to justify the
sample size that was used.
 Data. A table of the experimental data with a
description of the table and its organization. Special
mention should be made of missing values and any
observations that have associated special causes. The
location of the original data in its paper or electronic
form should be indicated.
 Statistical analysis. A description of the statistical
analysis with explicit references to all of the computer
analyses and supporting graphs. Discussion of the
analysis is integrated into this section.
 Recommendations. A recommendations section
should be included in the report if a follow-up study is
required or ambiguities remain after the analysis. This
section may also include a focused interpretation of
the analysis to address a specific problem or goal of
the experiment, for example, to optimize response.

5. Attachments
Research and Development Guidelines

 Experiment report
6. References
 Antony, J. (2014). Design of Experiments for Engineers and Scientists (Elsevier Insights) (2nd ed.).
Elsevier.
 Bower, J. A. (2013). Statistical Methods for Food Science: Introductory Procedures for the Food
Practitioner (2nd ed.). Wiley-Blackwell.
 Gupta, B. C. (2021b). Statistical Quality Control: Using MINITAB, R, JMP and Python (1st ed.). Wiley.
 Lesik, S. A. (2018). Applied Statistical Inference with MINITAB®, Second Edition. Amsterdam University
Press.
Research and Development Guidelines

Company Name
Experiment report

Reference #: Date Initiated

Department Target Close Date
Location: Actual Close Date

Experiment title

Findings

Research problem background (Problem description and purposes of the research)

Research and Development Guidelines

Experiment design
Research and Development Guidelines

Data
Research and Development Guidelines

Statistical analysis results

Research and Development Guidelines

Recommendations

References

Project team
Research and Development Guidelines

Project Project members

leader