ABSTRACT

This Project comes up with the applications of Random Forest techniques for
detecting the 'fake news', that is, misleading news stories that comes from the non-reputable
sources. Only by building a model based on a count vectorizer (using word tallies) or a (Term
Frequency Inverse Document Frequency) tfidf matrix, (word tallies relative to how often
they’re used in other articles in your dataset) can only get you so far. But these models do not
consider the important qualities like word ordering and context. It is very possible that two
articles that are similar in their word count will be completely different in their meaning. The
data science community has responded by taking actions against the problem. There is a
Kaggle competition called as the “Fake News Challenge” and Facebook is employing AI to
filter fake news stories out of users’ feeds. Combatting the fake news is a classic text
classification project with a straight forward proposition. Is it possible for you to build a
model that can differentiate between “Real “news and “Fake” news? So a proposed work on
assembling a dataset of both fake and real news and employ a Random Forestclassifier in
order to create a model to classify an article into fake or real based on its words and phrases.
OBJECTIVE
The main objective is to detect the fake news, which is a classic text classification
problem with a straight forward proposition. It is needed to build a model that can
differentiate between “Real” news and “Fake” news
INTRODUCTION
These days’ fake news is creating different issues from sarcastic articles to a
fabricated news and plan government propaganda in some outlets. Fake news and lack of
trust in the media are growing problems with huge ramifications in our society. Obviously, a
purposely misleading story is “fake news “but lately blathering social media’s discourse is
changing its definition. Some of them now use the term to dismiss the facts counter to their
preferred viewpoints.
The importance of disinformation within American political discourse was the
subject of weighty attention, particularly following the American president election. The term
'fake news' became common parlance for the issue, particularly to describe factually incorrect
and misleading articles published mostly for the purpose of making money through page
views. In this paper, it is seemed to produce a model that can accurately predict the likelihood
that a given article is fake news.
Facebook has been at the epicenter of much critique following media attention. They
have already implemented a feature to flag fake news on the site when a user sees’s it; they
have also said publicly they are working on to to distinguish these articles in an automated
way. Certainly, it is not an easy task. A given algorithm must be politically unbiased – since
fake news exists on both ends of the spectrum – and also give equal balance to legitimate
news sources on either end of the spectrum. In addition, the question of legitimacy is a
difficult one. However, in order to solve this problem, it is necessary to have an
understanding on what Fake News is. Later, it is needed to look into how the techniques in
the fields of machine learning, natural language processing help us to detect fake news.
EXISTING SYSTEM
There exists a large body of research on the topic of machine learning methods for
deception detection, most of it has been focusing on classifying online reviews and publicly
available social media posts. Particularly since late 2016 during the American Presidential
election, the question of determining 'fake news' has also been the subject of particular
attention within the literature.
Conroy, Rubin, and Chen [1] outlines several approaches that seem promising towards
the aim of perfectly classify the misleading articles. They note that simple content-related n-
grams and shallow parts-of-speech (POS) tagging have proven insufficient for the
classification task, often failing to account for important context information. Rather, these
methods have been shown useful only in tandem with more complex methods of analysis.
Deep Syntax analysis using Probabilistic Context Free Grammars (PCFG) have been shown
to be particularly valuable in combination with n-gram methods. Feng, Banerjee, and
Choi [2] are able to achieve 85%-91% accuracy in deception related classification tasks using
online review corpora.
Feng and Hirst implemented a semantic analysis looking at 'object:descriptor' pairs for
contradictions with the text on top of Feng's initial deep syntax model for additional
improvement. Rubin, Lukoianova and Tatiana analyze rhetorical structure using a vector
space model with similar success. Ciampaglia et al. employ language pattern similarity
networks requiring a pre-existing knowledge base.

PROPOSED SYSTEM
In this paper a model is build based on the count vectorizer or a tfidf matrix ( i.e )
word tallies relatives to how often they are used in other artices in your dataset ) can help .
Since this problem is a kind of text classification, implementing a Random Forest classifier
will be best as this is standard for text-based processing. The actual goal is in developing a
model which was the text transformation (count vectorizervstfidfvectorizer) and choosing
which type of text to use (headlines vs full text). Now the next step is to extract the most
optimal features for countvectorizer or tfidf-vectorizer, this is done by using a n-number of
the most used words, and/or phrases, lower casing or not, mainly removing the stop words
which are common words such as “the”, “when”, and “there” and only using those words that
appear at least a given number of times in a given text dataset.
COLLECTING DATA
So, there must be two parts to the data-acquisition process, “fake news” and “real
news”. Collecting the fake news was easy as Kaggle released a fake news dataset consisting
of 13,000 articles published during the 2016 election cycle. Now the later part is very
difficult. That is to get the real news for the fake news dataset. It requires huge work around
many Sites because it was the only way to do web scraping thousands of articles from
numerous websites. With the help of web scraping a total of 5279 articles, real news dataset
was generated, mostly from media organizations (New York Times, WSJ, Bloomberg, NPR,
and the Guardian) which were published around 2015 – 2016.

HARDWARE SPECIFICATION
 Main Processor : 2GHz
 Ram : 512 MB (min)
 Hard Disk : 80 GB

SOFTWARE SPECIFICATION
 Language : Python
 Web Server : Jupyter notebook (Anaconda3)
 Operating System : Windows 10 32 or 64 Bit
Modules

Data Pre-processing

This file contains all the pre-processing functions needed to process all input documents and
texts. First we read the train, test and validation data files then performed some pre-
processing like tokenizing, stemming etc. There are some exploratory data analysis is
performed like response variable distribution and data quality checks like null or missing
values etc.

Feature Extraction

In this file we have performed feature extraction and selection methods from sci-kit learn
python libraries. For feature selection, we have used methods like simple bag-of-words and
n-grams and then term frequency like tf-tdf weighting. We have also used word2vec and POS
tagging to extract the features, though POS tagging and word2vec has not been used at this
point in the project.

Classification

Here we have built all the classifiers for predicting the fake news detection. The extracted
features are fed into different classifiers. We have used Naive-bayes, Logistic Regression,
Linear SVM, Stochastic gradient decent and Random forest classifiers from sklearn. Each of
the extracted features were used in all of the classifiers. Once fitting the model, we compared
the f1 score and checked the confusion matrix. After fitting all the classifiers, 2 best
performing models were selected as candidate models for fake news classification. We have
performed parameter tuning by implementing GridSearchCV methods on these candidate
models and chosen best performing parameters for these classifier. Finally selected model
was used for fake news detection with the probability of truth. In Addition to this, we have
also extracted the top 50 features from our term-frequency tfidf vectorizer to see what words
are most and important in each of the classes. We have also used Precision-Recall and
learning curves to see how training and test set performs when we increase the amount of
data in our classifiers.

Prediction

Our finally selected and best performing classifier was algorithm which was then saved on
disk with name final_model.sav. Once you close this repository, this model will be copied to
user's machine and will be used by prediction.py file to classify the fake news. It takes a news
article as input from user then model is used for final classification output that is shown to
user along with probability of truth.

LITERATURE SURVEY
 1. A. Nematzadeh, E. Ferrara, A. Flammini, Y. Y. Ahn, "Optimal network
modularity for information diffusion", Phys. Rev. Lett., vol. 113, no. 8, pp.
088701, 2014.
With the widespread of social networks, the risk of information sharing has become
inevitable. Sharing a user's particular information in social networks is an all-or-none
decision. Users receiving friendship invitations from others may decide to accept this
request and share their information or reject it in which case none of their information will
be shared. Access control in social networks is a challenging topic. Social network users
would want to determine the optimum level of details at which they share their personal
information with other users based on the risk associated with the process. In this paper,
we formulate the problem of data sharing in social networks using two different models:
(i) a model based on \emph{diffusion kernels}, and (ii) a model based on access control.
We show that it is hard to apply the former in practice and explore the latter. We prove
that determining the optimal levels of information sharing is an NP-hard problem and
propose an approximation algorithm that determines to what extent social network users
share their own information. We propose a trust-based model to assess the risk of
sharing sensitive information and use it in the proposed algorithm. Moreover, we prove
that the algorithm could be solved in polynomial time. Our results rely heavily on
adopting the super modularity property of the risk function, which allows us to employ
techniques from convex optimization. To evaluate our model, we conduct a user study to
collect demographic information of several social networks users and get their
perceptions on risk and trust. In addition, through experimental studies on synthetic
data, we compare our proposed algorithm with the optimal algorithm both in terms of risk
and time. We show that the proposed algorithm is scalable and that the sacrifice in risk
is outweighed by the gain in efficiency.

2. J. Firmstone, S. Coleman, "The changing role of the local news media in

enabling citizens to engage in local democracies", Journalism Pract., vol. 8,
no. 5, pp. 596-606, 2014.
Understand the role ethics plays in standards development and application as well as what
questions to ask when deciding whether or not a decision is ethical. Learn what options you have
when faced with a standards related ethical dilemma and what resources are available to assist
you in making ethical decisions. Gain an understanding of the different possible types of
standards abuse and learn about real-life examples of the effects of bad ethical decisions in
standards related situations.

3. S. Della Vigna, E. Kaplan, "The Fox News effect: Media bias and
voting", Quart. J. Econ., vol. 122, no. 3, pp. 1187-1234, 2007.
We investigate the complex relations existing within news content in the 27 countries of
the European Union (EU). In particular we are interested in detecting and modelling any
biases in the patterns of content that appear in news outlets of different countries. We
make use of a large scale infrastructure to gather, translate and analyse data from the
most representative news outlets of each country in the EU. In order to model the
relations found in this data, we extract from it different networks expressing relations
between countries: one based on similarities in the choice of news stories, the other
based on the amount of attention paid by one country to another. We develop methods
to test the significance of the patterns we detect, and to explain them in terms of other
networks we created based on trade, geographic proximity and Eurovision voting
patterns. We show that media content networks are 1) stable over time, and hence well
defined as patterns of the news media sphere; 2) significantly related to trade,
geography and Eurovision voting patterns; 3) by combining all the relevant side
information, it is possible to predict the structure of the media content network. In order
to achieve the above results, we develop various pattern analysis methods to quantify
and test the non-metric, non-symmetric pairwise relations involved in this data. These
methods are general and likely to be useful in many other domains.

4. M. Gentzkow, J. M. Shapiro, "Media bias and reputation", J. Political

Economy, vol. 114, no. 2, pp. 280-316, 2006.
Fiber optics is an amazing technology that allows our daily voice, video and data transmissions
to occur with ever-increasing quality and lower costs. In this tutorial, you will be presented with a
brief history that covers how fiber optics evolved from the 19th century to the present. This will be
followed by a short primer on light basics that examines basic optical theory. Next, will be a
discussion of basic fiber structures of optical fibers followed by an overview of available
equipment and tasks required to correctly perform a variety of fiber optic tests. Fiber optic active
and passive devices will also be reviewed as well as system loss budgets. After completing this
tutorial, you should have an understanding of: the history of fiber optics; light basics; fiber
structures; fiber optic testing; fiber optic active devices; fiber optic passive devices; and system
loss budgets.

5. M. Karsai et al., "Small but slow world: How network topology and
burstiness slow down spreading", Phys. Rev. E Stat. Phys. Plasmas Fluids
Relat. Interdiscip. Top., vol. 83, no. 2, pp. 025102, 2011.
We investigate two competing products with network effects diffusion on heterogeneous
Newman-Watts (HNW) network based on a computer simulation model. Consumerspsila
behaviors in our model are determined not only by their individual preferences and
purchasing resistances but also by their interactions with others. To analyze the
determinant factors in products diffusion we divide network effects into two kinds - direct
network effects and indirect network effects. The outcomes of simulations show that
direct network effects act as a brake on the winner-take-all process, often lead to market
sharing. Consumers in the market are cliquish and each faction adopts the same
product when market sharing outcome occurs. Indirect network effects are great impetus
on the winner-take-all process, but the winner-take-all process can be slowed for the
heterogeneity of purchasing resistances among consumers.

6. F. D. Gilliam, S. Iyengar, "Prime suspects: The influence of local television

news on the viewing public", Amer. J. Political Sci., vol. 44, no. 3, pp. 560-
573, 2000.
MANETs are an upcoming technology that is gaining momentum in recent years. Due to
their unique characteristics, MANETs are suffering from wide range of security attacks.
Wormhole is a common security issue encounter in MANETs routing protocol. A new
routing protocol naming extended prime product number (EPPN) based on the hop
count model is proposed in this article. Here hop count between source & destination is
obtained depending upon the current active route. This hop count model is integrated
into AODV protocol. In the proposed scheme firstly the route is selected on the basis of
RREP and then hop count model calculates the hop count between source &
destination. Finally wormhole DETECTION procedure will be started if the calculated
hop count is greater than the received hop count in the route to get out the suspected
nodes.

7. M. Girvan, M. E. J. Newman, "Community structure in social and biological

networks", Proc. Nat. Acad. Sci. USA, vol. 99, no. 12, pp. 7821-7826, Apr.
2002.
Social networks are merely a reflection of certain realities among people that have been
identified. But in order for people or even computer systems (such as expert systems) to
make sense of the social network, it needs to be analyzed with various methods so that
the characteristics of the social network can be understood in a meaningful context. This
is challenging not only due to the number of people that can be on social networks, but
the changes in relationships between people on the social network over time. In this
paper, we develop a method to help make sense of dynamic social networks. This is
achieved by establishing a hierarchical community structure where each level represents
a community partition at a specific granularity level. By organizing each level of the
hierarchical community structure by granularity level, a person can essentially “zoom in”
to view more detailed (smaller) communities and “zoom out” to view less detailed (larger)
communities. Communities consisting of one or more subsets of people having relatively
extensive links with other communities are identified and represented as overlapping
community structures. Mechanisms are also in place to enable modifications to the
social network to be dynamically updated on the hierarchical and overlapping community
structure without recreating it in real time for every modification. The experimental
results show that the genetic algorithm approach can effectively detect hierarchical and
overlapping community structures.

8. J. Yang, J. Leskovec, "Overlapping community detection at scale: A

nonnegative matrix factorization approach", Proc. 6th ACM Int. Conf. Web
Search Data Mining, pp. 587-596, 2013.
One of the main organizing principles in real-world networks is that of network
communities, where sets of nodes organize into densely linked clusters. Communities in
networks often overlap as nodes can belong to multiple communities at once. Identifying
such overlapping communities is crucial for the understanding the structure as well as
the function of real-world networks. Even though community structure in networks has
been widely studied in the past, practically all research makes an implicit assumption
that overlaps between communities are less densely connected than the non-
overlapping parts themselves. Here we validate this assumption on 6 large scale social,
collaboration and information networks where nodes explicitly state their community
memberships. By examining such ground-truth communities we find that the community
overlaps are more densely connected than the non-overlapping parts, which is in sharp
contrast to the conventional wisdom that community overlaps are more sparsely
connected than the communities themselves. Practically all existing community detection
methods fail to detect communities with dense overlaps. We propose Community-
Affiliation Graph Model, a model-based community detection method that builds on
bipartite node-community affiliation networks. Our method successfully captures
overlapping, non-overlapping as well as hierarchically nested communities, and
identifies relevant communities more accurately than the state-of-the-art methods in
networks ranging from biological to social and information networks.

9. S. Ji, N. Satish, S. Li, P. Dubey, Parallelizing word2vec in shared and distributed

memory, 2016, [online] Available
Word2Vec (Word to Vector) processes natural language by calculating the cosine
similarity. However, the serial algorithm of original Word2Vec fails to satisfy the
demands of training of corpus text because of the explosive growth of information. It has
become the bottleneck owing to its comparatively low processing efficiency. The High
Performance Computing (HPC) specializes in improving the calculation efficiency;
therefore, the training efficiency of corpus texts can be greatly improved by parallelizing
Word2Vec algorithm. After analyzing the characteristics of the Word2Vec algorithm in
detail, we design and implement a parallel Word2Vec algorithm and use it to train corpus
text on HPC. Furthermore, the corpus texts of different sizes are collected and trained,
and the speed-up ratio is calculated by using the serial algorithm and parallel algorithm
of Word2Vec, respectively. The experimental results show that there is a higher speed-
up ratio when using the Word2Vec parallel algorithm running on HPC.

10 T. Mikolov, I. C. K. Sutskever, G. S. Corrado, J. Dean, "Distributed

representations of words and phrases and their compositionality", Proc. Adv.
Neural Inf. Process. Syst., pp. 3111-3119, 2013.

In this paper, by learning the origin of the word distributed representation, knowing the
distributed representation is one of the bridges of natural language processing mapping
to mathematical calculations. Through the learning distributed representation model:
neural network language model, CBOW model and Skip-gram model, the advantages
and disadvantages of each model are clarified. Through the experiment, we can
understand the mapping relationship between distributed representation calculation and
natural language processing, and also clarify the research direction of the next step, that
is, the use of distributed representation to sentence, paragraph, article modeling.

DATA FLOW DIAGRAM

LEVEL 0

Dataset
Collection

Pre-
processing

Random
selection

Trained &
Testing
dataset
LEVEL 1

Dataset
collection

Pre-
processing

Feature

Extraction

Apply

Algorithm
 LEVEL 2

Classify
the
dataset

Accuracy
of Result

Prediction of
UML DIAGRAMS churn
customer

Finalize the

accuracy
churn
customer
Use case diagram

Class diagram
Sequence diagram
ACTIVITY DIAGRAM
Data mining using binary classifier (Naive Bayes):

Binary classifiers are generated for each class of event

using relevant features for the class and classification
algorithm .Binary classifiers are derived from the
training sample by considering all classes other than the
current class as other, e.g., Cnormal will consider two
classes: normal and other. The purpose of this phase is
to select different features for different classes by
applying the information gain or gain ratio in order to
identify relevant features for each binary classifier.

Moreover, applying the information gain or gain

ratio will return all the features that contain more
information for separating the current class from all
other classes. The output of this ensemble of binary
classifiers will be decided using arbitration function
based on the confidence level of the output of
individual binary classifiers. C4.5 builds decision trees
from a set of training data in the same way as ID3,
using the concept of information entropy. The training
data is a set S = s1,s2,... of already classified samples.
Each sample si = x1,x2,... is a vector where x1,x2,...
represent attributes or features of the sample. The
training data is augmented with a vector C = c1,c2,...
where c1,c2,... represent the class to which each sample
belongs.

At each node of the tree, C4.5 chooses one attribute

of the data that most effectively splits its set of samples
into subsets enriched in one class or the other. Its
criterion is the normalized information gain (difference
in entropy) that results from choosing an attribute for
splitting the data. The attribute with the highest
normalized information gain is chosen to make the
decision. The naivyebayes algorithm then recourse on
the smaller sub lists.

This algorithm has a few base cases.

 All the samples in the list belong to the same class.

When this happens, it simply creates a leaf node for
the decision tree saying to choose that class.
  None of the features provide any information gain.
In this case, C4.5 creates a decision node higher up
the tree using the expected value of the class.
 Instance of previously-unseen class encountered.
Again, C4.5 creates a decision node higher up the
tree using the expected value.

Efficiency Calculation – Weka tool:

The effect of combining different classifiers can be

explained with the theory of bias-variance
decomposition. Bias refers to an error due to a learning
algorithm while variance refers to an error due to the
learned model. This is why the idea emerged of
combining both in order to profit from the advantages
of both algorithms and obtain an overall error
reduction.

The concept of bagging (voting for classification,

averaging for regression-type problems with
continuous dependent variables of interest) applies to
the area of predictive data mining, to combine the
predicted classifications (prediction) from multiple
models, or from the same type of model for different
learning data. It is also used to address the inherent
instability of results when applying complex models to
relatively small data sets. Suppose your data mining
task is to build a model for predictive classification,
and the dataset from which to train the model (learning
data set, which contains observed classifications) is
relatively small. It could repeatedly sub-sample (with
replacement) from the dataset, and apply, for example,
a tree classifier to the successive samples.

In practice, very different trees will often be grown for

the different samples, illustrating the instability of
models often evident with small datasets. One method
of deriving a single prediction (for new observations)
is to use all trees found in the different samples, and to
apply some simple voting: The final classification is
the one most often predicted by the different trees.
Note that some weighted combination of predictions
(weighted vote, weighted average) is also possible, and
commonly used.

Experimental Results:

To examine the execution of classification, the accuracy

and AUC measures are embraced. Four cases are
considered as the consequence of classifier.
TP (True Positive): the number of samples accurately
classified to that class.
TN (True Negative): the number of samples accurately
rejected from that class.
FP (False Positive): the number of samples
inaccurately rejected from that class.
FN (False Negative): the number of samples
inaccurately classified to that class.

The level of adequacy of the classification model is

ascertained with the number of correct and incorrect
classifications in every conceivable estimation of the
variables being classified. Accuracy can be calculated
by using below. We have used medical datasets such as
diabetes and heart disease datasets. All these datasets
are taken from UCI machine learning repository
datasets [10, 12]. The goal is to classify the diseases
based on parameter selection and to compare all the
classifiers such as Naïve Bayes, SVM, Decision tree.
Accuracy of different classifiers for diabetes dataset is
given in Table I and Table II respectively.
Testing

The various levels of testing are

1. White Box Testing

2. Black Box Testing
3. Unit Testing
4. Functional Testing
5. Performance Testing
 6. Integration Testing
7. Objective
8. Integration Testing
9. Validation Testing
10. System Testing
11. Structure Testing
12. Output Testing
13. User Acceptance Testing

White Box Testing

White-box testing (also known as clear box

testing, glass box testing, transparent box testing,
and structural testing) is a method of
testing software that tests internal structures or
workings of an application, as opposed to its
functionality (i.e. black-box testing). In white-box
testing an internal perspective of the system, as well as
programming skills, are used to design test cases. The
tester chooses inputs to exercise paths through the code
and determine the appropriate outputs. This is
analogous to testing nodes in a circuit, e.g. in-circuit
testing (ICT).

While white-box testing can be applied at

the unit, integration and system levels of the software
testing process, it is usually done at the unit level. It can
test paths within a unit, paths between units during
integration, and between subsystems during a system–
level test. Though this method of test design can
uncover many errors or problems, it might not detect
unimplemented parts of the specification or missing
requirements.

White-box test design techniques include:

 Control flow testing

 Data flow testing
 Branch testing
 Path testing
 Statement coverage
 Decision coverage
 White-box testing is a method of testing the
application at the level of the source code. The test
cases are derived through the use of the design
techniques mentioned above: control flow testing, data
flow testing, branch testing, path testing, statement
coverage and decision coverage as well as modified
condition/decision coverage. White-box testing is the
use of these techniques as guidelines to create an error
free environment by examining any fragile code.

These White-box testing techniques are the building

blocks of white-box testing, whose essence is the
careful testing of the application at the source code
level to prevent any hidden errors later on. These
different techniques exercise every visible path of the
source code to minimize errors and create an error-free
environment. The whole point of white-box testing is
the ability to know which line of the code is being
executed and being able to identify what the correct
output should be.

Levels
1. Unit testing. White-box testing is done during unit
testing to ensure that the code is working as
intended, before any integration happens with
previously tested code. White-box testing during
unit testing catches any defects early on and aids in
any defects that happen later on after the code is
integrated with the rest of the application and
therefore prevents any type of errors later on.
2. Integration testing. White-box testing at this level
are written to test the interactions of each interface
with each other. The Unit level testing made sure
that each code was tested and working accordingly
in an isolated environment and integration
examines the correctness of the behaviour in an
open environment through the use of white-box
testing for any interactions of interfaces that are
known to the programmer.
3. Regression testing. White-box testing during
regression testing is the use of recycled white-box
test cases at the unit and integration testing levels.
 White-box testing's basic procedures involve the
understanding of the source code that you are testing at
a deep level to be able to test them. The programmer
must have a deep understanding of the application to
know what kinds of test cases to create so that every
visible path is exercised for testing. Once the source
code is understood then the source code can be analysed
for test cases to be created. These are the three basic
steps that white-box testing takes in order to create test
cases:

1.Input, involves different types of requirements,

functional specifications, detailed designing of
documents, proper source code, security
specifications. This is the preparation stage of
white-box testing to layout all of the basic
information.
2.Processing Unit, involves performing risk analysis
to guide whole testing process, proper test plan,
execute test cases and communicate results. This is
the phase of building test cases to make sure they
 thoroughly test the application the given results are
recorded accordingly.
3.Output, prepare final report that encompasses all of
the above preparations and results.

Black Box Testing

Black-box testing is a method of software

testing that examines the functionality of an application
(e.g. what the software does) without peering into its
internal structures or workings (see white-box testing).
This method of test can be applied to virtually every
level of software
testing: unit, integration,system and acceptance. It
typically comprises most if not all higher level testing,
but can also dominate unit testing as well

Test procedures

Specific knowledge of the application's

code/internal structure and programming knowledge in
general is not required. The tester is aware of what the
software is supposed to do but is not aware of how it
does it. For instance, the tester is aware that a particular
input returns a certain, invariable output but is not
aware of how the software produces the output in the
first place.

Test cases
Test cases are built around specifications and
requirements, i.e., what the application is supposed to
do. Test cases are generally derived from external
descriptions of the software, including specifications,
requirements and design parameters. Although the tests
used are primarily functional in nature, non-
functional tests may also be used. The test designer
selects both valid and invalid inputs and determines the
correct output without any knowledge of the test
object's internal structure.

Test design techniques

Typical black-box test design techniques include:
 Decision table testing
 All-pairs testing
 State transition tables
 Equivalence partitioning
 Boundary value analysis
Unit testing

In computer programming, unit testing is a method

by which individual units of source code, sets of one or
more computer program modules together with
associated control data, usage procedures, and operating
procedures are tested to determine if they are fit for use.
Intuitively, one can view a unit as the smallest testable
part of an application. In procedural programming, a
unit could be an entire module, but is more commonly
an individual function or procedure. In object-oriented
programming, a unit is often an entire interface, such as
a class, but could be an individual method. Unit tests
are created by programmers or occasionally by white
box testers during the development process.
 Ideally, each test case is independent from the
others. Substitutes such as method stubs, mock
objects, fakes, and test harnesses can be used to assist
testing a module in isolation. Unit tests are typically
written and run by software developers to ensure that
code meets its design and behaves as intended. Its
implementation can vary from being very manual
(pencil and paper)to being formalized as part of build
automation.

Testing will not catch every error in the program,

since it cannot evaluate every execution path in any but
the most trivial programs. The same is true for unit
testing. Additionally, unit testing by definition only
tests the functionality of the units themselves.
Therefore, it will not catch integration errors or broader
system-level errors (such as functions performed across
multiple units, or non-functional test areas such
as performance).

Unit testing should be done in conjunction with

other software testing activities, as they can only show
the presence or absence of particular errors; they cannot
prove a complete absence of errors. In order to
guarantee correct behavior for every execution path and
every possible input, and ensure the absence of errors,
other techniques are required, namely the application of
formal methods to proving that a software component
has no unexpected behaviour.

Software testing is a combinatorial problem. For

example, every Boolean decision statement requires at
least two tests: one with an outcome of "true" and one
with an outcome of "false". As a result, for every line of
code written, programmers often need 3 to 5 lines of
test code.

This obviously takes time and its investment may

not be worth the effort. There are also many problems
that cannot easily be tested at all – for example those
that are nondeterministic or involve multiple threads. In
addition, code for a unit test is likely to be at least as
buggy as the code it is testing. Fred Brooks in The
Mythical Man-Month quotes: never take two
chronometers to sea. Always take one or
three. Meaning, if two chronometers contradict, how do
you know which one is correct?

Another challenge related to writing the unit tests is

the difficulty of setting up realistic and useful tests. It is
necessary to create relevant initial conditions so the part
of the application being tested behaves like part of the
complete system. If these initial conditions are not set
correctly, the test will not be exercising the code in a
realistic context, which diminishes the value and
accuracy of unit test results.

To obtain the intended benefits from unit testing,

rigorous discipline is needed throughout the software
development process. It is essential to keep careful
records not only of the tests that have been performed,
but also of all changes that have been made to the
source code of this or any other unit in the software.
Use of a version control system is essential. If a later
version of the unit fails a particular test that it had
previously passed, the version-control software can
provide a list of the source code changes (if any) that
have been applied to the unit since that time.

It is also essential to implement a sustainable

process for ensuring that test case failures are reviewed
daily and addressed immediately if such a process is not
implemented and ingrained into the team's workflow,
the application will evolve out of sync with the unit test
suite, increasing false positives and reducing the
effectiveness of the test suite.

Unit testing embedded system software presents a

unique challenge: Since the software is being developed
on a different platform than the one it will eventually
run on, you cannot readily run a test program in the
actual deployment environment, as is possible with
desktop programs.[7]

Functional testing

Functional testing is a quality assurance (QA)

process and a type of black box testing that bases its test
cases on the specifications of the software component
under test. Functions are tested by feeding them input
and examining the output, and internal program
structure is rarely considered (not like in white-box
testing). Functional Testing usually describes what the
system does.

Functional testing differs from system testing in that

functional testing "verifies a program by checking it
against ... design document(s) or specification(s)",
while system testing "validate a program by checking it
against the published user or system requirements"
(Kane, Falk, Nguyen 1999, p. 52).

Functional testing typically involves five steps .The

identification of functions that the software is expected
to perform

1.The creation of input data based on the function's

specifications
2.The determination of output based on the function's
specifications
3.The execution of the test case
 4.The comparison of actual and expected outputs

Performance testing

In software engineering, performance testing is in

general testing performed to determine how
a system performs in terms of responsiveness and
stability under a particular workload. It can also serve to
investigate, measure, validate or verify
other quality attributes of the system, such
as scalability, reliability and resource usage.

Performance testing is a subset of performance

engineering, an emerging computer science practice
which strives to build performance into the
implementation, design and architecture of a system.

Testing types

Load testing
Load testing is the simplest form of performance
testing. A load test is usually conducted to understand
the behaviour of the system under a specific expected
load. This load can be the expected concurrent number
of users on the application performing a specific
number of transactions within the set duration. This test
will give out the response times of all the important
business critical transactions. If
the database, application server, etc. are also monitored,
then this simple test can itself point
towards bottlenecks in the application software.

Stress testing
Stress testing is normally used to understand the
upper limits of capacity within the system. This kind of
test is done to determine the system's robustness in
terms of extreme load and helps application
administrators to determine if the system will perform
sufficiently if the current load goes well above the
expected maximum.

Soak testing
Soak testing, also known as endurance testing, is
usually done to determine if the system can sustain the
continuous expected load. During soak tests, memory
utilization is monitored to detect potential leaks. Also
important, but often overlooked is performance
degradation. That is, to ensure that the throughput
and/or response times after some long period of
sustained activity are as good as or better than at the
beginning of the test. It essentially involves applying a
significant load to a system for an extended, significant
period of time. The goal is to discover how the system
behaves under sustained use.

Spike testing
Spike testing is done by suddenly increasing the
number of or load generated by, users by a very large
amount and observing the behaviour of the system. The
goal is to determine whether performance will suffer,
the system will fail, or it will be able to handle dramatic
changes in load.
Configuration testing
Rather than testing for performance from the
perspective of load, tests are created to determine the
effects of configuration changes to the system's
components on the system's performance and
behaviour. A common example would be
experimenting with different methods of load-
balancing.

Isolation testing

Isolation testing is not unique to performance

testing but involves repeating a test execution that
resulted in a system problem. Often used to isolate and
confirm the fault domain.

Integration testing

Integration testing (sometimes called integration

and testing, abbreviated I&T) is the phase in software
testing in which individual software modules are
combined and tested as a group. It occurs after unit
testing and before validation testing. Integration testing
takes as its input modules that have been unit tested,
groups them in larger aggregates, applies tests defined
in an integration test plan to those aggregates, and
delivers as its output the integrated system ready
for system testing.

Purpose

The purpose of integration testing is to verify

functional, performance, and
reliability requirements placed on major design items.
These "design items", i.e. assemblages (or groups of
units), are exercised through their interfaces using black
box testing, success and error cases being simulated via
appropriate parameter and data inputs. Simulated usage
of shared data areas and inter-process communication is
tested and individual subsystems are exercised through
their input interface.
 Test cases are constructed to test whether all the
components within assemblages interact correctly, for
example across procedure calls or process activations,
and this is done after testing individual modules, i.e.
unit testing. The overall idea is a "building block"
approach, in which verified assemblages are added to a
verified base which is then used to support the
integration testing of further assemblages.

Some different types of integration testing are big

bang, top-down, and bottom-up. Other Integration
Patterns are: Collaboration Integration, Backbone
Integration, Layer Integration, Client/Server
Integration, Distributed Services Integration and High-
frequency Integration.

Big Bang
In this approach, all or most of the developed
modules are coupled together to form a complete
software system or major part of the system and then
used for integration testing. The Big Bang method is
very effective for saving time in the integration testing
process. However, if the test cases and their results are
not recorded properly, the entire integration process will
be more complicated and may prevent the testing team
from achieving the goal of integration testing.

A type of Big Bang Integration testing is

called Usage Model testing. Usage Model Testing can
be used in both software and hardware integration
testing. The basis behind this type of integration testing
is to run user-like workloads in integrated user-like
environments. In doing the testing in this manner, the
environment is proofed, while the individual
components are proofed indirectly through their use.

Usage Model testing takes an optimistic approach

to testing, because it expects to have few problems with
the individual components. The strategy relies heavily
on the component developers to do the isolated unit
testing for their product. The goal of the strategy is to
avoid redoing the testing done by the developers, and
instead flesh-out problems caused by the interaction of
the components in the environment.
 For integration testing, Usage Model testing can be
more efficient and provides better test coverage than
traditional focused functional integration testing. To be
more efficient and accurate, care must be used in
defining the user-like workloads for creating realistic
scenarios in exercising the environment. This gives
confidence that the integrated environment will work as
expected for the target customers.

Top-down and Bottom-up

Bottom Up Testing is an approach to integrated
testing where the lowest level components are tested
first, then used to facilitate the testing of higher level
components. The process is repeated until the
component at the top of the hierarchy is tested.

All the bottom or low-level modules, procedures or

functions are integrated and then tested. After the
integration testing of lower level integrated modules,
the next level of modules will be formed and can be
used for integration testing. This approach is helpful
only when all or most of the modules of the same
development level are ready. This method also helps to
determine the levels of software developed and makes it
easier to report testing progress in the form of a
percentage.

Top Down Testing is an approach to integrated

testing where the top integrated modules are tested and
the branch of the module is tested step by step until the
end of the related module.

Sandwich Testing is an approach to combine top

down testing with bottom up testing.

The main advantage of the Bottom-Up approach is that

bugs are more easily found. With Top-Down, it is easier
to find a missing branch link

Verification and validation

Verification and Validation are independent

procedures that are used together for checking that a
product, service, or system meets requirements
and specifications and that it full fills its intended
purpose. These are critical components of a quality
management system such as ISO 9000. The words
"verification" and "validation" are sometimes preceded
with "Independent" (or IV&V), indicating that the
verification and validation is to be performed by
a disinterested third party.

It is sometimes said that validation can be

expressed by the query "Are you building the right
thing?" and verification by "Are you building it
right?"In practice, the usage of these terms varies.
Sometimes they are even used interchangeably.

The PMBOK guide, an IEEE standard, defines them as

follows in its 4th edition

 "Validation. The assurance that a product, service, or

system meets the needs of the customer and other
identified stakeholders. It often involves acceptance
and suitability with external customers. Contrast
with verification."
 "Verification. The evaluation of whether or not a
product, service, or system complies with a
regulation, requirement, specification, or imposed
condition. It is often an internal process. Contrast
with validation."
 Verification is intended to check that a product,
service, or system (or portion thereof, or set
thereof) meets a set of initial design specifications.
In the development phase, verification procedures
involve performing special tests to model or
simulate a portion, or the entirety, of a product,
service or system, then performing a review or
analysis of the modelling results. In the post-
development phase, verification procedures involve
regularly repeating tests devised specifically to
ensure that the product, service, or system
continues to meet the initial design requirements,
specifications, and regulations as time progresses.
It is a process that is used to evaluate whether a
product, service, or system complies with
regulations, specifications, or conditions imposed
at the start of a development phase. Verification
 can be in development, scale-up, or production.
This is often an internal process.

 Validation is intended to check that development

and verification procedures for a product, service,
or system (or portion thereof, or set thereof) result
in a product, service, or system (or portion thereof,
or set thereof) that meets initial requirements. For a
new development flow or verification flow,
validation procedures may involve modelling either
flow and using simulations to predict faults or gaps
that might lead to invalid or incomplete verification
or development of a product, service, or system (or
portion thereof, or set thereof). A set of validation
requirements, specifications, and regulations may
then be used as a basis for qualifying a
development flow or verification flow for a
product, service, or system (or portion thereof, or
set thereof). Additional validation procedures also
include those that are designed specifically to
ensure that modifications made to an existing
 qualified development flow or verification flow
will have the effect of producing a product, service,
or system (or portion thereof, or set thereof) that
meets the initial design requirements,
specifications, and regulations; these validations
help to keep the flow qualified. It is a process of
establishing evidence that provides a high degree of
assurance that a product, service, or system
accomplishes its intended requirements. This often
involves acceptance of fitness for purpose with end
users and other product stakeholders. This is often
an external process.

 It is sometimes said that validation can be

expressed by the query "Are you building the right
thing?" and verification by "Are you building it
right?". "Building the right thing" refers back to the
user's needs, while "building it right" checks that
the specifications are correctly implemented by the
system. In some contexts, it is required to have
written requirements for both as well as formal
 procedures or protocols for determining
compliance.

 It is entirely possible that a product passes when

verified but fails when validated. This can happen
when, say, a product is built as per the
specifications but the specifications themselves fail
to address the user’s needs.

Activities

Verification of machinery and equipment usually

consists of design qualification (DQ), installation
qualification (IQ), operational qualification (OQ), and
performance qualification (PQ). DQ is usually a
vendor's job. However, DQ can also be performed by
the user, by confirming through review and testing that
the equipment meets the written acquisition
specification. If the relevant document or manuals of
machinery/equipment are provided by vendors, the later
3Q needs to be thoroughly performed by the users who
work in an industrial regulatory environment.
Otherwise, the process of IQ, OQ and PQ is the task of
validation. The typical example of such a case could be
the loss or absence of vendor's documentation for
legacy equipment or do-it-yourself (DIY) assemblies
(e.g., cars, computers etc.) and, therefore, users should
endeavour to acquire DQ document beforehand. Each
template of DQ, IQ, OQ and PQ usually can be found
on the internet respectively, whereas the DIY
qualifications of machinery/equipment can be assisted
either by the vendor's training course materials and
tutorials, or by the published guidance books, such
as step-by-step series if the acquisition of
machinery/equipment is not bundled with on- site
qualification services. This kind of the DIY approach is
also applicable to the qualifications of software,
computer operating systems and a manufacturing
process. The most important and critical task as the last
step of the activity is to generating and archiving
machinery/equipment qualification reports for auditing
purposes, if regulatory compliances are mandatory.
 Qualification of machinery/equipment is venue
dependent, in particular items that are shock sensitive
and require balancing or calibration, and re-
qualification needs to be conducted once the objects are
relocated. The full scales of some equipment
qualifications are even time dependent as consumables
are used up (i.e. filters) or springs stretch out,
requiring recalibration, and hence re-certification is
necessary when a specified due time lapse Re-
qualification of machinery/equipment should also be
conducted when replacement of parts, or coupling with
another device, or installing a new application
software and restructuring of the computer which
affects especially the pre-settings, such as
on BIOS, registry, disk drive partition table,
dynamically-linked (shared) libraries, or an ini file etc.,
have been necessary. In such a situation, the
specifications of the parts/devices/software and
restructuring proposals should be appended to the
qualification document whether the
parts/devices/software are genuine or not.

Torres and Hyman have discussed the suitability of

non-genuine parts for clinical use and provided
guidelines for equipment users to select appropriate
substitutes which are capable to avoid adverse
effects. In the case when genuine parts/devices/software
are demanded by some of regulatory requirements, then
re-qualification does not need to be conducted on the
non-genuine assemblies. Instead, the asset has to be
recycled for non-regulatory purposes.

When machinery/equipment qualification is

conducted by a standard endorsed third party such as by
an ISO standard accredited company for a particular
division, the process is called certification. Currently,
the coverage of ISO/IEC 15408 certification by
an ISO/IEC 27001 accredited organization is limited;
the scheme requires a fair amount of efforts to get
popularized.

System testing
 System testing of software or hardware is testing
conducted on a complete, integrated system to evaluate
the system's compliance with its specified requirements.
System testing falls within the scope of black box
testing, and as such, should require no knowledge of the
inner design of the code or logic.

As a rule, system testing takes, as its input, all of

the "integrated" software components that have
passed integration testing and also the software system
itself integrated with any applicable hardware
system(s). The purpose of integration testing is to detect
any inconsistencies between the software units that are
integrated together (called assemblages) or between any
of the assemblages and the hardware. System testing is
a more limited type of testing; it seeks to detect defects
both within the "inter-assemblages" and also within the
system as a whole.

System testing is performed on the entire system in

the context of a Functional
Requirement Specification(s) (FRS) and/or a System
Requirement Specification (SRS). System testing tests
not only the design, but also the behavior and even the
believed expectations of the customer. It is also
intended to test up to and beyond the bounds defined in
the software/hardware requirements specification

Types of tests to include in system testing

The following examples are different types of testing
that should be considered during System testing:

 Graphical user interface testing

 Usability testing
 Software performance testing
 Compatibility testing
 Exception handling
 Load testing
 Volume testing
 Stress testing
 Security testing
 Scalability testing
 Sanity testing
 Smoke testing
 Exploratory testing
 Ad hoc testing
 Regression testing
 Installation testing
 Maintenance testing Recovery testing and failover
testing.
 Accessibility testing, including compliance with:
 Americans with Disabilities Act of 1990
 Section 508 Amendment to the Rehabilitation Act
of 1973
 Web Accessibility Initiative (WAI) of the World
Wide Web Consortium (W3C)
Although different testing organizations may prescribe
different tests as part of System testing, this list serves
as a general framework or foundation to begin with.

Structure Testing:

It is concerned with exercising the internal logic of

a program and traversing particular execution paths.
Output Testing:

 Output of test cases compared with the expected

results created during design of test cases.
 Asking the user about the format required by them
tests the output generated or displayed by the
system under consideration.
 Here, the output format is considered into two
was, one is on screen and another one is printed
format.
 The output on the screen is found to be correct as
the format was designed in the system design
phase according to user needs.
 The output comes out as the specified
requirements as the user’s hard copy.

User acceptance Testing:

  Final Stage, before handling over to the customer
which is usually carried out by the customer where
the test cases are executed with actual data.
 The system under consideration is tested for user
acceptance and constantly keeping touch with the
prospective system user at the time of developing
and making changes whenever required.
 It involves planning and execution of various types
of test in order to demonstrate that the
implemented software system satisfies the
requirements stated in the requirement
document.
Two set of acceptance test to be run:

1. Those developed by quality assurance

group.
2. Those developed by customer.