Autoencoders import tensorflow as tf

message = tf.constant('Hello World!')

sess = tf.Session()
print(sess.run(message))

Output:
Hello World!

 One way to improve the performance of deep net is to implement it in terms of vectors and
matrices. This avoids too many loops and increases the computation speed significantly. This is
known as parallel programming which is implemented at a software level.
 Parallelism can also be implemented at a hardware level, which is called parallel processing.
CPUs with multiple cores and GPUs support parallel processing.
 Complex deep nets perform best when parallel processing is used. It can be either shared
memory or distributed computing.
 GPUs, FPGAs, and ASICs are some of the shared memory options available to train a deep net.
o Graphical processing unit (GPU) consists of thousands of core in the same unit where
each core are process instructions in parallel. Hence they are suitable for deep learning
application. They consume more power.
o Field programmable gate arrays (FPGA) and Application-specific Integrated specific
(ASIC) are best options for customized deep net since they can be programmed at a
hardware level and they are power efficient.
 Data parallelism, model parallelism, and pipeline parallelism are some of the options for
distributed computing for deep learning.
Deep net performance can be improved at software level through parallel programming. Below are
some of the ways we can implement parallel programming:

 By decomposing the model into chunks and each chunk is fed in parallel to perform an
independent task.
 Identify tasks that have dependencies and group them. By creating multiple groups with no
dependencies, each group of tasks can be processed in parallel.
 By implementing tasks or task groups as threads and run them in parallel.

 CPU executes instruction sequentially though instruction is independent of each other. Hence, it

takes more time to process information for complex algorithms. It has a limited number of cores.

 GPU, in contrast, has thousands of cores that identify independent instructions, groups them and
execute in parallel. Hence they are more suitable for deep learning kind of applications.

 Theano is a Python library.

 It was developed by a machine learning group headed by Yoshua Bengio at the University of
Montreal.
 It is more a research platform than a deep learning library. You must perform more work by
yourself to generate the models that you want. It does not have any deep learning classes within
itself.
Theano - Features

 Theano allows us to define mathematical expressions as a set of vectors and matrices. This
avoids too many for loops in our code and greatly reduces the computation time.
 Theano is best suited when we are going to build everything from scratch. It just aids in
representing our deep net in terms of vectors and matrices.
 Some of the optimization techniques used by Theano are the use of GPU for computations,
arithmetic simplification, constant folding, using memory aliasing to avoid calculation, etc.
 Popular libraries like Keras, Lasagne, Blocks, and Pylearn2 are built on top of Theano.

 DeepLearning4J (DL4J) is a Deep Learning framework created in Java and JVM languages for
using in commercial deep learning projects. Adam Gibson developed DL4J.

 DL4J is utilized in business environments on distributed CPUs and GPUs, making it ideal

for commercial-grade applications.

Features

 DL4J runs on distributed GPUs and CPUs.

 It allows us to tune the deep net by selecting values for hyperparameters
 It supports most of the deep nets like DBN, RBN, CNN, RNTN, autoencoders, Recurrent net and
vanilla MLP.
 It also includes vectorization library called Canova and distributed multi-node map reduce
procedure for training the model.

 Torch is a Lua deep learning framework developed by Koray Kavukcuoglu, Clement Farabet and
Ronan Collobert for research and development activities into deep learning algorithms.
 Torch is written in LuaJit (framework in Lua programming language) with an underlying C
implementation.
 It has also been further contributed by Facebook, Google DeepMind, Twitter and a host of others.
 Popular applications of Torch are for supervised image problems with Convolutional Neural
Networks and agents in more complex domains with deep reinforcement learning.

Features

 Torch allows us to set up, train, and model deep net by configuring its hyperparameters.
 Fast and efficient GPU support.
 Provides built-in functions for indexing, transposing, slicing and numeric optimization.
 Embeddable, with ports to iOS and Android backends.

 Caffe library was developed by Yangqing Jia at the Berkeley Vision and Learning Center for
supervised computer vision problems.
 It is written in C++ with a Python interface
 It is mainly suited for machine vision tasks and also supports speech and text, reinforcement
learning and recurrent nets.
Features

 An application can easily switch between CPU and GPU since Caffe is written in C++ with CUDA
(a parallel computing platform).
 You can build extremely complex deep net since it allows to perform highly sophisticated
configuration on each layer.
 It also includes Matlab and Python interfaces.
 Caffe stores, communicates and manipulates the information as blobs that provides
synchronization between CPU and GPU.

 TensorFlow is an open-source deep learning library from Google. It is available on GitHub.

 Its flexible architecture permits to use computation to one or more GPUs or CPUs in a server,
desktop or mobile device with one API.

Features

 It is similar to the computational graph where edges carry data as N-dimensional vector known as
tensor and nodes represent mathematical operation which acts on the tensors.
 Tensorflow lets you deploy parallel computing devices to execute operations quicker. Operations
at the nodes are automatically scheduled for parallel computing.
 It includes features such as auto differentiation, shared and symbolic variables, and common
subexpression elimination.
 It comes with visualization tools for graphically viewing different levels of the network, changes
over time throughout the training process.

Configuration Parameters

 When building a deep net, you need to decide on various parameters for its better performance.
These are called hyperparameters.
 Unlike weights and biases hyperparameters are not learned by itself, you need to choose these
parameters when building a deep net.

 Input layer: This layer is nothing but our input data, more the data better the learning of deep
net.

 Hidden layers: There are many different hidden layers to choose from like convolution, pooling,
activation, loss layer, etc.

 Output layer: At this layer, you need to decide on cost functions base on which our model gets
trained. Some of the cost functions are the sum of squares, exponential cost, cross-entropy, etc.

Choosing the Number of Neurons

You need to decide on the number of neurons in each layer. There are two ways to choose an optimized
number of neurons: growing and pruning.
 Growing: In this approach, you start with a small number of neurons and keep on adding them
until there is no improvement in the cost function.
 Pruning: It is exactly opposite of what you follow in growing. You start with excess neurons and
remove them step by step until there is a cost difference.
A Neural Network without Activation function would simply be a Linear regression model,
which has limited power and does not perform right most of the times. Most popular types of activation
functions are

 Sigmoid
 Tanh
 ReLu.

Regularization

 We apply regularization to avoid overfitting of our model (a case where your model does not
perform well on test data).
 Regularization, significantly decreases the variance of the model, without a substantial increase
in its bias.
 Ridge and Lasso regularization are the most commonly used regularization techniques.

 During the time of backpropagation, the learning rate must not be too small.

 This is because since it takes more time for the algorithm to learn and it should not be too large,
which may skip the global minimum of the cost function.

Fact Extraction and Text Translation

 Deep nets can be used to extract facts and relations in the text.
 Named Entity Recognition (NER), Part Of Speech (POS) tagging or sentiment analysis are some
of the problems where neural network models have outperformed traditional approaches.
 Texts can be translated from one language to other using RNN. The encoder-decoder recurrent
neural network architecture is the core technology inside Google’s translate service.

Sentiment Analysis

 Using sentiment analysis, the underlying intent of the text can be extracted.
 With social media channels, it is possible to automate and measure what public feels on a given
news story, topic, brand or product.
 Positive sentiment can be identified thereby allowing the identification of product advocates, or to
see which parts of a business strategy are working.

Medical Applications

 Deep nets can be trained to detect cancerous cells and tumors from scanned images.
 They are applied in drug discovery by training nets with molecular structure and chemical
compositions.
 Using MRI and CT scans neural networks can be trained to differentiate between benign and
malignant tumors.

Other Applications
  Deep nets are used in stock predictions and building portfolios and risk allocation.
 In digital advertising to optimally bid an ad space in a web page.
 In fraud detection to detect suspicious transaction.
 Using images from satellites and sensor data deep nets can be trained to detect problematic
environmental conditions in agriculture.

Which one of the following libraries are mainly designed for machine vision? - Caffe

What is a Tensor? – N-Dimensional Array

Deep learning software platforms are available as a downloadable package. - True

Which one of the following libraries uses computations graph of deep learning? – Tensor Flow

Which one of the following libraries comes with a visualization tool for deep learning? - Tensor Flow

Which one of the following libraries stores and communicates data using blobs? – Caffe

Learning rate is learned by the network when training. – True

Deep learning is not suitable for text analysis. – False

Deep learning performs better when sequential processing is used. – False

Formula for recall is – TP/(TP+FN)

FPGAs are power efficient when compared to GPU. – True

Which one of the functions always maps the values between 0 and 1? – Sigmoid

A convolutional neural network is mainly used for image recognition. - True