Let’s start with Artificial Neural Network. Computer Vision has found it place in our day-to-day life due to Deep Learning, and the basic element of Deep Learning algorithm is an Artificial Neural Network…
What are Artificial Neural Network (ANN) ?
Two contradictory aspects are present in the comparison of computer programs and human being. With faultless ease, a young child can recognize faces even though it does not remember the names of those persons. However, data processing functions of face recognition are difficult to program using rule-based logic. On the other hand, complex arithmetic functions that asks for a great effort even for intelligent adults are simple to program. These aspects unfolds the blending of biological sophistication (perceptual recognition, generalization, recall, and thinking [1] and primitiveness of human being [2].
In order to imitate these aspects in computer programs, several ‘self-organized’ networks have been constructed [3]. These networks are the replication of a human brain. Biological neurons, the part of a human brain that has dendrites transmit signals to a cell body. The cell processes the signal. Then, the signal passes to other neurons through connections called axon [4].
The element which replicates a biological neuron was named “perceptron”, which can be illustrated as shown in above Figure [1] , the networks are called as artificial neural networks.
Artificial neural networks are computing models that are a set of Perceptron, which recognize patterns and learn representations when fed with raw data. Representation learning was done before 1986 using simple methods such as the perceptron-convergence procedure [5]. The deepen neural networks became possible after the invention of back-propagation, using which representations can be learned by adjusting the weights of the connections in the network [5]. The gradient-based learning, the combination of the stochastic gradient descent algorithm and back-propagation algorithm has improved the capability of extracting features from high-dimensional patterns using Convolutional Neural Networks (CNNs)-based methods, and these methods has eliminated hand-crafted feature extraction methods [6].
Elements of an ANNs:
An artificial neural network, the vital part of a Deep Learning algorithm can be divided into three segments: 1) Input Layer, 2) Hidden Layer, 3) Output Layer as shown in above Figure.
A Input layer, also known as a “passive” layer, is the very first layer of the network. The system generates the weights of passive neurons of the input layer. And, these weights bring the information first time into the system. A hidden layer, a sandwich layer in between the input layer and output layer, contains artificial neurons. These neurons have specific weighted inputs and get updated in the process. Also, these neurons could be two types: Linear Processing Unit, or Nonlinear Layer Activation. A output layer, the last layer of an artificial neural network, produces required number of outputs that we want from the system.
Further, These layers and the other parts of deep learning algorithm are explained in here: Convolutional Neural Networks.
References:
- Frank Rosenblatt. “The perceptron: a probabilistic model for information storage and organization in the brain.” In: Psychological review 65.6 (1958), p. 386
- Michael J Apter. The computer simulation of behaviour. Routledge, 2018.
- BWAC Farley and W do Clark. “Simulation of self-organizing systems by digital computer”. In: Transactions of the IRE Professional Group on Information Theory 4.4 (1954), pp. 76–84.
- Richard E Neapolitan and Xia Jiang. Artificial intelligence: With an introduction to machine learning. CRC Press, 2018.
- David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. “Learning representations by back-propagating errors”. In: nature 323.6088 (1986), pp. 533–536
- Yann LeCun et al. “Gradient-based learning applied to document recognition”. In: Proceedings of the IEEE 86.11 (1998), pp. 2278–2324.
