Skip to content

Commit 82572fe

Browse files
Updated answer no. 7 with a more relevant answer (andrewekhalel#13)
* Updated answer no. 7 with a more relevant answer * Reverted a line deletion --------- Co-authored-by: Andrew Khalel <andrew_emel@hotmail.com>
1 parent ae12a01 commit 82572fe

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,14 +55,15 @@ Principal Component Analysis (PCA) is a dimensionality reduction technique used
5555
* Computation Efficiency:
5656
As ReLU is a simple threshold the forward and backward path will be faster.
5757
* Reduced Likelihood of Vanishing Gradient:
58-
Gradient of ReLU is 1 for positive values and 0 for negative values.
58+
Gradient of ReLU is 1 for positive values and 0 for negative values while Sigmoid activation saturates (gradients close to 0) quickly with slightly higher or lower inputs leading to vanishing gradients.
5959
* Sparsity:
6060
Sparsity happens when the input of ReLU is negative. This means fewer neurons are firing ( sparse activation ) and the network is lighter.
6161

6262

6363
[[src1]](https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0) [[src2]](https://stats.stackexchange.com/questions/126238/what-are-the-advantages-of-relu-over-sigmoid-function-in-deep-neural-networks)
6464

6565

66+
6667
#### 8) Given stride S and kernel sizes for each layer of a (1-dimensional) CNN, create a function to compute the [receptive field](https://www.quora.com/What-is-a-receptive-field-in-a-convolutional-neural-network) of a particular node in the network. This is just finding how many input nodes actually connect through to a neuron in a CNN. [[src](https://www.reddit.com/r/computervision/comments/7gku4z/technical_interview_questions_in_cv/)]
6768

6869
The receptive field are defined portion of space within an inputs that will be used during an operation to generate an output.

0 commit comments

Comments
 (0)