17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

Optional Lab: Python, NumPy and Vectorization

A brief introduction to some of the scientific computing used in this course. In particular the NumPy scientific computing package and its use with python.

Outline
1.1 Goals
1.2 Useful References
2 Python and NumPy
3 Vectors
3.1 Abstract
3.2 NumPy Arrays
3.3 Vector Creation
3.4 Operations on Vectors
4 Matrices
4.1 Abstract
4.2 NumPy Arrays
4.3 Matrix Creation
4.4 Operations on Matrices

In [1]: import numpy as np

import time

In [2]: import numpy as np # it is an unofficial standard to use np for numpy

import time

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 1/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

1.1 Goals
In this lab, you will:

Review the features of NumPy and Python that are used in Course 1

1.2 Useful References

NumPy Documentation including a basic introduction: NumPy.org (https://NumPy.org/doc/stable/)
A challenging feature topic: NumPy Broadcasting (https://NumPy.org/doc/stable/user/basics.broadcasting.html)

2 Python and NumPy

Python is the programming language we will be using in this course. It has a set of numeric data types and arithmetic operations. NumPy is a library that extends
the base capabilities of python to add a richer data set including more numeric types, vectors, matrices, and many matrix functions. NumPy and python work
together fairly seamlessly. Python arithmetic operators work on NumPy data types and many NumPy functions will accept python data types.

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 2/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

3 Vectors
3.1 Abstract
Vectors, as you will use them in this course, are ordered arrays of numbers. In notation, vectors are
denoted with lower case bold letters such as 𝐱. The elements of a vector are all the same type. A vector does not, for example, contain both characters and
numbers. The number of elements in the array is often referred to as the dimension though mathematicians may prefer rank. The vector shown has a dimension
of 𝑛. The elements of a vector can be referenced with an index. In math settings, indexes typically run from 1 to n. In computer science and these labs, indexing
𝑡ℎ
will typically run from 0 to n-1. In notation, elements of a vector, when referenced individually will indicate the index in a subscript, for example, the 0 element,
of the vector 𝐱 is 𝑥0 . Note, the x is not bold in this case.

3.2 NumPy Arrays

NumPy's basic data structure is an indexable, n-dimensional array containing elements of the same type ( dtype ). Right away, you may notice we have
overloaded the term 'dimension'. Above, it was the number of elements in the vector, here, dimension refers to the number of indexes of an array. A one-
dimensional or 1-D array has one index. In Course 1, we will represent vectors as NumPy 1-D arrays.

1-D array, shape (n,): n elements indexed [0] through [n-1]

3.3 Vector Creation

In [ ]:

Data creation routines in NumPy will generally have a first parameter which is the shape of the object. This can either be a single value for a 1-D result or a tuple
(n,m,...) specifying the shape of the result. Below are examples of creating vectors using these routines.

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 3/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [11]: a = np.zeros(4); print(f"np.zeroes(4) : a = {a} a shape = {a.shape} a data type = {a.dtype}")

a = np.zeros((4,)); print(f"np.zeros((4,)): a = {a} a shape = {a.shape} a data type = {a.dtype}")
a = np.random.random_sample(4); print(f"np.random.random_sample(4): a = {a} a shape = {a.shape}, a data
type = {a.dtype}")

np.zeroes(4) : a = [0. 0. 0. 0.] a shape = (4,) a data type = float64

np.zeros((4,)): a = [0. 0. 0. 0.] a shape = (4,) a data type = float64
np.random.random_sample(4): a = [0.06499332 0.49098514 0.96876626 0.87880798] a shape = (4,), a data type
= float64

In [6]: # NumPy routines which allocate memory and fill arrays with value
a = np.zeros(4); print(f"np.zeros(4) : a = {a}, a shape = {a.shape}, a data type = {a.dty
pe}")
a = np.zeros((4,)); print(f"np.zeros(4,) : a = {a}, a shape = {a.shape}, a data type = {a.dty
pe}")
a = np.random.random_sample(4); print(f"np.random.random_sample(4): a = {a}, a shape = {a.shape}, a data t
ype = {a.dtype}")

np.zeros(4) : a = [0. 0. 0. 0.], a shape = (4,), a data type = float64

np.zeros(4,) : a = [0. 0. 0. 0.], a shape = (4,), a data type = float64
np.random.random_sample(4): a = [0.2260514 0.39265849 0.412455 0.57422791], a shape = (4,), a data type
= float64

Some data creation routines do not take a shape tuple:

In [ ]: a = np.arange(4.); print(f"np.arrange(4.): a = {a} a shape = {a.shape} a data type = {a.dtype}")

In [13]: # NumPy routines which allocate memory and fill arrays with value but do not accept shape as input argumen
t
a = np.arange(4.); print(f"np.arange(4.): a = {a}, a shape = {a.shape}, a data type = {a.
dtype}")
a = np.random.rand(4); print(f"np.random.rand(4): a = {a}, a shape = {a.shape}, a data type = {a.
dtype}")

np.arange(4.): a = [0. 1. 2. 3.], a shape = (4,), a data type = float64

np.random.rand(4): a = [0.27234005 0.54894909 0.68909301 0.39371212], a shape = (4,), a data type = float6
4

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 4/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

values can be specified manually as well.

In [ ]:

In [19]: a = np.array([1,4,2,6,5]); print(f"{a.shape} {a.dtype}")

(5,) int64

In [20]: # NumPy routines which allocate memory and fill with user specified values
a = np.array([5,4,3,2]); print(f"np.array([5,4,3,2]): a = {a}, a shape = {a.shape}, a data type =
{a.dtype}")
a = np.array([5.,4,3,2]); print(f"np.array([5.,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dty
pe}")

np.array([5,4,3,2]): a = [5 4 3 2], a shape = (4,), a data type = int64

np.array([5.,4,3,2]): a = [5. 4. 3. 2.], a shape = (4,), a data type = float64

These have all created a one-dimensional vector a with four elements. a.shape returns the dimensions. Here we see a.shape = (4,) indicating a 1-d array
with 4 elements.

3.4 Operations on Vectors

Let's explore some operations using vectors.

3.4.1 Indexing
Elements of vectors can be accessed via indexing and slicing. NumPy provides a very complete set of indexing and slicing capabilities. We will explore only the
basics needed for the course here. Reference Slicing and Indexing (https://NumPy.org/doc/stable/reference/arrays.indexing.html) for more details.
Indexing means referring to an element of an array by its position within the array.
Slicing means getting a subset of elements from an array based on their indices.
NumPy starts indexing at zero so the 3rd element of an vector 𝐚 is a[2] .

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 5/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]:

In [31]: a = np.arange(10); print(f"{a}")

try:
c = a[10] ; print(f"{c}")
except Exception as e:
print(e)

[0 1 2 3 4 5 6 7 8 9]
index 10 is out of bounds for axis 0 with size 10

In [ ]: #vector indexing operations on 1-D vectors

a = np.arange(10)
print(a)

#access an element
print(f"a[2].shape: {a[2].shape} a[2] = {a[2]}, Accessing an element returns a scalar")

# access the last element, negative indexes count from the end
print(f"a[-1] = {a[-1]}")

#indexs must be within the range of the vector or they will produce and error
try:
c = a[10]
except Exception as e:
print("The error message you'll see is:")
print(e)

3.4.2 Slicing
Slicing creates an array of indices using a set of three values ( start:stop:step ). A subset of values is also valid. Its use is best explained by example:

In [37]: a = np.arange(10)
c = a[:]; print(c)

[0 1 2 3 4 5 6 7 8 9]

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 6/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]: #vector slicing operations

a = np.arange(10)
print(f"a = {a}")

#access 5 consecutive elements (start:stop:step)

c = a[2:7:1]; print("a[2:7:1] = ", c)

# access 3 elements separated by two

c = a[2:7:2]; print("a[2:7:2] = ", c)

# access all elements index 3 and above

c = a[3:]; print("a[3:] = ", c)

# access all elements below index 3

c = a[:3]; print("a[:3] = ", c)

# access all elements

c = a[:]; print("a[:] = ", c)

3.4.3 Single vector operations

There are a number of useful operations that involve operations on a single vector.

In [44]: a = np.array([1,2,3,4])
print(f"a :{a}")
b = a**2
print(f"b : {b}")

a :[1 2 3 4]
b : [ 1 4 9 16]

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 7/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]: a = np.array([1,2,3,4])
print(f"a : {a}")
# negate elements of a
b = -a
print(f"b = -a : {b}")

# sum all elements of a, returns a scalar

b = np.sum(a)
print(f"b = np.sum(a) : {b}")

b = np.mean(a)
print(f"b = np.mean(a): {b}")

b = a**2
print(f"b = a**2 : {b}")

3.4.4 Vector Vector element-wise operations

Most of the NumPy arithmetic, logical and comparison operations apply to vectors as well. These operators work on an element-by-element basis. For example
𝑐𝑖 = 𝑎𝑖 + 𝑏𝑖

In [45]: a = np.array([ 1, 2, 3, 4])

b = np.array([-1,-2, 3, 4])
print(f"Binary operators work element wise: {a + b}")

Binary operators work element wise: [0 0 6 8]

Of course, for this to work correctly, the vectors must be of the same size:

In [ ]:

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 8/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [46]: a = np.array([1,2,3])
c = np.array([1,2])
a+c

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-46-433a64d462bd> in <module>
1 a = np.array([1,2,3])
2 c = np.array([1,2])
----> 3 a+c

ValueError: operands could not be broadcast together with shapes (3,) (2,)

In [47]: #try a mismatched vector operation

c = np.array([1, 2])
try:
d = a + c
except Exception as e:
print("The error message you'll see is:")
print(e)

The error message you'll see is:

operands could not be broadcast together with shapes (3,) (2,)

3.4.5 Scalar Vector operations

Vectors can be 'scaled' by scalar values. A scalar value is just a number. The scalar multiplies all the elements of the vector.

In [48]: a = np.array([1, 2, 3, 4])

# multiply a by a scalar
b = 5 * a
print(f"b = 5 * a : {b}")

b = 5 * a : [ 5 10 15 20]

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 9/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

3.4.6 Vector Vector dot product

The dot product is a mainstay of Linear Algebra and NumPy. This is an operation used extensively in this course and should be well understood. The dot product
is shown below.

The dot product multiplies the values in two vectors element-wise and then sums the result. Vector dot product requires the dimensions of the two vectors to be
the same.

Let's implement our own version of the dot product below:

Using a for loop, implement a function which returns the dot product of two vectors. The function to return given inputs 𝑎 and 𝑏 :
𝑛−1
𝑥 = ∑ 𝑎𝑖 𝑏𝑖
𝑖=0
Assume both a and b are the same shape.

In [ ]: def my_dot(a, b):

"""
Compute the dot product of two vectors

Args:
a (ndarray (n,)): input vector
b (ndarray (n,)): input vector with same dimension as a

Returns:
x (scalar):
"""
x=0
for i in range(a.shape[0]):
x = x + a[i] * b[i]
return x

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 10/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [51]: # test 1-D

a = np.array([1, 2, 3, 4])
b = np.array([-1, 4, 3, 2])
c = 0
for i in range(0,a.shape[0]):
c = c + a[i]*b[i]

print(np.dot(a,b))

24

Note, the dot product is expected to return a scalar value.

Let's try the same operations using np.dot .

In [ ]: # test 1-D
a = np.array([1, 2, 3, 4])
b = np.array([-1, 4, 3, 2])
c = np.dot(a, b)
print(f"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} ")
c = np.dot(b, a)
print(f"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} ")

Above, you will note that the results for 1-D matched our implementation.

3.4.7 The Need for Speed: vector vs for loop

We utilized the NumPy library because it improves speed memory efficiency. Let's demonstrate:

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 11/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [5]: import numpy as np

import time
np.random.seed(1)
a = np.random.rand(100000000)
b = np.random.rand(100000000)
tic = time.time()
x = np.dot(a,b)
toc = time.time()
print(f"{x} {toc -tic}")
temp_1 = toc-tic
tic = time.time()
f = 0
for i in range(0,a.shape[0]):
f = f + a[i]*b[i]
toc = time.time()
temp_2 = toc - tic
print(f"{f} {toc - tic}")
print(temp_2/temp_1)

25000669.807339136 0.48903322219848633

---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
<ipython-input-5-7a9af421ea5a> in <module>
12 f = 0
13 for i in range(0,a.shape[0]):
---> 14 f = f + a[i]*b[i]
15 toc = time.time()
16 temp_2 = toc - tic

KeyboardInterrupt:

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 12/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]: np.random.seed(1)
a = np.random.rand(10000000) # very large arrays
b = np.random.rand(10000000)

tic = time.time() # capture start time

c = np.dot(a, b)
toc = time.time() # capture end time

print(f"np.dot(a, b) = {c:.4f}")
print(f"Vectorized version duration: {1000*(toc-tic):.4f} ms ")

tic = time.time() # capture start time

c = my_dot(a,b)
toc = time.time() # capture end time

print(f"my_dot(a, b) = {c:.4f}")
print(f"loop version duration: {1000*(toc-tic):.4f} ms ")

del(a);del(b) #remove these big arrays from memory

So, vectorization provides a large speed up in this example. This is because NumPy makes better use of available data parallelism in the underlying hardware.
GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel. This is critical in
Machine Learning where the data sets are often very large.

3.4.8 Vector Vector operations in Course 1

Vector Vector operations will appear frequently in course 1. Here is why:

Going forward, our examples will be stored in an array, X_train of dimension (m,n). This will be explained more in context, but here it is important to note
it is a 2 Dimensional array or matrix (see next section on matrices).
w will be a 1-dimensional vector of shape (n,).
we will perform operations by looping through the examples, extracting each example to work on individually by indexing X. For example: X[i]
X[i] returns a value of shape (n,), a 1-dimensional vector. Consequently, operations involving X[i] are often vector-vector.

That is a somewhat lengthy explanation, but aligning and understanding the shapes of your operands is important when performing vector operations.

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 13/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]: # show common Course 1 example

X = np.array([[1],[2],[3],[4]])
w = np.array([2])
c = np.dot(X[1], w)

print(f"X[1] has shape {X[1].shape}")

print(f"w has shape {w.shape}")
print(f"c has shape {c.shape}")

4 Matrices

4.1 Abstract
Matrices, are two dimensional arrays. The elements of a matrix are all of the same type. In notation, matrices are denoted with capitol, bold letter such as 𝐗. In
this and other labs, m is often the number of rows and n the number of columns. The elements of a matrix can be referenced with a two dimensional index. In
math settings, numbers in the index typically run from 1 to n. In computer science and these labs, indexing will run from 0 to n-1.

missing
Generic Matrix Notation, 1st index is row, 2nd is column

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 14/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

4.2 NumPy Arrays

NumPy's basic data structure is an indexable, n-dimensional array containing elements of the same type ( dtype ). These were described earlier. Matrices have
a two-dimensional (2-D) index [m,n].

In Course 1, 2-D matrices are used to hold training data. Training data is 𝑚 examples by 𝑛 features creating an (m,n) array. Course 1 does not do operations
directly on matrices but typically extracts an example as a vector and operates on that. Below you will review:

data creation
slicing and indexing

4.3 Matrix Creation

The same functions that created 1-D vectors will create 2-D or n-D arrays. Here are some examples

Below, the shape tuple is provided to achieve a 2-D result. Notice how NumPy uses brackets to denote each dimension. Notice further than NumPy, when
printing, will print one row per line.

In [14]:

[[19 22]
[43 50]]

In [ ]:

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 15/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]: a = np.zeros((1, 5))
print(f"a shape = {a.shape}, a = {a}")

a = np.zeros((2, 1))
print(f"a shape = {a.shape}, a = {a}")

a = np.random.random_sample((1, 1))
print(f"a shape = {a.shape}, a = {a}")

One can also manually specify data. Dimensions are specified with additional brackets matching the format in the printing above.

In [ ]: # NumPy routines which allocate memory and fill with user specified values
a = np.array([[5], [4], [3]]); print(f" a shape = {a.shape}, np.array: a = {a}")
a = np.array([[5], # One can also
[4], # separate values
[3]]); #into separate rows
print(f" a shape = {a.shape}, np.array: a = {a}")

4.4 Operations on Matrices

Let's explore some operations using matrices.

4.4.1 Indexing

Matrices include a second index. The two indexes describe [row, column]. Access can either return an element or a row/column. See below:

In [28]: import numpy as np

a = np.arange(6).reshape(-1,2)
print(f"a[0].shape {type(a[0][1])}")

a[0].shape <class 'numpy.int64'>

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 16/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [15]: import numpy as np

#vector indexing operations on matrices
a = np.arange(6).reshape(-1, 2) #reshape is a convenient way to create matrices
print(f"a.shape: {a.shape}, \na= {a}")

#access an element
print(f"\na[2,0].shape: {a[2, 0].shape}, a[2,0] = {a[2, 0]}, type(a[2,0]) = {type(a[2, 0])} Accessin
g an element returns a scalar\n")

#access a row
print(f"a[2].shape: {a[2].shape}, a[2] = {a[2]}, type(a[2]) = {type(a[2])}")

a.shape: (3, 2),

a= [[0 1]
[2 3]
[4 5]]

a[2,0].shape: (), a[2,0] = 4, type(a[2,0]) = <class 'numpy.int64'> Accessing an element returns a sc

alar

a[2].shape: (2,), a[2] = [4 5], type(a[2]) = <class 'numpy.ndarray'>

It is worth drawing attention to the last example. Accessing a matrix by just specifying the row will return a 1-D vector.

Reshape
The previous example used reshape (https://numpy.org/doc/stable/reference/generated/numpy.reshape.html) to shape the array.
a = np.arange(6).reshape(-1, 2)
This line of code first created a 1-D Vector of six elements. It then reshaped that vector into a 2-D array using the reshape command. This could have been
written:
a = np.arange(6).reshape(3, 2)
To arrive at the same 3 row, 2 column array. The -1 argument tells the routine to compute the number of rows given the size of the array and the number of
columns.

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 17/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

4.4.2 Slicing
Slicing creates an array of indices using a set of three values ( start:stop:step ). A subset of values is also valid. Its use is best explained by example:

In [36]: import numpy as np

a = np.arange(20).reshape(-1,10)
print(f"a = {a}")
print(f"{a[:,0:1].shape}")

a = [[ 0 1 2 3 4 5 6 7 8 9]
[10 11 12 13 14 15 16 17 18 19]]
(2, 1)

In [ ]: #vector 2-D slicing operations

a = np.arange(20).reshape(-1, 10)
print(f"a = \n{a}")

#access 5 consecutive elements (start:stop:step)

print("a[0, 2:7:1] = ", a[0, 2:7:1], ", a[0, 2:7:1].shape =", a[0, 2:7:1].shape, "a 1-D array")

#access 5 consecutive elements (start:stop:step) in two rows

print("a[:, 2:7:1] = \n", a[:, 2:7:1], ", a[:, 2:7:1].shape =", a[:, 2:7:1].shape, "a 2-D array")

# access all elements

print("a[:,:] = \n", a[:,:], ", a[:,:].shape =", a[:,:].shape)

# access all elements in one row (very common usage)

print("a[1,:] = ", a[1,:], ", a[1,:].shape =", a[1,:].shape, "a 1-D array")
# same as
print("a[1] = ", a[1], ", a[1].shape =", a[1].shape, "a 1-D array")

Congratulations!
In this lab you mastered the features of Python and NumPy that are needed for Course 1.

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 18/19

17/03/2025, 13:53 C1_W2_Lab01_Python_Numpy_Vectorization_Soln

In [ ]:

file:///Users/abhishek.luthra/Downloads/C1_W2_Lab01_Python_Numpy_Vectorization_Soln (1).html 19/19