PyTorch Essentials: Autograd, Tensors, and Training Loop Skills Quiz Quiz

Assess your foundational knowledge of PyTorch basics, focusing on tensors, automatic differentiation with autograd, and constructing simple training loops. This quiz is ideal for learners seeking to reinforce their understanding of key PyTorch operations, gradient computation, and model training workflows.

1. Understanding Tensor Creation

   Which of the following lines correctly creates a 2D tensor of zeros with shape (3, 2)?

   1. Tensor.zeros([3,2])
   2. torch.zeros((3, 2))
   3. torch.zeroes([3, 2])
   4. tensor.zeros(3,2)

   Explanation: The correct syntax for creating a tensor filled with zeros in a specified shape is torch.zeros(), passing the shape as a tuple. The option tensor.zeros(3,2) is incorrect because 'tensor' is not the right namespace and the shape should be a tuple. torch.zeroes([3, 2]) contains a typo in the function name ('zeroes' should be 'zeros'). Tensor.zeros([3,2]) uses the wrong case and object reference. Only torch.zeros((3, 2)) accurately follows the common Python and PyTorch syntax.

2. Gradient Tracking in Autograd

   How can you enable a tensor to track gradients for automatic differentiation?

   1. By saving the tensor as a NumPy array
   2. By calling grad() on the tensor
   3. By setting requires_grad=True when creating the tensor
   4. By multiplying the tensor by 2

   Explanation: Setting requires_grad=True during tensor creation tells the autograd system to track operations on this tensor for gradient computation. Calling grad() does not enable gradient tracking; instead, it's a property to access computed gradients. Multiplying the tensor or saving it as a NumPy array has no effect on gradient tracking. Only the correct option enables the tensor to be part of the computation graph.

3. Autograd and Backward Propagation

   After computing a loss value in a training step, what method is commonly called to compute gradients for parameters involved?

   1. backward()
   2. compute_gradients()
   3. autograd()
   4. backprop()

   Explanation: The backward() method is used to backpropagate the error and compute gradients of tensors with requires_grad set to True. Neither compute_gradients() nor backprop() are valid PyTorch methods, and autograd() is not a function. The backward() method specifically triggers the automatic differentiation process.

4. Accessing Computed Gradients

   Where can the gradients of a parameter tensor be found after calling backward()?

   1. As a separate gradients array
   2. In the tensor's .grad attribute
   3. By re-running the optimizer
   4. In the tensor's .backprop property

   Explanation: After calling backward(), gradients are populated in the .grad attribute of each tensor with requires_grad set to True. There is no .backprop property, nor are gradients stored in a separate array by default. Running the optimizer does not store gradients but typically uses them to update parameters. The .grad attribute is the direct way to access these values.

5. Tensor Conversion to NumPy Arrays

   What is the correct way to convert a PyTorch tensor 'x' to a NumPy array?

   1. np.array(x)
   2. x.numpy()
   3. x.to_numpy()
   4. x.asnumpy()

   Explanation: The method for converting a tensor to a NumPy array is x.numpy(), which is efficient if the tensor is on the CPU. x.to_numpy() and x.asnumpy() are not valid PyTorch tensor methods. np.array(x) works but creates a copy and does not share memory, which can be less efficient. Using x.numpy() is the direct and recommended method.

6. Zeroing Gradients in Training Loops

   Why is it important to zero gradients (e.g., optimizer.zero_grad()) in a training loop?

   1. To initialize model parameters before training
   2. Because gradients accumulate across backward passes by default
   3. To save memory space for tensors
   4. Because it makes the training loop faster

   Explanation: Gradients accumulate by default to enable gradient accumulation over multiple backward passes, so they must be manually zeroed at the start of each iteration. This is not done to save memory or initialize parameters, nor does it directly make training faster. Forgetting to zero gradients may lead to incorrect parameter updates.

7. Element-wise Operations on Tensors

   Given tensors a = [1, 2, 3] and b = [4, 5, 6], what is the result of a + b?

   1. [1, 2, 3, 4, 5, 6]
   2. [5, 7, 9]
   3. [1, 2, 3, 10, 12, 15]
   4. [4, 5, 6]

   Explanation: The + operator for tensors performs element-wise addition, so the result is [1+4, 2+5, 3+6] which is [5, 7, 9]. Concatenation would result in [1, 2, 3, 4, 5, 6], which is not what this operator does. The option [4, 5, 6] only lists one of the original tensors, and [1, 2, 3, 10, 12, 15] does not match any valid operation. Element-wise addition gives [5, 7, 9].

8. Disabling Gradient Tracking

   Which context disables gradient tracking for the code within its block?

   1. with torch.no_grad():
   2. with torch.stop_grad():
   3. with no_grad():
   4. with torch.nograd():

   Explanation: The correct way to prevent gradient tracking is to use 'with torch.no_grad()' as a context manager. torch.nograd(), torch.stop_grad(), and no_grad() are invalid names and will cause errors. Only 'with torch.no_grad():' is recognized for this purpose, making it useful for inference or evaluation phases.

9. Optimizers in Training Loops

   Which method is commonly called on the optimizer to update model parameters after gradients are calculated?

   1. optimizer.update()
   2. optimizer.backward()
   3. optimizer.step()
   4. optimizer.apply_gradients()

   Explanation: Calling optimizer.step() applies the computed gradients to update the parameters. optimizer.update() and optimizer.apply_gradients() do not exist in this library's API, and optimizer.backward() is incorrect because backward() is a method on tensors, not optimizers. Only optimizer.step() has the effect of performing a parameter update.

10. Checking Tensor Device

    How can you check whether a tensor is currently stored on the CPU or the GPU?

    1. By printing tensor.device
    2. By using tensor.check_device()
    3. By checking tensor.location
    4. By calling tensor.is_gpu()

    Explanation: Accessing the tensor.device property reveals whether the tensor is on the CPU or GPU. The tensor.is_gpu() function does not exist, and tensor.location or tensor.check_device() are not standard methods or attributes. Only tensor.device provides this information concisely.