# What is a neural network? 什么是神经网络？ A neural network is a collection of **neurons** that are connected by layers. Each neuron is a small computing unit that performs simple calculations to collectively solve a problem. Neurons are organized in three types of layers: input layer, hidden layer, and output layer. The hidden and output layers contain a number of neurons. Neural networks mimic the way a human brain processes information. 神经网络是一组通过层连接的**神经元**的集合。每个神经元是一个小型计算单元，执行简单的计算来集体解决问题。神经元被组织成三种类型的层：输入层、隐藏层和输出层。隐藏层和输出层中包含多个神经元。神经网络模拟人脑处理信息的方式。 ## Components of a neural network 神经网络的组成部分 - An **activation function** determines whether a neuron should be activated or not. The computations that happen in a neural network include applying an activation function. If a neuron activates, then it means the input is important. There are different kinds of activation functions. The choice of which activation function to use depends on what you want the output to be. Another important role of an activation function is to add non-linearity to the model. - **激活函数**决定了一个神经元是否应该被激活。神经网络中的计算包括应用激活函数。如果一个神经元被激活，这意味着输入是重要的。激活函数有多种类型，选择哪种激活函数取决于你希望输出的结果。激活函数的另一个重要作用是为模型增加非线性特性。 - _Binary_ is used to set an output node to 1 if the function result is positive and 0 if the function result is zero or negative. $f(x)= {\small \begin{cases} 0, & \text{if } x < 0\\ 1, & \text{if } x\geq 0\\ \end{cases}}$ - _Binary_ （二值）用于将输出节点设置为1（如果函数结果为正），或设置为0（如果函数结果为0或负）。$f(x)= {\small \begin{cases} 0, & \text{if } x < 0\\ 1, & \text{if } x\geq 0\\ \end{cases}}$ - _Sigmoid_ is used to predict the probability of an output node being between 0 and 1. $f(x) = {\large \frac{1}{1+e^{-x}}} $ - _Sigmoid_（S形函数）用于预测输出节点在0到1之间的概率。$f(x) = {\large \frac{1}{1+e^{-x}}}$ - _Tanh_ is used to predict whether an output node is between 1 and -1, for classification use cases. $f(x) = {\large \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}} $ - _Tanh_（双曲正切函数）用于预测输出节点在1到-1之间的值，常用于分类场景。$f(x) = {\large \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}}$ - _ReLU_ (*rectified linear activation function*) is used to set the output node to 0 if the function result is negative and keeps the result value if the result is a positive value. $f(x)= {\small \begin{cases} 0, & \text{if } x < 0\\ x, & \text{if } x\geq 0\\ \end{cases}}$ - _ReLU_（修正线性激活函数）用于将输出节点设置为0（如果函数结果为负），并保留正值的结果。$f(x)= {\small \begin{cases} 0, & \text{if } x < 0\\ x, & \text{if } x\geq 0\\ \end{cases}}$ - **Weights** influence how close our network's output is to the expected output value. As an input enters the neuron, it gets multiplied by a weight value, and the resulting output is either observed or passed to the next layer in the neural network. Weights for all neurons in a layer are organized into one tensor. - **权重**影响神经网络输出与期望输出的接近程度。当输入进入神经元时，它会被一个权重值所乘，结果要么被观察，要么传递到神经网络的下一层。每一层中所有神经元的权重组织成一个张量。 - **Bias** makes up the difference between the activation function's output and its intended output. A low bias suggests that the network is making more assumptions about the output's form, whereas a high bias value makes fewer assumptions about the output's form. - **偏置**用于弥补激活函数输出与期望输出之间的差距。较低的偏置意味着网络对输出的形式做了更多假设，而较高的偏置值则对输出的形式假设较少。 We can say that an output $y$ of a neural network layer with weights $W$ and bias $b$ is computed as summation of the inputs multiplied by the weights plus the bias. $x = \sum{(weights * inputs) + bias} $, where $f(x)$ is the activation function. 我们可以说，具有权重$W$和偏置$b$的神经网络层的输出$y$是输入乘以权重再加上偏置的总和。$x = \sum{(weights * inputs) + bias}$，其中$f(x)$是激活函数。 # Build a neural network 构建神经网络 Neural networks are comprised of layers and modules that perform operations on data. The `torch.nn` namespace provides all the building blocks you need to build your own neural network. Every module in PyTorch subclasses `nn.Module`. A neural network is itself a module that consists of other modules (layers). This nested structure allows for building and managing complex architectures easily. 神经网络由执行数据操作的层和模块组成。`torch.nn`命名空间提供了构建神经网络所需的所有基础组件。PyTorch 中的每个模块都继承自 `nn.Module`。神经网络本身也是一个模块，由其他模块（层）组成。这样的嵌套结构可以轻松构建和管理复杂的架构。 In the following sections, we'll build a neural network to classify images in the FashionMNIST dataset. 在接下来的部分中，我们将构建一个神经网络，用于对 FashionMNIST 数据集中的图像进行分类。 ```python %matplotlib inline import os import torch from torch import nn from torch.utils.data import DataLoader from torchvision import datasets, transforms ``` ## Get a hardware device for training 获取用于训练的硬件设备 We want to be able to train our model on a hardware accelerator like a GPU, if one is available. Let's check to see whether `torch.cuda` is available; if not, we'll continue to use the CPU. 我们希望能够在硬件加速器（如 GPU）上训练模型，如果可用的话。让我们检查一下 `torch.cuda` 是否可用；如果不可用，我们将继续使用 CPU。 ```python try: import torch_directml device = torch_directml.device() except ImportError: device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu" print("Using {} device".format(device)) ``` Using privateuseone:0 device ## Define the class 定义类 We define our neural network by subclassing `nn.Module`, and initialize the neural network layers in `__init__`. Every `nn.Module` subclass implements the operations on input data in the `forward` method. 我们通过继承 `nn.Module` 来定义神经网络，并在 `__init__` 方法中初始化神经网络层。每个 `nn.Module` 子类都在 `forward` 方法中实现对输入数据的操作。 Our neural network is composed of the following: - The input layer with 28x28 or 784 features/pixels. - The first linear module takes the input 784 features and transforms it to a hidden layer with 512 features. - The ReLU activation function will be applied in the transformation. - The second linear module takes 512 features as input from the first hidden layer and transforms it to the next hidden layer with 512 features. - The ReLU activation function will be applied in the transformation. - The third linear module take 512 features as input from the second hidden layer and transforms those features to the output layer with 10, which is the number of classes. - The ReLU activation function will be applied in the transformation. 我们的神经网络由以下部分组成： - 输入层有 28x28 或 784 个特征/像素。 - 第一个线性模块将 784 个输入特征转换为具有 512 个特征的隐藏层。 - 在转换过程中应用 ReLU 激活函数。 - 第二个线性模块将第一个隐藏层的 512 个特征作为输入，并将其转换为具有 512 个特征的下一个隐藏层。 - 在转换过程中应用 ReLU 激活函数。 - 第三个线性模块将第二个隐藏层的 512 个特征作为输入，并将这些特征转换为输出层，输出层具有 10 个类（即分类数）。 - 在转换过程中应用 ReLU 激活函数。 ```python class NeuralNetwork(nn.Module): def __init__(self): super(NeuralNetwork, self).__init__() self.flatten = nn.Flatten() self.linear_relu_stack = nn.Sequential( nn.Linear(28*28, 512), nn.ReLU(), nn.Linear(512, 512), nn.ReLU(), nn.Linear(512, 10), nn.ReLU() ) def forward(self, x): x = self.flatten(x) logits = self.linear_relu_stack(x) return logits ``` We create an instance of `NeuralNetwork`, move it to the `device`, and print its structure. 我们创建一个 `NeuralNetwork` 实例，将其移动到 `device` 上，并打印其结构。 ```python model = NeuralNetwork().to(device) print(model) ``` NeuralNetwork( (flatten): Flatten(start_dim=1, end_dim=-1) (linear_relu_stack): Sequential( (0): Linear(in_features=784, out_features=512, bias=True) (1): ReLU() (2): Linear(in_features=512, out_features=512, bias=True) (3): ReLU() (4): Linear(in_features=512, out_features=10, bias=True) (5): ReLU() ) ) To use the model, we pass it the input data. This executes the model's `forward`, along with some background operations. However, don't call `model.forward()` directly! Calling the model on the input returns a 10-dimensional tensor with raw predicted values for each class. 要使用模型，我们将输入数据传递给它。这将执行模型的 `forward` 方法，并在后台进行一些操作。但不要直接调用 `model.forward()`！将输入传递给模型会返回一个包含每个类别的原始预测值的 10 维张量。 We get the prediction densities by passing it through an instance of the `nn.Softmax`. 我们通过将其传递给 `nn.Softmax` 的实例来获得预测的概率分布。 ```python X = torch.rand(1, 28, 28, device=device) logits = model(X) pred_probab = nn.Softmax(dim=1)(logits) y_pred = pred_probab.argmax(1) print(f"Predicted class: {y_pred}") ``` Predicted class: tensor([6], device='privateuseone:0') ## Weight and Bias 权重和偏置 The `nn.Linear` module randomly initializes the ${weights}$ and ${bias}$ for each layer and internally stores the values in Tensors. `nn.Linear` 模块会随机初始化每一层的 ${weights}$ 和 ${bias}$，并将这些值内部存储在张量中。 ```python print(f"First Linear weights: {model.linear_relu_stack[0].weight} \n") print(f"First Linear biases: {model.linear_relu_stack[0].bias} \n") ``` First Linear weights: Parameter containing: tensor([[ 0.0016, 0.0144, -0.0148, ..., -0.0311, 0.0349, -0.0070], [ 0.0113, 0.0346, 0.0020, ..., 0.0019, -0.0127, -0.0244], [-0.0143, 0.0023, -0.0221, ..., 0.0085, 0.0353, 0.0259], ..., [-0.0298, -0.0305, 0.0248, ..., 0.0074, -0.0256, -0.0231], [ 0.0101, -0.0337, -0.0350, ..., 0.0216, -0.0129, 0.0260], [ 0.0147, -0.0342, 0.0310, ..., -0.0301, -0.0048, -0.0288]], device='privateuseone:0', requires_grad=True) First Linear biases: Parameter containing: tensor([ 4.6210e-03, -4.8784e-03, -1.4250e-02, -5.5684e-03, -2.4937e-02, 1.4589e-02, 7.8008e-03, 1.0860e-02, -2.9434e-02, 4.0715e-04, 5.5757e-03, -1.2573e-02, -2.6717e-02, -1.4656e-02, 2.4371e-02, 2.6097e-03, 2.0497e-02, 2.8763e-02, -3.1914e-02, -1.9835e-02, -1.1525e-02, -8.0463e-03, -2.9528e-02, -1.9453e-02, -3.1493e-02, -1.0137e-03, 2.7029e-03, 3.0929e-02, 1.6575e-02, -1.0592e-02, 1.8419e-02, -3.3271e-02, -8.5039e-03, 1.1255e-03, 3.5259e-02, 2.5678e-02, 3.1693e-02, 3.8354e-03, 2.5844e-02, 2.7779e-02, -3.1347e-02, -7.6530e-03, -8.9866e-04, 6.2070e-03, 2.8353e-02, 3.5572e-02, -1.4487e-02, 2.4054e-02, -2.0622e-02, 1.2845e-02, -3.3719e-02, 3.3352e-02, 2.6608e-02, -2.2926e-02, 3.3618e-02, 2.3416e-02, 5.7970e-03, -5.4303e-03, -1.4419e-02, 2.6956e-03, 2.1294e-02, 6.2779e-03, 2.7432e-02, 1.0559e-02, -2.3688e-02, -2.2409e-02, 1.3598e-02, 7.6210e-03, -9.0783e-03, -2.3644e-02, 3.5181e-02, 2.4644e-02, 1.5393e-03, -3.3321e-02, -2.7582e-02, 1.0291e-02, 9.9860e-03, -1.4350e-02, -3.4415e-02, 5.6420e-03, 7.9971e-03, 1.7016e-03, 1.8598e-02, 7.3437e-03, 1.9815e-02, -1.7384e-02, 2.8861e-02, -1.3477e-02, 6.2217e-03, 1.2929e-03, -1.7984e-02, -2.5472e-02, 1.9770e-02, -2.8329e-03, 2.5418e-03, -1.7753e-02, -6.1870e-03, -2.9540e-02, -2.3303e-02, -7.7504e-03, 2.0856e-02, 2.2112e-02, -7.3416e-03, 2.2720e-03, 3.3436e-03, -5.0838e-03, -2.2531e-02, -1.8400e-03, -3.3608e-02, -1.0948e-02, 2.0605e-02, -2.8476e-02, -3.2736e-02, -2.0323e-02, 2.3324e-02, 3.2242e-02, 3.5159e-02, 3.5215e-02, 9.7593e-03, 2.3426e-02, 3.2070e-02, 1.8215e-02, -2.6789e-02, -3.3101e-02, -7.4461e-03, 2.4071e-02, 2.6295e-02, -2.3088e-02, 3.0321e-03, -2.9472e-02, -1.7763e-02, -1.9219e-02, -2.2132e-02, -2.7860e-02, 7.5992e-03, -7.0964e-03, 1.9382e-02, -1.4148e-02, 2.1809e-02, 1.6966e-02, 1.7949e-02, 2.7910e-02, 1.5970e-02, -3.0151e-02, -1.2758e-02, -3.3399e-02, -3.2098e-02, -1.0773e-02, 2.7361e-02, -2.1495e-02, -4.2179e-03, 3.4759e-02, -2.5232e-02, -1.7100e-02, -1.7625e-02, 1.3297e-02, -2.2669e-02, -2.0267e-02, 1.5720e-02, -1.4704e-02, -2.0415e-03, 4.1788e-03, -8.0999e-03, -1.2767e-02, 3.0385e-02, 1.6899e-02, -2.8761e-02, 2.7879e-02, 3.2565e-02, 2.6173e-02, 1.2173e-03, 2.6603e-02, -2.7333e-03, -6.6666e-03, -1.0879e-03, 9.8097e-03, 1.0042e-03, 3.4978e-02, 3.5589e-02, 2.4971e-05, -1.7491e-02, 1.0292e-02, -2.9348e-02, 2.2382e-02, 2.3650e-02, -3.2808e-02, -2.5504e-02, 3.2992e-02, -2.6580e-02, 1.6403e-02, 3.5502e-02, 5.6130e-03, -2.3188e-02, 1.2800e-02, -3.4104e-02, -3.2618e-02, -2.8509e-02, 2.8677e-02, -2.0576e-02, -1.5760e-02, 3.1213e-02, 4.4589e-03, -1.7961e-03, 2.0230e-02, -3.0797e-02, -1.6037e-02, 2.8899e-02, 2.5120e-02, -5.7101e-03, -3.1578e-02, 2.0123e-02, -2.4081e-02, 3.2989e-02, 6.7291e-03, -1.6732e-02, -9.8674e-03, -5.9818e-03, 2.6199e-02, -1.7941e-03, 3.0034e-02, -2.0785e-02, 1.9704e-02, -3.2263e-02, 2.8272e-02, 3.0563e-02, -1.6757e-02, -1.8474e-02, -2.2929e-02, 2.8403e-02, -2.2048e-02, 1.9925e-02, -9.5924e-03, 4.1058e-04, -2.0782e-02, 1.0660e-02, -2.8603e-02, -2.1806e-02, 1.9110e-02, -1.4392e-02, -1.4515e-02, -3.1525e-02, 3.3669e-02, -2.8914e-02, -1.9269e-02, -2.2796e-02, 9.9256e-03, 6.3308e-03, -1.9162e-02, -1.1942e-02, 2.5439e-02, -4.8769e-03, 1.3723e-02, -1.0561e-02, -2.9748e-02, -1.1906e-02, -3.3194e-02, -7.6452e-03, 1.3707e-02, 2.8308e-02, -1.9873e-02, 2.8642e-02, -3.0204e-02, -3.3586e-03, 1.7056e-02, -3.3790e-03, -1.8661e-02, -9.8581e-03, 3.7137e-04, -1.0028e-02, 3.1406e-02, -2.1060e-02, 8.6699e-03, -2.2983e-03, -1.4178e-02, 3.1254e-02, -2.4956e-03, 2.6758e-02, -2.9156e-03, 3.3374e-02, -3.1758e-02, 1.9481e-02, 2.0265e-02, 2.5305e-02, 6.3926e-03, 5.5478e-03, -3.2102e-02, 3.4670e-02, 2.1389e-02, -2.5373e-02, 2.2137e-02, 1.6891e-02, 1.0222e-02, 1.4526e-02, 3.1266e-04, 7.7932e-03, -4.1716e-03, -3.5018e-02, -9.9369e-03, -1.5133e-02, 1.7765e-02, 3.5236e-02, -2.1198e-02, -7.6979e-03, -2.1444e-02, 1.1148e-02, -2.0982e-02, -2.3188e-02, -4.9198e-03, -6.5885e-03, -1.5400e-02, 2.1331e-02, -3.4980e-02, 4.3496e-03, 2.2685e-02, 1.2298e-02, -1.7279e-02, 2.5880e-02, 2.5065e-02, 3.5510e-02, -1.4370e-02, -5.4901e-03, 2.8861e-02, -2.6165e-02, -3.0833e-02, -3.1507e-02, -3.5380e-02, -5.5276e-03, -3.5327e-02, 1.2894e-03, -1.7455e-02, 1.3338e-03, 1.3200e-02, 2.6399e-02, 2.9759e-02, -2.4413e-02, -7.0529e-03, -2.1699e-02, -1.0140e-02, 1.5638e-02, -3.0176e-02, 2.9083e-02, -1.9109e-02, -2.4341e-02, -9.2860e-03, -1.0878e-03, -2.0820e-02, 3.0168e-02, 1.9014e-02, -2.0647e-02, 1.0465e-03, -3.1491e-02, -1.7900e-02, 2.3971e-02, 3.2904e-02, -6.2390e-03, 6.2233e-03, -1.5210e-03, 2.4757e-02, 6.3047e-03, 2.5119e-02, -1.3470e-03, 2.9565e-02, 9.7634e-03, 6.7590e-03, 1.9044e-02, -1.2122e-02, -3.4269e-02, -1.2673e-02, -3.1275e-02, -5.3968e-04, -1.0034e-02, 2.8800e-02, 2.2795e-02, 3.3328e-02, -6.7631e-03, -1.6673e-02, 2.3123e-02, -3.2199e-02, 9.4170e-03, -2.2273e-02, -1.7387e-02, -3.5340e-02, 2.1455e-03, 2.5793e-02, 8.8271e-03, 1.2975e-02, -5.9093e-03, 1.8990e-02, 2.2633e-02, 1.6573e-02, -1.2716e-03, 2.3819e-02, 1.1007e-03, -4.3171e-03, 3.3581e-02, 2.0196e-02, 3.9881e-03, 2.7017e-02, -2.5353e-02, -6.2545e-03, -1.6453e-02, -4.0607e-03, 2.1773e-02, -1.0500e-02, 1.4073e-02, 1.0861e-02, -2.9600e-02, 6.2740e-03, -2.8184e-02, -5.4537e-03, 4.4250e-04, 2.6535e-02, -2.5258e-02, -2.0346e-02, -2.3957e-02, -3.3210e-02, -1.1227e-02, 2.6316e-02, 3.2828e-02, -2.1154e-02, 1.7546e-02, -2.6868e-02, 6.8745e-03, -3.2300e-02, -3.4238e-02, 2.1192e-03, 3.4135e-02, 2.4180e-02, 4.3352e-04, -2.4903e-02, -2.8769e-02, -5.0413e-03, -3.5233e-02, -3.4034e-02, -3.1925e-03, 1.6053e-02, 1.8069e-03, -3.8671e-03, 2.7840e-02, -9.8077e-03, -9.8580e-03, 2.9139e-02, 5.6035e-03, 2.7007e-02, -2.1433e-02, -3.3953e-02, 2.8704e-02, -8.2298e-03, 1.9656e-02, 3.9317e-03, 2.5742e-02, 1.3947e-02, 2.3737e-03, -1.3314e-02, -1.0077e-03, -3.5845e-03, 1.9549e-02, -1.5887e-02, 7.5714e-03, 1.0778e-02, 2.9843e-02, -2.8599e-02, -3.4904e-02, -2.6831e-02, 2.6895e-02, -9.6588e-03, 2.5203e-02, 1.7947e-04, -2.7425e-03, -2.7128e-02, 1.3563e-02, -3.1757e-02, -3.8123e-03, 2.2252e-02, 1.1445e-02, -1.1469e-02, 1.4986e-02, -1.4992e-03, -1.9054e-03, -1.6516e-02, 1.5346e-02, 3.0842e-02, -3.2813e-02, -2.4596e-02, 4.0143e-03, -1.4619e-02, -1.7058e-02, 2.5064e-02, 1.0040e-02, 2.3835e-02, -3.4938e-02, 2.9771e-02, 2.5211e-02, -1.8383e-02, 1.6897e-02, 1.1886e-02, -7.5648e-03, -2.3954e-02, 1.4190e-02, -2.5697e-02, 3.1411e-02, -3.1370e-02, 1.9279e-02, -1.7164e-02, 3.4065e-02, 1.5399e-02, -1.7762e-02, 1.0139e-02, -1.8695e-02, 4.9170e-03, -3.1555e-02, -1.7381e-03], device='privateuseone:0', requires_grad=True) ## Model layers 模型层 Let's break down the layers in the FashionMNIST model. To illustrate it, we take a sample minibatch of three images of size **28x28** and see what happens to it as we pass it through the network. 让我们详细分析 FashionMNIST 模型中的各个层。为了说明这一点，我们取一个包含三张 **28x28** 尺寸图像的样本小批量，并观察它在通过网络时会发生什么。 ```python input_image = torch.rand(3,28,28) print(input_image.size()) ``` torch.Size([3, 28, 28]) ### nn.Flatten We initialize the `nn.Flatten` layer to convert each two-dimensional 28x28 image into a contiguous array of 784 pixel values; that is, the minibatch dimension (at dim=0) is maintained. Each of the pixels are passed to the neural network's input layer. 我们初始化 `nn.Flatten` 层，以将每张二维的 28x28 图像转换为一个连续的 784 像素值数组；即，保持小批量维度（在 dim=0 处）。每个像素都被传递到神经网络的输入层。 ```python flatten = nn.Flatten() flat_image = flatten(input_image) print(flat_image.size()) ``` torch.Size([3, 784]) ### nn.Linear The linear layer is a module that applies a linear transformation on the input using its stored weights and biases. The grayscale value of each pixel in the input layer is connected to neurons in the hidden layer for calculation. The calculation the transformation uses is ${{weight * input + bias}}$. 线性层是一个模块，它使用存储的权重和偏置对输入应用线性变换。输入层中每个像素的灰度值与隐藏层中的神经元连接以进行计算。变换使用的计算公式为 ${{weight * input + bias}}$。 ```python layer1 = nn.Linear(in_features=28*28, out_features=20) hidden1 = layer1(flat_image) print(hidden1.size()) ``` torch.Size([3, 20]) ### nn.ReLU Nonlinear activations are what create the complex mappings between the model's inputs and outputs. They're applied after linear transformations to introduce *nonlinearity*, helping neural networks learn a wide variety of phenomena. In this model, we use `nn.ReLU` between our linear layers, but there's other activations to introduce nonlinearity in your model. 非线性激活函数在模型的输入和输出之间创建复杂的映射。它们在进行线性变换后应用，以引入 *非线性*，帮助神经网络学习多种现象。在这个模型中，我们在线性层之间使用 `nn.ReLU`，但还有其他激活函数可以在模型中引入非线性。 The ReLU activation function takes the output from the linear layer calculation and replaces the negative values with zeros. ReLU 激活函数将线性层计算的输出中的负值替换为零。 Linear output: 线性输出：${ x = {weight * input + bias}} $. ReLU: $f(x)= \begin{cases} 0, & \text{if } x < 0\\ x, & \text{if } x\geq 0\\ \end{cases} $ ```python print(f"Before ReLU: {hidden1}\n\n") hidden1 = nn.ReLU()(hidden1) print(f"After ReLU: {hidden1}") ``` Before ReLU: tensor([[-0.2975, 0.4203, 0.0824, 0.3017, -0.0590, 0.1329, 0.2476, -0.3945, -0.4045, 0.1827, -0.3776, 0.8676, 0.1698, -0.0057, 0.3674, -0.2054, 0.0282, -0.3115, 0.1861, -0.2039], [-0.0822, 0.4195, 0.5260, 1.0648, -0.2423, 0.1298, 0.0811, -0.0503, -0.0605, 0.2147, 0.0151, 0.6730, -0.2151, 0.1955, 0.0893, -0.3837, 0.0376, -0.5318, -0.0472, 0.0991], [-0.2026, 0.4443, 0.1924, 1.0782, -0.5161, 0.1429, 0.0521, -0.1417, -0.0918, -0.2046, -0.0145, 0.6576, 0.1293, 0.1070, 0.2631, -0.1609, -0.0072, -0.4469, -0.1536, 0.1167]], grad_fn=) After ReLU: tensor([[0.0000, 0.4203, 0.0824, 0.3017, 0.0000, 0.1329, 0.2476, 0.0000, 0.0000, 0.1827, 0.0000, 0.8676, 0.1698, 0.0000, 0.3674, 0.0000, 0.0282, 0.0000, 0.1861, 0.0000], [0.0000, 0.4195, 0.5260, 1.0648, 0.0000, 0.1298, 0.0811, 0.0000, 0.0000, 0.2147, 0.0151, 0.6730, 0.0000, 0.1955, 0.0893, 0.0000, 0.0376, 0.0000, 0.0000, 0.0991], [0.0000, 0.4443, 0.1924, 1.0782, 0.0000, 0.1429, 0.0521, 0.0000, 0.0000, 0.0000, 0.0000, 0.6576, 0.1293, 0.1070, 0.2631, 0.0000, 0.0000, 0.0000, 0.0000, 0.1167]], grad_fn=) ### nn.Sequential `nn.Sequential` is an ordered container of modules. The data is passed through all the modules in their defined order. You can use sequential containers to put together a quick network like `seq_modules`. `nn.Sequential` 是一个有序的模块容器。数据按定义的顺序通过所有模块进行传递。您可以使用顺序容器快速组装网络，例如 `seq_modules`。 ```python seq_modules = nn.Sequential( flatten, layer1, nn.ReLU(), nn.Linear(20, 10) ) input_image = torch.rand(3,28,28) logits = seq_modules(input_image) ``` ### nn.Softmax The last linear layer of the neural network returns `logits` (the raw values in \[`-infty`, `infty`]), which are passed to the `nn.Softmax` module. The Softmax activation function calculates the probability of the output from the neural network. It's only used on a neural network's output layer. The results are scaled to values \[0, 1\], representing the model's predicted densities for each class. The `dim` parameter indicates the dimension along which the result values must sum to 1. The node with the highest probability predicts the desired output. 神经网络的最后一个线性层返回 `logits`（原始值在 \[`-infty`, `infty`] 范围内），这些值会传递给 `nn.Softmax` 模块。Softmax 激活函数计算神经网络输出的概率。它仅用于神经网络的输出层。结果被缩放到 \[0, 1\] 的值，表示模型对每个类别的预测概率。`dim` 参数指示结果值必须在某个维度上相加为 1。具有最高概率的节点预测所需的输出。 ```python softmax = nn.Softmax(dim=1) pred_probab = softmax(logits) ``` ## Model parameters 模型参数 Many layers inside a neural network are *parameterized*; that is, the layers have associated weights and biases that are optimized during training. Subclassing `nn.Module` automatically tracks all fields defined inside your model object, and makes all parameters accessible using your model's `parameters()` or `named_parameters()` methods. 神经网络中的许多层是 *参数化的*；也就是说，这些层具有与之关联的权重和偏置，这些权重和偏置在训练过程中被优化。继承 `nn.Module` 会自动跟踪模型对象中定义的所有字段，并使所有参数可以通过模型的 `parameters()` 或 `named_parameters()` 方法访问。 In this example, we iterate over each parameter and print its size and a preview of its values. 在这个示例中，我们遍历每个参数并打印其大小及其值的预览。 ```python print("Model structure: ", model, "\n\n") for name, param in model.named_parameters(): print(f"Layer: {name} | Size: {param.size()} | Values : {param[:2]} \n") ``` Model structure: NeuralNetwork( (flatten): Flatten(start_dim=1, end_dim=-1) (linear_relu_stack): Sequential( (0): Linear(in_features=784, out_features=512, bias=True) (1): ReLU() (2): Linear(in_features=512, out_features=512, bias=True) (3): ReLU() (4): Linear(in_features=512, out_features=10, bias=True) (5): ReLU() ) ) Layer: linear_relu_stack.0.weight | Size: torch.Size([512, 784]) | Values : tensor([[ 0.0016, 0.0144, -0.0148, ..., -0.0311, 0.0349, -0.0070], [ 0.0113, 0.0346, 0.0020, ..., 0.0019, -0.0127, -0.0244]], device='privateuseone:0', grad_fn=) Layer: linear_relu_stack.0.bias | Size: torch.Size([512]) | Values : tensor([ 0.0046, -0.0049], device='privateuseone:0', grad_fn=) Layer: linear_relu_stack.2.weight | Size: torch.Size([512, 512]) | Values : tensor([[ 0.0261, -0.0426, 0.0266, ..., -0.0137, 0.0381, -0.0074], [ 0.0182, 0.0181, 0.0260, ..., 0.0159, 0.0016, -0.0265]], device='privateuseone:0', grad_fn=) Layer: linear_relu_stack.2.bias | Size: torch.Size([512]) | Values : tensor([0.0118, 0.0111], device='privateuseone:0', grad_fn=) Layer: linear_relu_stack.4.weight | Size: torch.Size([10, 512]) | Values : tensor([[-0.0089, 0.0099, -0.0148, ..., -0.0348, -0.0260, -0.0272], [-0.0123, -0.0107, -0.0066, ..., -0.0278, 0.0342, 0.0255]], device='privateuseone:0', grad_fn=) Layer: linear_relu_stack.4.bias | Size: torch.Size([10]) | Values : tensor([-0.0050, -0.0234], device='privateuseone:0', grad_fn=)

Search