深度学习每周学习总结J6（ResNeXt-50 算法实战与解析 - 猴痘识别）

🍨 本文为🔗365天深度学习训练营中的学习记录博客
🍖 原作者：K同学啊 | 接辅导、项目定制

0. 总结

数据导入及处理部分：本次数据导入没有使用torchvision自带的数据集，需要将原始数据进行处理包括数据导入，查看数据分类情况，定义transforms，进行数据类型转换等操作。

划分数据集：划定训练集测试集后，再使用torch.utils.data中的DataLoader()分别加载上一步处理好的训练及测试数据，查看批处理维度.

模型构建部分：ResNeXt-50

设置超参数：在这之前需要定义损失函数，学习率（动态学习率），以及根据学习率定义优化器（例如SGD随机梯度下降），用来在训练中更新参数，最小化损失函数。

定义训练函数：函数的传入的参数有四个，分别是设置好的DataLoader(),定义好的模型，损失函数，优化器。函数内部初始化损失准确率为0，接着开始循环，使用DataLoader()获取一个批次的数据，对这个批次的数据带入模型得到预测值，然后使用损失函数计算得到损失值。接下来就是进行反向传播以及使用优化器优化参数，梯度清零放在反向传播之前或者是使用优化器优化之后都是可以的，一般是默认放在反向传播之前。

定义测试函数：函数传入的参数相比训练函数少了优化器，只需传入设置好的DataLoader(),定义好的模型，损失函数。此外除了处理批次数据时无需再设置梯度清零、返向传播以及优化器优化参数，其余部分均和训练函数保持一致。

训练过程：定义训练次数，有几次就使用整个数据集进行几次训练，初始化四个空list分别存储每次训练及测试的准确率及损失。使用model.train()开启训练模式，调用训练函数得到准确率及损失。使用model.eval()将模型设置为评估模式，调用测试函数得到准确率及损失。接着就是将得到的训练及测试的准确率及损失存储到相应list中并合并打印出来，得到每一次整体训练后的准确率及损失。

结果可视化

模型的保存，调取及使用。在PyTorch中，通常使用 torch.save(model.state_dict(), ‘model.pth’) 保存模型的参数，使用 model.load_state_dict(torch.load(‘model.pth’)) 加载参数。

需要改进优化的地方：确保模型和数据的一致性，都存到GPU或者CPU;注意numclasses不要直接用默认的1000，需要根据实际数据集改进；实例化模型也要注意numclasses这个参数；此外注意测试模型需要用（3,224,224）3表示通道数，这和tensorflow定义的顺序是不用的（224,224,3），做代码转换时需要注意。

关于调优（十分重要）：本次将测试集准确率提升到了94.87%（随机种子设置为42）
1：使用多卡不一定比单卡效果好，需要继续调优
2：本次微调参数主要调整了两点一是初始学习率从1e-4 增大为了3e-4;其次是原来图片预处理只加入了随机水平翻转，本次加入了小角度的随机翻转，随机缩放剪裁，光照变化等，发现有更好的效果。测试集准确率有了很大的提升。从训练后的准确率图像也可以看到，训练准确率和测试准确率很接近甚至能够超过。之前没有做这个改进之前，都是训练准确率远大于测试准确率。

关键代码示例：

import torchvision.transforms as transforms
 
# 定义猴痘识别的 transforms
train_transforms = transforms.Compose([
    transforms.Resize([224, 224]),            # 统一图片尺寸
    transforms.RandomHorizontalFlip(p=0.5),  # 随机水平翻转
    transforms.RandomRotation(degrees=15),   # 小角度随机旋转
    transforms.RandomResizedCrop(size=224, scale=(0.8, 1.2)),  # 随机缩放裁剪
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.1),  # 光照变化
    transforms.ToTensor(),                   # 转换为 Tensor 格式
    transforms.Normalize(                    # 标准化
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    )
])

ResNeXt基本介绍

ResNeXt是一个基于卷积神经网络（CNN）的深度学习模型，最早由Facebook AI Research（FAIR）团队在2017年提出。它是ResNet（残差网络）的一个变种，通过引入"cardinality"（基数）这一概念，进一步提升了模型的性能。

ResNeXt的关键创新

Cardinality（基数）：
- 传统的卷积神经网络通常通过增加层数或者每层的通道数（宽度）来提升模型的表现，但这可能导致计算和内存开销的大幅增加。ResNeXt通过引入"cardinality"（基数）的概念，指的是每个残差块中并行的路径数量。通过增加并行路径的数量，ResNeXt能够在不显著增加计算量的情况下提升网络的表达能力。
- 具体来说，ResNeXt在每个残差块中使用了多个分支，每个分支都是相同的网络结构。通过调整分支的数量（即基数）来提高网络的表达能力。
分组卷积（Group Convolution）：
- ResNeXt使用了分组卷积，这使得计算量更加高效。分组卷积通过将输入通道分为若干组进行卷积操作，减少了计算量和内存开销。
结构设计：
- 在ResNeXt中，残差块的结构是通过多路径结构来增强模型的表现。每个路径相当于一个独立的卷积操作，最终将它们的输出进行合并。这种方法与传统的单路径ResNet不同。

与传统神经网络的对比

传统CNN（例如AlexNet、VGG等）：
- 传统的CNN网络通过加深网络层数或增加每一层的神经元来增强网络的表达能力，但这种做法面临梯度消失、过拟合等问题。因此，随着层数的增加，传统CNN的训练变得越来越困难。
ResNet与ResNeXt的优势：
- ResNet通过残差连接解决了深度神经网络训练时的梯度消失问题，使得网络可以很深而不容易退化。ResNeXt继承了ResNet的优点，但通过引入“基数”来进一步提升性能。相比于简单地增加网络深度或宽度，ResNeXt能够更高效地利用网络容量。
- ResNeXt通过分支结构使得每个残差块更具表达能力，相较于传统网络和单路径的ResNet，ResNeXt在相同的计算量下通常能够得到更好的效果。

下图是ResNet（左）与ResNeXt（右）block的差异。在ResNet中，输入的具有256个通道的特征经过1×1卷积压缩4倍到64个通道，之后3×3的卷积核用于处理特征，经1×1卷积扩大通道数与原特征残差连接后输出。ResNeXt也是相同的处理策略，但在ResNeXt中，输入的具有256个通道的特征被分为32个组，每组被压缩64倍到4个通道后进行处理。32个组相加后与原特征残差连接后输出。这里cardinatity指的是一个block中所具有的相同分支的数目。

import torch
import torch.nn as nn
import torchvision
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
import torchvision.models as models
import torch.nn.functional as F
from collections import OrderedDict 


import os,PIL,pathlib
import matplotlib.pyplot as plt
import warnings

warnings.filterwarnings('ignore') # 忽略警告信息

plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False   # 用来正常显示负号
plt.rcParams['figure.dpi'] = 100 # 分辨率

1. 设置GPU

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device

device(type='cuda')

2. 导入数据及处理部分

# 获取数据分布情况
path_dir = './data/mpox_recognize/'
path_dir = pathlib.Path(path_dir)

paths = list(path_dir.glob('*'))
# classNames = [str(path).split("\\")[-1] for path in paths] # ['Bananaquit', 'Black Skimmer', 'Black Throated Bushtiti', 'Cockatoo']
classNames = [path.parts[-1] for path in paths]
classNames

['Monkeypox', 'Others']

# 定义transforms 并处理数据
# train_transforms = transforms.Compose([
#     transforms.Resize([224,224]),      # 将输入图片resize成统一尺寸
#     transforms.RandomHorizontalFlip(), # 随机水平翻转
#     transforms.ToTensor(),             # 将PIL Image 或 numpy.ndarray 装换为tensor,并归一化到[0,1]之间
#     transforms.Normalize(              # 标准化处理 --> 转换为标准正太分布（高斯分布），使模型更容易收敛
#         mean = [0.485,0.456,0.406],    # 其中 mean=[0.485,0.456,0.406]与std=[0.229,0.224,0.225] 从数据集中随机抽样计算得到的。
#         std = [0.229,0.224,0.225]
#     )
# ])

# 定义猴痘识别的 transforms 并处理数据
train_transforms = transforms.Compose([
    transforms.Resize([224, 224]),            # 统一图片尺寸
    transforms.RandomHorizontalFlip(p=0.5),  # 随机水平翻转
    transforms.RandomRotation(degrees=15),   # 小角度随机旋转
    transforms.RandomResizedCrop(size=224, scale=(0.8, 1.2)),  # 随机缩放裁剪
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.1),  # 光照变化
    transforms.ToTensor(),                   # 转换为 Tensor 格式
    transforms.Normalize(                    # 标准化
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    )
])

test_transforms = transforms.Compose([
    transforms.Resize([224,224]),
    transforms.ToTensor(),
    transforms.Normalize(
        mean = [0.485,0.456,0.406],
        std = [0.229,0.224,0.225]
    )
])
total_data = datasets.ImageFolder('./data/mpox_recognize/',transform = train_transforms)
total_data

Dataset ImageFolder
    Number of datapoints: 2142
    Root location: ./data/mpox_recognize/
    StandardTransform
Transform: Compose(
               Resize(size=[224, 224], interpolation=bilinear, max_size=None, antialias=True)
               RandomHorizontalFlip(p=0.5)
               RandomRotation(degrees=[-15.0, 15.0], interpolation=nearest, expand=False, fill=0)
               RandomResizedCrop(size=(224, 224), scale=(0.8, 1.2), ratio=(0.75, 1.3333), interpolation=bilinear, antialias=True)
               ColorJitter(brightness=(0.8, 1.2), contrast=(0.8, 1.2), saturation=(0.9, 1.1), hue=None)
               ToTensor()
               Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
           )

3. 划分数据集

# 设置随机种子
torch.manual_seed(42)

# 划分数据集
train_size = int(len(total_data) * 0.8)
test_size = len(total_data) - train_size

train_dataset,test_dataset = torch.utils.data.random_split(total_data,[train_size,test_size])
train_dataset,test_dataset

(<torch.utils.data.dataset.Subset at 0x7c9ba5755670>,
 <torch.utils.data.dataset.Subset at 0x7c9ba5755790>)

# 定义DataLoader用于数据集的加载

batch_size = 32 # 如使用多显卡，请确保 batch_size 是显卡数量的倍数。

train_dl = torch.utils.data.DataLoader(
    train_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)
test_dl = torch.utils.data.DataLoader(
    test_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)

# 观察数据维度
for X,y in test_dl:
    print("Shape of X [N,C,H,W]: ",X.shape)
    print("Shape of y: ", y.shape,y.dtype)
    break

Shape of X [N,C,H,W]:  torch.Size([32, 3, 224, 224])
Shape of y:  torch.Size([32]) torch.int64

4. 模型构建部分

import torch
import torch.nn as nn
import torch.nn.functional as F


# 定义分组卷积模块
class GroupedConvBlock(nn.Module):
    def __init__(self, in_channels, groups, g_channels, stride):
        super(GroupedConvBlock, self).__init__()
        self.groups = groups
        self.group_convs = nn.ModuleList([
            nn.Conv2d(g_channels, g_channels, kernel_size=3, stride=stride, padding=1, bias=False)
            for _ in range(groups)
        ])
        self.bn = nn.BatchNorm2d(in_channels)
        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        # 分组数据
        split_x = torch.split(x, x.size(1) // self.groups, dim=1)
        group_out = [conv(g) for g, conv in zip(split_x, self.group_convs)]
        # 合并数据
        x = torch.cat(group_out, dim=1)
        x = self.bn(x)
        x = self.relu(x)
        return x


# 定义残差模块
class ResNeXtBlock(nn.Module):
    def __init__(self, in_channels, filters, groups=32, stride=1, conv_shortcut=True):
        super(ResNeXtBlock, self).__init__()
        self.conv_shortcut = conv_shortcut
        self.groups = groups
        self.g_channels = filters // groups

        # Shortcut分支
        if conv_shortcut:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_channels, filters * 2, kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(filters * 2),
            )
        else:
            self.shortcut = nn.Identity()

        # 主分支
        self.conv1 = nn.Sequential(
            nn.Conv2d(in_channels, filters, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(filters),
            nn.ReLU(inplace=True)
        )
        self.grouped_conv = GroupedConvBlock(filters, groups, self.g_channels, stride)
        self.conv3 = nn.Sequential(
            nn.Conv2d(filters, filters * 2, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(filters * 2),
        )
        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        shortcut = self.shortcut(x)
        x = self.conv1(x)
        x = self.grouped_conv(x)
        x = self.conv3(x)
        x += shortcut
        x = self.relu(x)
        return x


# 定义 ResNeXt-50 模型
class ResNeXt50(nn.Module):
    def __init__(self, num_classes=1000):
        super(ResNeXt50, self).__init__()
        self.stem = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
        )

        # 堆叠ResNeXt模块
        self.layer1 = self._make_layer(64, 128, 3, stride=1)
        self.layer2 = self._make_layer(256, 256, 4, stride=2)
        self.layer3 = self._make_layer(512, 512, 6, stride=2)
        self.layer4 = self._make_layer(1024, 1024, 3, stride=2)

        # 全局平均池化和分类层
        self.global_avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Linear(2048, num_classes)

    def _make_layer(self, in_channels, filters, blocks, stride):
        layers = [ResNeXtBlock(in_channels, filters, stride=stride)]
        for _ in range(1, blocks):
            layers.append(ResNeXtBlock(filters * 2, filters, stride=1))
        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.stem(x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
        x = self.global_avg_pool(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = ResNeXt50(num_classes=len(classNames)).to(device)
model

ResNeXt50(
  (stem): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  )
  (layer1): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer2): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (3): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer3): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (3): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (4): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (5): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer4): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(2048, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(2048, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (global_avg_pool): AdaptiveAvgPool2d(output_size=1)
  (fc): Linear(in_features=2048, out_features=2, bias=True)
)

# 查看模型详情
import torchsummary as summary
summary.summary(model,(3,224,224))

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 64, 112, 112]           9,408
       BatchNorm2d-2         [-1, 64, 112, 112]             128
              ReLU-3         [-1, 64, 112, 112]               0
         MaxPool2d-4           [-1, 64, 56, 56]               0
            Conv2d-5          [-1, 256, 56, 56]          16,384
       BatchNorm2d-6          [-1, 256, 56, 56]             512
            Conv2d-7          [-1, 128, 56, 56]           8,192
       BatchNorm2d-8          [-1, 128, 56, 56]             256
              ReLU-9          [-1, 128, 56, 56]               0
           Conv2d-10            [-1, 4, 56, 56]             144
           Conv2d-11            [-1, 4, 56, 56]             144
           Conv2d-12            [-1, 4, 56, 56]             144
           Conv2d-13            [-1, 4, 56, 56]             144
           Conv2d-14            [-1, 4, 56, 56]             144
           Conv2d-15            [-1, 4, 56, 56]             144
           Conv2d-16            [-1, 4, 56, 56]             144
           Conv2d-17            [-1, 4, 56, 56]             144
           Conv2d-18            [-1, 4, 56, 56]             144
           Conv2d-19            [-1, 4, 56, 56]             144
           Conv2d-20            [-1, 4, 56, 56]             144
           Conv2d-21            [-1, 4, 56, 56]             144
           Conv2d-22            [-1, 4, 56, 56]             144
           Conv2d-23            [-1, 4, 56, 56]             144
           Conv2d-24            [-1, 4, 56, 56]             144
           Conv2d-25            [-1, 4, 56, 56]             144
           Conv2d-26            [-1, 4, 56, 56]             144
           Conv2d-27            [-1, 4, 56, 56]             144
           Conv2d-28            [-1, 4, 56, 56]             144
           Conv2d-29            [-1, 4, 56, 56]             144
           Conv2d-30            [-1, 4, 56, 56]             144
           Conv2d-31            [-1, 4, 56, 56]             144
           Conv2d-32            [-1, 4, 56, 56]             144
           Conv2d-33            [-1, 4, 56, 56]             144
           Conv2d-34            [-1, 4, 56, 56]             144
           Conv2d-35            [-1, 4, 56, 56]             144
           Conv2d-36            [-1, 4, 56, 56]             144
           Conv2d-37            [-1, 4, 56, 56]             144
           Conv2d-38            [-1, 4, 56, 56]             144
           Conv2d-39            [-1, 4, 56, 56]             144
           Conv2d-40            [-1, 4, 56, 56]             144
           Conv2d-41            [-1, 4, 56, 56]             144
      BatchNorm2d-42          [-1, 128, 56, 56]             256
             ReLU-43          [-1, 128, 56, 56]               0
 GroupedConvBlock-44          [-1, 128, 56, 56]               0
           Conv2d-45          [-1, 256, 56, 56]          32,768
      BatchNorm2d-46          [-1, 256, 56, 56]             512
             ReLU-47          [-1, 256, 56, 56]               0
     ResNeXtBlock-48          [-1, 256, 56, 56]               0
           Conv2d-49          [-1, 256, 56, 56]          65,536
      BatchNorm2d-50          [-1, 256, 56, 56]             512
           Conv2d-51          [-1, 128, 56, 56]          32,768
      BatchNorm2d-52          [-1, 128, 56, 56]             256
             ReLU-53          [-1, 128, 56, 56]               0
           Conv2d-54            [-1, 4, 56, 56]             144
           Conv2d-55            [-1, 4, 56, 56]             144
           Conv2d-56            [-1, 4, 56, 56]             144
           Conv2d-57            [-1, 4, 56, 56]             144
           Conv2d-58            [-1, 4, 56, 56]             144
           Conv2d-59            [-1, 4, 56, 56]             144
           Conv2d-60            [-1, 4, 56, 56]             144
           Conv2d-61            [-1, 4, 56, 56]             144
           Conv2d-62            [-1, 4, 56, 56]             144
           Conv2d-63            [-1, 4, 56, 56]             144
           Conv2d-64            [-1, 4, 56, 56]             144
           Conv2d-65            [-1, 4, 56, 56]             144
           Conv2d-66            [-1, 4, 56, 56]             144
           Conv2d-67            [-1, 4, 56, 56]             144
           Conv2d-68            [-1, 4, 56, 56]             144
           Conv2d-69            [-1, 4, 56, 56]             144
           Conv2d-70            [-1, 4, 56, 56]             144
           Conv2d-71            [-1, 4, 56, 56]             144
           Conv2d-72            [-1, 4, 56, 56]             144
           Conv2d-73            [-1, 4, 56, 56]             144
           Conv2d-74            [-1, 4, 56, 56]             144
           Conv2d-75            [-1, 4, 56, 56]             144
           Conv2d-76            [-1, 4, 56, 56]             144
           Conv2d-77            [-1, 4, 56, 56]             144
           Conv2d-78            [-1, 4, 56, 56]             144
           Conv2d-79            [-1, 4, 56, 56]             144
           Conv2d-80            [-1, 4, 56, 56]             144
           Conv2d-81            [-1, 4, 56, 56]             144
           Conv2d-82            [-1, 4, 56, 56]             144
           Conv2d-83            [-1, 4, 56, 56]             144
           Conv2d-84            [-1, 4, 56, 56]             144
           Conv2d-85            [-1, 4, 56, 56]             144
      BatchNorm2d-86          [-1, 128, 56, 56]             256
             ReLU-87          [-1, 128, 56, 56]               0
 GroupedConvBlock-88          [-1, 128, 56, 56]               0
           Conv2d-89          [-1, 256, 56, 56]          32,768
      BatchNorm2d-90          [-1, 256, 56, 56]             512
             ReLU-91          [-1, 256, 56, 56]               0
     ResNeXtBlock-92          [-1, 256, 56, 56]               0
           Conv2d-93          [-1, 256, 56, 56]          65,536
      BatchNorm2d-94          [-1, 256, 56, 56]             512
           Conv2d-95          [-1, 128, 56, 56]          32,768
      BatchNorm2d-96          [-1, 128, 56, 56]             256
             ReLU-97          [-1, 128, 56, 56]               0
           Conv2d-98            [-1, 4, 56, 56]             144
           Conv2d-99            [-1, 4, 56, 56]             144
          Conv2d-100            [-1, 4, 56, 56]             144
          Conv2d-101            [-1, 4, 56, 56]             144
          Conv2d-102            [-1, 4, 56, 56]             144
          Conv2d-103            [-1, 4, 56, 56]             144
          Conv2d-104            [-1, 4, 56, 56]             144
          Conv2d-105            [-1, 4, 56, 56]             144
          Conv2d-106            [-1, 4, 56, 56]             144
          Conv2d-107            [-1, 4, 56, 56]             144
          Conv2d-108            [-1, 4, 56, 56]             144
          Conv2d-109            [-1, 4, 56, 56]             144
          Conv2d-110            [-1, 4, 56, 56]             144
          Conv2d-111            [-1, 4, 56, 56]             144
          Conv2d-112            [-1, 4, 56, 56]             144
          Conv2d-113            [-1, 4, 56, 56]             144
          Conv2d-114            [-1, 4, 56, 56]             144
          Conv2d-115            [-1, 4, 56, 56]             144
          Conv2d-116            [-1, 4, 56, 56]             144
          Conv2d-117            [-1, 4, 56, 56]             144
          Conv2d-118            [-1, 4, 56, 56]             144
          Conv2d-119            [-1, 4, 56, 56]             144
          Conv2d-120            [-1, 4, 56, 56]             144
          Conv2d-121            [-1, 4, 56, 56]             144
          Conv2d-122            [-1, 4, 56, 56]             144
          Conv2d-123            [-1, 4, 56, 56]             144
          Conv2d-124            [-1, 4, 56, 56]             144
          Conv2d-125            [-1, 4, 56, 56]             144
          Conv2d-126            [-1, 4, 56, 56]             144
          Conv2d-127            [-1, 4, 56, 56]             144
          Conv2d-128            [-1, 4, 56, 56]             144
          Conv2d-129            [-1, 4, 56, 56]             144
     BatchNorm2d-130          [-1, 128, 56, 56]             256
            ReLU-131          [-1, 128, 56, 56]               0
GroupedConvBlock-132          [-1, 128, 56, 56]               0
          Conv2d-133          [-1, 256, 56, 56]          32,768
     BatchNorm2d-134          [-1, 256, 56, 56]             512
            ReLU-135          [-1, 256, 56, 56]               0
    ResNeXtBlock-136          [-1, 256, 56, 56]               0
          Conv2d-137          [-1, 512, 28, 28]         131,072
     BatchNorm2d-138          [-1, 512, 28, 28]           1,024
          Conv2d-139          [-1, 256, 56, 56]          65,536
     BatchNorm2d-140          [-1, 256, 56, 56]             512
            ReLU-141          [-1, 256, 56, 56]               0
          Conv2d-142            [-1, 8, 28, 28]             576
          Conv2d-143            [-1, 8, 28, 28]             576
          Conv2d-144            [-1, 8, 28, 28]             576
          Conv2d-145            [-1, 8, 28, 28]             576
          Conv2d-146            [-1, 8, 28, 28]             576
          Conv2d-147            [-1, 8, 28, 28]             576
          Conv2d-148            [-1, 8, 28, 28]             576
          Conv2d-149            [-1, 8, 28, 28]             576
          Conv2d-150            [-1, 8, 28, 28]             576
          Conv2d-151            [-1, 8, 28, 28]             576
          Conv2d-152            [-1, 8, 28, 28]             576
          Conv2d-153            [-1, 8, 28, 28]             576
          Conv2d-154            [-1, 8, 28, 28]             576
          Conv2d-155            [-1, 8, 28, 28]             576
          Conv2d-156            [-1, 8, 28, 28]             576
          Conv2d-157            [-1, 8, 28, 28]             576
          Conv2d-158            [-1, 8, 28, 28]             576
          Conv2d-159            [-1, 8, 28, 28]             576
          Conv2d-160            [-1, 8, 28, 28]             576
          Conv2d-161            [-1, 8, 28, 28]             576
          Conv2d-162            [-1, 8, 28, 28]             576
          Conv2d-163            [-1, 8, 28, 28]             576
          Conv2d-164            [-1, 8, 28, 28]             576
          Conv2d-165            [-1, 8, 28, 28]             576
          Conv2d-166            [-1, 8, 28, 28]             576
          Conv2d-167            [-1, 8, 28, 28]             576
          Conv2d-168            [-1, 8, 28, 28]             576
          Conv2d-169            [-1, 8, 28, 28]             576
          Conv2d-170            [-1, 8, 28, 28]             576
          Conv2d-171            [-1, 8, 28, 28]             576
          Conv2d-172            [-1, 8, 28, 28]             576
          Conv2d-173            [-1, 8, 28, 28]             576
     BatchNorm2d-174          [-1, 256, 28, 28]             512
            ReLU-175          [-1, 256, 28, 28]               0
GroupedConvBlock-176          [-1, 256, 28, 28]               0
          Conv2d-177          [-1, 512, 28, 28]         131,072
     BatchNorm2d-178          [-1, 512, 28, 28]           1,024
            ReLU-179          [-1, 512, 28, 28]               0
    ResNeXtBlock-180          [-1, 512, 28, 28]               0
          Conv2d-181          [-1, 512, 28, 28]         262,144
     BatchNorm2d-182          [-1, 512, 28, 28]           1,024
          Conv2d-183          [-1, 256, 28, 28]         131,072
     BatchNorm2d-184          [-1, 256, 28, 28]             512
            ReLU-185          [-1, 256, 28, 28]               0
          Conv2d-186            [-1, 8, 28, 28]             576
          Conv2d-187            [-1, 8, 28, 28]             576
          Conv2d-188            [-1, 8, 28, 28]             576
          Conv2d-189            [-1, 8, 28, 28]             576
          Conv2d-190            [-1, 8, 28, 28]             576
          Conv2d-191            [-1, 8, 28, 28]             576
          Conv2d-192            [-1, 8, 28, 28]             576
          Conv2d-193            [-1, 8, 28, 28]             576
          Conv2d-194            [-1, 8, 28, 28]             576
          Conv2d-195            [-1, 8, 28, 28]             576
          Conv2d-196            [-1, 8, 28, 28]             576
          Conv2d-197            [-1, 8, 28, 28]             576
          Conv2d-198            [-1, 8, 28, 28]             576
          Conv2d-199            [-1, 8, 28, 28]             576
          Conv2d-200            [-1, 8, 28, 28]             576
          Conv2d-201            [-1, 8, 28, 28]             576
          Conv2d-202            [-1, 8, 28, 28]             576
          Conv2d-203            [-1, 8, 28, 28]             576
          Conv2d-204            [-1, 8, 28, 28]             576
          Conv2d-205            [-1, 8, 28, 28]             576
          Conv2d-206            [-1, 8, 28, 28]             576
          Conv2d-207            [-1, 8, 28, 28]             576
          Conv2d-208            [-1, 8, 28, 28]             576
          Conv2d-209            [-1, 8, 28, 28]             576
          Conv2d-210            [-1, 8, 28, 28]             576
          Conv2d-211            [-1, 8, 28, 28]             576
          Conv2d-212            [-1, 8, 28, 28]             576
          Conv2d-213            [-1, 8, 28, 28]             576
          Conv2d-214            [-1, 8, 28, 28]             576
          Conv2d-215            [-1, 8, 28, 28]             576
          Conv2d-216            [-1, 8, 28, 28]             576
          Conv2d-217            [-1, 8, 28, 28]             576
     BatchNorm2d-218          [-1, 256, 28, 28]             512
            ReLU-219          [-1, 256, 28, 28]               0
GroupedConvBlock-220          [-1, 256, 28, 28]               0
          Conv2d-221          [-1, 512, 28, 28]         131,072
     BatchNorm2d-222          [-1, 512, 28, 28]           1,024
            ReLU-223          [-1, 512, 28, 28]               0
    ResNeXtBlock-224          [-1, 512, 28, 28]               0
          Conv2d-225          [-1, 512, 28, 28]         262,144
     BatchNorm2d-226          [-1, 512, 28, 28]           1,024
          Conv2d-227          [-1, 256, 28, 28]         131,072
     BatchNorm2d-228          [-1, 256, 28, 28]             512
            ReLU-229          [-1, 256, 28, 28]               0
          Conv2d-230            [-1, 8, 28, 28]             576
          Conv2d-231            [-1, 8, 28, 28]             576
          Conv2d-232            [-1, 8, 28, 28]             576
          Conv2d-233            [-1, 8, 28, 28]             576
          Conv2d-234            [-1, 8, 28, 28]             576
          Conv2d-235            [-1, 8, 28, 28]             576
          Conv2d-236            [-1, 8, 28, 28]             576
          Conv2d-237            [-1, 8, 28, 28]             576
          Conv2d-238            [-1, 8, 28, 28]             576
          Conv2d-239            [-1, 8, 28, 28]             576
          Conv2d-240            [-1, 8, 28, 28]             576
          Conv2d-241            [-1, 8, 28, 28]             576
          Conv2d-242            [-1, 8, 28, 28]             576
          Conv2d-243            [-1, 8, 28, 28]             576
          Conv2d-244            [-1, 8, 28, 28]             576
          Conv2d-245            [-1, 8, 28, 28]             576
          Conv2d-246            [-1, 8, 28, 28]             576
          Conv2d-247            [-1, 8, 28, 28]             576
          Conv2d-248            [-1, 8, 28, 28]             576
          Conv2d-249            [-1, 8, 28, 28]             576
          Conv2d-250            [-1, 8, 28, 28]             576
          Conv2d-251            [-1, 8, 28, 28]             576
          Conv2d-252            [-1, 8, 28, 28]             576
          Conv2d-253            [-1, 8, 28, 28]             576
          Conv2d-254            [-1, 8, 28, 28]             576
          Conv2d-255            [-1, 8, 28, 28]             576
          Conv2d-256            [-1, 8, 28, 28]             576
          Conv2d-257            [-1, 8, 28, 28]             576
          Conv2d-258            [-1, 8, 28, 28]             576
          Conv2d-259            [-1, 8, 28, 28]             576
          Conv2d-260            [-1, 8, 28, 28]             576
          Conv2d-261            [-1, 8, 28, 28]             576
     BatchNorm2d-262          [-1, 256, 28, 28]             512
            ReLU-263          [-1, 256, 28, 28]               0
GroupedConvBlock-264          [-1, 256, 28, 28]               0
          Conv2d-265          [-1, 512, 28, 28]         131,072
     BatchNorm2d-266          [-1, 512, 28, 28]           1,024
            ReLU-267          [-1, 512, 28, 28]               0
    ResNeXtBlock-268          [-1, 512, 28, 28]               0
          Conv2d-269          [-1, 512, 28, 28]         262,144
     BatchNorm2d-270          [-1, 512, 28, 28]           1,024
          Conv2d-271          [-1, 256, 28, 28]         131,072
     BatchNorm2d-272          [-1, 256, 28, 28]             512
            ReLU-273          [-1, 256, 28, 28]               0
          Conv2d-274            [-1, 8, 28, 28]             576
          Conv2d-275            [-1, 8, 28, 28]             576
          Conv2d-276            [-1, 8, 28, 28]             576
          Conv2d-277            [-1, 8, 28, 28]             576
          Conv2d-278            [-1, 8, 28, 28]             576
          Conv2d-279            [-1, 8, 28, 28]             576
          Conv2d-280            [-1, 8, 28, 28]             576
          Conv2d-281            [-1, 8, 28, 28]             576
          Conv2d-282            [-1, 8, 28, 28]             576
          Conv2d-283            [-1, 8, 28, 28]             576
          Conv2d-284            [-1, 8, 28, 28]             576
          Conv2d-285            [-1, 8, 28, 28]             576
          Conv2d-286            [-1, 8, 28, 28]             576
          Conv2d-287            [-1, 8, 28, 28]             576
          Conv2d-288            [-1, 8, 28, 28]             576
          Conv2d-289            [-1, 8, 28, 28]             576
          Conv2d-290            [-1, 8, 28, 28]             576
          Conv2d-291            [-1, 8, 28, 28]             576
          Conv2d-292            [-1, 8, 28, 28]             576
          Conv2d-293            [-1, 8, 28, 28]             576
          Conv2d-294            [-1, 8, 28, 28]             576
          Conv2d-295            [-1, 8, 28, 28]             576
          Conv2d-296            [-1, 8, 28, 28]             576
          Conv2d-297            [-1, 8, 28, 28]             576
          Conv2d-298            [-1, 8, 28, 28]             576
          Conv2d-299            [-1, 8, 28, 28]             576
          Conv2d-300            [-1, 8, 28, 28]             576
          Conv2d-301            [-1, 8, 28, 28]             576
          Conv2d-302            [-1, 8, 28, 28]             576
          Conv2d-303            [-1, 8, 28, 28]             576
          Conv2d-304            [-1, 8, 28, 28]             576
          Conv2d-305            [-1, 8, 28, 28]             576
     BatchNorm2d-306          [-1, 256, 28, 28]             512
            ReLU-307          [-1, 256, 28, 28]               0
GroupedConvBlock-308          [-1, 256, 28, 28]               0
          Conv2d-309          [-1, 512, 28, 28]         131,072
     BatchNorm2d-310          [-1, 512, 28, 28]           1,024
            ReLU-311          [-1, 512, 28, 28]               0
    ResNeXtBlock-312          [-1, 512, 28, 28]               0
          Conv2d-313         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-314         [-1, 1024, 14, 14]           2,048
          Conv2d-315          [-1, 512, 28, 28]         262,144
     BatchNorm2d-316          [-1, 512, 28, 28]           1,024
            ReLU-317          [-1, 512, 28, 28]               0
          Conv2d-318           [-1, 16, 14, 14]           2,304
          Conv2d-319           [-1, 16, 14, 14]           2,304
          Conv2d-320           [-1, 16, 14, 14]           2,304
          Conv2d-321           [-1, 16, 14, 14]           2,304
          Conv2d-322           [-1, 16, 14, 14]           2,304
          Conv2d-323           [-1, 16, 14, 14]           2,304
          Conv2d-324           [-1, 16, 14, 14]           2,304
          Conv2d-325           [-1, 16, 14, 14]           2,304
          Conv2d-326           [-1, 16, 14, 14]           2,304
          Conv2d-327           [-1, 16, 14, 14]           2,304
          Conv2d-328           [-1, 16, 14, 14]           2,304
          Conv2d-329           [-1, 16, 14, 14]           2,304
          Conv2d-330           [-1, 16, 14, 14]           2,304
          Conv2d-331           [-1, 16, 14, 14]           2,304
          Conv2d-332           [-1, 16, 14, 14]           2,304
          Conv2d-333           [-1, 16, 14, 14]           2,304
          Conv2d-334           [-1, 16, 14, 14]           2,304
          Conv2d-335           [-1, 16, 14, 14]           2,304
          Conv2d-336           [-1, 16, 14, 14]           2,304
          Conv2d-337           [-1, 16, 14, 14]           2,304
          Conv2d-338           [-1, 16, 14, 14]           2,304
          Conv2d-339           [-1, 16, 14, 14]           2,304
          Conv2d-340           [-1, 16, 14, 14]           2,304
          Conv2d-341           [-1, 16, 14, 14]           2,304
          Conv2d-342           [-1, 16, 14, 14]           2,304
          Conv2d-343           [-1, 16, 14, 14]           2,304
          Conv2d-344           [-1, 16, 14, 14]           2,304
          Conv2d-345           [-1, 16, 14, 14]           2,304
          Conv2d-346           [-1, 16, 14, 14]           2,304
          Conv2d-347           [-1, 16, 14, 14]           2,304
          Conv2d-348           [-1, 16, 14, 14]           2,304
          Conv2d-349           [-1, 16, 14, 14]           2,304
     BatchNorm2d-350          [-1, 512, 14, 14]           1,024
            ReLU-351          [-1, 512, 14, 14]               0
GroupedConvBlock-352          [-1, 512, 14, 14]               0
          Conv2d-353         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-354         [-1, 1024, 14, 14]           2,048
            ReLU-355         [-1, 1024, 14, 14]               0
    ResNeXtBlock-356         [-1, 1024, 14, 14]               0
          Conv2d-357         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-358         [-1, 1024, 14, 14]           2,048
          Conv2d-359          [-1, 512, 14, 14]         524,288
     BatchNorm2d-360          [-1, 512, 14, 14]           1,024
            ReLU-361          [-1, 512, 14, 14]               0
          Conv2d-362           [-1, 16, 14, 14]           2,304
          Conv2d-363           [-1, 16, 14, 14]           2,304
          Conv2d-364           [-1, 16, 14, 14]           2,304
          Conv2d-365           [-1, 16, 14, 14]           2,304
          Conv2d-366           [-1, 16, 14, 14]           2,304
          Conv2d-367           [-1, 16, 14, 14]           2,304
          Conv2d-368           [-1, 16, 14, 14]           2,304
          Conv2d-369           [-1, 16, 14, 14]           2,304
          Conv2d-370           [-1, 16, 14, 14]           2,304
          Conv2d-371           [-1, 16, 14, 14]           2,304
          Conv2d-372           [-1, 16, 14, 14]           2,304
          Conv2d-373           [-1, 16, 14, 14]           2,304
          Conv2d-374           [-1, 16, 14, 14]           2,304
          Conv2d-375           [-1, 16, 14, 14]           2,304
          Conv2d-376           [-1, 16, 14, 14]           2,304
          Conv2d-377           [-1, 16, 14, 14]           2,304
          Conv2d-378           [-1, 16, 14, 14]           2,304
          Conv2d-379           [-1, 16, 14, 14]           2,304
          Conv2d-380           [-1, 16, 14, 14]           2,304
          Conv2d-381           [-1, 16, 14, 14]           2,304
          Conv2d-382           [-1, 16, 14, 14]           2,304
          Conv2d-383           [-1, 16, 14, 14]           2,304
          Conv2d-384           [-1, 16, 14, 14]           2,304
          Conv2d-385           [-1, 16, 14, 14]           2,304
          Conv2d-386           [-1, 16, 14, 14]           2,304
          Conv2d-387           [-1, 16, 14, 14]           2,304
          Conv2d-388           [-1, 16, 14, 14]           2,304
          Conv2d-389           [-1, 16, 14, 14]           2,304
          Conv2d-390           [-1, 16, 14, 14]           2,304
          Conv2d-391           [-1, 16, 14, 14]           2,304
          Conv2d-392           [-1, 16, 14, 14]           2,304
          Conv2d-393           [-1, 16, 14, 14]           2,304
     BatchNorm2d-394          [-1, 512, 14, 14]           1,024
            ReLU-395          [-1, 512, 14, 14]               0
GroupedConvBlock-396          [-1, 512, 14, 14]               0
          Conv2d-397         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-398         [-1, 1024, 14, 14]           2,048
            ReLU-399         [-1, 1024, 14, 14]               0
    ResNeXtBlock-400         [-1, 1024, 14, 14]               0
          Conv2d-401         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-402         [-1, 1024, 14, 14]           2,048
          Conv2d-403          [-1, 512, 14, 14]         524,288
     BatchNorm2d-404          [-1, 512, 14, 14]           1,024
            ReLU-405          [-1, 512, 14, 14]               0
          Conv2d-406           [-1, 16, 14, 14]           2,304
          Conv2d-407           [-1, 16, 14, 14]           2,304
          Conv2d-408           [-1, 16, 14, 14]           2,304
          Conv2d-409           [-1, 16, 14, 14]           2,304
          Conv2d-410           [-1, 16, 14, 14]           2,304
          Conv2d-411           [-1, 16, 14, 14]           2,304
          Conv2d-412           [-1, 16, 14, 14]           2,304
          Conv2d-413           [-1, 16, 14, 14]           2,304
          Conv2d-414           [-1, 16, 14, 14]           2,304
          Conv2d-415           [-1, 16, 14, 14]           2,304
          Conv2d-416           [-1, 16, 14, 14]           2,304
          Conv2d-417           [-1, 16, 14, 14]           2,304
          Conv2d-418           [-1, 16, 14, 14]           2,304
          Conv2d-419           [-1, 16, 14, 14]           2,304
          Conv2d-420           [-1, 16, 14, 14]           2,304
          Conv2d-421           [-1, 16, 14, 14]           2,304
          Conv2d-422           [-1, 16, 14, 14]           2,304
          Conv2d-423           [-1, 16, 14, 14]           2,304
          Conv2d-424           [-1, 16, 14, 14]           2,304
          Conv2d-425           [-1, 16, 14, 14]           2,304
          Conv2d-426           [-1, 16, 14, 14]           2,304
          Conv2d-427           [-1, 16, 14, 14]           2,304
          Conv2d-428           [-1, 16, 14, 14]           2,304
          Conv2d-429           [-1, 16, 14, 14]           2,304
          Conv2d-430           [-1, 16, 14, 14]           2,304
          Conv2d-431           [-1, 16, 14, 14]           2,304
          Conv2d-432           [-1, 16, 14, 14]           2,304
          Conv2d-433           [-1, 16, 14, 14]           2,304
          Conv2d-434           [-1, 16, 14, 14]           2,304
          Conv2d-435           [-1, 16, 14, 14]           2,304
          Conv2d-436           [-1, 16, 14, 14]           2,304
          Conv2d-437           [-1, 16, 14, 14]           2,304
     BatchNorm2d-438          [-1, 512, 14, 14]           1,024
            ReLU-439          [-1, 512, 14, 14]               0
GroupedConvBlock-440          [-1, 512, 14, 14]               0
          Conv2d-441         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-442         [-1, 1024, 14, 14]           2,048
            ReLU-443         [-1, 1024, 14, 14]               0
    ResNeXtBlock-444         [-1, 1024, 14, 14]               0
          Conv2d-445         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-446         [-1, 1024, 14, 14]           2,048
          Conv2d-447          [-1, 512, 14, 14]         524,288
     BatchNorm2d-448          [-1, 512, 14, 14]           1,024
            ReLU-449          [-1, 512, 14, 14]               0
          Conv2d-450           [-1, 16, 14, 14]           2,304
          Conv2d-451           [-1, 16, 14, 14]           2,304
          Conv2d-452           [-1, 16, 14, 14]           2,304
          Conv2d-453           [-1, 16, 14, 14]           2,304
          Conv2d-454           [-1, 16, 14, 14]           2,304
          Conv2d-455           [-1, 16, 14, 14]           2,304
          Conv2d-456           [-1, 16, 14, 14]           2,304
          Conv2d-457           [-1, 16, 14, 14]           2,304
          Conv2d-458           [-1, 16, 14, 14]           2,304
          Conv2d-459           [-1, 16, 14, 14]           2,304
          Conv2d-460           [-1, 16, 14, 14]           2,304
          Conv2d-461           [-1, 16, 14, 14]           2,304
          Conv2d-462           [-1, 16, 14, 14]           2,304
          Conv2d-463           [-1, 16, 14, 14]           2,304
          Conv2d-464           [-1, 16, 14, 14]           2,304
          Conv2d-465           [-1, 16, 14, 14]           2,304
          Conv2d-466           [-1, 16, 14, 14]           2,304
          Conv2d-467           [-1, 16, 14, 14]           2,304
          Conv2d-468           [-1, 16, 14, 14]           2,304
          Conv2d-469           [-1, 16, 14, 14]           2,304
          Conv2d-470           [-1, 16, 14, 14]           2,304
          Conv2d-471           [-1, 16, 14, 14]           2,304
          Conv2d-472           [-1, 16, 14, 14]           2,304
          Conv2d-473           [-1, 16, 14, 14]           2,304
          Conv2d-474           [-1, 16, 14, 14]           2,304
          Conv2d-475           [-1, 16, 14, 14]           2,304
          Conv2d-476           [-1, 16, 14, 14]           2,304
          Conv2d-477           [-1, 16, 14, 14]           2,304
          Conv2d-478           [-1, 16, 14, 14]           2,304
          Conv2d-479           [-1, 16, 14, 14]           2,304
          Conv2d-480           [-1, 16, 14, 14]           2,304
          Conv2d-481           [-1, 16, 14, 14]           2,304
     BatchNorm2d-482          [-1, 512, 14, 14]           1,024
            ReLU-483          [-1, 512, 14, 14]               0
GroupedConvBlock-484          [-1, 512, 14, 14]               0
          Conv2d-485         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-486         [-1, 1024, 14, 14]           2,048
            ReLU-487         [-1, 1024, 14, 14]               0
    ResNeXtBlock-488         [-1, 1024, 14, 14]               0
          Conv2d-489         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-490         [-1, 1024, 14, 14]           2,048
          Conv2d-491          [-1, 512, 14, 14]         524,288
     BatchNorm2d-492          [-1, 512, 14, 14]           1,024
            ReLU-493          [-1, 512, 14, 14]               0
          Conv2d-494           [-1, 16, 14, 14]           2,304
          Conv2d-495           [-1, 16, 14, 14]           2,304
          Conv2d-496           [-1, 16, 14, 14]           2,304
          Conv2d-497           [-1, 16, 14, 14]           2,304
          Conv2d-498           [-1, 16, 14, 14]           2,304
          Conv2d-499           [-1, 16, 14, 14]           2,304
          Conv2d-500           [-1, 16, 14, 14]           2,304
          Conv2d-501           [-1, 16, 14, 14]           2,304
          Conv2d-502           [-1, 16, 14, 14]           2,304
          Conv2d-503           [-1, 16, 14, 14]           2,304
          Conv2d-504           [-1, 16, 14, 14]           2,304
          Conv2d-505           [-1, 16, 14, 14]           2,304
          Conv2d-506           [-1, 16, 14, 14]           2,304
          Conv2d-507           [-1, 16, 14, 14]           2,304
          Conv2d-508           [-1, 16, 14, 14]           2,304
          Conv2d-509           [-1, 16, 14, 14]           2,304
          Conv2d-510           [-1, 16, 14, 14]           2,304
          Conv2d-511           [-1, 16, 14, 14]           2,304
          Conv2d-512           [-1, 16, 14, 14]           2,304
          Conv2d-513           [-1, 16, 14, 14]           2,304
          Conv2d-514           [-1, 16, 14, 14]           2,304
          Conv2d-515           [-1, 16, 14, 14]           2,304
          Conv2d-516           [-1, 16, 14, 14]           2,304
          Conv2d-517           [-1, 16, 14, 14]           2,304
          Conv2d-518           [-1, 16, 14, 14]           2,304
          Conv2d-519           [-1, 16, 14, 14]           2,304
          Conv2d-520           [-1, 16, 14, 14]           2,304
          Conv2d-521           [-1, 16, 14, 14]           2,304
          Conv2d-522           [-1, 16, 14, 14]           2,304
          Conv2d-523           [-1, 16, 14, 14]           2,304
          Conv2d-524           [-1, 16, 14, 14]           2,304
          Conv2d-525           [-1, 16, 14, 14]           2,304
     BatchNorm2d-526          [-1, 512, 14, 14]           1,024
            ReLU-527          [-1, 512, 14, 14]               0
GroupedConvBlock-528          [-1, 512, 14, 14]               0
          Conv2d-529         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-530         [-1, 1024, 14, 14]           2,048
            ReLU-531         [-1, 1024, 14, 14]               0
    ResNeXtBlock-532         [-1, 1024, 14, 14]               0
          Conv2d-533         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-534         [-1, 1024, 14, 14]           2,048
          Conv2d-535          [-1, 512, 14, 14]         524,288
     BatchNorm2d-536          [-1, 512, 14, 14]           1,024
            ReLU-537          [-1, 512, 14, 14]               0
          Conv2d-538           [-1, 16, 14, 14]           2,304
          Conv2d-539           [-1, 16, 14, 14]           2,304
          Conv2d-540           [-1, 16, 14, 14]           2,304
          Conv2d-541           [-1, 16, 14, 14]           2,304
          Conv2d-542           [-1, 16, 14, 14]           2,304
          Conv2d-543           [-1, 16, 14, 14]           2,304
          Conv2d-544           [-1, 16, 14, 14]           2,304
          Conv2d-545           [-1, 16, 14, 14]           2,304
          Conv2d-546           [-1, 16, 14, 14]           2,304
          Conv2d-547           [-1, 16, 14, 14]           2,304
          Conv2d-548           [-1, 16, 14, 14]           2,304
          Conv2d-549           [-1, 16, 14, 14]           2,304
          Conv2d-550           [-1, 16, 14, 14]           2,304
          Conv2d-551           [-1, 16, 14, 14]           2,304
          Conv2d-552           [-1, 16, 14, 14]           2,304
          Conv2d-553           [-1, 16, 14, 14]           2,304
          Conv2d-554           [-1, 16, 14, 14]           2,304
          Conv2d-555           [-1, 16, 14, 14]           2,304
          Conv2d-556           [-1, 16, 14, 14]           2,304
          Conv2d-557           [-1, 16, 14, 14]           2,304
          Conv2d-558           [-1, 16, 14, 14]           2,304
          Conv2d-559           [-1, 16, 14, 14]           2,304
          Conv2d-560           [-1, 16, 14, 14]           2,304
          Conv2d-561           [-1, 16, 14, 14]           2,304
          Conv2d-562           [-1, 16, 14, 14]           2,304
          Conv2d-563           [-1, 16, 14, 14]           2,304
          Conv2d-564           [-1, 16, 14, 14]           2,304
          Conv2d-565           [-1, 16, 14, 14]           2,304
          Conv2d-566           [-1, 16, 14, 14]           2,304
          Conv2d-567           [-1, 16, 14, 14]           2,304
          Conv2d-568           [-1, 16, 14, 14]           2,304
          Conv2d-569           [-1, 16, 14, 14]           2,304
     BatchNorm2d-570          [-1, 512, 14, 14]           1,024
            ReLU-571          [-1, 512, 14, 14]               0
GroupedConvBlock-572          [-1, 512, 14, 14]               0
          Conv2d-573         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-574         [-1, 1024, 14, 14]           2,048
            ReLU-575         [-1, 1024, 14, 14]               0
    ResNeXtBlock-576         [-1, 1024, 14, 14]               0
          Conv2d-577           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-578           [-1, 2048, 7, 7]           4,096
          Conv2d-579         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-580         [-1, 1024, 14, 14]           2,048
            ReLU-581         [-1, 1024, 14, 14]               0
          Conv2d-582             [-1, 32, 7, 7]           9,216
          Conv2d-583             [-1, 32, 7, 7]           9,216
          Conv2d-584             [-1, 32, 7, 7]           9,216
          Conv2d-585             [-1, 32, 7, 7]           9,216
          Conv2d-586             [-1, 32, 7, 7]           9,216
          Conv2d-587             [-1, 32, 7, 7]           9,216
          Conv2d-588             [-1, 32, 7, 7]           9,216
          Conv2d-589             [-1, 32, 7, 7]           9,216
          Conv2d-590             [-1, 32, 7, 7]           9,216
          Conv2d-591             [-1, 32, 7, 7]           9,216
          Conv2d-592             [-1, 32, 7, 7]           9,216
          Conv2d-593             [-1, 32, 7, 7]           9,216
          Conv2d-594             [-1, 32, 7, 7]           9,216
          Conv2d-595             [-1, 32, 7, 7]           9,216
          Conv2d-596             [-1, 32, 7, 7]           9,216
          Conv2d-597             [-1, 32, 7, 7]           9,216
          Conv2d-598             [-1, 32, 7, 7]           9,216
          Conv2d-599             [-1, 32, 7, 7]           9,216
          Conv2d-600             [-1, 32, 7, 7]           9,216
          Conv2d-601             [-1, 32, 7, 7]           9,216
          Conv2d-602             [-1, 32, 7, 7]           9,216
          Conv2d-603             [-1, 32, 7, 7]           9,216
          Conv2d-604             [-1, 32, 7, 7]           9,216
          Conv2d-605             [-1, 32, 7, 7]           9,216
          Conv2d-606             [-1, 32, 7, 7]           9,216
          Conv2d-607             [-1, 32, 7, 7]           9,216
          Conv2d-608             [-1, 32, 7, 7]           9,216
          Conv2d-609             [-1, 32, 7, 7]           9,216
          Conv2d-610             [-1, 32, 7, 7]           9,216
          Conv2d-611             [-1, 32, 7, 7]           9,216
          Conv2d-612             [-1, 32, 7, 7]           9,216
          Conv2d-613             [-1, 32, 7, 7]           9,216
     BatchNorm2d-614           [-1, 1024, 7, 7]           2,048
            ReLU-615           [-1, 1024, 7, 7]               0
GroupedConvBlock-616           [-1, 1024, 7, 7]               0
          Conv2d-617           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-618           [-1, 2048, 7, 7]           4,096
            ReLU-619           [-1, 2048, 7, 7]               0
    ResNeXtBlock-620           [-1, 2048, 7, 7]               0
          Conv2d-621           [-1, 2048, 7, 7]       4,194,304
     BatchNorm2d-622           [-1, 2048, 7, 7]           4,096
          Conv2d-623           [-1, 1024, 7, 7]       2,097,152
     BatchNorm2d-624           [-1, 1024, 7, 7]           2,048
            ReLU-625           [-1, 1024, 7, 7]               0
          Conv2d-626             [-1, 32, 7, 7]           9,216
          Conv2d-627             [-1, 32, 7, 7]           9,216
          Conv2d-628             [-1, 32, 7, 7]           9,216
          Conv2d-629             [-1, 32, 7, 7]           9,216
          Conv2d-630             [-1, 32, 7, 7]           9,216
          Conv2d-631             [-1, 32, 7, 7]           9,216
          Conv2d-632             [-1, 32, 7, 7]           9,216
          Conv2d-633             [-1, 32, 7, 7]           9,216
          Conv2d-634             [-1, 32, 7, 7]           9,216
          Conv2d-635             [-1, 32, 7, 7]           9,216
          Conv2d-636             [-1, 32, 7, 7]           9,216
          Conv2d-637             [-1, 32, 7, 7]           9,216
          Conv2d-638             [-1, 32, 7, 7]           9,216
          Conv2d-639             [-1, 32, 7, 7]           9,216
          Conv2d-640             [-1, 32, 7, 7]           9,216
          Conv2d-641             [-1, 32, 7, 7]           9,216
          Conv2d-642             [-1, 32, 7, 7]           9,216
          Conv2d-643             [-1, 32, 7, 7]           9,216
          Conv2d-644             [-1, 32, 7, 7]           9,216
          Conv2d-645             [-1, 32, 7, 7]           9,216
          Conv2d-646             [-1, 32, 7, 7]           9,216
          Conv2d-647             [-1, 32, 7, 7]           9,216
          Conv2d-648             [-1, 32, 7, 7]           9,216
          Conv2d-649             [-1, 32, 7, 7]           9,216
          Conv2d-650             [-1, 32, 7, 7]           9,216
          Conv2d-651             [-1, 32, 7, 7]           9,216
          Conv2d-652             [-1, 32, 7, 7]           9,216
          Conv2d-653             [-1, 32, 7, 7]           9,216
          Conv2d-654             [-1, 32, 7, 7]           9,216
          Conv2d-655             [-1, 32, 7, 7]           9,216
          Conv2d-656             [-1, 32, 7, 7]           9,216
          Conv2d-657             [-1, 32, 7, 7]           9,216
     BatchNorm2d-658           [-1, 1024, 7, 7]           2,048
            ReLU-659           [-1, 1024, 7, 7]               0
GroupedConvBlock-660           [-1, 1024, 7, 7]               0
          Conv2d-661           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-662           [-1, 2048, 7, 7]           4,096
            ReLU-663           [-1, 2048, 7, 7]               0
    ResNeXtBlock-664           [-1, 2048, 7, 7]               0
          Conv2d-665           [-1, 2048, 7, 7]       4,194,304
     BatchNorm2d-666           [-1, 2048, 7, 7]           4,096
          Conv2d-667           [-1, 1024, 7, 7]       2,097,152
     BatchNorm2d-668           [-1, 1024, 7, 7]           2,048
            ReLU-669           [-1, 1024, 7, 7]               0
          Conv2d-670             [-1, 32, 7, 7]           9,216
          Conv2d-671             [-1, 32, 7, 7]           9,216
          Conv2d-672             [-1, 32, 7, 7]           9,216
          Conv2d-673             [-1, 32, 7, 7]           9,216
          Conv2d-674             [-1, 32, 7, 7]           9,216
          Conv2d-675             [-1, 32, 7, 7]           9,216
          Conv2d-676             [-1, 32, 7, 7]           9,216
          Conv2d-677             [-1, 32, 7, 7]           9,216
          Conv2d-678             [-1, 32, 7, 7]           9,216
          Conv2d-679             [-1, 32, 7, 7]           9,216
          Conv2d-680             [-1, 32, 7, 7]           9,216
          Conv2d-681             [-1, 32, 7, 7]           9,216
          Conv2d-682             [-1, 32, 7, 7]           9,216
          Conv2d-683             [-1, 32, 7, 7]           9,216
          Conv2d-684             [-1, 32, 7, 7]           9,216
          Conv2d-685             [-1, 32, 7, 7]           9,216
          Conv2d-686             [-1, 32, 7, 7]           9,216
          Conv2d-687             [-1, 32, 7, 7]           9,216
          Conv2d-688             [-1, 32, 7, 7]           9,216
          Conv2d-689             [-1, 32, 7, 7]           9,216
          Conv2d-690             [-1, 32, 7, 7]           9,216
          Conv2d-691             [-1, 32, 7, 7]           9,216
          Conv2d-692             [-1, 32, 7, 7]           9,216
          Conv2d-693             [-1, 32, 7, 7]           9,216
          Conv2d-694             [-1, 32, 7, 7]           9,216
          Conv2d-695             [-1, 32, 7, 7]           9,216
          Conv2d-696             [-1, 32, 7, 7]           9,216
          Conv2d-697             [-1, 32, 7, 7]           9,216
          Conv2d-698             [-1, 32, 7, 7]           9,216
          Conv2d-699             [-1, 32, 7, 7]           9,216
          Conv2d-700             [-1, 32, 7, 7]           9,216
          Conv2d-701             [-1, 32, 7, 7]           9,216
     BatchNorm2d-702           [-1, 1024, 7, 7]           2,048
            ReLU-703           [-1, 1024, 7, 7]               0
GroupedConvBlock-704           [-1, 1024, 7, 7]               0
          Conv2d-705           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-706           [-1, 2048, 7, 7]           4,096
            ReLU-707           [-1, 2048, 7, 7]               0
    ResNeXtBlock-708           [-1, 2048, 7, 7]               0
AdaptiveAvgPool2d-709           [-1, 2048, 1, 1]               0
          Linear-710                    [-1, 2]           4,098
================================================================
Total params: 37,555,522
Trainable params: 37,555,522
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 444.08
Params size (MB): 143.26
Estimated Total Size (MB): 587.92
----------------------------------------------------------------

5. 设置超参数：定义损失函数，学习率，以及根据学习率定义优化器等

# loss_fn = nn.CrossEntropyLoss() # 创建损失函数

# learn_rate = 1e-3 # 初始学习率
# def adjust_learning_rate(optimizer,epoch,start_lr):
#     # 每两个epoch 衰减到原来的0.98
#     lr = start_lr * (0.92 ** (epoch//2))
#     for param_group in optimizer.param_groups:
#         param_group['lr'] = lr
        
# optimizer = torch.optim.Adam(model.parameters(),lr=learn_rate)

# 调用官方接口示例
loss_fn = nn.CrossEntropyLoss()

# learn_rate = 1e-4  
learn_rate = 3e-4
lambda1 = lambda epoch:(0.92**(epoch//2))

optimizer = torch.optim.Adam(model.parameters(),lr = learn_rate)
scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer,lr_lambda=lambda1) # 选定调整方法

6. 训练函数

# 训练函数
def train(dataloader,model,loss_fn,optimizer):
    size = len(dataloader.dataset) # 训练集大小
    num_batches = len(dataloader) # 批次数目
    
    train_loss,train_acc = 0,0
    
    for X,y in dataloader:
        X,y = X.to(device),y.to(device)
        
        # 计算预测误差
        pred = model(X)
        loss = loss_fn(pred,y)
        
        # 反向传播
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        # 记录acc与loss
        train_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
        train_loss += loss.item()
        
    train_acc /= size
    train_loss /= num_batches
    
    return train_acc,train_loss

7. 测试函数

# 测试函数
def test(dataloader,model,loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    
    test_acc,test_loss = 0,0
    
    with torch.no_grad():
        for X,y in dataloader:
            X,y = X.to(device),y.to(device)
            
            # 计算loss
            pred = model(X)
            loss = loss_fn(pred,y)
            
            test_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
            test_loss += loss.item()
            
    test_acc /= size
    test_loss /= num_batches
    
    return test_acc,test_loss

8. 正式训练

import copy

epochs = 60

train_acc = []
train_loss = []
test_acc = []
test_loss = []

best_acc = 0.0

# 检查 GPU 可用性并打印设备信息
if torch.cuda.is_available():
    for i in range(torch.cuda.device_count()):
        print(f"GPU {i}: {torch.cuda.get_device_name(i)}")
        print(f"Initial Memory Allocated: {torch.cuda.memory_allocated(i)/1024**2:.2f} MB")
        print(f"Initial Memory Cached: {torch.cuda.memory_reserved(i)/1024**2:.2f} MB")
else:
    print("No GPU available. Using CPU.")

# 多显卡设置 当前使用的是使用 PyTorch 自带的 DataParallel，后续如有需要可以设置为DistributedDataParallel，这是更加高效的方式
# 且多卡不一定比单卡效果就好，需要调整优化
# if torch.cuda.device_count() > 1:
#     print(f"Using {torch.cuda.device_count()} GPUs")
#     model = nn.DataParallel(model)
# model = model.to('cuda')

for epoch in range(epochs):
    
    # 更新学习率——使用自定义学习率时使用
    # adjust_learning_rate(optimizer,epoch,learn_rate)
    
    model.train()
    epoch_train_acc,epoch_train_loss = train(train_dl,model,loss_fn,optimizer)
    scheduler.step() # 更新学习率——调用官方动态学习率时使用
    
    model.eval()
    epoch_test_acc,epoch_test_loss = test(test_dl,model,loss_fn)
    
    # 保存最佳模型到 best_model
    if epoch_test_acc > best_acc:
        best_acc = epoch_test_acc
        best_model = copy.deepcopy(model)
    
    train_acc.append(epoch_train_acc)
    train_loss.append(epoch_train_loss)
    test_acc.append(epoch_test_acc)
    test_loss.append(epoch_test_loss)
    
    # 获取当前学习率
    lr = optimizer.state_dict()['param_groups'][0]['lr']
    
    template = ('Epoch:{:2d},Train_acc:{:.1f}%,Train_loss:{:.3f},Test_acc:{:.1f}%,Test_loss:{:.3f},Lr:{:.2E}')
    print(template.format(epoch+1,epoch_train_acc*100,epoch_train_loss,epoch_test_acc*100,epoch_test_loss,lr))

    # 实时监控 GPU 状态
    if torch.cuda.is_available():
        for i in range(torch.cuda.device_count()):
            print(f"GPU {i} Usage:")
            print(f"  Memory Allocated: {torch.cuda.memory_allocated(i)/1024**2:.2f} MB")
            print(f"  Memory Cached: {torch.cuda.memory_reserved(i)/1024**2:.2f} MB")
            print(f"  Max Memory Allocated: {torch.cuda.max_memory_allocated(i)/1024**2:.2f} MB")
            print(f"  Max Memory Cached: {torch.cuda.max_memory_reserved(i)/1024**2:.2f} MB")

print('Done','best_acc: ',best_acc)

GPU 0: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 151.84 MB
Initial Memory Cached: 422.00 MB
GPU 1: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
GPU 2: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
GPU 3: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
Epoch: 1,Train_acc:56.1%,Train_loss:0.737,Test_acc:51.5%,Test_loss:0.845,Lr:3.00E-04
GPU 0 Usage:
  Memory Allocated: 737.77 MB
  Memory Cached: 5010.00 MB
  Max Memory Allocated: 4545.57 MB
  Max Memory Cached: 5010.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 2,Train_acc:61.8%,Train_loss:0.668,Test_acc:66.9%,Test_loss:0.655,Lr:2.76E-04
GPU 0 Usage:
  Memory Allocated: 739.44 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 3,Train_acc:63.4%,Train_loss:0.650,Test_acc:60.8%,Test_loss:0.658,Lr:2.76E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 4,Train_acc:65.7%,Train_loss:0.637,Test_acc:61.3%,Test_loss:0.668,Lr:2.54E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 5,Train_acc:66.8%,Train_loss:0.633,Test_acc:65.0%,Test_loss:0.618,Lr:2.54E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 6,Train_acc:66.6%,Train_loss:0.615,Test_acc:62.7%,Test_loss:0.629,Lr:2.34E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 7,Train_acc:66.7%,Train_loss:0.608,Test_acc:64.6%,Test_loss:0.611,Lr:2.34E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 8,Train_acc:68.7%,Train_loss:0.592,Test_acc:67.1%,Test_loss:0.598,Lr:2.15E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 9,Train_acc:69.0%,Train_loss:0.598,Test_acc:67.1%,Test_loss:0.579,Lr:2.15E-04
GPU 0 Usage:
  Memory Allocated: 738.74 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:10,Train_acc:68.7%,Train_loss:0.575,Test_acc:66.7%,Test_loss:0.561,Lr:1.98E-04
GPU 0 Usage:
  Memory Allocated: 738.24 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:11,Train_acc:70.5%,Train_loss:0.571,Test_acc:72.7%,Test_loss:0.559,Lr:1.98E-04
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:12,Train_acc:71.7%,Train_loss:0.565,Test_acc:68.8%,Test_loss:0.558,Lr:1.82E-04
GPU 0 Usage:
  Memory Allocated: 737.04 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:13,Train_acc:71.2%,Train_loss:0.570,Test_acc:73.4%,Test_loss:0.532,Lr:1.82E-04
GPU 0 Usage:
  Memory Allocated: 738.40 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:14,Train_acc:72.4%,Train_loss:0.553,Test_acc:70.2%,Test_loss:0.544,Lr:1.67E-04
GPU 0 Usage:
  Memory Allocated: 738.67 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:15,Train_acc:71.5%,Train_loss:0.554,Test_acc:70.2%,Test_loss:0.551,Lr:1.67E-04
GPU 0 Usage:
  Memory Allocated: 733.94 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:16,Train_acc:72.8%,Train_loss:0.551,Test_acc:69.5%,Test_loss:0.557,Lr:1.54E-04
GPU 0 Usage:
  Memory Allocated: 735.85 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:17,Train_acc:72.8%,Train_loss:0.528,Test_acc:70.6%,Test_loss:0.573,Lr:1.54E-04
GPU 0 Usage:
  Memory Allocated: 735.85 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:18,Train_acc:72.6%,Train_loss:0.546,Test_acc:70.6%,Test_loss:0.550,Lr:1.42E-04
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:19,Train_acc:74.0%,Train_loss:0.523,Test_acc:74.1%,Test_loss:0.544,Lr:1.42E-04
GPU 0 Usage:
  Memory Allocated: 737.51 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:20,Train_acc:75.7%,Train_loss:0.494,Test_acc:73.4%,Test_loss:0.532,Lr:1.30E-04
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:21,Train_acc:77.6%,Train_loss:0.469,Test_acc:74.4%,Test_loss:0.506,Lr:1.30E-04
GPU 0 Usage:
  Memory Allocated: 738.05 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:22,Train_acc:76.6%,Train_loss:0.488,Test_acc:76.7%,Test_loss:0.478,Lr:1.20E-04
GPU 0 Usage:
  Memory Allocated: 736.08 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:23,Train_acc:77.1%,Train_loss:0.485,Test_acc:72.3%,Test_loss:0.497,Lr:1.20E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:24,Train_acc:79.5%,Train_loss:0.454,Test_acc:76.0%,Test_loss:0.519,Lr:1.10E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:25,Train_acc:77.3%,Train_loss:0.476,Test_acc:76.9%,Test_loss:0.466,Lr:1.10E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:26,Train_acc:80.7%,Train_loss:0.436,Test_acc:79.7%,Test_loss:0.452,Lr:1.01E-04
GPU 0 Usage:
  Memory Allocated: 736.58 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:27,Train_acc:79.0%,Train_loss:0.429,Test_acc:78.8%,Test_loss:0.429,Lr:1.01E-04
GPU 0 Usage:
  Memory Allocated: 737.29 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:28,Train_acc:78.9%,Train_loss:0.433,Test_acc:79.3%,Test_loss:0.405,Lr:9.34E-05
GPU 0 Usage:
  Memory Allocated: 737.48 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:29,Train_acc:82.4%,Train_loss:0.389,Test_acc:85.5%,Test_loss:0.368,Lr:9.34E-05
GPU 0 Usage:
  Memory Allocated: 737.78 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:30,Train_acc:82.8%,Train_loss:0.388,Test_acc:81.6%,Test_loss:0.409,Lr:8.59E-05
GPU 0 Usage:
  Memory Allocated: 738.97 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.16 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:31,Train_acc:84.6%,Train_loss:0.361,Test_acc:83.2%,Test_loss:0.408,Lr:8.59E-05
GPU 0 Usage:
  Memory Allocated: 739.15 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.16 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:32,Train_acc:85.7%,Train_loss:0.336,Test_acc:84.8%,Test_loss:0.379,Lr:7.90E-05
GPU 0 Usage:
  Memory Allocated: 739.40 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.39 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:33,Train_acc:86.9%,Train_loss:0.306,Test_acc:86.9%,Test_loss:0.340,Lr:7.90E-05
GPU 0 Usage:
  Memory Allocated: 736.35 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:34,Train_acc:86.8%,Train_loss:0.311,Test_acc:88.1%,Test_loss:0.329,Lr:7.27E-05
GPU 0 Usage:
  Memory Allocated: 738.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:35,Train_acc:87.4%,Train_loss:0.312,Test_acc:82.3%,Test_loss:0.394,Lr:7.27E-05
GPU 0 Usage:
  Memory Allocated: 738.49 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:36,Train_acc:87.0%,Train_loss:0.313,Test_acc:87.2%,Test_loss:0.318,Lr:6.69E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:37,Train_acc:88.6%,Train_loss:0.280,Test_acc:88.6%,Test_loss:0.286,Lr:6.69E-05
GPU 0 Usage:
  Memory Allocated: 738.21 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:38,Train_acc:88.8%,Train_loss:0.270,Test_acc:86.9%,Test_loss:0.321,Lr:6.15E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:39,Train_acc:88.6%,Train_loss:0.283,Test_acc:83.9%,Test_loss:0.338,Lr:6.15E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:40,Train_acc:90.0%,Train_loss:0.249,Test_acc:89.0%,Test_loss:0.249,Lr:5.66E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:41,Train_acc:91.0%,Train_loss:0.226,Test_acc:91.8%,Test_loss:0.211,Lr:5.66E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:42,Train_acc:90.1%,Train_loss:0.242,Test_acc:91.1%,Test_loss:0.233,Lr:5.21E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:43,Train_acc:92.1%,Train_loss:0.196,Test_acc:89.5%,Test_loss:0.245,Lr:5.21E-05
GPU 0 Usage:
  Memory Allocated: 736.57 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:44,Train_acc:93.0%,Train_loss:0.198,Test_acc:90.0%,Test_loss:0.232,Lr:4.79E-05
GPU 0 Usage:
  Memory Allocated: 738.29 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:45,Train_acc:92.6%,Train_loss:0.195,Test_acc:92.3%,Test_loss:0.227,Lr:4.79E-05
GPU 0 Usage:
  Memory Allocated: 738.55 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:46,Train_acc:93.0%,Train_loss:0.184,Test_acc:91.4%,Test_loss:0.263,Lr:4.41E-05
GPU 0 Usage:
  Memory Allocated: 736.11 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:47,Train_acc:93.5%,Train_loss:0.164,Test_acc:93.0%,Test_loss:0.186,Lr:4.41E-05
GPU 0 Usage:
  Memory Allocated: 736.08 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:48,Train_acc:93.9%,Train_loss:0.163,Test_acc:91.8%,Test_loss:0.220,Lr:4.06E-05
GPU 0 Usage:
  Memory Allocated: 737.04 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:49,Train_acc:94.2%,Train_loss:0.163,Test_acc:93.2%,Test_loss:0.223,Lr:4.06E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:50,Train_acc:93.8%,Train_loss:0.161,Test_acc:92.5%,Test_loss:0.203,Lr:3.73E-05
GPU 0 Usage:
  Memory Allocated: 736.50 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:51,Train_acc:92.3%,Train_loss:0.174,Test_acc:92.8%,Test_loss:0.178,Lr:3.73E-05
GPU 0 Usage:
  Memory Allocated: 735.31 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:52,Train_acc:95.1%,Train_loss:0.134,Test_acc:92.3%,Test_loss:0.191,Lr:3.43E-05
GPU 0 Usage:
  Memory Allocated: 736.88 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:53,Train_acc:95.2%,Train_loss:0.148,Test_acc:93.7%,Test_loss:0.164,Lr:3.43E-05
GPU 0 Usage:
  Memory Allocated: 737.10 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:54,Train_acc:95.0%,Train_loss:0.119,Test_acc:93.0%,Test_loss:0.180,Lr:3.16E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:55,Train_acc:94.2%,Train_loss:0.143,Test_acc:91.8%,Test_loss:0.197,Lr:3.16E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:56,Train_acc:96.1%,Train_loss:0.111,Test_acc:93.0%,Test_loss:0.183,Lr:2.91E-05
GPU 0 Usage:
  Memory Allocated: 737.33 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:57,Train_acc:96.1%,Train_loss:0.102,Test_acc:94.9%,Test_loss:0.170,Lr:2.91E-05
GPU 0 Usage:
  Memory Allocated: 738.05 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:58,Train_acc:95.2%,Train_loss:0.120,Test_acc:93.5%,Test_loss:0.201,Lr:2.67E-05
GPU 0 Usage:
  Memory Allocated: 739.19 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:59,Train_acc:95.3%,Train_loss:0.136,Test_acc:94.9%,Test_loss:0.157,Lr:2.67E-05
GPU 0 Usage:
  Memory Allocated: 738.25 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:60,Train_acc:95.5%,Train_loss:0.106,Test_acc:93.5%,Test_loss:0.167,Lr:2.46E-05
GPU 0 Usage:
  Memory Allocated: 739.19 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Done best_acc:  0.9487179487179487

9. 结果可视化

epochs_range = range(epochs)

plt.figure(figsize = (12,3))

plt.subplot(1,2,1)
plt.plot(epochs_range,train_acc,label = 'Training Accuracy')
plt.plot(epochs_range,test_acc,label = 'Test Accuracy')
plt.legend(loc = 'lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1,2,2)
plt.plot(epochs_range,train_loss,label = 'Test Accuracy')
plt.plot(epochs_range,test_loss,label = 'Test Loss')
plt.legend(loc = 'lower right')
plt.title('Training and validation Loss')
plt.show()

findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei

在这里插入图片描述

10. 模型的保存

# 自定义模型保存
# 状态字典保存
torch.save(model.state_dict(),'./模型参数/J6_ResNeXt50_model_state_dict2.pth') # 仅保存状态字典

# 定义模型用来加载参数
best_model = ResNeXt50(num_classes=len(classNames)).to(device)

best_model.load_state_dict(torch.load('./模型参数/J6_ResNeXt50_model_state_dict2.pth')) # 加载状态字典到模型

<All keys matched successfully>

11.使用训练好的模型进行预测

# 指定路径图片预测
from PIL import Image
import torchvision.transforms as transforms

classes = list(total_data.class_to_idx) # classes = list(total_data.class_to_idx)

def predict_one_image(image_path,model,transform,classes):
    
    test_img = Image.open(image_path).convert('RGB')
    # plt.imshow(test_img) # 展示待预测的图片
    
    test_img = transform(test_img)
    img = test_img.to(device).unsqueeze(0)
    
    model.eval()
    output = model(img)
    print(output) # 观察模型预测结果的输出数据
    
    _,pred = torch.max(output,1)
    pred_class = classes[pred]
    print(f'预测结果是:{pred_class}')

# 预测训练集中的某张照片
predict_one_image(image_path='./data/mpox_recognize/Monkeypox/M01_01_04.jpg',
                 model = model,
                 transform = test_transforms,
                 classes = classes
                 )

tensor([[ 2.6236, -2.9544]], device='cuda:0', grad_fn=<AddmmBackward0>)
预测结果是:Monkeypox

classes

['Monkeypox', 'Others']