Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping (CVPR 2024)在3080上复现-笔记

conda create -n zgp_cross python=3.9
cd /home/zgp/Documents/crossmodal-feature-mapping-main

在这里插入图片描述

processing暂时使用旧的

在这里插入图片描述

checkpoint

1.Create a folder named checkpoints/checkpoints_CFM_mvtec in the project directory;
2.Download the weights https://t.ly/DZ-o1;

wandb

conda 切换

                     /home/hanmh/anaconda3/envs/zgp_cross

base /home/xiongzc/miniconda3
balance /home/xiongzc/miniconda3/envs/balance

zgp@hanmh1804:~$ source ~/.bashrc
(base) zgp@hanmh1804:~$

conda environments:

base * /home/hanmh/anaconda3
fed_hmh /home/hanmh/anaconda3/envs/fed_hmh
hgat /home/hanmh/anaconda3/envs/hgat
pytorch2.1 /home/hanmh/anaconda3/envs/pytorch2.1
selfsuper /home/hanmh/anaconda3/envs/selfsuper
semantic /home/hanmh/anaconda3/envs/semantic
structure-clip /home/hanmh/anaconda3/envs/structure-clip
u2pl /home/hanmh/anaconda3/envs/u2pl
veh311 /home/hanmh/anaconda3/envs/veh311
veh_sem /home/hanmh/anaconda3/envs/veh_sem
zgp_cross /home/hanmh/anaconda3/envs/zgp_cross
zgp_cvc /home/hanmh/anaconda3/envs/zgp_cvc

(base) zgp@hanmh1804:~$
(zgp_cross) zgp@hanmh1804:~$

pip install -r requirements.txt
注释掉重复的
注释掉torch单独安装
#torch1.13.1
#torchvision0.14.1

pip install torch1.10.0+cu113 torchvision0.11.1+cu113 torchaudio --extra-index-url https://download.pytorch.org/whl/cu113 -i https://pypi.tuna.tsinghua.edu.cn/simple

参考《远程服务器上，再次配笔记1、2、11、12相同的深度学习和gcc环境》
(zgp_cross) zgp@hanmh1804:~$ pip install torch1.10.0+cu113 torchvision0.11.1+cu113 torchaudio --extra-index-url https://download.pytorch.org/whl/cu113 -i https://pypi.tuna.tsinghua.edu.cn/simple
ERROR: Invalid requirement: ‘torch1.10.0+cu113’: Expected end or semicolon (after name and no valid version specifier)
torch1.10.0+cu113
^
修改为
(zgp_cross) zgp@hanmh1804:~$ pip install torch1.10.0+cu113 torchvision0.11.1+cu113 torchaudio --extra-index-url https://download.pytorch.org/whl/cu113 -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install -r requirements.txt

python3 -m pip install /home/zgp/Documents/crossmodal-feature-mapping-main/Pointnet2_PyTorch-master/Pointnet2_PyTorch-master/pointnet2_ops_lib

报错

(zgp_cross) zgp@hanmh1804:~/Documents/crossmodal-feature-mapping-main$ python cfm_inference.py
Traceback (most recent call last):
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/cfm_inference.py", line 226, in <module>
    infer_CFM(args)
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/cfm_inference.py", line 37, in infer_CFM
    feature_extractor = MultimodalFeatures()
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/features.py", line 20, in __init__
    self.deep_feature_extractor = FeatureExtractors(device = self.device, 
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/full_models.py", line 25, in __init__
    self.rgb_backbone = timm.create_model(model_name=rgb_backbone_name, pretrained=True, pretrained_cfg_overlay=dict( file='checkpoints/dino_vitbase8_pretrain.pth'),
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/_factory.py", line 117, in create_model
    model = create_fn(
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/vision_transformer.py", line 2066, in vit_base_patch8_224
    model = _create_vision_transformer('vit_base_patch8_224', pretrained=pretrained, **dict(model_args, **kwargs))
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/vision_transformer.py", line 1946, in _create_vision_transformer
    return build_model_with_cfg(
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/_builder.py", line 418, in build_model_with_cfg
    load_pretrained(
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/_builder.py", line 168, in load_pretrained
    state_dict = load_state_dict(pretrained_loc)
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/timm/models/_helpers.py", line 42, in load_state_dict
    checkpoint = torch.load(checkpoint_path, map_location=device)
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/torch/serialization.py", line 600, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/home/hanmh/anaconda3/envs/zgp_cross/lib/python3.9/site-packages/torch/serialization.py", line 242, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

pip install --upgrade timm
无效

观察下载发现服务器上的模型大小小于实际大小

在这里插入图片描述

在这里插入图片描述
重新上传，可以了
Extracting feature from class: bagel.: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 110/110 [00:11<00:00, 9.49it/s]
Metrics for class bagel with 50ep_4bs
AUPRO@30% | AUPRO@10% | AUPRO@5% | AUPRO@1% | P-AUROC | I-AUROC
0.978 | 0.934 | 0.872 | 0.453 | 0.997 | 0.990
Extracting feature from class: cable_gland.: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 108/108 [00:09<00:00, 10.97it/s]
Metrics for class cable_gland with 50ep_4bs
AUPRO@30% | AUPRO@10% | AUPRO@5% | AUPRO@1% | P-AUROC | I-AUROC
0.956 | 0.870 | 0.770 | 0.372 | 0.986 | 0.880

train

修改数据集路径和class_name parser.add_argument(‘–dataset_path’, default = ‘/home/zgp/Documents/m3dmpre/datasets/mvtec3d’, type = str,
help = ‘Dataset path.’)
报错
RuntimeError: CUDA out of memory. Tried to allocate 736.00 MiB (GPU 0; 9.78 GiB total capacity; 6.75 GiB already allocated; 360.19 MiB free; 7.29 GiB reserved in total by PyTorch
batch4改为2
parser.add_argument(‘–batch_size’, default = 2, type = int,
help = ‘Batch dimension. Usually 16 is around the max.’)
测试batch2
Extracting feature from class: bagel.: 100%|██| 110/110 [00:11<00:00, 9.46it/s]
Metrics for class bagel with 50ep_2bs
AUPRO@30% | AUPRO@10% | AUPRO@5% | AUPRO@1% | P-AUROC | I-AUROC
0.977 | 0.931 | 0.868 | 0.446 | 0.997 | 0.995

再次尝试

基于veh_sem虚拟环境安装
pip install “git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib”

  WARNING: Generating metadata for package pointnet2_ops produced metadata for project name pointnet2. Fix your #egg=pointnet2_ops fragments.
Discarding git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops: Requested pointnet2 from git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops has inconsistent name: filename has 'pointnet2-ops', but metadata has 'pointnet2'
ERROR: Could not find a version that satisfies the requirement pointnet2-ops (unavailable) (from versions: none)
ERROR: No matching distribution found for pointnet2-ops (unavailable)

python3 -m pip install /home/zgp/Documents/crossmodal-feature-mapping-main/Pointnet2_PyTorch-master/Pointnet2_PyTorch-master/pointnet2_ops_lib

修改数据集路径

python cfm_inference.py

报错

  File "cfm_inference.py", line 37, in infer_CFM
    feature_extractor = MultimodalFeatures()
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/features.py", line 22, in __init__
    group_size = group_size, num_group = num_group)
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/full_models.py", line 24, in __init__
    self.rgb_backbone = timm.create_model(model_name = rgb_backbone_name, pretrained = True, **kwargs)
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/factory.py", line 67, in create_model
    raise RuntimeError('Unknown model (%s)' % model_name)
RuntimeError: Unknown model (vit_base_patch8_224.dino)

pip install --upgrade timm
可以了，下一个报错

huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.

上传修改了checkpoint地址的文件覆盖之前的文件
可以了，下一个报错

  File "cfm_inference.py", line 37, in infer_CFM
    feature_extractor = MultimodalFeatures()
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/features.py", line 22, in __init__
    group_size = group_size, num_group = num_group)
  File "/home/zgp/Documents/crossmodal-feature-mapping-main/models/full_models.py", line 26, in __init__
    **kwargs)
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/_factory.py", line 121, in create_model
    **kwargs,
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/vision_transformer.py", line 1883, in vit_base_patch8_224
    model = _create_vision_transformer('vit_base_patch8_224', pretrained=pretrained, **dict(model_args, **kwargs))
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/vision_transformer.py", line 1770, in _create_vision_transformer
    **kwargs,
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/_builder.py", line 403, in build_model_with_cfg
    strict=pretrained_strict,
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/_builder.py", line 167, in load_pretrained
    state_dict = load_state_dict(pretrained_loc)
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/timm/models/_helpers.py", line 42, in load_state_dict
    checkpoint = torch.load(checkpoint_path, map_location=device)
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/torch/serialization.py", line 600, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/home/hanmh/anaconda3/envs/veh_sem/lib/python3.7/site-packages/torch/serialization.py", line 242, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

模型不完整