Skip to content

Commit

Permalink
[Improvement] Update searchable model (#438)
Browse files Browse the repository at this point in the history 
* bugfix search save_subnet

* update link

* clean

---------

Co-authored-by: aptsunny <[email protected]>
  • Loading branch information
@aptsunny
aptsunny and aptsunny authored Feb 17, 2023
1 parent f4d4478 commit 18754f3
Show file tree
Hide file tree
Showing 5 changed files with 16 additions and 23 deletions.
10 changes: 5 additions & 5 deletions configs/nas/mmcls/bignas/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ CUDA_VISIBLE_DEVICES=0 PORT=29500 ./tools/dist_test.sh \

## Results and models

| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 | Config | Download | Remarks |
| :------: | :------------------: | :--------------------------------------------------------------------------------------------: | :--------------------: | :------------------: | :---------------------: | :----------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------: |
| ImageNet | AttentiveMobileNetV3 | [search space](../../../../configs/_base_/nas_backbones/attentive_mobilenetv3_supernet.py) | 11.56(min) / 23.3(max) | 414(min) / 1944(max) | 76.88(min) / 81.42(max) | [config](../../../../configs/nas/mmcls/bignas/attentive_mobilenet_supernet_32xb64_in1k.py) | [model\*](https://download.openmmlab.com/mmrazor/v0.1/nas/bignas/attentive_mobilenet_subnet_8xb256_in1k/attentive_mobilenet_supernet_32xb64_in1k_flops-2G_acc-81.72_20221229_200440-954772a3.pth) | [log](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_supernet_32xb64_in1k_20221227_175800-bcf94eaa.json) (`sandwich rule`) |
| ImageNet | AttentiveMobileNetV3 | [AttentiveNAS-A0\*](https://download.openmmlab.com/mmrazor/v1/bignas/ATTENTIVE_SUBNET_A0.yaml) | 11.559 | 414 | 77.19 | [config](../../../../configs/nas/mmcls/bignas/attentive_mobilenet_subnet_8xb256_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_subnet_8xb256_in1k_flops-0.21G_acc-77.19_20221229_200440-282a1f70.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | [AttentiveNAS-A6\*](https://download.openmmlab.com/mmrazor/v1/bignas/ATTENTIVE_SUBNET_A6.yaml) | 16.476 | 1163 | 80.81 | [config](../../../../configs/nas/mmcls/bignas/attentive_mobilenet_subnet_8xb256_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_subnet_8xb256_in1k_flops-0.93G_acc-80.81_20221229_200440-73d92cc6.pth) | Converted from the repo |
| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 | Config | Download | Remarks |
| :------: | :------------------: | :-------------------------------------------------------------------------------------------------------------------------------: | :--------------------: | :------------------: | :---------------------: | :-------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------: |
| ImageNet | AttentiveMobileNetV3 | [search space](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/_base_/nas_backbones/attentive_mobilenetv3_supernet.py) | 11.56(min) / 23.3(max) | 414(min) / 1944(max) | 76.88(min) / 81.42(max) | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/bignas/attentive_mobilenet_supernet_32xb64_in1k.py) | [model\*](https://download.openmmlab.com/mmrazor/v0.1/nas/bignas/attentive_mobilenet_subnet_8xb256_in1k/attentive_mobilenet_supernet_32xb64_in1k_flops-2G_acc-81.72_20221229_200440-954772a3.pth) | [log](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_supernet_32xb64_in1k_20221227_175800-bcf94eaa.json) (`sandwich rule`) |
| ImageNet | AttentiveMobileNetV3 | [AttentiveNAS-A0\*](https://download.openmmlab.com/mmrazor/v1/bignas/ATTENTIVE_SUBNET_A0.yaml) | 11.559 | 414 | 77.19 | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/bignas/attentive_mobilenet_subnet_8xb256_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_subnet_8xb256_in1k_flops-0.21G_acc-77.19_20221229_200440-282a1f70.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | [AttentiveNAS-A6\*](https://download.openmmlab.com/mmrazor/v1/bignas/ATTENTIVE_SUBNET_A6.yaml) | 16.476 | 1163 | 80.81 | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/bignas/attentive_mobilenet_subnet_8xb256_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/bignas/attentive_mobilenet_subnet_8xb256_in1k_flops-0.93G_acc-80.81_20221229_200440-73d92cc6.pth) | Converted from the repo |

*Models with * are converted from the [official repo](https://github.com/facebookresearch/AttentiveNAS). The config files of these models
are only for inference. We support training the supernet by `sandwich rule`, which is different from `rejection sampling` in [official repo](https://github.com/facebookresearch/AttentiveNAS), and welcome you to contribute your reproduction results.*
Expand Down
10 changes: 5 additions & 5 deletions configs/nas/mmcls/onceforall/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ CUDA_VISIBLE_DEVICES=0 PORT=29500 ./tools/dist_test.sh \

## Results and models

| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 | Config | Download | Remarks |
| :------: | :------------------: | :----------------------------------------------------------------------: | :-------: | :------: | :---: | :---------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------: |
| ImageNet | AttentiveMobileNetV3 | [search space](configs/_base_/nas_backbones/ofa_mobilenetv3_supernet.py) | 7.6 | 747.8 | 77.5 | [config](./detnas_subnet_frcnn_shufflenetv2_fpn_1x_coco.py) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/ofa/ofa_mobilenet_supernet_d234_e346_k357_w1_0.py_20221214_0940-d0ebc66f.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | note8_lat@[email protected]_finetune@25 | 4.3 | 70.9 | 70.3 | [config](./OFA_SUBNET_NOTE8_LAT22.yaml) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/ofa/ofa_mobilenet_subnet_8xb256_in1k_note8_lat%4022ms_top1%4070.4_finetune%4025.py_20221214_0938-fb7fb84f.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | note8_lat@[email protected]_finetune@25 | 4.6 | 105.4 | 72.6 | [config](./OFA_SUBNET_NOTE8_LAT31.yaml) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/ofa/ofa_mobilenet_subnet_8xb256_in1k_note8_lat%4031ms_top1%4072.8_finetune%4025.py_20221214_0939-981a8b2a.pth) | Converted from the repo |
| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 | Config | Download | Remarks |
| :------: | :------------------: | :-------------------------------------------------------------------------------------------------------------------------: | :-------: | :------: | :---: | :-----------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------: |
| ImageNet | AttentiveMobileNetV3 | [search space](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/_base_/nas_backbones/ofa_mobilenetv3_supernet.py) | 7.6 | 747.8 | 77.5 | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/onceforall/ofa_mobilenet_supernet_32xb64_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/ofa/ofa_mobilenet_supernet_d234_e346_k357_w1_0.py_20221214_0940-d0ebc66f.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | note8_lat@[email protected]_finetune@25 | 4.3 | 70.9 | 70.3 | [config](https://download.openmmlab.com/mmrazor/v1/ofa/rtmdet/OFA_SUBNET_NOTE8_LAT22.yaml) | [model](https://download.openmmlab.com/mmrazor/v1/ofa/ofa_mobilenet_subnet_8xb256_in1k_note8_lat%4022ms_top1%4070.4_finetune%4025.py_20221214_0938-fb7fb84f.pth) | Converted from the repo |
| ImageNet | AttentiveMobileNetV3 | note8_lat@[email protected]_finetune@25 | 4.6 | 105.4 | 72.6 | [config](https://download.openmmlab.com/mmrazor/v1/ofa/rtmdet/OFA_SUBNET_NOTE8_LAT31.yaml) | [model](https://download.openmmlab.com/mmrazor/v1/ofa/ofa_mobilenet_subnet_8xb256_in1k_note8_lat%4031ms_top1%4072.8_finetune%4025.py_20221214_0939-981a8b2a.pth) | Converted from the repo |

**Note**:

Expand Down
8 changes: 4 additions & 4 deletions configs/nas/mmcls/spos/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ CUDA_VISIBLE_DEVICES=0 PORT=29500 ./tools/dist_test.sh \

## Results and models

| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 (%) | Top-5 (%) | Config | Download | Remarks |
| :------: | :--------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-------: | :------: | :-------: | :-------: | :--------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------------------------------------------------------: |
| ImageNet | ShuffleNetV2 | [mutable](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_shufflenetv2_subnet_8xb128_in1k/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-454627be_mutable_cfg.yaml) | 3.35 | 0.33 | 73.87 | 91.6 | [config](./spos_subnet_shufflenetv2_8xb128_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_shufflenetv2_subnet_8xb128_in1k/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d.pth) \| [log](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_shufflenetv2_subnet_8xb128_in1k/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d.log.json) | MMRazor searched |
| ImageNet | MobileNet-ProxylessGPU | [mutable](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_mobilenet_subnet/spos_angelnas_flops_0.49G_acc_75.98_20220307-54f4698f_mutable_cfg.yaml) | 5.94 | 0.49\* | 75.98 | 92.77 | [config](./spos_mobilenet_for_check_ckpt_from_anglenas.py) | | [AngleNAS](https://github.com/megvii-model/AngleNAS) searched |
| Dataset | Supernet | Subnet | Params(M) | Flops(G) | Top-1 (%) | Top-5 (%) | Config | Download | Remarks |
| :------: | :--------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------: | :-------: | :------: | :-------: | :-------: | :-------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :-----------------------------------------------------------: |
| ImageNet | ShuffleNetV2 | [mutable](https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20220715-aa94d5ef_subnet_cfg_v3.yaml) | 3.35 | 0.33 | 73.87 | 91.6 | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/spos/spos_subnet_shufflenetv2_8xb128_in1k.py) | [model](https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d_v3.pth) \| [log](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_shufflenetv2_subnet_8xb128_in1k/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d.log.json) | MMRazor searched |
| ImageNet | MobileNet-ProxylessGPU | [mutable](https://download.openmmlab.com/mmrazor/v0.1/nas/spos/spos_mobilenet_subnet/spos_angelnas_flops_0.49G_acc_75.98_20220307-54f4698f_mutable_cfg.yaml) | 5.94 | 0.49\* | 75.98 | 92.77 | [config](https://github.com/open-mmlab/mmrazor/blob/dev-1.x/configs/nas/mmcls/spos/spos_mobilenet_subnet_8xb128_in1k.py) | | [AngleNAS](https://github.com/megvii-model/AngleNAS) searched |

**Note**:

Expand Down
4 changes: 2 additions & 2 deletions configs/nas/mmcls/spos/metafile.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ Models:
In Collection: SPOS
Metadata:
FLOPs: 330 MB
Subnet: https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20220715-aa94d5ef_subnet_cfg_v1.yaml
Subnet: https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d_subnet_cfg_v3.yaml
Results:
- Task: Image Classification
Dataset: ImageNet-1k
Metrics:
Top 1 Accuracy: 73.87
Top 5 Accuracy: 91.60
Config: configs/nas/mmcls/spos/spos_shufflenet_subnet_8xb128_in1k.py
Weights: https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d_v1.pth
Weights: https://download.openmmlab.com/mmrazor/v1/spos/spos_shufflenetv2_subnet_8xb128_in1k_flops_0.33M_acc_73.87_20211222-1f0a0b4d_v3.pth
7 changes: 0 additions & 7 deletions mmrazor/engine/runner/subnet_val_loop.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from typing import Dict, List, Optional, Union

from mmengine.evaluator import Evaluator
from mmengine.hooks import CheckpointHook
from mmengine.runner import ValLoop
from torch.utils.data import DataLoader

Expand Down Expand Up @@ -53,12 +52,6 @@ def __init__(
self.calibrate_sample_num = calibrate_sample_num
self.estimator = TASK_UTILS.build(estimator_cfg)

# remove CheckpointHook to avoid extra problems.
for hook in self.runner._hooks:
if isinstance(hook, CheckpointHook):
self.runner._hooks.remove(hook)
break

def run(self):
"""Launch validation."""
self.runner.call_hook('before_val')
Expand Down

0 comments on commit 18754f3

Please sign in to comment.