Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AttributeError: 'NoneType' object has no attribute 'name' in FusedBatchNorm #925

Open
machanic opened this issue Jun 28, 2021 · 3 comments
Open

AttributeError: 'NoneType' object has no attribute 'name' in FusedBatchNorm #925

machanic opened this issue Jun 28, 2021 · 3 comments

Comments

@machanic
Copy link

Platform (like ubuntu 16.04/win10): ubuntu 16.04

Python version: 3.7.3

Source framework with version (like Tensorflow 1.4.1 with GPU): Tensorflow 1.4.0

Destination framework with version (like CNTK 2.3 with GPU): PyTorch 1.7.1

Pre-trained model path (webpath or webdisk path): https://drive.google.com/file/d/1awFM8y4A9jWcfUFaVs6S3CBGZramLBkY/view?usp=sharing
Running scripts: mmconvert -sf tensorflow -in model-50.meta -iw model-50 --dstNode resnet10/logits/BiasAdd -df pytorch -om model-50.pth
@machanic
Copy link
Author

It seems that this is a bug that caused by Fused Batch Normalization layer, but I did not use that layer.

I only use tf.layers.batch_normalization as the BN layer in ResNet-10, but I didn't pass fused argument, which is set to None by default. None means it uses a faster, fused implementation if possible

This was referenced Jun 28, 2021
Open
Closed
@JiahaoYao
Copy link
Contributor

JiahaoYao commented Jun 28, 2021 •
edited
Loading

Hi @machanic , I looked into the network. Here is a temporal solution:

You might want to replace these two liners with

        mean = self.get_parent(source_node.name, [3], True)
        var = self.get_parent(source_node.name, [4], True)

also, replace these two liners with

            self.set_weight(source_node.name, 'mean', self.ckpt_data[mean.name.replace('Const', 'moving_mean')])

            self.set_weight(source_node.name, 'var', self.ckpt_data[var.name.replace('Const_1', 'moving_variance')])

You can find your src path via import mmdnn; print(mmdnn.__path__)

Most likely, after the temporal hotfixes, you can get

    def forward(self, x):
        resnet10_Pad    = F.pad(x, (3, 3, 3, 3), mode = 'constant', value = 0)
        resnet10_conv1_Conv2D = self.resnet10_conv1_Conv2D(resnet10_Pad)
        resnet10_pool1_MaxPool, resnet10_pool1_MaxPool_idx = F.max_pool2d(resnet10_conv1_Conv2D, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_pool1_MaxPool)
        resnet10_block1_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block1_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Pad)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block1_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_add = resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block1_unit_1_bottleneck_v2_add)
        resnet10_block2_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block2_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Pad)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block2_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_add = resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block2_unit_1_bottleneck_v2_add)
        resnet10_block3_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block3_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Pad)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block3_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_add = resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block3_unit_1_bottleneck_v2_add)
        resnet10_block4_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad = F.pad(resnet10_block4_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1))
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_add = resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_batch_normalization_FusedBatchNorm = self.resnet10_batch_normalization_FusedBatchNorm(resnet10_block4_unit_1_bottleneck_v2_add)
        resnet10_Relu   = F.relu(resnet10_batch_normalization_FusedBatchNorm)
        resnet10_pool5  = torch.mean(resnet10_Relu, 3, True)
        resnet10_pool5  = torch.mean(resnet10_pool5, 2, True)
        resnet10_logits_Conv2D = self.resnet10_logits_Conv2D(resnet10_pool5)
        return resnet10_logits_Conv2D

Good Luck!

@JiahaoYao
Copy link
Contributor

Hi @machanic, did this solve your issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@machanic @JiahaoYao