MXNet convertorToIR don't parse 'attrs' layer parameters #85
Comments
|
I change functions rename_FullyConnected, rename_LeakyReLU and rename_Convolution in mxnet_parser.py in the following way: this solve this issue. These changes should be applied to all mxnet functions. |
|
Hi @belgraviton , cool! I fixed it and checked it in. Could you help to try if it works in newest code now? Thanks! |
|
I use already implemented '_get_layer_attr' function in all 'rename' functions. |
|
Appreciate what you have done! |
|
I'm trying to use @belgraviton PR to convert the same model LResNet50E-IR but I am getting the following error: |
|
Hi @d4nst , there are two operators not support by current framework. Warning: MXNet Parser has not supported operator _minus_scalar with name _minusscalar0.
Warning: MXNet Parser has not supported operator _mul_scalar with name _mulscalar0.Could you provide the related files for us to implement and test it? Thanks. |
|
I used the same model posted by @belgraviton: LResNet50E-IR |
|
_minusscalar0 and _mulscalar0 can be replaced to _copy in json file. There are some other problems during 'LResNet50E-IR' MXNet to IR conversion. One of them is solved by PR #96 |
|
Hi @belgraviton @d4nst I am working on these two op -> IR stuff. Will ping you if it is done. Thanks. |
|
@belgraviton @d4nst . Fixed. Please check the newest code and recent commit for detail info. But I didn't check the correctness. |
|
It now runs without any errors. Thanks! |
|
There is no errors. OK. Thank you |
|
@d4nst @belgraviton Were you guys able to convert the IR representation to any other framework? I tried converting it (for the same model) to CoreML, but that gives vastly different outputs. When trying to convert the IR to Tensorflow, I get the following errors: |
|
hi，i also use the LResNet50E-IR mxnet model，and i success to convert it to IR，and then i convert models from IR to Tensorflow code snippet use follow script like that： log show that： but when i convert models from IR to Tensorflow model，i met the below error： for resolve the raise error， then i modified the tf_resnet50.py： to that help to fix the raise ValueError，but another error raise，like that： it show that the file tf_resnet50.py have a bud，local in file： consider all of the scripts and executed log， maybe error bring by the step “convert models from IR to Tensorflow code snippet”，and the log “TensorflowEmitter has not supported operator [_copy]. update: if i modifiy the code "minusscalar0 = id - _minusscalar0_second" to "minusscalar0 = data - _minusscalar0_second", that will help to solve the error, whitch 'data' is a placeholder of input, but then i met the error same as @galli-leo. |
|
@xsr-ai I just removed the whole minus and mul scalar nodes, since they are just preprocessing the images. But yeah, stuck on the error I received then. When I get some more time, I will try and debug it, I am guessing somehow the weights are wrongly saved. |
|
@galli-leo i add “bn1 = tf.reshape(bn1, (1, 25088))” before tf.layers.dense in the tf_resnet50.py，then fix the error “ValueError: Too many elements provided. Needed at most 262144, but received 12845056”， but when i test the converted tf model it have a poor performance，lfw Acc just reach to 0.573+-0.021，i don‘t know why the test performance so poor. |
|
@xsr-ai We probably need to fix the weights themselves not the layer. I should get some time next week to look at this. |
|
@galli-leo look forward to your great work. |
|
@galli-leo hi, gali, are you get any progress？ |
|
@xsr-ai @galli-leo Could you provide your test code? I guess I know how to handle it, just try transpose the weight. |
|
@kitstar we use the tensorflow model https://drive.google.com/open?id=1x0-EiYX9jMUKiq-n1Bd9OCK4fVB3a54v or https://pan.baidu.com/s/1mj6X7MK, you can look at 14 days ago which i mentioned to repeat the issue.finally, thanks you kindly help me. |
|
@galli-leo hi, gali, are you getting any progress？ |
|
i add "bn1 = tf.reshape(bn1, [-1, 7*7*512])", which fix the error “ValueError: Too many elements provided. Needed at most 262144, but received 12845056”. |
|
@stormand thanks! @JiahaoYao could you try to confirm the fix and try to fix it in code? Thanks! |
|
To sum up, I have compared the inference results of converted tensorflow and original mxnet. They are exactly the same. Here comes the IR structure. Secondly, just as @xsr-ai mentioned, Here comes the tensorflow snippet. However, Two things have to be modified in this snippet.
_mulscalar0_second = tf.constant(__weights_dict['_mulscalar0_second']['value'], name='_mulscalar0_second')
_minusscalar0_second = tf.constant(__weights_dict['_minusscalar0_second']['value'], name='_minusscalar0_second')
data = tf.placeholder(tf.float32, shape = (None, 112, 112, 3), name = 'data')
minusscalar0 = data - _minusscalar0_second
mulscalar0 = minusscalar0 * _mulscalar0_second
Then the code should be like mulscalar0_second = tf.constant(__weights_dict['_mulscalar0_second']['value'], name='mulscalar0_second')
minusscalar0_second = tf.constant(__weights_dict['_minusscalar0_second']['value'], name='minusscalar0_second')
data = tf.placeholder(tf.float32, shape = (None, 112, 112, 3), name = 'data')
minusscalar0 = data - minusscalar0_second
mulscalar0 = minusscalar0 * mulscalar0_secondIn my view, this may because the node name can not begin with '_' in tensorflow.
The code (line 255-260) should be converted from bn1 = batch_normalization(plus23, variance_epsilon=1.99999994948e-05, name='bn1')
pre_fc1 =... to bn1 = batch_normalization(plus23, variance_epsilon=1.99999994948e-05, name='bn1')
bn1 = tf.reshape(bn1, [-1, 7*7*512])
pre_fc1 = ...In my opinion, this is because in this mxnet json, there is no flatten operator before the fc layer. That is why there is no flatten layer when converted to tensorflow. Here, we add this flatten layer to the tensorflow code. As to the Sanity checks, I compare the inference result of the two. # inference with mxnet
import mxnet as mx
from tensorflow.contrib.keras.api.keras.preprocessing import image
import numpy as np
from collections import namedtuple
Batch = namedtuple('Batch', ['data'])
ctx = mx.cpu()
# load model
sym, arg_params, aux_params = mx.model.load_checkpoint('/Users/kit/Downloads/model-r50-am-lfw-n/model', 0)
mod = mx.mod.Module(symbol = sym, context= ctx, label_names= None)
mod.bind(for_training=False, data_shapes=[('data', (1, 3, 112, 112))], label_shapes= mod._label_shapes)
mod.set_params(arg_params, aux_params, allow_missing= True)
path = '/Users/kit/github/MMdnn/mmdnn/conversion/examples/data/seagull.jpg'
# load image with BGRTranspose=True
img = image.load_img(path, target_size = (112, 112))
img = image.img_to_array(img)
img = img[..., ::-1]
# channel first in mxnet
img = np.expand_dims(img, 0).transpose((0,3,1,2))
# compute the predict probabilities
mod.forward(Batch([mx.nd.array(img)]))
prob = mod.get_outputs()[0].asnumpy()
prob = np.squeeze(prob)
print(prob)For tensorflow, Exploit the script # inference with tensorflow
from __future__ import absolute_import
import argparse
import numpy as np
from six import text_type as _text_type
from tensorflow.contrib.keras.api.keras.preprocessing import image
import tensorflow as tf
parser = argparse.ArgumentParser()
parser.add_argument('-n', type=_text_type, default='kit_imagenet',
help='Network structure file name.')
parser.add_argument('-w', type=_text_type, required=True,
help='Network weights file name')
parser.add_argument('--image', '-i',
type=_text_type, help='Test image path.',
default="mmdnn/conversion/examples/data/seagull.jpg"
)
args = parser.parse_args()
if args.n.endswith('.py'):
args.n = args.n[:-3]
# import converted model
model_converted = __import__(args.n).KitModel(args.w)
input_tf, model_tf = model_converted
# load img with BGRTranspose=True
img = image.load_img(args.image, target_size = (112, 112))
img = image.img_to_array(img)
img = img[..., ::-1]
input_data = np.expand_dims(img, 0)
# inference with tensorflow
with tf.Session() as sess:
init = tf.global_variables_initializer()
sess.run(init)
predict = sess.run(model_tf, feed_dict = {input_tf : input_data})
print(predict)one can get the inference by running python tensorflow_inference.py -n tf_resnet50.py -w model-descr512.npy -i /Users/kit/github/MMdnn/mmdnn/conversion/examples/data/seagull.jpgTo sum up, the two results are exactly the same, |
|
We have already got a pull request to solve the above two problems. |
|
Hi, thank you for your great effort. But I still encounter some errors. First step: Second step is fine, too. With the script: However, in the third step with script: |
|
I change the third script from
But got a new error: Besides, my generated tensorflow file is different from @JiahaoYao |
|
@tengerye You are welcome, and I wonder if you can show me your generated tensorflow file. |
|
Thank you for your kind reply @JiahaoYao . My tensorflow file starts with Do you need to read all the content? |
|
@tengerye MMdnn aims at converting one framework to another framework. After your second step, you get the tensorflow snippet and parse file. Thus, the model is already converted. First, try the newest version by Second, convert the model to intermediate representation format. Third, convert to tensorflow code Then you already get tensorflow code If you would like to check whether the conversion is right, you can follow Sanity checks in my last post. Always remember to change the path of the model and image. Hoping this address your problem~ |
|
It works. Thank you so much @JiahaoYao . |
|
Great! It is my pleasure~ |
|
@JiahaoYao I thought the comparison you did in previous post cannot prove the whole conversion process is correct. |
|
@JiahaoYao Sorry, I made a mistakes during the conversion. LResNet50E-IR: Converted Tensorflow Model: |
|
@sczhengyabin |
|
@xmuszq Just use the tensorflow freeze_graph tool. |
|
I did a test on lfw, the accuracy is 55%. |
|
@xmuszq Could you share the scripts that convert the tensorflow code snippet and .npy file into .ckpt or .pb files with me ? Thank u very much! |
|
Hi, I'm still getting an error when running the first command. Any help would be greatly appreciated ! Platform: Ununtu 16.04 Python version: 3.6 Source framework with version: MXNet 1.0.0 with GPU Pre-trained : LResNet50E-IR, same as everyone Running scripts:
Error message : |
|
Hi @StanislasBertrand , Please refer to this issue |
|
@JiahaoYao your conversion and inference code works great in python. However, do you have any sample code for performing inference with the converted model in C++. I've tried the session API but haven't been able to make it work with the model. |
|
Figured out how to do it. For those wondering, when loading the model with the python code above, you can make the following modification to save the model: You can then use the C++ session API |
Platform: Ununtu 16.04
Python version: 3.6
Source framework with version: MXNet 1.0.0 with GPU
Destination framework with version: TensorFlow 1.4.1 with GPU
Pre-trained model path: LResNet50E-IR
Model json-file contain layers in the following form:
{ "op": "Convolution", "name": "conv0", "attrs": { "kernel": "(3, 3)", "no_bias": "True", "num_filter": "64", "pad": "(1, 1)", "stride": "(1, 1)", "workspace": "256" }, "inputs": [[3, 0, 0], [4, 0, 0]] }Running scripts:
python -m mmdnn.conversion._script.convertToIR -f mxnet -n model-symbol.json -w model-0000.params -d model-descr512 --inputShape 3 112 112Error:
The text was updated successfully, but these errors were encountered: