Total nitpick: Not all TorchScript models are traced - q.v., torch.jit.script(), method decorations. I'd name this var torchscript_model or similar for clarity & accuracy.

There are valid TorchScript models (googlenet & inception_v3 in torchvision) that segfault on this line. Other than that, the instructions work and there's a modest (<=2.2%) improvement in file size for these models:

mobilenet_v2
resnet18
alexnet
squeezenet1_0
vgg16
densenet161
shufflenet_v2_x1_0
mnasnet1_0

Two things:

1. PMM is going to come back and say that we should write it TorchScript.
2. The code below does not pass a TorchScript module to fuse_modules() - that MobileNet v2 from TorchVision is not a subclass of ScriptModule.

Passing in the original torchvision module itself or a version of it processed by torch.jit.script() works, with the latter giving ~2% file size improvement.

Should we be guiding people to torch.jit.trace()? torch.jit.script() preserves control flow, trace() does not. trace() is still there for cases where script() hits an unsupported op.

y is never used, which means we could do without x as well.

Two things:

1. Is this the quantization workflow we want to show? This looks like it's just pulling down a pre-quantized version of the model, in which case this recipe is only useful if you want to quantize the torchvision models. In the general case, I'd think people would want to be able to quantize their own trained models for mobile deployment.
2. On MacOS, this line throws a warning:

/Users/bradheintz/anaconda2/envs/pyto16pre/lib/python3.8/site-packages/torch/nn/quantized/modules/utils.py:8: UserWarning: 0quantize_tensor_per_tensor_affine current rounding mode is not set to round-to-nearest-ties-to-even (FE_TONEAREST). This will cause accuracy issues in quantized models. (Triggered internally at  ../aten/src/ATen/native/quantized/affine_quantizer.cpp:25.)
  qweight = torch.quantize_per_tensor(
/Users/bradheintz/anaconda2/envs/pyto16pre/lib/python3.8/site-packages/torch/quantization/observer.py:134: UserWarning: must run observer before calling calculate_qparams.                                    Returning default scale and zero point
  warnings.warn(

(And yes, it really looks like that in my terminal.) The warning came up in this env:

# torch 1.6.0a0+55bcb5d built from master with USE_CUDA=0
# torchvision 0.7.0a0+148bac2 built from master with USE_CUDA=0
# python 3.8.0
# MacOS 10.15.4

We should show a workflow where we start with a floating point model and then do the quantization. The steps are:

# Start with a fully trained floating point model
# The model code is modified to enable eager mode quantization, for more details
# please see the quantization tutorials at: https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html

model = torchvision.models.quantization.__dict__['resnet18'](pretrained=True, quantize=False)
torch.backends.quantized.engine='qnnpack'
# We convert the float model with the appropriate_Qconfig
model.eval()
model.qconfig = torch.quantization.get_default_qconfig('qnnpack')
torch.quantization.prepare(model)
# Run model with representative data for calibration
# model(calibration_data)
torch.quantization.convert(model)
script_model = torch.jit.script(model)

# Export to mobile
script_model._save_for_lite_interpreter("model.bc")

I don't actually know the answer to this: Is it preferable to do quantization before or after TorchScript conversion, or does it matter at all?

Currently, pytorch only supports doing quantization prior to scripting. We are working on adding support for quantization after scripting, but it is not part of release 1.6 yet.

Including a sample model here is a great idea - this will help users generalize to their own use case.

prepare() has inplace=False by default, and the same goes for convert(), so this whole method is a no-op except for setting model.qconfig.

We either need to do:

model = torch.quantization.prepare(model)
# calibration
return torch.quantization.convert(model)

or:

torch.quantization.prepare(model, inplace=True)
# calibration
torch.quantization.convert(model, inplace=True)