You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _mobile/ios.md
+20-18
Original file line number
Diff line number
Diff line change
@@ -12,11 +12,11 @@ published: true
12
12
13
13
To get started with PyTorch on iOS, we recommend exploring the following [HelloWorld](https://github.com/pytorch/ios-demo-app/tree/master/HelloWorld).
14
14
15
-
## Quickstart with a Hello World example
15
+
## Quickstart with a Hello World Example
16
16
17
17
HelloWorld is a simple image classification application that demonstrates how to use PyTorch C++ libraries on iOS. The code is written in Swift and uses Objective-C as a bridge.
18
18
19
-
### Model preparation
19
+
### Model Preparation
20
20
21
21
Let's start with model preparation. If you are familiar with PyTorch, you probably should already know how to train and save your model. In case you don't, we are going to use a pre-trained image classification model - Resnet18, which is already packaged in [TorchVision](https://pytorch.org/docs/stable/torchvision/index.html). To install it, run the command below.
22
22
@@ -34,7 +34,7 @@ python trace_model.py
34
34
35
35
If everything works well, we should have our model - `model.pt` generated in the `HelloWorld` folder. Now copy the model file to our application folder `HelloWorld/model`.
36
36
37
-
> To find out more details about TorchScript, please visit [tutorials on pytorch.org](https://pytorch.org/docs/stable/jit.html)
37
+
> To find out more details about TorchScript, please visit [tutorials on pytorch.org](https://pytorch.org/tutorials/advanced/cpp_export.html)
38
38
39
39
### Install PyTorch C++ libraries via Cocoapods
40
40
@@ -46,11 +46,13 @@ pod install
46
46
47
47
Now it's time to open the `HelloWorld.xcworkspace` in XCode, select an iOS simulator and launch it (cmd + R). If everything works well, we should see a wolf picture on the simulator screen along with the prediction result.
In this part, we are going to walk through the code step by step.
52
54
53
-
#### Image loading
55
+
#### Image Loading
54
56
55
57
Let's begin with image loading.
56
58
@@ -76,9 +78,9 @@ for i in 0 ..< w * h {
76
78
}
77
79
```
78
80
79
-
The code might look weird at first glance, but it’ll make sense once we understand our model. The input data of our model is a 3-channel RGB image of shape (3 x H x W), where H and W are expected to be at least 224. The image have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].
81
+
The code might look weird at first glance, but it’ll make sense once we understand our model. The input data is a 3-channel RGB image of shape (3 x H x W), where H and W are expected to be at least 224. The image has to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].
80
82
81
-
#### TorchScript module
83
+
#### TorchScript Module
82
84
83
85
Now that we have preprocessed our input data and we have a pre-trained TorchScript model, the next step is to use them to run predication. To do that, we'll first load our model into the application.
84
86
@@ -97,7 +99,7 @@ Note that the `TorchModule` Class is an Objective-C wrapper of `torch::jit::scri
Since Swift can not talk to C++ directly, we have to either use an Objective-C class as a bride, or create a C wrapper for the C++ library. For the demo purpose, we're going to wrap everything in this Objective-C class, but we are working on bringing the Swift wrapper to PyTorch.
102
+
Since Swift can not talk to C++ directly, we have to either use an Objective-C class as a bridge, or create a C wrapper for the C++ library. For demo purpose, we're going to wrap everything in this Objective-C class, but we are working on bringing the Swift wrapper to PyTorch.
The C++ function `torch::from_blob` will create an input tensor from the pixel buffer. Note that the shpae of the tensor is `{1,3,224,224}` which represents `NxCxWxH` as we discuessed in above section.
122
+
The C++ function `torch::from_blob` will create an input tensor from the pixel buffer. Note that the shape of the tensor is `{1,3,224,224}` which represents `NxCxWxH` as we discussed in above section.
The above two lines tells the PyTorch engine to do inference only. This is beacuse By default, PyTorch has built-in support for doing auto-differentiation, which is also known as autograd. Since we don't do training on mobile, we can just disable the autograd mode.
128
+
The above two lines tells the PyTorch engine to do inference only. This is because by default, PyTorch has built-in support for doing auto-differentiation, which is also known as autograd. Since we don't do training on mobile, we can just disable the autograd mode.
127
129
128
130
Finally, we can call this `forward` function to get the output tensor as the results.
129
131
130
132
```cpp
131
133
auto outputTensor = _impl.forward({tensor}).toTensor();
132
134
```
133
135
134
-
### Collect results
136
+
### Collect Results
135
137
136
138
The output tensor is a one-dimensional float array of shape 1x1000, where each value represents the confidence that a label is predicted from the image. The code below sorts the array and retrieves the top three results.
137
139
@@ -140,28 +142,28 @@ let zippedResults = zip(labels.indices, outputs)
140
142
let sortedResults = zippedResults.sorted { $0.1.floatValue > $1.1.floatValue }.prefix(3)
141
143
```
142
144
143
-
### PyTorch demo app
145
+
### PyTorch Demo App
144
146
145
-
For more complex use cases, we recommend to check out the PyTorch demo application. The demo app contains two showcases. A camera app that runs a quantized model to predict the images coming from device’s rear-facing camera in real time. And a text-based app that uses a text classififcation model to predict the topic from the input string.
147
+
For more complex use cases, we recommend to check out the [PyTorch demo application](https://github.com/pytorch/ios-demo-app). The demo app contains two showcases. A camera app that runs a quantized model to predict the images coming from device’s rear-facing camera in real time. And a text-based app that uses a text classififcation model to predict the topic from the input string.
146
148
147
-
## Build PyTorch iOS libraries from source
149
+
## Build PyTorch iOS Libraries from Source
148
150
149
151
To track the latest progress on mobile, we can always build the PyTorch iOS libraries from the source. Follow the steps below.
150
152
151
-
### Setup local Python development environment
153
+
### Setup Local Python Development Environment
152
154
153
155
Follow the PyTorch Github page to set up the Python environment. Make sure you have `cmake` and Python installed correctly on your local machine.
154
156
155
-
### Build LibTorch.a for iOS simulator
157
+
### Build LibTorch for iOS Simulators
156
158
157
159
Open terminal and navigate to the PyTorch root directory. Run the following command
After the build succeed, all static libraries and header files will be generated under `build_ios/install`
172
174
173
-
### XCode setup
175
+
### XCode Setup
174
176
175
177
Open your project in XCode, copy all the static libraries as well as header files to your project. Navigate to the project settings, set the value **Header Search Paths** to the path of header files you just copied in the first step.
0 commit comments