fixed quantization script, pull bytes out of Float32Array when quantization is skipped

justadudewhohacks · justadudewhohacks · commit 234acd42d331 · 2018-10-26T10:18:54.000+02:00
diff --git a/tools/quantize/views/quantizeModel.html b/tools/quantize/views/quantizeModel.html
@@ -58,7 +58,7 @@
         }
 
         if (isSkipQuantization) {
-          quantizedTensorArrays.push(tensor.dataSync())
+          quantizedTensorArrays.push(new Uint8Array(tensor.dataSync().buffer))
           weightEntries.push(weightEntry)
           return
         }

Original file line number	Diff line number	Diff line change
`@@ -58,7 +58,7 @@`
`58`	`58`	`}`
`59`	`59`
`60`	`60`	`if (isSkipQuantization) {`
`61`		`- quantizedTensorArrays.push(tensor.dataSync())`
	`61`	`+ quantizedTensorArrays.push(new Uint8Array(tensor.dataSync().buffer))`
`62`	`62`	`weightEntries.push(weightEntry)`
`63`	`63`	`return`
`64`	`64`	`}`