Scribd
Upload
193 views17 pages

ml5.js: Friendly Machine Learning For The Web

Uploaded by

Exdy Xd

ml5.js is a friendly machine learning library for the web that provides access to machine learning algorithms and models in the browser. It aims to make machine learning approachable for artists, students, and others. Key features include running pre-trained models, creating and training neural networks, and building custom models. The library wraps TensorFlow.js and is inspired by Processing and p5.js.

Copyright:

© All Rights Reserved
Download as PDF, TXT or read online from Scribd
Download as pdf or txt

ml5.js: Friendly Machine Learning For The Web

Uploaded by

Exdy Xd
193 views17 pages
ml5.js is a friendly machine learning library for the web that provides access to machine learning algorithms and models in the browser. It aims to make machine learning approachable for artists, students, and others. Key features include running pre-trained models, creating and training neural networks, and building custom models. The library wraps TensorFlow.js and is inspired by Processing and p5.js.

Original Title

ys_ml5

Copyright

© © All Rights Reserved

Available Formats

PDF, TXT or read online from Scribd

Did you find this document useful?

Is this content inappropriate?

ml5.js is a friendly machine learning library for the web that provides access to machine learning algorithms and models in the browser. It aims to make machine learning approachable for artists, students, and others. Key features include running pre-trained models, creating and training neural networks, and building custom models. The library wraps TensorFlow.js and is inspired by Processing and p5.js.

Copyright:

© All Rights Reserved
Download as PDF, TXT or read online from Scribd
Download as pdf or txt
193 views17 pages

ml5.js: Friendly Machine Learning For The Web

Uploaded by

Exdy Xd
ml5.js is a friendly machine learning library for the web that provides access to machine learning algorithms and models in the browser. It aims to make machine learning approachable for artists, students, and others. Key features include running pre-trained models, creating and training neural networks, and building custom models. The library wraps TensorFlow.js and is inspired by Processing and p5.js.

Copyright:

© All Rights Reserved
Download as PDF, TXT or read online from Scribd
Download as pdf or txt
You are on page 1/ 17

ml5.

js: Friendly Machine Learning for the Web

Yining Shi
Adjunct professor at New York University
Software Engineer at RunwayML

1
Friendly Machine Learning for the Web
A neighborly approach to creating and exploring artificial
intelligence in the browser.
Developing ml5 is not just about developing machine learning software, it is about making machine learning
approachable for a broad audience of artists, creative coders, and students.

The library provides access to machine learning algorithms and models in the browser, building on top of
TensorFlow.js.

ml5.js is heavily inspired by Processing and p5.js.

ml5js.org

2
What does ml5.js do?
1. Running pre-trained models
2. Creating and training neural networks

3
imageClassifier(‘MobileNet’)
ported by Cristobal Valenzuela

const classifier = ml5.imageClassifier(‘MobileNet’);

classifier.classify(video, gotResult);

function gotResult(error, result) {


console.log(result);
}

4
PoseNet
ported by Cristobal Valenzuela, Maya Man, Dan Oved.

const posenet = ml5.poseNet(video);

posenet.on('pose', function(results) {
poses = results;
});

function draw() {
if (poses.length > 0) {
circle(poses[0].nose.x, poses[0].nose.y);
}
}

5
Style Transfer
ported by Yining Shi

const cubist = ml5.styleTransfer(‘models/cubist’, modelReady);

function modelReady() {
cubist.transfer(video, gotImage)
}

function gotImage(error, result) {


image(result.image, 0, 0);
}

6
DIY Neural Network
Wrapped by Dan Shiffman, Joey Lee & Yining Shi

const options = {
dataUrl: 'data/colorData.json',
inputs: ['r', 'g', 'b'],
outputs: ['label'],
task: 'classification'
};

// Create Neural Network


neuralNetwork = ml5.neuralNetwork(options);

neuralNetwork.train(whileTraining, finishedTraining);

neuralNetwork.classify(data, gotResults);

7
image sound text helpers
imageClassifier CartoonGAN Pitch Detection CharRNN NeuralNetwork
ObjectDetector FaceApi SoundClassifier Word Vectorization FeatureExtractor
poseNet Facemesh Sentiment KNNClassifier
BodyPix Handpose UniversalSentence KMeans
UNET CVAE Encoder
YOLO DCGAN
StyleTransfer SketchRNN
Pix2Pix

ml5.js version 0.5.0 https://ml5js.org 8


How do we use ml5.js?

9
Step 1: Call your ml5 function.
const myClassifier = await ml5.imageClassifier(‘MobileNet’);

Step 2: Apply your ml5 function - e.g. to an image, video, or text.


const results = await myClassifier.classify(myCatImage);

Step 3: Do something with the results.


// An array of objects with “label” and “confidence”
// [ { label: ‘cat’, confidence: 0.74 } ]
console.log(results);

ml5.js in 3 simple steps 10


How is ml5.js built?

11
Core

ml5-library ml5-examples

Education / Training / models

training-* ml5-website intro-to-ml-arts


ml5-data-and-models-server

ml5 ecosystem https://github.com/ml5js


12
13
The case for ML in the browser:
● Accessibility + familiarity of web apps
● Ease of creation of interactive use cases
○ Low effort in using existing browser APIs such as webcams
and microphones with the ability of rendering outputs easily
to image, canvas, audio, or text elements on the DOM.

14
https://ml5js.org/community
Challenge 1: Native support for converting browser
I/O streams to model input/output data structures
● Accessibility + familiarity of web apps
● Ease of creation of interactive use cases
○ Better support for real-time interaction with webcam,
microphones as inputs and image, canvas, audio, text as
outputs

15
https://ml5js.org/community
Challenge 2: Standardization of model format
● Factors to determine whether pre-trained models can be ported
to ml5.js:
○ Small model size
○ Low-latency
○ Portable model format
● Workflow of porting:
1 2 3

16
Thank you!

https://ml5js.org
https://github.com/ml5js
Twitter @ml5js

17

You might also like