How does Machine Learning 2.0 work?

Machine Learning 2.0 integrates a decent number of pre-trained models. Through TensorFlow.js, the extension carries out transfer learning based on your uploaded data. Let's simplify the idea by understanding it as follows: the pre-trained models resemble a seasoned musician who masters various instruments. The data you provided is like a new type of instrument. The musician uses the previously mastered knowledge of playing music (pre-trained dataset) to learn the new instrument (newly given data) quickly and accurately. This is what we call transfer learning.

What types of models can I train?

Image, audio, and pose models are supported currently. In the coming future, more models will be integrated. Stay tuned!

Why does my model not work as expected?

Multiple factors can affect your model training results. Here are some suggestions for you:

• Find another place. For image and pose models, lighting and contrast may affect the training results because the pictures are collected by the webcam. If your model does not work well, you are advised to retrain the model by collecting data in a different place.
• Change or fix your position. Pose models can not only identify your poses, but also track your position within the webcam frame. If your model does not work well, you are advised to change or fix your position during the training or preview phase.
• Replace the microphone or adjust the distance between it and the sound source. If your audio model does not work well, you are advised to use another microphone and recollect the sounds. Alternatively, you can adjust the distance between the microphone and the sound source.

How many models can I train?

In this version, you can create one model for each task. A model will be overwritten if you create a new one for the same task. In our plan, you will be allowed to create more models for each task in later versions.

Will the uploaded data be stored?

No. The whole training process runs in your local browser, and the training data you upload in Machine Learning 2.0 is not sent to our servers. Even if you save your projects using our cloud service, only your trained models will be stored. In other words, your training data will not be collected.

How do I train a model?

Tutorials for three types of machine learning tasks are ready. For details, see:

• How to Create an Image Model
• How to Create an Audio Model
• How to Create a Pose Model

How do I use my trained model in mBlock?

After training a model, click Use model in the upper-right corner of the training page to use it in mBlock. The currently provided blocks include:

• start recognition: You can click this block to start collecting real-time image and audio samples. The recognition result is stored in the recognition result block. Each time you click this block refreshes values of the recognition result, confidence of, and recognition result is blocks.

• recognition result: This block stores the recognition result of the start recognition block. If you haven't clicked the start recognition block, this block contains no value.

• confidence of: This block stores the probability that an object recognized belongs to a class each time the start recognition block is clicked.

• recognition result is: This block returns a boolean value. You can determine whether the object identified belongs to a certain class using this block. If yes, true is returned; otherwise, false is returned.