I am using an ml5.js setup in my React app where I click on a button to train a model and then I click on another button to make predictions. The Test button works on the first time but it throws an error on the second time:
TypeError: Cannot read property 'classify' of undefined
I believe this is because once I run the prediction part, the model gets deleted? Because the classifier is now undefined. How can I modify it such that I can repeatedly click on the Test button and obtain new predictions each time.
I tried the save() function but apparently it only downloads the model and not save it for the app.
export const Video: React.FC<ComponentProps> = (props: ComponentProps) => {
const [prediction, setPrediction] = useState<string>();
let capture: p5Types.Element;
let classifier: any;
const setup = (p5: p5Types, canvasParentRef: Element) => {
capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
const featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
classifier = featureExtractor.classification(capture, videoReady);
}
const draw = (p5: p5Types) => {
}
function gotResult() {
classifier.classify(capture, (err: any, result: any) => {
setPrediction(result[0].label);
});
}
function train() {
classifier.train((lossValue: any) => {
console.log('Loss is', lossValue);
});
//classifier.save();
}
return (<div><Sketch setup={setup} draw={draw} className="sketch" />
<div className="button">
<Button variant="contained" color="primary" onClick={() => classifier.addImage('first')}>First</Button>
<Button variant="contained" color="primary" onClick={() => classifier.addImage('second')}>Second</Button>
</div>
<div className="secondbutton">
<Button variant="contained" color="primary" onClick={() => train()}>Train!</Button>
<Button variant="contained" color="primary" onClick={() => gotResult()}>Test!</Button>
<br />
<span>Prediction: {prediction}</span>
</div>
</div>)
;
};
Codesandbox:
https://codesandbox.io/s/hardcore-solomon-zb34l?file=/src/Component.tsx
Updated code:
export const VideoComponent: React.FC<ComponentProps> = (props: ComponentProps) => {
const [prediction, setPrediction] = useState<string>();
const [confidence, setConfidence] = useState<string>();
const [trainingComplete, setTrainingComplete] = useState<boolean>();
const captureRef = useRef<p5Types.Element>();
const classifierRef = useRef<any>();
const setup = (p5: p5Types, canvasParentRef: Element) => {
const capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
const featureExtractor = ml5.featureExtractor("MobileNet", modelReady);
captureRef.current = capture;
classifierRef.current = featureExtractor.classification(capture, videoReady);
}
const draw = (p5: p5Types) => {
}
function gotResult() {
console.log('classifier in results', classifierRef.current);
classifierRef.current.classify(captureRef.current, (err: any, result: any) => {
setPrediction(result[0].label);
setConfidence(result[0].confidence);
});
}
function train() {
console.log('classifier in train', classifierRef.current);
classifierRef.current?.classify.train((lossValue: any) => {
console.log('Loss is', lossValue);
if (lossValue == null) {
//setTrainingComplete(true);
console.log('training complete')
}
});
}
return (
<div>
<Sketch setup={setup} draw={draw} className="sketch" />
<div className="button">
<Button variant="contained" color="primary" onClick={() => { classifierRef.current?.classifier.addImage('first'); console.log('image added') }}>First</Button>
<Button variant="contained" color="primary" onClick={() => { classifierRef.current?.classifier.addImage('second'); console.log('image added') }}>Second</Button>
</div>
<div className="secondbutton">
<Button variant="contained" color="primary" onClick={() => train()}>Train!</Button>
<Button variant="contained" color="primary" onClick={() => gotResult()}>Test!</Button>
<br />
{trainingComplete && (<span>Training Complete!</span>)}<br />
<span>Prediction: {prediction}</span><br />
</div>
</div>)
;
};
The problem is the two
let
variables,capture
andclassifier
, which you are using to store stateful data.These variables get re-created on every re-render. If you want to have instance variables which persist across re-renders then you need to use
useState
oruseRef
. Since these two variables come from thep5
package and might be modified by other functions I thinkuseRef
is probably what you want here. You can read more aboutuseRef
in the React docs.You need to add some extra conditionally checks in your code since
captureRef.current
could be either anElement
orundefined
, so you need to make sure that you have anElement
before using it. Forclassifier
you can use the optional chaining operator like this:classifierRef.current?.classify()
.You should try to find or create an actual type for
classifier
instead ofany
. It looks like there is no types package forml5
, but someone created a draft.