I tried to use handpose 2.0 on my previous sketches, “Drum Machine” and “Pond Scum” but I ran into a couple of issues.
I had trouble using both versions of ml5 in the same sketch.
I was able to add handpose 2.0 to Drum Machine BUT, it became very laggy, almost to the point where my computer seemed like it was going to crash. I suspect part of the reason is that ml5 and p5 sound have some processes which interfere with eachother
However, I was still very determined to do something involving sounds, and ‘buttons’ so I created a 2x4 grid of ‘buttons’ which would play sounds according to where the user moves their hand on the canvas.
Despite my best efforts and doing as minimal conditions as possible, the sketch is still quite glitchy, but it does technically work. I would like to do further research about why as soon as I try to combine sound and ml5, things start getting so laggy, while either of them work so smoothly on their own.
I was stubborn, and decided to test out a very minimal version of “Drum Machine” and combining p5 sound with ml5 handpose.
Experience using handPose2.0
As we observed in class, in the ml5 next-gen library, handpose works a bit differently. One of the first challenges I encountered was trying to pass additional options (flipHorizontal) into the model. Due to the changes, I was initially confused at where to put the options parameter, but I quickly figured that out.
For some reason, handpose seemed harder to use than posenet or faceAPI. However, I think this is due to there being so many points, and perhaps because of how glitchy my experiments with p5 sound became.
