Three.js Hand Recognition Panel Using MediaPipe and WebGL

Abstract

The rapid evolution of human-computer interaction (HCI) technologies has created demand for natural, contactless interfaces that transcend the limitations of traditional input devices. This paper presents the design and implementation of a real-time gesture-based interaction system that integrates Google’s MediaPipe Hands framework with the Three.js WebGL rendering library. The proposed system, titled Three.js Hand Recognition Panel, captures live video from a standard webcam and detects up to two hands simultaneously, extracting 21 three-dimensional landmark points per hand. Distancebased and positional gesture recognition algorithms map these landmarks to interactive commands that drive a WebGL-rendered 3D virtual environment. Operating entirely within a web browser without specialized hardware, the system achieves 50–60 FPS on standard desktops, gesture recognition accuracy exceeding 88%, and average response latency below 70 ms. Experimental results validate the feasibility of deploying production-quality gesture interfaces through open-source web technologies. The work contributes a modular, extensible architecture suitable for applications in gaming, virtual reality, healthcare, and assistive technologies.