Virtual Mouse Using Hand Gesture

The virtual mouse using hand gesture project is a Python-based application that enables users to control the computer mouse cursor using hand gestures captured through a webcam. This project harnesses computer vision techniques and machine learning to interpret hand movements in real-time and translate them into mouse actions, offering a hands-free alternative for interacting with digital interfaces.

Python Code

import cv2
    import mediapipe as mp
    import pyautogui
    cap = cv2.VideoCapture(0)
    hand_detector = mp.solutions.hands.Hands()
    drawing_utils = mp.solutions.drawing_utils

screen_width, screen_height = pyautogui.size()
    index_y = 0
    while True:
        _, frame = cap.read()
        frame = cv2.flip(frame,1)
        frame_height,frame_width,_ = frame.shape
        rgd_frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
        output = hand_detector.process(rgd_frame)
        hands = output.multi_hand_landmarks

if hands:
            for hand in hands:
                drawing_utils.draw_landmarks(frame,hand)
                landmarks = hand.landmark
                for id, landmark in enumerate(landmarks):
                    x = int(landmark.x * frame_width)
                    y = int(landmark.y * frame_height)
                    if id == 8:
                        cv2.circle(img=frame,center=(x,y),radius=10,color=(0,255,255))
                        index_x = screen_width/frame_width*x
                        index_y = screen_height/frame_height*y
                    if id == 4:
                        cv2.circle(img=frame,center=(x,y),radius=10,color=(0,255,255))
                        thumb_x = screen_width/frame_width*x
                        thumb_y = screen_height/frame_height*y
                        print('outside',abs(index_y - thumb_y))
                        if abs(index_y - thumb_y)<20:
                            pyautogui.click()
                            pyautogui.sleep(1)
                        elif abs(index_y - thumb_y)<100:
                              pyautogui.moveTo(index_x,index_y)

cv2.imshow('Virtual Mouse',frame)
        cv2.waitKey(1)

Key Features

Hand Gesture Recognition

Utilizes the OpenCV library in Python to capture live video frames from a webcam.

Implements computer vision algorithms to detect and track the user's hand within the video stream.

Recognizes specific hand gestures (e.g., pointing, clicking, dragging) using image processing techniques.

Mouse Cursor Control

Maps detected hand gestures to corresponding mouse cursor movements and actions.

Calculates the position and displacement of the hand to simulate cursor navigation and interaction on the computer screen.

Gesture-based Commands

Two-finger pinch gesture to perform right-click or scroll actions, depending on the context.

Real-time Interaction

Provides immediate feedback by updating the mouse cursor position and executing actions in real-time based on detected gestures.

Enables users to interact with desktop applications, web browsers, and graphical user interfaces (GUIs) without physical mouse devices.

Technology Stack

Python: Primary programming language for implementing the virtual mouse project.

OpenCV (Open Source Computer Vision Library):Used for capturing video frames, hand detection, and image processing.

PyAutoGUI: Interfaces with the PyAutoGUI library to programmatically control the mouse cursor and simulate mouse actions.

Workflow

Hand Detection and Tracking

Applies image processing techniques (e.g., thresholding, contour detection) to identify and track the user's hand.

Gesture Recognition

Analyzes the position, orientation, and movement of the detected hand to recognize predefined gestures.

Maps recognized gestures to corresponding mouse actions (e.g., cursor movement, clicking).

Mouse Simulation

Simulates mouse movements, clicks, and scroll actions on the computer screen based on detected gestures.

Utilizes PyAutoGUI to programmatically control the mouse cursor position and execute actions.

Potential Use Cases

Accessibility Tool: Assists individuals with physical disabilities in navigating computer interfaces using hand gestures.

Interactive Presentations:Enables presenters to control slideshows or multimedia content using intuitive hand movements.

Gaming and Entertainment: Provides an immersive gaming experience by allowing players to interact with virtual environments using gestures.

Conclusion

The virtual mouse using hand gesture project showcases the application of computer vision and machine learning techniques in enabling hands-free mouse control through real-time hand gestures. This project serves as a practical and innovative solution for enhancing user interaction with digital interfaces using Python-based technologies.

Now let's Build Virtual Mouse Using Eye Gesture using Python

Python Code

import cv2
      import mediapipe as mp
      import pyautogui
      cam = cv2.VideoCapture(0)
      face_mesh = mp.solutions.face_mesh.FaceMesh(refine_landmarks = True)
      screen_w, screen_h = pyautogui.size()
      while True:
          _, frame = cam.read()
          frame = cv2.flip(frame,1)
          rgd_frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
          output = face_mesh.process(rgd_frame)
          landmark_points = output.multi_face_landmarks
          frame_h,frame_w, _ = frame.shape
          if landmark_points:
              landmarks = landmark_points[0].landmark
              for id, landmark in enumerate(landmarks[474:478]):
                  x = int(landmark.x* frame_w)
                  y = int(landmark.y * frame_h)
                  cv2.circle(frame,(x,y),3,(0,255,0))
                  if id == 1:
                      screen_x = screen_w/ frame_w * x
                      screen_y = screen_h/ frame_h * y
                      pyautogui.moveTo(screen_x,screen_y)
              left = [landmarks[145],landmarks[159]]
              for landmark in left:
                   x = int(landmark.x* frame_w)
                   y = int(landmark.y * frame_h)
                   cv2.circle(frame,(x,y),3,(0,255,255))
              if(left[0].y - left[1].y) < 0.004:
                  pyautogui.click()
                  pyautogui.sleep(1)

cv2.imshow('Eye Controlled Mouse',frame)
          cv2.waitKey(1)

Virtual Mouse Using Hand Gesture using Python

By Neelkanth

Project Overview

Python Code

Key Features

Hand Gesture Recognition

Mouse Cursor Control

Gesture-based Commands

Real-time Interaction

Technology Stack

Workflow

Hand Detection and Tracking

Gesture Recognition

Mouse Simulation

Potential Use Cases

Conclusion

Now let's Build Virtual Mouse Using Eye Gesture using Python

Python Code

Virtual Mouse Using Hand Gesture using Python By Neelkanth

Project Overview

Python Code

Key Features

Hand Gesture Recognition

Mouse Cursor Control

Gesture-based Commands

Real-time Interaction

Technology Stack

Workflow

Hand Detection and Tracking

Gesture Recognition

Mouse Simulation

Potential Use Cases

Conclusion

Now let's Build Virtual Mouse Using Eye Gesture using Python

Python Code

Virtual Mouse Using Hand Gesture using Python

By Neelkanth