✨博客主页：王乐予🎈
✨年轻人要：Living for the moment（活在当下）！💪
🏆推荐专栏：【图像处理】【千锤百炼Python】【深度学习】【排序算法】

😺一、MediaPipe概述

MediaPipe 是一款由 Google Research 开发并开源的多媒体机器学习模型应用框架。

MediaPipe目前支持的解决方案(Solution)及支持的平台如下图所示：

😺二、MediaPipe手部特征点检测

官网描述：

https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker
The MediaPipe Hand Landmarker task lets you detect the landmarks of the hands in an image. You can use this task to locate key points of hands and render visual effects on them. This task operates on image data with a machine learning (ML) model as static data or a continuous stream and outputs hand landmarks in image coordinates, hand landmarks in world coordinates and handedness(left/right hand) of multiple detected hands.

该模型会跟踪手部 21 个关键点位置：

😺三、任务描述

本文将通过 Mediapipe 检测出手部关键点，并通过对各种关键点的位置判别，以达到手势识别的目的。本文将对如下 6 种手势进行判定识别：

1. OK
2. Return
3. Left
4. Right
5. Like
6. Pause

😺四、代码实现

🐶4.1 度量函数

🦄4.1.1 距离度量

计算两点之间的距离如下：

"""
计算两个点之间的距离：L2距离（欧式距离）
"""
def points_distance(x0, y0, x1, y1):
    return math.sqrt((x0 - x1) ** 2 + (y0 - y1) ** 2)

🦄4.1.2 角度度量

计算两向量之间的角度如下：

"""
计算两条线段之间的夹角，以弧度表示
"""
def compute_angle(x0, y0, x1, y1, x2, y2, x3, y3):
    AB = [x1 - x0, y1 - y0]
    CD = [x3 - x2, y3 - y2]

    dot_product = AB[0] * CD[0] + AB[1] * CD[1]

    AB_distance = points_distance(x0, y0, x1, y1) + 0.001   # 防止分母出现0
    CD_distance = points_distance(x2, y2, x3, y3) + 0.001

    cos_theta = dot_product / (AB_distance * CD_distance)

    theta = math.acos(cos_theta)

    return theta

🐶4.2 工作流程

1. 通过 Mediapipe 获取所有手部关键点；
2. 检测每根手指的状态（弯曲 or 伸直）；
3. 判断当前手势（定义多组手势判断函数）；
4. 如果连续30帧均为同一种手势，则可视化手势内容。

🐶4.3 代码实现

🦄4.3.1 定义手指状态函数（弯曲 or 伸直）

def detect_all_finger_state(all_points):

    finger_first_angle_bend_threshold = math.pi * 0.25  # 大拇指弯曲阈值
    finger_other_angle_bend_threshold = math.pi * 0.5  # 其他手指弯曲阈值
    finger_other_angle_straighten_threshold = math.pi * 0.2  # 其他手指伸直阈值

    first_is_bend = False
    first_is_straighten = False
    second_is_bend = False
    second_is_straighten = False
    third_is_bend = False
    third_is_straighten = False
    fourth_is_bend = False
    fourth_is_straighten = False
    fifth_is_bend = False
    fifth_is_straighten = False

    finger_first_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point1'][0], all_points['point1'][1],
                                        all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])
    finger_sencond_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                        all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    finger_third_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point9'][0], all_points['point9'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    finger_fourth_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point13'][0], all_points['point13'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    finger_fifth_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])

    if finger_first_angle > finger_first_angle_bend_threshold:              # 判断大拇指是否弯曲
        first_is_bend = True
        first_is_straighten = False
    else:
        first_is_bend = False
        first_is_straighten = True

    if finger_sencond_angle > finger_other_angle_bend_threshold:            # 判断食指是否弯曲
        second_is_bend = True
    elif finger_sencond_angle < finger_other_angle_straighten_threshold:
        second_is_straighten = True
    else:
        second_is_bend = False
        second_is_straighten = False

    if finger_third_angle > finger_other_angle_bend_threshold:              # 判断中指是否弯曲
        third_is_bend = True
    elif finger_third_angle < finger_other_angle_straighten_threshold:
        third_is_straighten = True
    else:
        third_is_bend = False
        third_is_straighten = False

    if finger_fourth_angle > finger_other_angle_bend_threshold:             # 判断无名指是否弯曲
        fourth_is_bend = True
    elif finger_fourth_angle < finger_other_angle_straighten_threshold:
        fourth_is_straighten = True
    else:
        fourth_is_bend = False
        fourth_is_straighten = False

    if finger_fifth_angle > finger_other_angle_bend_threshold:              # 判断小拇指是否弯曲
        fifth_is_bend = True
    elif finger_fifth_angle < finger_other_angle_straighten_threshold:
        fifth_is_straighten = True
    else:
        fifth_is_bend = False
        fifth_is_straighten = False

    # 将手指的弯曲或伸直状态存在字典中，简化后续函数的参数
    bend_states = {'first': first_is_bend, 'second': second_is_bend, 'third': third_is_bend, 'fourth': fourth_is_bend, 'fifth': fifth_is_bend}
    straighten_states = {'first': first_is_straighten, 'second': second_is_straighten, 'third': third_is_straighten, 'fourth': fourth_is_straighten, 'fifth': fifth_is_straighten}

    return bend_states, straighten_states

🦄4.3.2 定义 OK 手势判断函数

OK 手势判断流程如图：

def judge_OK(all_points, bend_states, straighten_states):

    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])

    if angle5_6_and_6_8 > 0.1 * math.pi and straighten_states['third'] and straighten_states['fourth'] and straighten_states['fifth']:

        distance4_and_8 = points_distance(all_points['point4'][0], all_points['point4'][1], all_points['point8'][0], all_points['point8'][1])
        distance2_and_6 = points_distance(all_points['point2'][0], all_points['point2'][1], all_points['point6'][0], all_points['point6'][1])
        distance4_and_6 = points_distance(all_points['point4'][0], all_points['point4'][1], all_points['point6'][0], all_points['point6'][1])

        if distance4_and_8 < distance2_and_6 and distance4_and_6 > distance4_and_8 and all_points['point11'][1] < all_points['point10'][1]:
            return 'OK'
        else:
            return False
    else:
        return False

🦄4.3.3 定义 Return 手势判断函数

Return 手势判断流程如图：

def judge_Return(all_points, bend_states, straighten_states):

    angle18_6_and_18_18_ = compute_angle(all_points['point18'][0], all_points['point18'][1], all_points['point6'][0], all_points['point6'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point18'][0] + 10, all_points['point18'][1])
    angle_6_18_and_6_6_ = compute_angle(all_points['point6'][0], all_points['point6'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point6'][0], all_points['point6'][1], all_points['point6'][0] + 10, all_points['point6'][1])
    angle_0_2_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point2'][0], all_points['point2'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])

    if (bend_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth'] and
            angle_0_2_and_0_17 > 0.15 * math.pi and
            all_points['point7'][1] > all_points['point6'][1] and all_points['point11'][1] > all_points['point10'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):

        if angle18_6_and_18_18_ < 0.1 * math.pi or angle_6_18_and_6_6_ < 0.1 * math.pi:
            return 'Return'
        else:
            return False
    else:
        return False

🦄4.3.4 定义 Left 手势判断函数

Left 手势判断流程如图：

def judge_Left(all_points, bend_states, straighten_states):

    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])
    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])

    if ((straighten_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or
        (straighten_states['first'] and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and
        angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi)):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if (angle0_5_and_0_17 > 0.15 * math.pi and angle0_0__and_0_4 > 0.7 * math.pi and all_points['point3'][0] < all_points['point2'][0] and
            angle0_6_and_0_4 > 0.1 * math.pi and all_points['point11'][1] > all_points['point10'][1] and all_points['point7'][1] > all_points['point6'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):
            return 'Left'
        else:
            return False
    else:
        return False

🦄4.3.5 定义 Right 手势判断函数

Right 手势判断流程如图：

def judge_Right(all_points, bend_states, straighten_states):

    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

    if ((straighten_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or
        (straighten_states['first'] and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and
        angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi)):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if (angle0_5_and_0_17 > 0.15 * math.pi and angle0_0__and_0_4 < 0.25 * math.pi and all_points['point3'][0] > all_points['point2'][0] and
            angle0_6_and_0_4 > 0.1 * math.pi and all_points['point11'][1] > all_points['point10'][1] and all_points['point7'][1] > all_points['point6'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):
            return 'Right'
        else:
            return False
    else:
        return False

🦄4.3.6 定义 Like 手势判断函数

Like 手势判断流程如图：

def judge_Like(all_points, bend_states, straighten_states):

    angle2_4_and_2_8 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point8'][0], all_points['point8'][1])
    angle2_3_and_3_4 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point3'][0], all_points['point3'][1],
                                    all_points['point3'][0], all_points['point3'][1], all_points['point4'][0], all_points['point4'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

    if (angle2_3_and_3_4 < 0.2 * math.pi and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or (angle2_3_and_3_4 < 0.2 * math.pi and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if angle0_0__and_0_4 > 0.25 * math.pi and angle0_0__and_0_4 < 0.75 * math.pi and all_points['point3'][1] > all_points['point4'][1] and all_points['point5'][1] < all_points['point9'][1] and all_points['point9'][1] < all_points['point13'][1] and all_points['point13'][1] < all_points['point17'][1] and all_points['point2'][1] < all_points['point5'][1] and angle0_6_and_0_4 > 0.1 * math.pi and angle2_4_and_2_8 > 0.1 * math.pi:
            return 'Like'
        else:
            return False
    else:
        return False

🦄4.3.7 定义 Pause 手势判断函数

Pause 手势判断流程如图：

def judge_Pause(all_points, bend_states, straighten_states):

    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])
    angle2_3_and_2_4 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point3'][0], all_points['point3'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle1_2_and_2_4 = compute_angle(all_points['point1'][0], all_points['point1'][1], all_points['point2'][0], all_points['point2'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])

    if angle2_3_and_2_4 < 0.2 * math.pi and angle5_6_and_6_8 < 0.07 * math.pi and angle9_10_and_10_12 < 0.07 * math.pi and angle13_14_and_14_16 < 0.07 * math.pi and angle17_18_and_18_20 < 0.07 * math.pi and angle9_10_and_10_12 < 0.07 * math.pi and angle1_2_and_2_4 < 0.25 * math.pi:
        if angle0_5_and_0_17 > 0.1 * math.pi and all_points['point3'][1] > all_points['point4'][1] and all_points['point6'][1] > all_points['point8'][1] and all_points['point10'][1] > all_points['point12'][1]:
            return 'Pause'
        else:
            return False
    else:
        return False

🦄4.3.8 定义检测当前手势函数

"""
检测当前手势，返回当前手势
"""
def detect_hand_state(all_points, bend_states, straighten_states):
    state_OK = judge_OK(all_points, bend_states, straighten_states)
    state_Return = judge_Return(all_points, bend_states, straighten_states)
    state_Left = judge_Left(all_points, bend_states, straighten_states)
    state_Right = judge_Right(all_points, bend_states, straighten_states)
    state_Like = judge_Like(all_points, bend_states, straighten_states)
    state_Pause = judge_Pause(all_points, bend_states, straighten_states)

    if state_OK == 'OK':
        return 'OK'
    elif state_Return == 'Return':
        return 'Return'
    elif state_Left == 'Left':
        return 'Left'
    elif state_Right == 'Right':
        return 'Right'
    elif state_Like == 'Like':
        return 'Like'
    elif state_Pause == 'Pause':
        return 'Pause'
    else:
        return 'None'

🐶4.4 完整代码

🦄4.4.1 gesture_judgment.py

import math


"""
计算两个点之间的距离：L2距离（欧式距离）
"""
def points_distance(x0, y0, x1, y1):
    return math.sqrt((x0 - x1) ** 2 + (y0 - y1) ** 2)


"""
计算两条线段之间的夹角，以弧度表示
"""
def compute_angle(x0, y0, x1, y1, x2, y2, x3, y3):
    AB = [x1 - x0, y1 - y0]
    CD = [x3 - x2, y3 - y2]

    dot_product = AB[0] * CD[0] + AB[1] * CD[1]

    AB_distance = points_distance(x0, y0, x1, y1) + 0.001   # 防止分母出现0
    CD_distance = points_distance(x2, y2, x3, y3) + 0.001

    cos_theta = dot_product / (AB_distance * CD_distance)

    theta = math.acos(cos_theta)

    return theta


"""
检测所有手指状态（判断每根手指弯曲 or 伸直）
大拇指只有弯曲和伸直两种状态，其他手指除了弯曲和伸直还包含第三种状态（手指没有伸直，但是也没有达到弯曲的标准），第三种状态是为了后续更新迭代用的，这里用不到
"""
def detect_all_finger_state(all_points):

    finger_first_angle_bend_threshold = math.pi * 0.25  # 大拇指弯曲阈值
    finger_other_angle_bend_threshold = math.pi * 0.5  # 其他手指弯曲阈值
    finger_other_angle_straighten_threshold = math.pi * 0.2  # 其他手指伸直阈值

    first_is_bend = False
    first_is_straighten = False
    second_is_bend = False
    second_is_straighten = False
    third_is_bend = False
    third_is_straighten = False
    fourth_is_bend = False
    fourth_is_straighten = False
    fifth_is_bend = False
    fifth_is_straighten = False

    finger_first_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point1'][0], all_points['point1'][1],
                                        all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])
    finger_sencond_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                        all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    finger_third_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point9'][0], all_points['point9'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    finger_fourth_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point13'][0], all_points['point13'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    finger_fifth_angle = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])

    if finger_first_angle > finger_first_angle_bend_threshold:              # 判断大拇指是否弯曲
        first_is_bend = True
        first_is_straighten = False
    else:
        first_is_bend = False
        first_is_straighten = True

    if finger_sencond_angle > finger_other_angle_bend_threshold:            # 判断食指是否弯曲
        second_is_bend = True
    elif finger_sencond_angle < finger_other_angle_straighten_threshold:
        second_is_straighten = True
    else:
        second_is_bend = False
        second_is_straighten = False

    if finger_third_angle > finger_other_angle_bend_threshold:              # 判断中指是否弯曲
        third_is_bend = True
    elif finger_third_angle < finger_other_angle_straighten_threshold:
        third_is_straighten = True
    else:
        third_is_bend = False
        third_is_straighten = False

    if finger_fourth_angle > finger_other_angle_bend_threshold:             # 判断无名指是否弯曲
        fourth_is_bend = True
    elif finger_fourth_angle < finger_other_angle_straighten_threshold:
        fourth_is_straighten = True
    else:
        fourth_is_bend = False
        fourth_is_straighten = False

    if finger_fifth_angle > finger_other_angle_bend_threshold:              # 判断小拇指是否弯曲
        fifth_is_bend = True
    elif finger_fifth_angle < finger_other_angle_straighten_threshold:
        fifth_is_straighten = True
    else:
        fifth_is_bend = False
        fifth_is_straighten = False

    # 将手指的弯曲或伸直状态存在字典中，简化后续函数的参数
    bend_states = {'first': first_is_bend, 'second': second_is_bend, 'third': third_is_bend, 'fourth': fourth_is_bend, 'fifth': fifth_is_bend}
    straighten_states = {'first': first_is_straighten, 'second': second_is_straighten, 'third': third_is_straighten, 'fourth': fourth_is_straighten, 'fifth': fifth_is_straighten}

    return bend_states, straighten_states


"""
判断是否为 OK 手势
"""
def judge_OK(all_points, bend_states, straighten_states):

    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])

    if angle5_6_and_6_8 > 0.1 * math.pi and straighten_states['third'] and straighten_states['fourth'] and straighten_states['fifth']:

        distance4_and_8 = points_distance(all_points['point4'][0], all_points['point4'][1], all_points['point8'][0], all_points['point8'][1])
        distance2_and_6 = points_distance(all_points['point2'][0], all_points['point2'][1], all_points['point6'][0], all_points['point6'][1])
        distance4_and_6 = points_distance(all_points['point4'][0], all_points['point4'][1], all_points['point6'][0], all_points['point6'][1])

        if distance4_and_8 < distance2_and_6 and distance4_and_6 > distance4_and_8 and all_points['point11'][1] < all_points['point10'][1]:
            return 'OK'
        else:
            return False
    else:
        return False


"""
判断是否为 Return 手势
"""
def judge_Return(all_points, bend_states, straighten_states):

    angle18_6_and_18_18_ = compute_angle(all_points['point18'][0], all_points['point18'][1], all_points['point6'][0], all_points['point6'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point18'][0] + 10, all_points['point18'][1])
    angle_6_18_and_6_6_ = compute_angle(all_points['point6'][0], all_points['point6'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point6'][0], all_points['point6'][1], all_points['point6'][0] + 10, all_points['point6'][1])
    angle_0_2_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point2'][0], all_points['point2'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])

    if (bend_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth'] and
            angle_0_2_and_0_17 > 0.15 * math.pi and
            all_points['point7'][1] > all_points['point6'][1] and all_points['point11'][1] > all_points['point10'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):

        if angle18_6_and_18_18_ < 0.1 * math.pi or angle_6_18_and_6_6_ < 0.1 * math.pi:
            return 'Return'
        else:
            return False
    else:
        return False


"""
判断是否为 Left 手势
"""
def judge_Left(all_points, bend_states, straighten_states):

    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])
    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])

    if ((straighten_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or
        (straighten_states['first'] and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and
        angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi)):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if (angle0_5_and_0_17 > 0.15 * math.pi and angle0_0__and_0_4 > 0.7 * math.pi and all_points['point3'][0] < all_points['point2'][0] and
            angle0_6_and_0_4 > 0.1 * math.pi and all_points['point11'][1] > all_points['point10'][1] and all_points['point7'][1] > all_points['point6'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):
            return 'Left'
        else:
            return False
    else:
        return False


"""
判断是否为 Right 手势
"""
def judge_Right(all_points, bend_states, straighten_states):

    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

    if ((straighten_states['first'] and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or
        (straighten_states['first'] and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and
        angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi)):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if (angle0_5_and_0_17 > 0.15 * math.pi and angle0_0__and_0_4 < 0.25 * math.pi and all_points['point3'][0] > all_points['point2'][0] and
            angle0_6_and_0_4 > 0.1 * math.pi and all_points['point11'][1] > all_points['point10'][1] and all_points['point7'][1] > all_points['point6'][1] and
            all_points['point15'][1] > all_points['point14'][1] and all_points['point19'][1] > all_points['point18'][1]):
            return 'Right'
        else:
            return False
    else:
        return False


"""
判断是否为 Like 手势
"""
def judge_Like(all_points, bend_states, straighten_states):

    angle2_4_and_2_8 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point8'][0], all_points['point8'][1])
    angle2_3_and_3_4 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point3'][0], all_points['point3'][1],
                                    all_points['point3'][0], all_points['point3'][1], all_points['point4'][0], all_points['point4'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle0_6_and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

    if (angle2_3_and_3_4 < 0.2 * math.pi and bend_states['second'] and bend_states['third'] and bend_states['fourth'] and bend_states['fifth']) or (angle2_3_and_3_4 < 0.2 * math.pi and angle5_6_and_6_8 > 0.2 * math.pi and angle9_10_and_10_12 > 0.2 * math.pi and angle13_14_and_14_16 > 0.2 * math.pi and angle17_18_and_18_20 > 0.2 * math.pi):

        angle0_0__and_0_4 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point0'][0] + 10, all_points['point0'][1],
                                        all_points['point0'][0], all_points['point0'][1], all_points['point4'][0], all_points['point4'][1])

        if angle0_0__and_0_4 > 0.25 * math.pi and angle0_0__and_0_4 < 0.75 * math.pi and all_points['point3'][1] > all_points['point4'][1] and all_points['point5'][1] < all_points['point9'][1] and all_points['point9'][1] < all_points['point13'][1] and all_points['point13'][1] < all_points['point17'][1] and all_points['point2'][1] < all_points['point5'][1] and angle0_6_and_0_4 > 0.1 * math.pi and angle2_4_and_2_8 > 0.1 * math.pi:
            return 'Like'
        else:
            return False
    else:
        return False


"""
判断是否为 Pause 手势
"""
def judge_Pause(all_points, bend_states, straighten_states):

    angle0_5_and_0_17 = compute_angle(all_points['point0'][0], all_points['point0'][1], all_points['point5'][0], all_points['point5'][1],
                                    all_points['point0'][0], all_points['point0'][1], all_points['point17'][0], all_points['point17'][1])
    angle2_3_and_2_4 = compute_angle(all_points['point2'][0], all_points['point2'][1], all_points['point3'][0], all_points['point3'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])
    angle5_6_and_6_8 = compute_angle(all_points['point5'][0], all_points['point5'][1], all_points['point6'][0], all_points['point6'][1],
                                    all_points['point6'][0], all_points['point6'][1], all_points['point8'][0], all_points['point8'][1])
    angle9_10_and_10_12 = compute_angle(all_points['point9'][0], all_points['point9'][1], all_points['point10'][0], all_points['point10'][1],
                                        all_points['point10'][0], all_points['point10'][1], all_points['point12'][0], all_points['point12'][1])
    angle13_14_and_14_16 = compute_angle(all_points['point13'][0], all_points['point13'][1], all_points['point14'][0], all_points['point14'][1],
                                        all_points['point14'][0], all_points['point14'][1], all_points['point16'][0], all_points['point16'][1])
    angle17_18_and_18_20 = compute_angle(all_points['point17'][0], all_points['point17'][1], all_points['point18'][0], all_points['point18'][1],
                                        all_points['point18'][0], all_points['point18'][1], all_points['point20'][0], all_points['point20'][1])
    angle1_2_and_2_4 = compute_angle(all_points['point1'][0], all_points['point1'][1], all_points['point2'][0], all_points['point2'][1],
                                    all_points['point2'][0], all_points['point2'][1], all_points['point4'][0], all_points['point4'][1])

    if angle2_3_and_2_4 < 0.2 * math.pi and angle5_6_and_6_8 < 0.07 * math.pi and angle9_10_and_10_12 < 0.07 * math.pi and angle13_14_and_14_16 < 0.07 * math.pi and angle17_18_and_18_20 < 0.07 * math.pi and angle9_10_and_10_12 < 0.07 * math.pi and angle1_2_and_2_4 < 0.25 * math.pi:
        if angle0_5_and_0_17 > 0.1 * math.pi and all_points['point3'][1] > all_points['point4'][1] and all_points['point6'][1] > all_points['point8'][1] and all_points['point10'][1] > all_points['point12'][1]:
            return 'Pause'
        else:
            return False
    else:
        return False


"""
检测当前手势，返回当前手势
"""
def detect_hand_state(all_points, bend_states, straighten_states):
    state_OK = judge_OK(all_points, bend_states, straighten_states)
    state_Return = judge_Return(all_points, bend_states, straighten_states)
    state_Left = judge_Left(all_points, bend_states, straighten_states)
    state_Right = judge_Right(all_points, bend_states, straighten_states)
    state_Like = judge_Like(all_points, bend_states, straighten_states)
    state_Pause = judge_Pause(all_points, bend_states, straighten_states)

    if state_OK == 'OK':
        return 'OK'
    elif state_Return == 'Return':
        return 'Return'
    elif state_Left == 'Left':
        return 'Left'
    elif state_Right == 'Right':
        return 'Right'
    elif state_Like == 'Like':
        return 'Like'
    elif state_Pause == 'Pause':
        return 'Pause'
    else:
        return 'None'

🦄4.4.2 main.py

import cv2
import mediapipe as mp
import time
from gesture_judgment import detect_all_finger_state, detect_hand_state


mp_hands = mp.solutions.hands
hands = mp_hands.Hands(static_image_mode=False, max_num_hands=1, min_detection_confidence=0.5, min_tracking_confidence=0.5)
mp_drawing = mp.solutions.drawing_utils

recent_states = [''] * 30    # 存储最近 30 次的手势判断结果

cap = cv2.VideoCapture(0)

prev_time = 0
while True:
    ret, frame = cap.read()
    frame = cv2.flip(frame, 1)  # Horizontal mirror flipping
    h, w = frame.shape[:2]
    image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    keypoints = hands.process(image)

    if keypoints.multi_hand_landmarks:
        lm = keypoints.multi_hand_landmarks[0]
        lmHand = mp_hands.HandLandmark

        landmark_list = [[] for _ in range(6)]  # landmark_list有 6 个子列表，分别存储着根节点坐标（0点）以及其它 5 根手指的关键点坐标

        for index, landmark in enumerate(lm.landmark):
            x = int(landmark.x * w)
            y = int(landmark.y * h)
            if index == lmHand.WRIST:
                landmark_list[0].append((x, y))
            elif 1 <= index <= 4:
                landmark_list[1].append((x, y))
            elif 5 <= index <= 8:
                landmark_list[2].append((x, y))
            elif 9 <= index <= 12:
                landmark_list[3].append((x, y))
            elif 13 <= index <= 16:
                landmark_list[4].append((x, y))
            elif 17 <= index <= 20:
                landmark_list[5].append((x, y))

        # 获取所有关节点的坐标
        point0 = landmark_list[0][0]
        point1, point2, point3, point4 = landmark_list[1][0], landmark_list[1][1], landmark_list[1][2], landmark_list[1][3]
        point5, point6, point7, point8 = landmark_list[2][0], landmark_list[2][1], landmark_list[2][2], landmark_list[2][3]
        point9, point10, point11, point12 = landmark_list[3][0], landmark_list[3][1], landmark_list[3][2], landmark_list[3][3]
        point13, point14, point15, point16 = landmark_list[4][0], landmark_list[4][1], landmark_list[4][2], landmark_list[4][3]
        point17, point18, point19, point20 = landmark_list[5][0], landmark_list[5][1], landmark_list[5][2], landmark_list[5][3]

        # 将所有关键点的坐标存储到一起，简化后续函数的参数
        all_points = {'point0': landmark_list[0][0],
                        'point1': landmark_list[1][0], 'point2': landmark_list[1][1], 'point3': landmark_list[1][2], 'point4': landmark_list[1][3],
                        'point5': landmark_list[2][0], 'point6': landmark_list[2][1], 'point7': landmark_list[2][2], 'point8': landmark_list[2][3],
                        'point9': landmark_list[3][0], 'point10': landmark_list[3][1], 'point11': landmark_list[3][2], 'point12': landmark_list[3][3],
                        'point13': landmark_list[4][0], 'point14': landmark_list[4][1], 'point15': landmark_list[4][2], 'point16': landmark_list[4][3],
                        'point17': landmark_list[5][0], 'point18': landmark_list[5][1], 'point19': landmark_list[5][2], 'point20': landmark_list[5][3]}

        # 调用函数，判断每根手指的弯曲或伸直状态
        bend_states, straighten_states = detect_all_finger_state(all_points)

        # 调用函数，检测当前手势
        current_state = detect_hand_state(all_points, bend_states, straighten_states)

        # 更新最近状态列表
        recent_states.pop(0)
        recent_states.append(current_state)

        # 检查列表中的所有状态是否相同
        if len(set(recent_states)) == 1:        # 如果连续30帧的手势状态都相同，则认为手势稳定，输出当前手势
            print("Detected consistent hand state:", recent_states[0])
            cv2.putText(frame, current_state, (10, 70), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2, cv2.LINE_AA)


        for hand_landmarks in keypoints.multi_hand_landmarks:       # keypoints.multi_hand_landmarks是一个列表，只有一个元素，存储着21个关键点的xyz坐标。使用for循环遍历
            mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS,
                                    mp_drawing.DrawingSpec(color=(0, 255, 0), thickness=2, circle_radius=4),
                                    mp_drawing.DrawingSpec(color=(255, 0, 0), thickness=2, circle_radius=2))

    curr_time = time.time()             # 计算帧率
    fps = 1 / (curr_time - prev_time)
    prev_time = curr_time

    # 在画面上绘制帧率
    cv2.putText(frame, f"FPS: {int(fps)}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)

    cv2.imshow("Hand Detection", frame)
    if cv2.waitKey(1) == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()