ML Project

10 min readMay 21, 2023

𝑬𝒙𝒄𝒊𝒕𝒊𝒏𝒈 𝑷𝒓𝒐𝒋𝒆𝒄𝒕 : 𝑫𝒐𝒄𝒌𝒆𝒓 𝑪𝒐𝒏𝒕𝒂𝒊𝒏𝒆𝒓𝒔 𝒘𝒊𝒕𝒉 𝑭𝒊𝒏𝒈𝒆𝒓𝒔! 🖐🚀

𝘐𝘮𝘢𝘨𝘪𝘯𝘦 that launch 𝘋𝘰𝘤𝘬𝘦𝘳 𝘤𝘰𝘯𝘵𝘢𝘪𝘯𝘦𝘳𝘴 𝘸𝘪𝘵𝘩 𝘫𝘶𝘴𝘵 𝘢 𝘸𝘢𝘷𝘦 𝘰𝘧 𝘺𝘰𝘶𝘳 𝘩𝘢𝘯𝘥! 🌟

Using advanced finger detection techniques and leveraging the power of AWS services, I’ve developed a solution that allows you to scale your Docker containers on EC2 Linux instances with a simple gesture.
explain my project in brief: -
🔺Camera click the photo and detect hand landmark.
🔺 Detect the hand and give result in variables.
🔺 continue photo work as video live streaming and the result of fingers in [0,1] from.
🔺Add some features in finger detection
According to my hand gesture detection it shows user locked or unlocked.
🔺 launched docker containers with hand gesture.
🔺As i showed my fingers
its start launching OS.
With the help of aws, ec2, lambda ,ssh, and such use cases of aws services.

I'm going to build A project based on ML integration to AWS cloud for administrator work.

The problem is to scale out the docker container just by showing fingers. The fingers up indicate the number of new containers that are needed to be launched.

Launching EC2 Instance

We know, every service in AWS is an individual service.
For connecting EC2 to SSM, We need to use IAM roles.

Go to the IAM dashboard, then go to Roles using the Navigator pane on the left side.

Create a role.
Select as shown (Search for SSM and select the policy for full access to SSM).
Set the name of the Role and create.
we can the role created.
Now, Launch an EC2 instance.
I am using Amazon Linux 2 for this demonstration.
Keep the rest as defaults, just make changes like the following.
Proceed without any key pair.
The Instance is launched.
Now let’s configure SSM to manage the ec2 manually first, then we will automate it also.

Come to AWS SYSTEM MANAGER Dashboard and go to Run Command from the navigation pane.

Click Run Command. Search for shell and select AWS-RunShellScript.then Scroll down and give the command for testing.Then scroll down and select as shown below.Scroll down, keep the rest as defaults, and Click on Run.The command was run successfully by SSM.

Configure docker service in Linux server.

Note: It’s not SSM that comes to the EC2 instance and configures the instance, EC2 comes to SSM and registers itself.
Due to being two independent services, EC2 needs the power to come to SSM. So, attach the IAM role to the EC2 instance and give full access to SSM.

Now run these commands.

#first  login as root 

# install docker service
yum install docker -y

# check the status of docker service

systemctl status docker

# start the service and enable on boot
systemctl start docker
systemctl enable docker

# pull container image
docker pull ubuntu:14.04

# you can use any other image.

Create a Lambda function.

We are using Lambda as a serverless service that connects to SSM and tell to run the necessary commands.

Write the code and deploy it.

Let’s understand this code in brief.

Import the required modules ‘json’ and ‘boto3’.
Define a function ‘lambda_handler’ that takes two parameters: ‘event’ and ‘context’. ‘event’ contains information about the event that triggered the function, while ‘context’ provides runtime information about the function.
Configure Lamda as a client of SSM (Systems Manager) using the ‘boto3’ module.
ssm_client = boto3.client(‘ssm’, region_name=’ap-south-1')
Create a command to be executed on an EC2 instance using the ‘send_command’ method of the SSM client.
First define the command and put it in a variable for further usage.
cmd = { “commands”: [ “docker run -dit ubuntu:14.04” ] }
The command is like a parameter, what command you want to run, store the command in the variable,
ssm_client.send_command(DocumentName=”AWS-RunShellScript”, InstanceIds=[“i-043cd8dfb03183120”], Parameters=cmd )
Here, the command is to create a docker container from the ubuntu container image, on the EC2 instance with the ID ‘i-043cd8dfb03183120’.
Return a response with a status code of 200 and a message in JSON format saying ‘Docker Launched!’
return {
‘statusCode’: 200,
‘body’: json.dumps(‘Docker Launched!’)
}

NOTE: Don’t forget create the IAM role for Lambda as Lambda and SSM are independent services. Here, Lambda is going to trigger SSM and tell to launch the docker containers.

If one role is created by default, then attach the respective policies.

Select AmazonSSMFullAccess policy and scroll down.

♦ Configuring API GATEWAY:

I am using the same Rest API that I used on ML Project-1.

As we can’t directly connect to our lambda function from our local system.
We have to introduce API GATEWAY as their intermediate service, so that over public internet we can connect to the respective API endpoint and invoke the lambda function.

One new resource we are creating as ‘/docker’ that on hit will launch containers.

Select the Lambda that is going to get triggered and then tell SSM to launch docker containers, in my case ‘MngDocker.py’ As we have made some changes, redeploy it.Hit the /docker API endpoint and see the result!check if one docker container is launched.

see, it works really well.

Till now we are just creating containers, let’s do something more.
Now, let’s introduce the Lambda function for the deletion of containers.

Note: Don’t forget to attach policies for accessing SSM.

Create API endpoint, just like shown before and select DockerRM as the Lambda function.

Now for the API to work, use Deploy API again as have made some changes.

Now testing.

Again hit the URL at this time at /rmvcont API endpoint.

AI MODEL CREATED !

We are going to use this AI model multiple times in future.

Developing The Python Code

The main code contains a lot of tasks so it’s better to discuss the subtasks first and then go to the main code. ( concept of Recursion, isn’t it? )

First, let’s talk about how to click a photo using Python code.

Capturing an Image :-

Overall, this code captures a frame from the webcam, displays it in a window named “my photo”, and waits for the Enter key to be pressed. Once the Enter key is pressed, the program exits, and the window is closed.

Let’s break down the code step by step:

`!pip install opencv-python`: This line installs the OpenCV library, specifically the `opencv-python` package, using the `pip` package manager. This package provides the necessary functions and classes for image and video processing tasks.
From the command prompt, just use “pip install opencv-python”. Here
“!” is commonly used in Jupyter Notebook or JupyterLab environments to execute shell commands directly from within the notebook.

2. `import cv2`: This line imports the OpenCV library, which is used for computer vision tasks like image and video processing.

3. `cap = cv2.VideoCapture(0)`: This line initializes the video capture object `cap` using `cv2.VideoCapture(0)`. It opens the default video capture device (usually the webcam) with an index of 0, indicating the first available camera. `0` means the in-built camera of system.

4. `ret, photo = cap.read()`: This line captures a frame from the video capture device using the `read()` method of the `cap` object. The return value `ret` indicates whether the frame was successfully captured, and `photo` contains the captured frame.
basically `cap.read()` returns two output and for that we used twp variables.

5. `cv2.imshow(“my photo”, photo)`: This line displays the captured frame in a window named “my photo” using the `imshow()` function from OpenCV. The `photo` variable holds the image data.

6. `if cv2.waitKey() == 13:`: This line waits for a key press event. The `waitKey()` function will block the program until a key is pressed. The value returned by `waitKey()` is then compared with the ASCII value of the Enter key (13).
You can use 27 also, it is the value for `esc` button on keyboard.

7. `cv2.destroyAllWindows()`: This line closes all open windows created by OpenCV. It is called when the Enter key is pressed(in this case as 13 is written here), indicated by the previous condition.

NOTE: Dont forget to release the camera when your work is done.

while loop` is continuously clicking photos every 10ms and showing the picture, providing a smooth video live streaming!!

Hand Detection source code for Program:-

Let’s break down the code step by step:

1. `import cv2`: This line imports the OpenCV library, which is used for computer vision tasks like image and video processing.

2. `from cvzone.HandTrackingModule import HandDetector`: This line imports a specific module called `HandDetector` from the `cvzone` library(dont import all the function but one specific one). This module provides functionality for hand detection and tracking.

3. `detector = HandDetector(maxHands=1, detectionCon=0.8)`: This line creates an instance of the `HandDetector` class with two parameters: `maxHands` and `detectionCon`. `maxHands` is set to 1, which means it will track only one hand. `detectionCon` is set to 0.8, which is the confidence threshold for hand detection. 0.8 means 80% here.

4. `cap = cv2.VideoCapture(0)`: This line initializes the capture object using `cv2.VideoCapture(0)`, which opens the default video capture device (typically the webcam) for capturing video frames.

Capturing a photo, this time with one hand, let’s know if my program can detect my hand!

ret, photo=cap.read()
cv2.imshow("my photo" , photo)
cv2.waitKey()

cv2.destroyAllWindows()

We are using `detector` object to call findHands() function and storing the details in another variable that is `hand`.

`hand = detector.findHands(photo, draw=False)`
The line of code is using a hand detection algorithm to detect and locate hands in an image represented by the “photo” variable.

The line assigns the result of the detection to the “hand” variable, which likely contains details about the detected hands, such as their positions, landmarks, and other relevant information. The “draw=False” parameter indicates that the detected hands should not be visually annotated or highlighted in the image.

You can use “draw=True” if you want to see all the annotations.

For detecting hands to get progress in project: -

This code is performing some operations on the detected hand stored in the “hand” variable.

The line “d = hand[0]” retrieves the first detected hand from the “hand” variable and assigns it to the variable “d”. This line assumes that there is at least one hand detected in the image. We are going to use variable `detecthand` instead of just d for better undestanding.
(As previously, we have set `maxHands=1` while creating `detector` object from the `HandDetector` class).

The next line, “detector.fingersUp(d)”, is calling a function named “fingersUp” of the “detector” object, passing the detected hand “d” as an argument. This function is designed to determine which fingers of the hand are raised or extended.

The “fingersUp” function is specifically identifying whether the second (index 1) and third (index 2) fingers of the hand are raised.(As you can see previosuly I have captured a picture where I kept two fingers up that is 2nd and third) The function returns a Boolean value or some form of representation indicating the state of each finger (raised or not raised).
In this case, returning an array of boolean values.

This code is performing some operations on the detected hand stored in the “hand” variable.

Now coming to the final source code and by integrating the AI model with the finger detection.

That’s it from this Blog you can easily made your project successful!

For this Project I would like to specially Thanks My Mentor Mr. Vimal Daga sir for providing us this amount of incredible knowledge and to whole team of Linux World for their support !

Keep Learning, Keep Growing!