Object Detection

Can you identify the actor in this image?

Yes, he is none other than Tom Cruise who has acted in many movies like the Mission Impossible series (Does he ever age?!).

But how were you able to identify him from the image? And how can a computer identify him? While we identified him using our eyes and brain (and the fact that he doesn’t seem to age), the computer uses computer vision to identify celebrity information from the image.

How? You will get the answer in this lesson!

Topic Covered in the Lesson

Human vision
Computer vision and its applications
How self-driving car works
Artificial Intelligence extension in PictoBlox
Computer vision blocks in PictoBlox

Key Learning Outcomes

At the end of the lesson, you will be able to:

Use Artificial Intelligence blocks in PictoBlox to identify the following in images:
1. Brand
2. Celebrity
3. Objects
4. Image Tags
5. Image Description

The following example shows a project that identifies the object in the image, i.e. a person, who that person is, i.e. a celebrity, his name, what he’s wearing, and the brand his clothes belong to.

Make AI projects in PictoBlox using computer vision.
Understand how computer vision works.
Understand the application of computer vision in various industries.

Let’s begin!

How Human Vision Works?

To understand computer vision, first, we must look at how human vision works.

Capture image: Humans capture images using their eyes. The image captured is formed on the retina which is similar to how the camera captures the image but in a very raw format.
Identify the objects and their features: The raw image is then transferred to the brain via the optical nerves for processing. The brain starts to identify different objects like a candle, human, chair, and many others along with their features such as size, color, shape, and others.
Extract information: In this step, our brain compares the features of the object to its past knowledge to gather information. E.g., it can differentiate between your father and your mother because you can distinguish their visual features.
Act: Once you get the higher-level information, you can start acting on it. E.g., if you can identify that a ball is coming to your face, then you can move aside to avoid hitting the ball.

All these steps happen at a very fast pace due to the perfection of the human eye and brain coordination.

Computer Vision

Computer vision deals with how computers can be made to gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human vision can do.

Computer vision also follows a similar approach as human vision.

Computer vision tasks include methods for acquiring, processing, analyzing, and understanding digital images.

Example: Self Driving Car Using Vision

A self-driving car is a vehicle that is capable of sensing its environment and moving safely with little or no human input.

For this example, let us consider that a self-driving car can go forward, turn left, right, or stop. Let’s see how the car would react if a pedestrian comes in front of it.

Acquire: Self-driving cars use cameras to acquire images. They acquire and process images are at a very high rate. Let’s consider that our camera has acquired this image:
Process: The computer starts to identify all the objects in the image and make a list of the objects with their position. In this case, there is something on the road. The computer still has no information about what object it is.
Analyze: The computer then classifies each object into different categories. In this case, it identifies the object as a girl. It also tags some information to the object as harmfulness, distance, and other parameters. These tags are the higher-level information used to make a decision.
Act: Based on the higher-level information the computer can act. In this case, the car will stop.

recognized () () () block

In feature recognition, you can get the location and other parameters of the recognized celebrity, brand, and object using the recognized () () () block. You can get the following parameters using this block:

x position: Reports the x position of the identified object.
y position: Reports the y position of the identified object.
width: Reports the width of the identified object.
height: Reports the height of the identified object.
confidence: Reports the confidence of the identified object. 0 is less likely and 1 is more likely.

Activity: Locating Objects

Let’s make a script that makes a bounding box on the identified object.

We’ll follow the following process:

Identifying the objects from the image.
Running the script in a loop for each object.
Making the bounding box for each object based on its X and Y location.

Let’s begin!

Setting Up the Stage

Open a new project in PictoBlox.
Select evive as your board from the Board tab on the menu bar.
Click the Add Extension button in the bottom left corner.
A modal will open will all the available extensions. Select the Artificial Intelligence extension from the library.
Download the image from here: https://learn.thestempedia.com/wp-content/uploads/2020/04/Kids-and-Bus.jpg
Upload the image as a backdrop.
Add a new sprite named Box from the sprite library:
Delete Tobi, select the Box sprite, and switch to the Code tab.
Add a when flag clicked block in the scripting area.
Snap a hide block from the Looks palette.
Add a recognize () in image from () block and select image features and stage as inputs.

Locating Objects on the Stage

We’ll continue with the same script. Make a variable named Object.
Add a set () to () block. Change the variable to Object and set the value as 0.
Add a repeat until () block. Drop an () = () block in the condition.
In the first input, add a recognized () count block and select object from the drop-down. In the second input, add the Object variable.
Inside the loop, add a change () by () block from the Variables palette. Change the variable to Object.
Snap a create clone of () block from the Control palette and select myself from the drop-down. Selecting myself means that you want to clone the same sprite you are writing the script for.
Add a wait () seconds block in the loop. With this, our main script is ready.

Drawing the Bounding Box

Follow the steps below for drawing the bounding box:

Add a when I start as a clone block into the scripting area from the Control palette.
Snap a set size to ()% block below the when I start as a clone block.
Add a recognized () () () block as the input of the set size to ()% block. Change the type to object and option to width. Next, add the Object variable as the input of the object number.
Next, add a set x to () block and a set y to () block and repeat step 3 for both the blocks as shown below:
Snap a show block from the Looks palette.
Add a say () block below the show block.
Add two join () () blocks.
Display the object name and object confidence using the say block as shown below.

Your project is complete! Click the green flag to run the script.

Explore: Add other backdrops from the internet and try to identify objects.

Application of Computer Vision

Computer Vision is used for various purposes in a wide range of industries such as healthcare, agriculture, insurance, automotive industries, etc.

Self Driving Cars: We have already had a look at self-driving cars and how computer vision helps them navigate.

Food Industries: Computer vision is used to identify food items and segregate the items using robots. Below is one example.
Agriculture: Computer vision is used in agriculture for identifying crop diseases and crop production. E.g., in this image, CV is used to calculate the grape production.