bobde_yagyesh

Computer Vision In Web | Intro To OpenCV JS

Computer Vision In Web | Intro To OpenCV JS

Dive into Computer Vision with OpenCV JS. Unleash the Power of Visual Recognition and Real-Time Image Processing for Web Applications

Yagyesh Bobde's photo
Yagyesh Bobde
·

7 min read

Table of contents

In this article, we'll be covering how to create computer vision projects that you can deploy to the web (which you directly share with others and add to your resume ✨)

Project Demo

Take a look at the deployed project right here.

Prerequisites

  1. HTML, CSS, JS (familiarity)

  2. Good to have prior knowledge of OpenCV (although not required)

OpenCV Js

Starting Our Project

Let's start with building our project.

  1. Initialize Project Folder

    Create your project folder. Open it in an editor of your choice and create this basic folder structure

    a. Resources: It contains the image assets that we will use in this project as well as the opencv.js file which is the script file needed for loading the opencvjs package in our project.

    b. Template: It contains our js code and the CSS file.

    c. index.html: lastly our website page

    You can find the GitHub Repo right here to download all the necessary files.

  2. Creating The UI

    In the project demo at the top of the page, we saw the UI of our site. We'll have a header at the top, our main page where we will show our tablet image as well as the image we render after applying the transformations (on canvas).

    1. index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>VisionGraphics</title>

    <!-- Link to css -->
    <link rel="stylesheet" href="./Template/style.css">
    <!-- Material Icons Google -->
    <link href="https://fonts.googleapis.com/icon?family=Material+Icons"
    rel="stylesheet">
    <!-- Add opencv -->
    <script async src="./Resources/opencv.js" onload="openCvReady()" ></script>
</head>
<body>
    <div class="main-page">

        <!-- Right panel -->
        <div class="right-panel">
            <!-- Header -->
            <div class="header">
                <!-- <div class="empty-side"></div> -->
                <h1>Developers Zone 🚀</h1>
                <!-- <div class="logo-container">
                    <img src="/Resources/Template/Logo.png" alt="logo" width="120">
                </div> -->
            </div>


            <!-- Tablet Container -->
            <div class="tablet-container">
                <img id="table-image" src="./Resources/Template/tablet.png"
                     alt="tablet frame"
                     width="840"
                     height="480">
                <canvas id="main-canvas">
                    <img 
                    id="img-main" 
                    src="./Resources/Template/scene.jpg" 
                    alt="car image"
                    width="760"
                    height="430">
                </canvas>
                <div id="file-upload" class="file-upload" >
                    <span class="material-icons">
                        add_photo_alternate
                    </span>
                    <label for="input-file">Upload Image</label>
                    <input type="file" name="input-file" id="input-file" accept="image/jpg, image/jpeg , image/png">
                </div>
            </div>
        </div>
    </div>
        <!-- Slide bar -->
        <div class="slide-bar">
            <span id="btn-rgb"
                class="circle">
                <i class="material-icons"
                style="font-size: 36px;color: white;">palette</i>
            </span>
            <span id="btn-gray"
                class="circle">
                <i class="material-icons"
                style="font-size: 36px;color: white;">contrast</i>
            </span>
            <span id="btn-blur"
                class="circle"><i class="material-icons"
                style="font-size: 36px;color: white;">visibility</i>
            </span>
            <span id="btn-canny"
                class="circle"><i class="material-icons"
                style="font-size: 36px;color: white;">auto_fix_normal</i>
            </span>
            <!-- <span id="btn-" class="circle blue" onclick="uploadToggle()" ><i class="material-icons " style="font-size: 36px;color: white;">upload</i> -->
            </span>

        </div>

    <!-- Link JS -->
    <script src="./Template/main.js"></script>
</body>
</html>

styles.css

Now that we have a basic HTML structure of our site ready, we'll apply styling to it now.

* {
    margin: 0px; 
    box-sizing: border-box;
}

:root{ 
    --bg : white;
    --bg-slide-bar: rgba(0,0,0,0.05);
    --bg-right-panel: white;

    --bg-circle-icon: #c7d0d8;
    --box-shadow: 1px 1px 10px rgb(185, 185, 185);
}

body{ 
    height: 100vh;
    overflow: hidden;
}

.main-page {
    /* background-color: red; */
    height: 100vh;
    display: flex;
}

.slide-bar {
    position: absolute;
    bottom: 0;
    background-color: var(--bg-side-bar);
    width: 100%;  
    max-height: 15%; 
    display: flex;
    justify-content: space-between;
    padding: 5% 15%;
    align-items: center;
    box-shadow: 1px 1px 10px rgb(185, 185, 185);
}
.circle{ 
    height: 50px;
    width: 50px;
    background-color: var(--bg-circle-icon);
    border-radius: 50%;
    display: flex;
    justify-content: center;
    align-items: center;
    transition: 256ms;
}
.circle:hover, .circle:active{ 
    cursor: pointer;
    background-color: var(--bg-side-line);
}

.circle.blue{ 
    background-color: var(--bg-side-line);
}

.right-panel{ 
    background-color: var(--bg-right-panel);
    height: 100%;
    width: 100%;
    display: flex;
    flex-direction: column;
}

/* header */
.header{ 
    background-color: var(--bg-side-line);
    display : flex;
    /* justify-content: space-between; */
    justify-content: center;
    align-items: center;
    padding: 1% 5%;
    box-shadow: var(--box-shadow);
    color: white;
}

.logo-container {
        display: flex;
        justify-content: center;
        align-items: center;
}

.header h1 {
    font-family: "Cairo", sans-serif;
    font-size: 1.5rem;
    letter-spacing: 0.2rem;
}


.tablet-container{ 
    display: flex;
    justify-content: center;
    align-items: center;
    width: 100%;
    height: 80%;
}
.tablet-container #table-image{ 
    position: absolute;

}
.tablet-container canvas ,.tablet-container canvas img{ 
    z-index: 4;
}
.file-upload{ 
    z-index: 5;
    position: absolute;
    min-width: 700px;
    min-height: 500px;
    flex-direction: column;
    justify-content: center;
    align-items: center;
    border : 5px dashed #37c9ef;
    display: none;
}

.showupload{
    display: flex;
}

.file-upload span { 
    font-size: 5rem;
}
.tablet-container label , .tablet-container input{ 
    z-index: 3;
}
.tablet-container input{ 
    display: none;
}
.tablet-container label{ 
    box-shadow: var(--box-shadow);
    padding: 1% 2%;
    background-color: orangered;
    color: white;
    font-size: 2rem;
    margin: 5% 0;
    font-weight: 600;
}

So, that's it for the UI part of our website. Now it's time to integrate the logic to apply the image transformations.

  1. Handling the Computer Vision

    main.js

    If you notice in the index.html file we have given a script tag with src as opencv.js file and we have provided openCvReady() as the onload function for this script.

So only after the OpenCV module is loaded do we want to do anything else right? So let's wait for the script to load first!

And note that after the script is loaded you can refer to the OpenCV module by using the cv object.

function openCvReady() {
    cv["onRuntimeInitialized"] = () => {
        console.log("Open Cv ready")
    }
}

The next thing we want is to read our image right? For this, we'll use the imread method provided by OpenCV.

  1. cv.imread(String img_element_id) :

if you have used OpenCV module before you might notice that you enter the file path as a parameter in the imread method but in this case, you provide the id of the img HTML element and it automatically takes the image file from the src of the element.

Returns an InputArray matrix.

We'll also have to show this image right? So, we'll use another method provided by the OpenCV module called imshow

  1. cv.imshow(String element_id, InputArray mat ):

    • This method is used to show image matrix in an html element. We pass in the id of the HTML element and the image matrix (that we get from imread() method) as parameters.

Returns none;

So, let's use these two methods to display our image in the main-canvas.

// Read an image
let imgMain = cv.imread('img-main')
let inputFile = document.getElementById("input-file");
cv.imshow("main-canvas", imgMain);
imgMain.delete();

Now to apply image transformations we'll be using the following methods provided by the OpenCV module:

  1. cv.cvtColor(InputArray input, InputArray output, code, dstCn)

    Parameters

    • input: input array

    • output: output image array of the same size and depth as the input array

    • code: color space conversion code (see cv.ColorConversionCodes)

    • dstCn: number of channels in the destination image; if the parameter is 0, the number of the channels is derived automatically from src and code

  2. cv.GaussianBlur(src, dst, ksize, sigmaX)

    Parameters

    • src − A Mat object representing the source (input image) for this operation.

    • dst − A Mat object representing the destination (output image) for this operation.

    • ksize − A Size object representing the size of the kernel.

    • sigmaX − A variable of the type double representing the Gaussian kernel standard deviation in X direction.

      (source: OpenCV - Gaussian Blur (tutorialspoint.com))

  3. cv.Canny(InputArray image, OutputArray edges, threshold1, threshold2, apertureSize 3, L2gradient false)

    This method is use to find the edges in the image.

So we'll use these methods to make the buttons on our page functional. We'll have to wrap these methods inside onclick event handler like this:

        document.getElementById("btn-gray").onclick = () => {
            // Read an image
            let imgMain = cv.imread('img-main')
            let imgGray = imgMain.clone();

            // converting image to grayscale
            cv.cvtColor(imgMain, imgGray, cv.COLOR_RGBA2GRAY, 0)

            cv.imshow("main-canvas", imgGray);
            imgMain.delete(); // to free memory allocated
            imgGray.delete(); // to free memory allocated
        }

Applying all the methods and merging all the code snippets we get the final main.js as follows:

function openCvReady() {
    cv["onRuntimeInitialized"] = () => {
        console.log("Open Cv ready")
        // Read an image
        let imgMain = cv.imread('img-main')
        let inputFile = document.getElementById("input-file");
        cv.imshow("main-canvas", imgMain);
        imgMain.delete(); 



        document.getElementById("btn-rgb").onclick = () => {
            // Read an image
            let imgMain = cv.imread('img-main')
            cv.imshow("main-canvas", imgMain);
            imgMain.delete(); // to free memory allocated
        }

        document.getElementById("btn-gray").onclick = () => {
            // Read an image
            let imgMain = cv.imread('img-main')
            let imgGray = imgMain.clone();

            // converting image to grayscale
            cv.cvtColor(imgMain, imgGray, cv.COLOR_RGBA2GRAY, 0)

            cv.imshow("main-canvas", imgGray);
            imgMain.delete(); // to free memory allocated
            imgGray.delete(); // to free memory allocated
        }

        document.getElementById("btn-blur").onclick = () => {
            // Read an image
            let imgMain = cv.imread('img-main')
            let imgBlur = imgMain.clone();

            // adding blur
            let ksize = new cv.Size(49 , 49);
            cv.GaussianBlur(imgMain, imgBlur, ksize, 0)

            cv.imshow("main-canvas", imgBlur);
            imgMain.delete(); // to free memory allocated
            imgBlur.delete(); // to free memory allocated
        }

        document.getElementById("btn-canny").onclick = () => {
            // Read an image
            let imgMain = cv.imread('img-main')
            let imgCanny = imgMain.clone();

            // adding blur
            cv.Canny(imgMain, imgCanny, 50, 100)

            cv.imshow("main-canvas", imgCanny);
            imgMain.delete(); // to free memory allocated
            imgCanny.delete(); // to free memory allocated
        }
    }
}

Thats It! You are now successfully able to use the OpenCv module inside your web applications.

What next?

Try out new stuff like handling video objects and using models to detect objects on real cam stream etc.

Keep Learning & Keep Building

See you guys 👋

Computer VisionopencvWeb DevelopmentMachine LearningDeep Learning