What is Computer Vision?

Object classification:

The ability to find or classify an object in visual data. At the most basic level, this would include the computer being able to identify the contents of an image—such as trees, the ocean, or a dog.

Object localization:

Object localization is the ability to locate a single object within visual data. For example, if you have a picture of a dog, the computer could identify and locate the dog within the image.

Object detection:

Object detection is a more advanced form of Computer Vision technology. Object detection capabilities allow computers to accurately identify and locate where multiple objects are within visual data. For example, imagine you have an image of both a dog and a cat together. With object detection, the computer could differentiate between the two objects and also accurately identify and locate both.

How does computer vision work?

Computer Vision works off of pattern recognition. It uses algorithms to scan an immense amount of visual data and make sense of it.

Training for Computer Vision involves exposing computers to massive amounts of visual data so they can learn to distinguish key features and patterns. For example—to teach a computer to identify boats, data scientists must feed it images of boats, as well as related objects. Over time, the computer learns to distinguish what is and isn’t a boat, as well as the key features of a boat.

Once the computer accurately identifies what an image is, it can then segment, detect, and classify it for various applications. Through advancements in AI, Computer Vision technology is as good as—and even oftentimes superior than—their human counterparts at image recognition.

Email security

Computer Vision enhances email security by analyzing images and identifying features typically used in phishing attacks like brand logos, QR codes, and suspicious copy. Image manipulation—a tactic hackers use where they slightly alter a company’s logo in order to bypass blacklists—is an increasing threat to organizations. Computer Vision provides an added layer of security to defend against these highly sophisticated email phishing attacks by recognizing image manipulation, even in remote images.

Facial recognition technology

Computer Vision powers software that analyzes a person’s facial features to verify their identity. While the use cases for facial recognition software are rapidly expanding, some popular ones include—security checkpoints such as airports and country borders, criminal suspect identification, and smartphone login.

Self-driving cars

Computer Vision is at the crux of self-driving technology. Self-driving cars like Tesla use object classification, identification, and localization to safely maneuver the vehicle with minimal need for human intervention.

Traffic light and pedestrian detection

Traffic-light detection is used to optimize the flow of traffic. The technology relies on digital sensors to recognize key traffic moments—cars waiting for the light to change, when emergency vehicles are nearby, or even when a car runs a red-light. Similarly, pedestrian detection is used to identify pedestrians at crosswalks and various points in the road.

Medical imaging (X-Rays, CT Scans, etc.)

Radiologists and other medical specialists rely on Computer Vision to help them more accurately interpret medical imaging. The AI model helps reveal abnormalities not easily detected by the human eye, vastly improving accuracy and even prognosis.

In 2019, the cybersecurity community turned towards Computer Vision technology to help with security threat detection. Using Machine Learning, researchers were able to create a program that could detect malware based on the images associated with it. Overtime, this AI technology evolved to include the detection of phishing websites, emails, and more.

At a high-level, Computer Vision works to identify cybersecurity threats by using binary visualization to identify detectable patterns. Binary visualization allows the software to detect patterns that are malicious or safe. Malicious files tend to include either ASCII characters or more vibrant imagery.

Traditionally, protecting against malicious websites involved blacklisting suspicious domains. Computer Vision enhances cybersecurity by detecting malware from images sent via phishing emails. This adds a layer of protection before the recipient clicks on the domain that leaves them susceptible to attacks.

Using Computer Vision technology, email security filters continually scans images found within emails to identify malicious content. Cybercriminals use three main techniques:

Image manipulation:

This tactic uses slightly altered images to evade email security solutions using fingerprint technology.

Text-based images:

This advanced phishing technique uses images that contain text that needs to be extracted with OCR (Optical Character Recognition) to bypass security analysis.

Remotely hosted images:

Remote images, images hosted elsewhere online, make it easier to sidestep email security since they’re often hosted on reputable sites like GitHub, making detecting malware more difficult.

What is Computer Vision?