An introduction to deep learning

Search engine technologies have evolved quite a lot since their beginnings in the 90s. Modern search engines are a lot more than keyword searches or website rankings; they take into account factors like user location, search history, and other factors to give results that users want. Present-day search engines can understand context and are capable of analyzing different forms of media. And they place a lot of emphasis on credibility and trustworthiness in their search results. The AI revolution and the leaps we’ve made with Deep Learning over the last two decades are the secret sauce leading these advances in search engine technology.

What is deep learning?

Deep learning is an approach to artificial intelligence that attempts to mimic how human brains work. Just like a human brain, an artificial neural network uses neural networks to make decisions. These neural networks are made of artificial neurons connected to each other and form different layers. If a neural network has more than three layers, it is referred to as a deep learning system.

Like the human brain, you can picture deep learning solutions as prediction engines. They use many different inputs to predict what the output would be like. For example, you can use neural network learning to predict weather conditions. Here it may consider factors like atmospheric pressure, today’s weather, humidity, or weather radar data. The neural network learns to understand how the different factors interact with each other and affect the final outcome.

Deep learning can be either supervised or unsupervised. In supervised learning, the input data, along with the actual output, is used to train the system.

In the case of unsupervised learning, the training data is presented without the expected output. Here the system figures out the actual outputs based on patterns in the training data.

Deep learning systems are behind almost every AI solution in existence. Search engines like Google (or Sinequa) use them to understand user queries and website content to offer better results.

How does deep learning work?

To understand how deep learning works, you have to understand the structure of a neural network.

Structure of a neural network

A neural network is composed of many layers of artificial neurons. Here, the first layer of neurons would collect the input, and the last layer would show the output. The neurons in the first layer would be connected to the ones in the second layer. This means that the output from the neurons in the first layer would serve as inputs for the neurons in the second layer. A single neural network could have as many layers as needed. If a neural network has more than three layers, it is referred to as a deep learning system.

Weights and connections in a neural network

The intelligence of a deep learning system is within the “strength” of connections between these neurons. The “strengths” of these connections are referred to as weights. We can consider the weights to represent the “importance” of a factor when making the decision.

For example, imagine a system trying to distinguish between cats and birds. Here the presence of wings would make it more likely for the picture to be that of a bird. But the presence of two eyes is not likely to be a distinguishing feature. So the connection between the neuron “has wings,” and the neuron “is a bird” would be strong. And the connection between the neuron “has two eyes,” and the neuron “is a bird” would be weak.

In an artificial neural network, the weights multiply the output from a neuron. In the above example, the neuron that detects wings would connect both to the neuron that says the image is that of a bird and the one that says it’s a cat. But the weight between the first pairs would be higher than the weight between the second pair. Let’s say that the neuron that detects wings produces an output of 5. If the weight between the first pair is 3, then the neuron that classifies the image as a bird gets an input of 15 (5 times 3).

Training a neural network

Like a human brain, a deep learning system learns from the data it gets. This process is called training, where we feed the system with both the input and the output until it learns to generate the correct output. For example, imagine we are developing a deep learning system for identifying objects in an image. Here, we will have to train the neural network learning system with hundreds or thousands of pictures with the objects tagged correctly.

During the training process, the weights are constantly adjusted until the system produces the expected output. The training is finished when the system can make the right decision for all the training data with an acceptable rate of error.

Keep in mind that all the examples we’ve used are oversimplified. In reality, different layers of a deep learning system usually perform different tasks. For instance, the first couple of layers may identify the edges of a picture, and then the next layers may identify the objects, and so on.

Deep learning vs. machine learning

Put simply, deep learning is a type of machine learning. Machine learning is an approach to artificial intelligence in which the system learns to do something from examples. The difference between machine learning and deep learning is mostly about the scale; that of the neural network, as well as that of the problem it’s solving.

For machine learning, the neural network has a limited number of layers, while there’s no maximum number of layers for deep learning. Deep learning also needs a larger data set for training compared to machine learning. IBM Watson, the ML system that won Jeopardy!, was trained on just 25,000 questions, while AlexNet, a Deep Learning system for image recognition, used a training set of around 1.2 million images.

The difference in the size of the training set can be partially attributed to the preprocessing that the data goes through before a machine learning system takes it as input. Machine learning systems typically cannot accept raw data as input.

But in the case of deep learning, it can take the raw image – jpg or PNG – and process it directly. In deep learning, the first couple of layers would learn to process the data into a simpler form, and the next layers take this as input. There’s no preprocessing when you use deep learning.

Applications of deep learning

The advances in deep learning over the last couple of years have made it suitable for many applications. Deep learning has become more mature and, at the same time, more accessible, encouraging more industries to adopt the tech. Here are a couple of applications of deep learning.

Healthcare and Life Sciences

The applications of deep learning in healthcare and life sciences are in the nascent stage. Many possible applications of the technology have been and are being explored. For example, IBM Watson developed deep learning models using X-ray data and EMR records to diagnose breast cancer. At the onset of the COVID-19 crisis, there were several attempts to use deep learning for diagnosing the disease from CT scan images, and the results were promising. According to the latest data, there are around 64 FDA-approved medical devices that use ML or deep learning. As technology improves, we can expect more medical devices and tech powered by deep learning to get to the market.

Manufacturing

Predictive maintenance is one of the most commonly deployed deep learning solutions in manufacturing industries. These solutions use neural networks to analyze vibration, sound, heat, and other data from equipment to predict when they may need maintenance.

Many automobile manufacturers use machine vision systems for quality control. These systems can identify manufacturing defects better and faster than the human eye. Porsche uses a sound analysis solution based on a deep learning model which can detect noise or unexpected sounds while testing vehicle components.

Designers and engineers are also using deep learning solutions to completely or partially automate the design process in many industries.

Aerospace and defense

While autopilot systems have been popular in the aerospace segment for quite a while, they’re getting an upgrade with deep learning. Most of the partially or completely autonomous aircraft under development is powered by deep learning models. The leaders of unmanned aircraft have all made significant investments in deep learning.

Besides flying itself, the aerospace and defense industry is using artificial intelligence systems for operations behind the scenes. They’re used to identify inefficiencies in their processes, conduct inspections, deliver a better experience to the end user, and many other applications.

Law and consulting firms

Law is not the first thing that comes to mind when you talk about the applications of deep learning. But if you look at it, research is a large part of practicing law; lawyers often have to go through vast amounts of documents and materials to prepare for their cases. They also have to research the laws and regulations pertaining to their case and look for precedents that support their clients.

Deep learning systems can help power this research. These systems can go through vast amounts of documents and annotate the important sections. Lawyers can quickly find out what they need and accelerate their research with these tools.

Finance

With its ability to analyze huge amounts of data and detect patterns, deep learning has found uses in detecting fraud and money laundering operations. They’re also often used for credit scoring and even for deciding if an applicant can be granted a loan. While researchers have raised ethical concerns about some of these applications, there definitely are benefits to using deep learning in finance.