https://miro.medium.com/max/4266/1*EESGJiqpopRk3A2QHd5NUw.png

(AI encompasses almost every work arena known to us nowadays.)

Hey all! A-student here! Sorry for being late. It's mainly because I thought of discontinuing this series and starting a bit more technical one. But since you people all voted for its continuation, I'm doing so! :thumbup:

In this post, we're gonna talk about the different paradigms of using AI - with more focus on Regression. In other words, what are the different broad purposes AI is used for! NOTE: Since I've got to explain the stuff to everyone, we'll only be scratching the surface!

So let's start!

What are the different paradigms of AI?

1. Classification
2. Regression
3. Support Vector Machines (SVM)

There are more, but these are the main ones!

1. Classification:

Well, if you've read my previous posts, you all already know about classification, and that too, in detail! No, I'm not joking! :-D

If you haven't, read the previous 3 posts and then come back.

To all those who have already read my previous posts in this series: Remember the perceptron model, we built for deciding whether someone can go to the birthday party or not? Yeah, that's a classification problem with 3 parameters!

Well, classification is actually a statistical problem, So this is what Wikipedia says:

A contributor to Wikipedia wrote:

In statistics, classification is the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, based on a training set of data containing observations (or instances) whose category membership is known. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient (gender, blood pressure, presence or absence of certain symptoms, etc.). Classification is an example of pattern recognition.

2. Regression

Well, regression is a bit more advanced topic: It involves attempts to predict future outcomes based on the data you already have. Sounds a bit like wizardry, doesn't it?

Fret not, I didn't intend to write all this stuff without explaining :rotfl:

Let us consider an example dataset that contains data on the heights (H) (in cm) of different students corresponding to their body weight (W) (in kg) in the form of (H, W), which are plotted like:

(Thanks to Alcula for plotting these scatter plots.)

Here, the Y-axis represents W, while the X-axis represents H. We call W and H the parameters of our dataset.

Now, the aim of applying regression in our model dataset is to predict the height of a student given his/her weight, or the other way round. So, we need to find a curve that can fits best into this data set, such that it can predict accurately (maybe not 100%, but as close as we can get

) either of the 2 parameters, given the other parameter.

Well, for simplicity, we will consider only Linear functions for now, hence the name Linear Regression. We now try to fit a line with all points deviating as little from it as possible, without compromising on the accuracy with respect to other points - we need to always see the big picture, don't we?

So, the best-fitting line looks similar to the one below:

(Thanks to Alcula again!

)

So, how do we design such a machine that sorts out a curve fitting the dataset?

Honestly, I was waiting to start this part!

Before diving into the design part, I just want to ensure you know some stuff so that you don't get stuck.

What are the characteristics of a straight line?

I'm sure that 99.99% of all the readers of this post would know what the characteristics of a straight line are. For the remaining 0.01%, who have forgotten it, a straight line is depicted by $y=\textbf{m}x + \textbf{c}$ , where m is the slope of the line and c is the intercept made by the line. Okay, that was all!

Now coming to the design part:

So, what we actually want to adjust are the values m and c. Keeping in mind the design of the 2-input perceptron model, we keep the value of c as one input with the weight of the connecting link as 1, and $x$ as the other input, with m acting as the weight corresponding to $x$ . So our output is nothing but the weighted sum of the inputs, i.e., $m \times x + 1 \times c.$ Now, the weighted sum itself is the output y, so no need for any activation function. (Too convenient

)

We now start with some arbitrary value of m and c. But why? Well, bear with me for some time.

But we need accuracy, don't we?

So how do we measure the accuracy of the line? This is where the error function $\text{erf}$ steps in.

What is the error function?

The error function is defined as:

$\text{erf} = \sum_{i=0}^{n}(y_i - Y_i)^2$

where $n$ is the number of observations in our data set, $y_i$ is the value predicted by our model for a particular value of $X_i$ , and $Y_i$ is the actual value in the dataset corresponding to $X_i$ . Hence, it's basically about optimizing the accuracy of measuring $y_i$ given a value of $X_i$ .

Now, we need to minimize the value of the $\text{erf}$ function to maximize our accuracy. We keep adjusting the values of m and c, until we reach a minimum value, and cannot improve further. We kinda give "feedback" to the computer as to "how much error is it making."

Now, think about it for a while. Why did we need to set arbitrary values of m and c initially? Programmers (anyone out there?

) will know that it is what they call Base Case, and it has to be fed to the program. It is the starting point of the feedback loop. You tell the computer: "For this combination of values of m and c, you have these values of $y_i$ and $Y_i$ for a particular $X_i$ . Minimize the error function now by trying to find an optimal combination of m and c." It gives you each time a particular combination, and you tell it "Try improving more." This continues until you cannot get a more fitting combination of m and c.

A piece of knowledge nugget: Computers can optimize cases, but they can never generate Base cases. :-D

I hope that I was able to make the point clear.

Nerd and Enthusiasts' (N&E) Section :coool:

Hey again, my kinsmen! You've read about measuring the accuracy of the error function. So is brute force the only way to measure the accuracy of the line? Turns out, there is an elegant algorithm that helps. It's called the Gradient Descent Algorithm.

Well, so what is the Gradient Descent Algorithm?

Well, this requires a bit of knowledge about differential calculus and its applications, especially maxima and minima. Now, I cannot teach you about maxima and minima in this post itself, but Professor Dave Explains has a video on it. You can check it out!

It also needs an intuitive understanding of planes and projections of curves on planes. Sorry, but they're too lengthy to be explained in one post.

It'll be the topic of discussion of another post (most probably another series.)

Further Reading:

1. Linear Regression - Wikipedia, the free encyclopaedia

2. Saishruthi Swaminathan: Linear Regression — Detailed View - Towards Data Science

3. What is Linear Regression? - StatTrek

4. David M. Lane : Introduction to Linear Regression - Online Stat Book

That's all for today folks! Do comment and give feedback! Suggestions are welcome! :bye:

Bye for now!
A-student