Are you ready to start your machine learning (ML) journey? In this course, I will
introduce you to this revolutionary field. With a focus on _"math light"_ explainers,
hands-on exercises and real-world examples, our self-paced approach will give you
a solid foundation in the core concepts of modern ML.

> We will focus on insights and practical tips so that you can apply what you learn immediately in your own projects.

**In this course we will cover:**

- An overview of what machine learning is (and what it isn't), and the different types of tasks machine learning can effectively solve.
- The Machine Learning workflow: from preparing data and training your model to deploying and validating it in the wild.
- Detailed demos of fundamental machine learning algorithms for you to try in your own projects.

**We‘ll work on the basis that you’re pretty new to Python**, but have some basic understanding of fundamental programming concepts and can run code locally on your machine.


### Section 1: Starting Blocks - An Introduction to Machine Learning

In this first section, we introduce the world of machine learning and consider:

- what are the core concepts in machine learning?
- what are machine learning algorithms trying to do?

We will work through some high profile examples and talk about what's going on under the hood. We will focus on understanding the different types of task machine learning can accomplish, and then introduce you to the first two ML algorithms!

<Image
	src={"/images/course-info/foundations-in-machine-learning/knn-explainer.jpg"}
	alt={"K Nearest Neighbour - digiLab Academy"}
	width={800}
	height={1071}
	caption={"Fig 1. A mind-map of k-Nearest Neighbour - one of our first models."}
/>

### Section 2: We're Off - Understanding the Machine Learning Workflow

Most machine learning projects have a natural flow. We want to give you a structure so that when you we start building machine learning models, the steps we take become familiar and second nature.

So, here we will walk you, step-by-step through the early data preparation and feature selection stages. We will then talk about model choice, training and tuning. Finally, we will focus on model validation and, ultimately, how you might deploy these models.

Are you ready to start your machine learning (ML) journey? This course will give you an accessible entry point into this revolutionary field. When you complete this course, you’ll understand core concepts in machine learning and, fundamentally, what are machine learning algorithms trying to do. With a focus on 'math light' explainers, hands-on exercises and real-world examples, our self-paced approach will give you a solid foundation in the core concepts of modern ML.

Prof Tim Dodwell

In this explainer, we focus on a high-level view of Machine Learning as a field and set of technologies. We explore why there is been an explosion in Machine Learning, how we already use machine learning in our everyday lives, and how it will help us push the frontiers in engineering and science. We then explore:

what machine learning is and how there are different classes of learning based on the task required.
we explore the key idea of generalisation as our understanding of "intelligence".
we look at different considerations we need to take when choosing which machine learning algorithms we might use in our machine learning for flow. Big data, sparse data, a large number of inputs or outputs or the amount of trust needed in the answer.
Finally, I present a few personal thoughts on how machine learning should be learnt.

This lecture is intended to be a presentation, with me as a talking head! Below are a set of summary notes for reference.

## Why Machine Learning?

The world has been witnessing an explosion in data generation, with the amount of data increasing by 50 folds between 2010 and 2020, and the total amount estimated to be around 44 ZB today. This phenomenon has been driven by several factors, including the availability of cheap data storage, the power of modern computing devices, and the proliferation of data-generating devices, ranging from sensors to remote systems. In this context, Machine Learning (ML) has emerged as a crucial field for extracting value, insights, and intelligence from this vast pool of information. By finding patterns in data, ML can enable us to analyze and understand complex phenomena, such as user behaviour, consumer preferences, market trends, disease outbreaks, and climate change. Therefore, the growing need for ML is aligned with the need for businesses, governments, and individuals to make sense of this exponential growth of data, and leverage it for innovation, productivity, and social impact.

## You use Machine Learning every day

Machine Learning isn't a new thing. It has been well integrated into many of the tools we use every data. Here are some examples

- Search Engines (e.g. Google)
- Recommender Systems (e.g. Netflix & Amazon)
- Speech Understanding (e.g. Siri, Alexa)
- Face Recognition
- Large Language Models (e.g. ChatGPT)
- Self Driving Cars

These are really all cool examples of ML. At digiLab, we are also excited by the opportunities ML has at the frontiers of Engineering & Science from fusion energy, to automated agriculture to medical diagnosis. There are very few areas not being touched by the advances in machine learning.

## What is Machine Learning?

When we talk about Machine Learning, it is important to understand that it is not just a single algorithm or workflow. Rather, it refers to a set of technologies that enable machines to learn from data and subsequently perform specific tasks. Machine Learning workflows essentially work as a pipeline that takes in data, trains on it, and then deploys the trained model for specific purposes.

An important or central concept in machine learning is the concept of intelligence in a machine learning algorithm. For me, the "intelligence" we assign is the ability of a Machine Learning algorithm or workflow to perform well on previously unseen tasks. This core concept is called Generalisation.

If an algorithm is just reproducing information it has been given, it is really nothing more than a rich look-up table. The fact that patterns, correlations and connections can be made and then use to bring insight into new cases is central to our understanding of intelligence as humans.

## Supervised, Unsupervised & Reinforcement Learning

There are different approaches for different machine learning tasks

They are often not used in isolation, and so a single machine learning workflow may bring together both unsupervised and supervised learning algorithms. This is really common.

Find the patterns in the data to simplify the data representation (unsupervised learning), and from this slimline representation fit a model which makes a good prediction from labelled examples (supervised learning).

Ok so I mention them here so what are the different types of learning

**Supervised machine learning** is a type of learning where an algorithm is trained using a labellabelledd dataset. In other words, the input data is already tagged with the correct output, allowing the algorithm to learn the mapping between the input and output variables. For instance, if we want to build an algorithm to predict the price of a house based on its size and location, we would provide a dataset containing examples of houses, their sizes, locations, and their corresponding prices. During the training process, the algorithm will learn the relationship between these features and the price and use this knowledge to make predictions on new, unseen data.

On the other hand, **unsupervised machine learning** is used when the input data is not labelled or categorized. Instead, the algorithm is tasked with finding patterns or structures in the data on its own. This is particularly useful in cases where there is no clear objective or target variable. An example of unsupervised learning is clustering, where the algorithm groups data points together based on their similarity. Another example is dimensionality reduction, where the algorithm reduces the number of input features while preserving the most important ones. Unsupervised learning is often used in exploratory data analysis or in cases where the data is too complex to be easily labelled.

Lastly, **reinforcement learning** is a type of learning that relies on an agent interacting with an environment and receiving feedback in the form of rewards or punishments. The agent learns by trial and error, and its goal is to find a policy that maximizes the cumulative reward over time. Reinforcement learning has been successfully used in areas such as game playing, robotics, and autonomous driving.

## Different Amounts of Types of Data

A central consideration in what methods we deploy in our ML workflow is how much data we have. For companies like Amazon for example capture an huge amount of data about how customers interact with their services. Their machine learning challenges are really about distilling patterns in these huge data sets, and because the data is so rich much less structure needs to be imposed on the methods they use. They allow free to have very flexible approaches, which are then sufficiently constrained by data.

In other applications, data can be very expensive to obtain. A good example that digiLab knows well is the outputs of mathematical simulations for Fusion, where a single simulation can take as long as a week on a national supercomputer (amazing). In this case, because data is limited the machine learning methods chosen need to be robust to limited data.

## The Curse of Dimensionality

In simple terms, the Curse of Dimensionality refers to the fact that as the number of features (or dimensions) in a dataset increases, the amount of data required to cover the feature space increases exponentially. This means that as the dimensions increase (models with large numbers of inputs and parameters) it becomes increasingly difficult to find patterns in the data. In other words, most points in a high-dimensional space are very far apart from each other - much further than we would expect in lower-dimensional spaces.

Suppose we have a one-dimensional input space, which takes values between 0 and 1. We ask for three equally spaced data points so that the distance between each data point is just 0.20.2. In two dimensions, the number of data points required to keep the same distance apart is 3^2=9 points.

In much higher dimensions, say 100 (not a big problem for ML) then the number of samples I need to retain a distance of 0.2 between each point is 3^100=5.15e47. This is an eye-watering amount of training data, even for the above example where the dimension of the input space is still relatively small.

Let's think about a common machine-learning problem, in which images are classified. For example, the classic question, "Is this picture, a picture of a Cat?" is a classic supervised (classification) problem. Take for example a picture made up of 640 by 480 pixels, in which each pixel is represented by 3 colour values - Red, Green, and Blue (RGB).

This gives a total input dimension of 640×480×3=921,600 values.

A central consideration in machine learning is how we can reduce the dimension of this input space, but still retain the maximum amount of information we need about the system.

## Do the Outputs Matter?

This is a central consideration when thinking about "how good a model has to be?" and influences are decision in

An important consideration in machine learning is to contemplate if the outputs of a model matter. For instance, if a recommender system suggests an unsuitable item for purchase, it may not matter much as the algorithm can learn and recommend a better item the next time.

However, in high-stakes applications (e.g. water quality control, cancer diagnosis, or drone control) the outputs can have serious consequences. In such cases, it is imperative for the application to provide not only predictions but also a measure of uncertainty or risk associated with the predictions. This can aid decision-makers in making informed and responsible choices by weighing the potential risks against the benefits.

If uncertainty measures are needed then this plays a key role in terms of choosing which algorithms are appropriate.

## Thoughts on Learning ML

When it comes to learning Machine Learning, there are a few things to keep in mind. First, it's important not to take a bottom-up approach, as this can lead to a lack of direction and understanding of the big picture. Instead, diving into learning on a specific problem allows for more effective learning, as each problem will require different types of machine learning algorithms and approaches.

Second, focus on understanding the big concepts of approaches before diving into the details. This approach is more effective, as the details can often become overwhelming, but understanding the larger concepts can provide a framework for tackling the details.

To learn Machine Learning, it's important to play around with the math and programming concepts by experimenting with different algorithms and approaches. This hands-on approach is a great way to learn through experience and to see how different techniques affect the performance of models.

It's also important to keep in mind that Machine Learning is a fast-moving and ever-growing field, so staying up to date with the latest developments and techniques is crucial for success.

In conclusion, learning Machine Learning is an ongoing process that requires an open mind, a willingness to learn, and a dedication to staying up to date with the latest developments in the field. By keeping these tips in mind, anyone can become an expert in Machine Learning and apply it to a variety of real-world problems.


Machine Learning 101 : Why, What and How?

In this quick video explainer we walk you through the outline of our first section of the course.

The primary learning outcomes are

- Understand High Level Concepts behind Machine Learning.

- Learn about some exciting areas where ML is having a meaningful impact in Science, Engineering & Technology.

- Understand the different types of methods and tasks that are performed using machine learning algorithms.

- Introduce you to your first supervised and unsupervised learning algorithms : K Nearest Neighbour and K Means

- Walk through your first python examples, with some real world data sets.


Section Overview

This explainer begins by contrasting supervised and unsupervised learning algorithms, but they're not in competition. The key takeaway for aspiring data scientists is that the choice between a supervised or unsupervised learning approach hinges on the specific task.

Frequently in a machine learning workflow, from raw data to the deployed model, both unsupervised and supervised learning models are utilised together.

## Supervised Learning: Learning from labeled data

So supervised learning is about learning from labeled data. We have inputs $X$ and associated outputs or target variables $y$ and we want to build an algorithm that mimics this process, for which it performs well on training data, but most importantly on unseen data. This idea of building a machine that can predict outcomes in new unseen situations or configurations is central to the concept of generalisation.

There are two principal supervised machine learning techniques: regression and classification algorithms. It's not that one is better than the other; rather, they are different types of supervised learning algorithms for different types of data. It all comes down to what the output data or variable looks like.

If the data is a class or a label, then the supervised learning algorithm is called a classification algorithm. Whilst if the output of the prediction is a number - aka a real or continuous value (e.g. temperature, stock price) then the supervised learning task is called regression.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img1.jpg"
	}
	alt={"digiLab.co.uk - Classification and Regression Methods"}
	width={1000}
	height={281}
	caption={
		"Figure 1. Examples of supervised learning task (LEFT) a machine learns how to identify a cat from labeled data of pictures of cats and dogs. supervised classification task. on the right is a response surface or curve fitting, an example of supervised regression."
	}
/>

A random forest, k nearest neighbour, deep neural network, support vector machine, linear regression, or logistic regression are all good examples of supervised learning algorithms.

The performance of supervised learning algorithms is typically easy to access, an algorithm that accurately predicts the output variables for unseen inputs can be computed and benchmarked against other approaches.

Let's take a bit of a deep dive and look at three supervised learning algorithms

### k Nearest Neighbour (aka kNN)

kNN is a really nice and conceptually simple supervised learning algorithm. Which can be used for both regression and classification tasks. In this little toy example, we are going to look at a classification problem.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img2.jpg"
	}
	alt={"digiLab.co.uk - KNN"}
	width={640}
	height={368}
	caption={"Figure 2. Simple toy representation of KNN"}
/>

In this picture what we have is a 2D space which represents the space of possible. data. For different inputs (locations in this space) each sample has been assigned a label. For some green triangles, for others yellow squares.

The method proceeds in a really simple way. First for a new location (not seen by the data before) - here marked X "New Instance", we look for the k (here k = 3) nearest neighbours to this point. We then make a prediction on the class label, by taking a majority vote, dependent on the class labels of the neighbours. Here in this pictured toy example, we see that 2 of the 3 neighbours are green, and with that, the supervised machine learning algorithm predicts a "green triangle" label for the previously unseen data.

In general, kNN is a really simple algorithm and a great starter algorithm to learn on your machine learning journey!

## Curve Fitting or Response Surfaces

Curve fitting is often not seen as a machine learning algorithm by itself. However, it is at the core of supervised machine learning models. For a response surface with simple constructions, such as the composition of polynomials, these models are often easy to train. They are naturally suited to regression-based tasks, in which they predict output data that is a real or continuous value. However, they can also be extended to classification algorithms, for example, by using logistic regression.

## Deep Neural Networks

Deep neural networks (DNNs) are a type of machine learning model. DNNs are based on the concept of artificial neurons — connected layers of nodes that mimic how nerve cells in the brain interact and process information. The connections between these nodes can be “trained” using a machine learning workflow.

The training process uses known inputs and outputs to adjust the weights of connections between nodes so that when presented with unseen data, the model can make accurate predictions. DNNs enable computers to learn complex patterns from data and make decisions based on these learned patterns. As such, DNNs are widely used in many areas of AI research.

Traditionally deep neural networks are used for supervised machine learning tasks, however, there are clear exceptions to the rule. Autoencoders are a widely used unsupervised learning model, whilst reinforcement learning is also dominated by DNNs. DNNs play a central role in modern machine learning.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img3.jpg"
	}
	alt={"digiLab.co.uk - Deep Learning"}
	width={1000}
	height={540}
	caption={
		"Figure 3. Diagram shows a visual demonstration of a deep neural network, applied to an image recognition problem of images, with our classic problem 'is it a cat?'"
	}
/>

## Unsupervised Learning: Finding Hidden Patterns

Unlike supervised learning, unsupervised learning uses unlabeled data points, and therefore only uses input data. Its purpose is to extract hidden patterns or inherent structure, which is often found in large data sets. By drawing out these patterns, unsupervised machine learning can assist in bringing new insights from data, or help fashion data representations that are more efficient and suitable for subsequently building supervised approaches.

Just like supervised learning, unsupervised learning algorithms are split into two primary classes.

### Unsupervised learning clustering algorithms.

Clustering algorithms are a set approach that takes unlabeled data and collects them into groups, with the aim that samples within each group are "alike". This allows data scientists to draw insights from the data sets, build better models (potentially by considering each cluster in turn) or can be used to guide sample collection - see semi-supervised learning.

There are various clustering algorithms, the one we briefly consider here is k-means. In k means, data is placed into the cluster whose centroid (average point) is closest. Here the k refers to the number of clusters and must be determined as part of the tuning process. It is one of the simplified unsupervised learning models and is a great one to know.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img4.jpg"
	}
	alt={"digiLab.co.uk - Unsupervised learning clustering algorithms"}
	width={650}
	height={523}
	caption={
		"Figure 4. Representation of k-means. splitting input data into three independent clusters. input data is assigned to the cluster whose centroid is close to the data point."
	}
/>

## Unsupervised Learning for Dimensionality Reduction

So a key challenge in machine learning is the curse of dimensionality - as the size or scale of the input data (the dimension) increases then the data requirements of the algorithms grow exponentially. Therefore a key component of building machine learning models is design methods, which reduce the dimension of the input data, so what is retained is reduced to the important or salient features which are sufficient to still make good predictions or decisions.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img5.jpg"
	}
	alt={"digiLab.co.uk - Dimension reduction"}
	width={470}
	height={340}
	caption={
		"Figure 5. Demonstration of the principle of dimension reduction, the simplification of reduction of input data into a salient feature space (shown here in the center in green). This reduced dimension allows much more efficient machine learning models to be built."
	}
/>

There are various methods for dimension reduction for feature selection, for example, some can be done as part of a supervised learning model (e.g. LASSO - look it up). Here we have a quick peek at Principle Component Analysis (aka PCA).

### Principal Component Analysis (aka PCA)

Let's first look at a simple picture and then say how it relates to PCA.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img6.jpg"
	}
	alt={"digiLab.co.uk - Principle Component Explainer"}
	width={590}
	height={515}
	caption={
		"Figure 6. Visualisation of principal components in a simple 2d synthetic data set."
	}
/>

What we see here is the principle component of a data set in 2-dimension. The principle components are a set of order vectors here denoted $v_1$ and $v_2$ in this example. The first direction (or vector) in the data aligns with the direction which drives the most variance in the data, the second direction here denoted $v_2$ is orthogonal (at right angles) to the first direction with the second largest variability in the data.

As we have only shown a 2D example, there are only two principal directions. The idea here is that if we want to reduce the model to a single dimension, we project (or collapse into) a single direction. Then the first principle component ($v_1$) retains the most variability in our data, therefore in an unsupervised setting (without knowing anything about our task) is a good way to reduce the dimension of data.

Importantly the PCA automatically gives the relative importance of each of the principle components, this is a measure called explained variance. With this, users can quantify how much of the "variance" in the data set is lost if a truncation to a smaller dimensional space is used.

## Semi-supervised learning

Semi-supervised learning is a type of Machine Learning workflow that uses both labeled and unlabeled data for training. It takes advantage of the large amounts of relatively inexpensive unlabeled data available, as well as the accuracy of models trained on labeled data, in order to improve the overall performance of the model.

By combining these two datasets, semi-supervised learning is able to provide accurate predictions with less data than traditional supervised learning methods. Semi-supervised learning can be used in a wide variety of applications, from text classification and image recognition to anomaly detection and natural language processing.

In addition, it has been successfully applied to deep neural networks for improved accuracy and speed. Ultimately, semi-supervised learning offers the potential for higher accuracy and faster training times than traditional supervised learning on similarly sized datasets. This makes it a powerful tool for data scientists who need to quickly create accurate models with limited resources.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img7.jpg"
	}
	alt={"digiLab.co.uk - Semi-supervised learning"}
	width={600}
	height={348}
	caption={
		"Figure 7. Demonstrates the core principle of semi-supervised learning. large amounts of unlabelled data are used to supplement limited labelled training data to build more data-efficient methods."
	}
/>

Let's look at it in a bit more detail, through a very simple pictorial example in which we seek to classify data. On the left, we see a labeled dataset, with labels from two different classes. In a purely supervised learning set, the best we can do is separate the input space, by placing a simple decision boundary halfway between each sample. Can we do better?

Suppose we also have lots of unlabelled data, how can we use it to make a better classification model? The idea in semi-supervised learning is shown on the right. We cluster unlabelled data with the cluster data, then by proxy (in terms of cluster) assign the same label (a pseudo-label) to the unlabelled examples. With this unlabelled data, we can now build a richer decision boundary, which we hope addresses the limited labelled data challenges, demonstrated in the setup shown on the left.

## Reinforcement Learning

Last but not least we branch out and look at a different type of learning which is increasingly getting more and more traction.

> As an aside, I write up this introductory explainer article on the day I met [Richard Sutton](http://incompleteideas.net/) whilst attending [CDL super session](https://creativedestructionlab.com/super-session/) in Toronto. If you don't know his work, then look him up. If you want an accessible route into the details of Reinforcement Learning, his book, [Sutton & Barto](http://incompleteideas.net/book/the-book-2nd.html) and the associated material, is great!

Let me first tell you about my favourite application of reinforcement learning.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img8.jpg"
	}
	alt={"digiLab.co.uk - Reinforcement Learning"}
	width={1000}
	height={559}
	caption={"Figure 8. My favourite example of RL"}
/>

So what are you looking at? You are looking at a new creature, a sort of two-legged giraffe. In this study, reinforcement learning methods are used to iteratively work out how the animal should control their muscles in a coordinated way to move forward whilst remaining stable and upright. The learning algorithm learns by experience, learning a "policy" - a strategy by which to act given observations of an environment and a sequence of positive and negative feedback. Over time, here 900 generations of policies, the animal learns how to effectively walk.

In a general setting machine learning, learns by reinforcing behaviour which leads to good outcomes. It more strongly aligns with how humans (particularly children) learn to do new tasks without directly supervised demonstration (although this can accelerate learning).

The reinforcement learning process involves a loop between an 'Agent' and the 'Environment' within which it operates. This loop consists of three cyclical components. First, the 'State,' which is the observable condition of the environment from the agent's perspective. Next, the 'Action,' which represents the choices the agent can make or the actions it takes. Finally, the 'Reward,' which is the feedback or outcome the agent receives based on its action.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img9.jpg"
	}
	alt={"digiLab.co.uk - Introduction to Reinforcement Learning"}
	width={1000}
	height={347}
	caption={
		"Figure 9. The general setup of a single agent reinforcement learning group. pacman a playground for building and testing new RL methods."
	}
/>

Innovation in Reinforcement Learning has primarily revolved around games. Not only are games enjoyable, but their rules, actions, and configurations are also well-defined, making them a productive platform for exploring new methods.

As Richard Sutton showed, we could write a book on these methods (and more!) Here we touch on the two primary approaches to reinforcement learning. Just like supervised learning and unsupervised learning, it is not either/or, and often the good approaches use them in combination.

### Value-Based Methods

The general principle around value-based methods is for an agent to take actions that move the system to the highest expected realizable value. The machine learning algorithm works by learning a model which maps from a "State" and "Action" pair to an expected value, a number.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img10.jpg"
	}
	alt={"digiLab.co.uk - value base learning"}
	width={1000}
	height={347}
	caption={
		"Figure 10. Value-based learning seeks to understand the value of taking an action in a particular observed state."
	}
/>

The value of taking an action, when in a state is calculated by experience. What is the expected future reward outcome if an action is made? We often follow this approach as humans. Here is a simple example in the final step of the noughts and crosses game.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img11.jpg"
	}
	alt={"digiLab.co.uk - value based learning example"}
	width={1000}
	height={347}
	caption={
		"Figure 11. Simple example of the final move of noughts and crosses to demo value-based learning."
	}
/>

In this case, we look at the decision of the one but last move of a "cross". The approach tries to assign a value to each of the actions, given a current state. We see that two actions lead to an equal possible chance of a win, whereas one move eliminates the chance of a win completely. This makes our action clear in this simple setting.

Value-based learning methods extend these principles to much more complex situations. To take a start in this space, look up Q-learning.

### Policy Based Methods

A policy is a mapping from "State" to "Action", what action should I do in a given state. Policies are learned to not minimise a loss function as in supervised learning, but instead to maximise the expected reward. Therefore many machine learning tools, as for Value-based learning, can be used in policy base RL methods; and often the most popular are deep neural networks (see above).

<Image
	src={
		"/images/courses/foundations-in-machine-learning/supervised-vs-unsupervised-learning-explainer/img12.jpg"
	}
	alt={"digiLab.co.uk - Policy Based Learning"}
	width={500}
	height={335}
	caption={
		"Figure 12. Policy based methods are about understanding the mapping from a state to action for an agent."
	}
/>

## Concluding Points

We take a fly-by tour of the core principles or ideas behind supervised and unsupervised learning. In no way is this a complete thesis, but lays the high-level landscape, so you can understand the difference in fundamental tasks that machine learning models can carry out, and what you might do with labeled and unlabeled datasets in your business or studies.

From this, we then take one step further and show that machine learning is not just this simple divide between supervised vs unsupervised learning, but of course, is more complex. This includes hybrid methods, called semi-supervised learning; which bring together for particular settings of a data set.

Secondly, we take a quick look at reinforcement learning, a field of machine learning where algorithms learn through experience; an exciting "new" area which is getting significant momentum in building new systems which display "intelligence" in operating or controlling complex tasks.


Supervised Vs Unsupervised Learning - Explainer

## K - Nearest Neighbours - Your first supervised Learning Algorithm

<Callout type='success' heading={null}>
	k-Nearest neighbours (kNN) is a simple non-parametric, supervised learning
	algorithm in machine learning.

In this explainer, we will:

- Explain the basic idea behind kNN
- Unpack what we mean by non-parametric and supervised learning.
- Introduce a simple example to explain the intuition behind KNN.
- Understanding a more general concept of "distance”.
- Understand what the 'k' stands for in kNN and how you might choose an appropriate value.

</Callout>

## The basics: kNN as a non-parametric, supervised learning algorithm.

Firstly, kNN is a supervised learning algorithm, this means that we have labelled data. For this discussion, we will call our input data

$$
{\bf X} := \{{\bf x}^{(0)}, {\bf x}^{(1)}, {\bf x}^{(2)}, \ldots {\bf x}^{(N)}\}
$$

and our associated target data

$$
{\bf y} := \{y^{(0)}, y^{(1)}, y^{(2)}, \ldots, y^{(N)}\}.
$$

In this dataset, we have pairs of input values and target values so that $(x^{(j)}, y^{(j)})$.

The challenge set out in supervised learning is, given a new unseen input $\tilde {\bf x}$, can we make a prediction of the target value $\tilde y$.

In kNN, the idea is very simple; we find the $k$ nearest points to ${\tilde x}$ which belong to our training data set $X$. We then use the average target values of those selected points as our prediction for the new point $\tilde y$.

<Callout type='info' heading={null}>
kNN is described as a non-parametric method. I remember being told this at first and thinking this is odd since it seems like $k$, the number of neighbours I choose to average over, is a parameter and I have to select it. More on $k$ in a minute. But what non-parametric means is that there are no underlying assumptions (parameterisations) of the data. kNN is therefore particularly effective if

1. If you have lots of data and in some sense have good "coverage" of your possible input space.
2. You have a poor understanding of the underlying distribution of your data; the data is easily interpretable and/or there are complex non-linear relations in your data which cannot be easily parameterised.

</Callout>

We will discuss how you pick your $k$ in a minute but first let's look at a picture to get a better idea of what we mean.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-explainer/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1200}
	height={516}
	caption={""}
/>

Here we have a training data set, which represented by blue dots (in the left figure).

We then want to make a prediction for input located at the green dot (right picture). In this example we see that the yellows are the $k = 5$ nearest neighbours.

Now a nice thing about kNN is we can use it for both **regression** and **classification** task. So let's look how this might work

### Regression

There are two simple variations of kNN - these are **average** vs **weighted average**. These are two variations of how predictions are made. The simple version is that the known target values for the k nearest neighbours are simply averaged. Each point contributes equally.

So here in this toy example above, the value at the green point $\tilde y$ is equal average of it's 5 neighbours. If the value at the 5 neigbours are $\hat y_1, \hat y_2, \ldots, \hat y_5$

$$
\tilde y = \frac{1}{5}\left(\hat y_1 + \hat y_2 + \ldots + \hat y_5 \right) = \frac{1}{5}\sum_{i=1}^5\hat y_i
$$



An alternative is we weight contributions, where more contributions are made by those training data points which are "closer" to.

So if we say $d_i$ denotes the distance between the $i$th closest neighbour and the new point, we can define the weight for the $i$th neighbour

$$
w_i = \frac{\exp(-d_i)}{\sum_{j=1}^k\exp(-d_j)}
$$


so then the preditions is made as follows

$$
\tilde y = \frac{1}{5}\left(w_1\hat y_1 + w_2\hat y_2 + \ldots + w_k\hat y_k \right) = \frac{1}{5}\sum_{i=1}^kw_i\hat y_i
$$


When you have a densely populated dataset, then the distance from an unseen point with it's neighbours will be almost equal. Hence the difference between these two approaches diminishes as a fuction of amount of data and k (number of nearest neighbours).

<Callout type="info" heading={null}>
	In general I would default to the weight distance, whilst a bit more to
	calculate, you will see that the packages that implement kNN do this for you,
	and the extra costs of including the weights is minimal.
</Callout>

### Classification

Classification is nice and easy, once you have found your $k$ nearest neighbours, you can do majority voting. What do we mean by this. So you will have got $k$ labels $y_i$ from your the neighnest neighbours. You tally up the number of each of the labels in that group, and the predicts is the one with the most "votes" for that label.

<Callout type="info" heading={null}>
	In the case where you had equal numbers of votes for a label, you could
	randomly sample, from the labels with equal weights.
</Callout>

There are more complex versions. Just like the regression task, the vote could be weighted by the distance from the point. This would work in a similar way. The "vote" count for each label would be the sum of the weights $w_i$ of all neighbours with the sample label. Then majority voting is applied in the same way.

### The good, the bad and the ugly.

It is clear to see that there are some **advantages** and **disadvantges** of kNN.

| Advantages                                                                                                                                                                                        | Disadvantages                                                                                                                                                                                                       |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Very Simple**. Easy to implement and easy to understand. Therefore as long as your data set isn't huge, then worth a try, before going more complex.                                            | A simple implementation of kNN would be very slow for large data sets. This is because a simple implementation requires calculating the distance between each pair of samples.                                      |
| It has a level of **identifiability** to it, in that the algorithm points to examples in a existing data set which inform the predicted value. These training points can be natural interograted. | The approach requires **significant about of data**, to make good predictions there needs to sufficient data, how much is problem dependent since it really depends how smooth the target value is being predicted. |

## What do we mean by distance?

Central to the knn algorithm is to find the distance between the new points $\tilde x$ and each of the existing training points. For a bit of notation we might write $d(\tilde x, x^{(j)})$.

<Callout type="info" heading={null}>
	In general when we say distance, we think of physical distance. Yet that only
	really makes sense when we are comparing physical coordinates. For example how
	close is the label 'cat' to 'dog' for example, our traditional understand. In
	mathematics this introduces a whole feel of understanding distances - which are
	called 'metrics' and importand concept in machine learning is this idea of
	'metric learning' - where by you find the best way to measure distance to give
	you the best model. We will talk about a simple example of this in a minute.
</Callout>

So here are the three most common measures of distance

1. **Euclidean** Distance. This is our traditional measure of distance between two points and is calculated

   $$
   d(\tilde {\bf x}, {\bf x}) = \sqrt{\sum_{i=0}^m (\tilde  x_i - x_i)^2}
   $$

   

2. **Manhattan Distance**: This is the distance between real vectors using the sum of their absolute difference, and so

   $$
   d(\tilde {\bf x}, {\bf x}) = \sum_{i=0}^m |\tilde  x_i - x_i |
   $$

  

3. **Hamming Distance**, which is used for discrete categorical variables. If two variables are the same the distance is zero, else 1.

   $$
   d(\tilde {\bf x}, {\bf x}) =
   \begin{cases}
   0 \quad \rm{if} \quad \tilde {\bf x} \equiv {\bf x}\\
   1 \quad \rm else
   \end{cases}
   $$

   
## How to choose k?

So the final piece of the jigsaw is how to choose the valye of k. A small value of k means that the algorithm is more sensitive to outliters and noise in the data, whilst a larger value of k, has a smoothing effect averaging noise out of a large number of data points, but with that reducing accuracy. The better value (not I don't say optimal) are a compromise.

The optimal value k is often determined through cross-valuidation (their others e.g. the elbow methods).

So how does **cross-validation** work?

The idea with cross-validation is to estimate the expected error - which we will call $e_k$. This can be done by dropping out a sample $({\bf x}^{(j)}, y^{(j)})$ from the training set, and then use kNN to predict the target value which $\tilde y$. The process is the repeated over all samples times, and we can then take the mean (or expected) error.

$$
e_k = \frac{1}{N}\sum_{i=1}^M |\tilde y^{(i)} -  y^{(i)}|.
$$

Note that we introduce a subscript to $e_k$ which denotes the choice in value of $k$.



<Callout type="info" heading={null}>
	If the data set is large often the error is approximated by randomly sample
	only a few samples, rather than estimating the error over the whole data set.
</Callout>

We can repeat this process of different values of k, so we would typically get a plot like the following (which I have knicked from the full example below).

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-explainer/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={540}
	height={374}
	caption={""}
/>

What we see is for a very low value of k (suppose $k=1$), the model overfits on the training data, which leads to a high error rate on the validation set. On the other hand, for a high value of k, the model performs poorly underfitting. If you observe closely, the validation error curve reaches a minima at a value of k = 15. This value of k is the optimum value of the model (but will clearly vary for different datasets or test/train splits of the same data set).

In the example below we can use a simple grid search to find a good value of k, which we show how to apply in the example below.

## Conclusions

One of the main advantages of the kNN algorithm is its simplicity. The algorithm is easy to implement and understand, and it does not require any complex mathematical operations. Additionally, it is a lazy learning algorithm, which means that it does not require any training data to be stored in memory. Instead, the algorithm only needs to store the training data and then use it to make predictions on new data.

However, there are also some limitations to the KNN algorithm. The algorithm can be computationally expensive when dealing with large datasets, as it needs to calculate the distance between the test point and all the points in the training set. Additionally, the algorithm can be sensitive to the scale of the features, and it may not perform well when dealing with high-dimensional data.


K-Nearest Neighbour Explainer

In this explainer we are going to look at a first example of using k Nearest Neighbour. This will give you a basic idea of how kNN is used and also a simple way to determine a good value of $k$.

### Add Libraries

As normal we start by adding some standard libraries for various functions we will use generally

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
```

## A dataset of sea snails!

Ok so important stuff, can we predict the age (equivalent to number of rings) of a sea shell based on various simple measurements.

Let us first download the dataset and get moving.

```python
url = (
    "https://archive.ics.uci.edu/ml/machine-learning-databases"
    "/abalone/abalone.data"
)
abalone = pd.read_csv(url, header = None)

abalone.columns = [
    'Sex',
    'Length',
    'Diameter',
    'Height',
    'Whole weight',
    'Shucked weight',
    'Viscera weight',
    'Shell weight',
    'Rings',
]

abalone = abalone.drop("Sex", axis = 1)
```

We dropped "Sex" as it is a categorical measurement, where all others are simple numbers. By dropping it, it makes life easier!

Ok, so let's look at our data

```python
abalone.head()
```

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-walkthrough/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={212}
	caption={""}
/>

```python
abalone.describe().T
```

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-walkthrough/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={800}
	height={273}
	caption={""}
/>

```python
X = abalone.drop("Rings", axis = 1)
y = abalone["Rings"]
```

We first want to rescale our data, we do this using an inbuilt frunction in sklearn. This transforms all variables to range of 0 to 1. It is vital in many ML algorithms, otherwise the inputs and their contribution to predictions are no handled fairly.

```python
from sklearn.preprocessing import scale

X_scaled = scale(X)
```

### Validation : Train / Test Split

So we are going to want to train our model, and then test it against held out data. Here we again use sklearn's inbuild functions, holding out 20% of the data for testing.

```python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, train_size=0.2, random_state=42)
```



## Fitting a Model

Now we are are in shape to fit a model. First we do this for a single given $k$ value, 10.

```python
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_squared_error

k = 10

knn_model = KNeighborsRegressor(n_neighbors=k).fit(X_train, y_train)

y_pred = knn_model.predict(X_test)

rmse = np.sqrt(mean_squared_error(y_test, y_pred))

print(rmse)
```



We see here the average root mean squared error is $2.34$ rings / years out.

So let's look at a validation plot over the whole of the testing set and see how we did.

```python
import matplotlib.pyplot as plt

plt.figure(figsize=(6,4))
plt.scatter(y_pred, y_test, c='cornflowerblue', alpha = 0.1)

perfect_line = np.linspace(3,23,2)

plt.plot(perfect_line, perfect_line, '-k', alpha=0.6)

plt.title('Model Validation for k = 10')
plt.xlim([2,19])
plt.xlabel("Predicted")
plt.ylabel("True")
```



<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-walkthrough/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={738}
	caption={""}
/>

You will note some banding in this, this is because the number of rings is an interger number rather, than a continous number. Hence in this case a kNN classifier might yield better results, but we wont worry here as we are primarily looking at the general principles.

# Choosing $k$

So now we have built the model for a single value of $k$, how do we explore the optimal value of $k$?

We now build models for all values of $k$ from $1$ all the way up to $125$

```python
k = np.arange(1,125, 1)

mse = []

for i, n_neigh in enumerate(k):

    knn_model = KNeighborsRegressor(n_neighbors=n_neigh).fit(X_train, y_train)

    y_pred = knn_model.predict(X_test)

    mse.append(mean_squared_error(y_test, y_pred))

optimal_k = k[np.argmin(mse)]

print(optimal_k)
```


The optimal is $k = 18$ in this study.

Let us also plot out all the others.

```python
plt.figure(figsize=(6,4))
plt.plot(k, mse, '-b', alpha = 0.6)
plt.scatter(optimal_k, np.min(mse), c='g')
plt.xlabel('Number of Neighbours')
plt.ylabel('Mean Square Error')
plt.legend()
```



<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-nearest-neighbour-walkthrough/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={700}
	caption={""}
/>


K-Nearest Neighbour Walkthrough

<Callout type='success' heading={null}>
k-Means is a simple and popular unsupervised learning algorithm in machine learning.

In this explainer, we will:

- Explain the basic idea behind k-Means as an unsupervised, clustering algorithm.
- Give a basic overview of how k-Means algorithms are trained.
- Tell you an approach for picking the number of clusters $k$.

</Callout>

We have introduced the idea of unsupervised learning. So as a short recap this is when we don't have labelled data we simply have a data set

$$
X := \{{\bf x}^{(1)}, {\bf x}^{(2)}\ldots, {\bf x}^{(N)}\}.
$$

Here $N$ is the total samples and each sample ${\bf x}^{(j)}$ is a D-dimensional vector.

In unsupervised learning we want to understand the patterns in this data. This can (primarily) mean one of two things:

1. **Dimension Reduction** - This is when we realise that actually there are correlations between each of the components that make ${\bf x}^{(j)}$, and hence we can seek methods to reduce the dimension of each data point, with little compromise to the total information in the dataset. This is often called **dimension reduction** or **feature extraction**. It is a fundamental part of building good machine learning models.

2. **Clustering** - is where we seek to naturally group samples from the data set into subsets or clusters with similar properties. There are various nice applications for clustering, one which we look at in this course called image segmentation.

So your first unsupervised learning algorithm is going to be K-means. This is a clustering algorithm.

From this we are seeking to partition the dataset into $K$ clusters, so using a bit of set notation

$$
X = X_0 \cup X_2 \cup \ldots \cup X_K \quad \text{where} \quad X_i \cap X_j = \emptyset.
$$

Which basically says the union of all the subgroups is the total set, and no two sets intersect. That is a sample can only be assigned to a single cluster (in maths interect of two clusters is the empty set).

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-means-explainer/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1200}
	height={517}
	caption={""}
/>

## Setting up the Mathematical Notation

So for the maths we need a nice way to say which set a sample belongs. To do this we introduce a flag $r_{ik}$ this tells us if the data point $i$ lives in cluster $k$. If data point $i$ does live in cluser $k$ then $r_{ik} = 1$ otherwise it is zero. We can write a bit of maths for this

$$
r_{ik} =
\begin{cases}
1 \quad \text{when} \quad {\bf x}^{(i)} \in X_{k} \\
0 \quad \text{otherwise}
\end{cases}
$$

Since a single data point only lives in a single cluster then if $r_{ik^\star} = 1$ then $r_{ik} = 0$ for all other clusters $k$ not equal to $k^\star$. This is the same statement as above, where we say the interect of any two clusters is zero.

Let's look at a picture first

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-means-explainer/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1200}
	height={598}
	caption={""}
/>

We can think of $r_{ik}$ as a matrix (a table of numbers) above - with $N$ rows, one per sample, and $K$ coloumns, one for each sample. Here we look at an example with $7$ samples and $K=3$ clusters. On the left, we see that each row, has a single value $1$. The coloumn in which the $1$ sits, indicates the cluster the sample belongs to. This is shown for one example that $r_{43} = 1$ but $r_{41} = r_{42} = 0$ which tells me samples ${\bf x}_4 \in X_3$ (belongs to the third cluster).

The picture on the right, shows me we can calculate the number of samples in each cluster by adding up all values in a column. We do this by summing up cover all rows. So here for cluster (or column) 3, we have $N_3$ the number in the third cluster is given by

$$
N_3 = r_{13} + r_{23} + \ldots + r_{73} = \sum_{i=1}^7r_{i3} = 3
$$

So basically we can pick out components of each cluster by summing over coloums.

For each subset or cluster we can introduce this concept of a **prototype**, **typical** or **mean** vector ${\boldsymbol \mu}_k$, where

$$
{\boldsymbol \mu}_k = \frac{\sum_{i=1}^{N} r_{ik}{\bf x}_i}{\sum_{i=1}^{N}r_{ik}}
$$



<Callout type="info" heading={null}>
	Does this make sense? If in doubt write it down for the simple example shown in
	the picture.
</Callout>

## What makes a good clustering?

So now we have written down some maths and definitions, so we can define belowing to a cluster. What is a good clustering?

So here is the idea behind k-means. Conceptally we might say that the best cluster minimises (reduces as much as possible) the distance of samples in a cluster from it's mean.

So we can write down an objective function for this

$$
\mathcal J = \sum_{i=1}^N\left[\sum_{k=1}^Kr_{ik} \|{\bf x}_i - {\boldsymbol \mu}_k\|^2\right].
$$



This objective function is also referred to as the **Inertia**. We see that we sum over all data points and all clusters.

Wouldn't really happen, but if every sample cluster was the same as it's mean then this would be zero.

<Callout type='info' heading={null}>
We could of actually waited to find out what the correct choice for $\boldsymbol{\mu}_k$. The mean come out by minimising the objective $\mathcal J$ for a fixed set assignment to clusters ($r_{ik}$).

For this reason the algorithm is called $K$-means.

</Callout>

## Training a K - Means Model

Here we won't go into too much detail about how we might train a K-means model. But the aim of the game is to set the values of $r_{ik}$ which minimise the inertia $\mathcal J$.

We notice that for a given $\boldsymbol{\mu}_k$ the objective ($\mathcal J$) is linear with respect to $r_{ik}$. The objective is then minimised by assigning the data point $i$ to the cluster $k$ for which it has minimum distance from the sample ${\bf x}_i$ to the mean $\boldsymbol{\mu}_k$.

There are lots of packages which will do K-means for you, but if you would like to know a little recipe for how it would be implemented as a simple version here you go

1. We set the number of clusters $K$

2. We randomly assign each data point to a cluster, i.e. we assign initial values for $r_{ik}$.

3. We start a loop until $\mathcal J$ the objective does not improve sufficiently. In this loop we
   a. First calculate the mean vectors $\boldsymbol{\mu}_k$
   b. We loop through each data point and calculate it's distance to each of the cluster means $\boldsymbol{\mu}_k$.
   c. We reassign data points to clusters for which their distance to the mean is smallest, i.e. we update $r_{ik}$.
   d. We recalculate $\mathcal J$.

<Callout type='info' heading={null}>
**Extra Challenges**

- Can you make a basic implementation yourself - even testing it against the example below.
- What do you think the computational cost of the algorithm is, i.e. where is all the effort? How might you speed it up?
- How do you pick K? More on that below.

</Callout>

## How to pick the number of clusters $K$

In the simple example that I drew above, it was clear that the number of clustered we wanted was three. In general though, in an abstract data set it might not be clear how many data sets a model should be clustered into.

### A Simple Approach to Determining $K$ - Elbow Method

A good k-means model is one with a low inertia (or objective function), but also has a low number of clusters. This is clearly a trade of as larger clusters will naturally reduce the error, all the way to zero when the number of clusters are actually the number of data points.

So let's look at a plot of inertia against number of clusters for this problem.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/k-means-explainer/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={550}
	height={445}
	caption={""}
/>

The first approach for picking $K$ is called the **elbow method** what does this do?

The elbow basically describes the transition in the gradient of the plot we have just plotted. What we are looking for is when the improvement in inertia is minimal for the increase in complexity. For our example we see this really is between 5 - 7 clusters. So let's go with $6$.

## Conclusions

k-means is a simple and often effective clustering algorithm. As an initial step clustering let's us explore if there are groups in our data, informing us how we might build a supervised machine learning model, or even how we should collect lables (e.g. semi-supervised learning). More over it can be used in computer vision tasks, such as image segmentation, and althogh not shown here compression.

A challenge often is to determine the number of clusters, and often this takes some engineering judgement, in which we plot inertia against number of clusters, and look for the "elbow".


K-Means Explainer

:::success
**📂 Resources**
Download the resources for this lesson <Link href="/resources/getting-started-in-machine-learning/trying-out-clustering-with-k-means/Resources.zip" download="Resources.zip">here.</Link>
:::

## Trying out Clustering with K-Means

<Callout type="success" heading={null}>
	In this walkthrough we can are going to explore K-means on a crime data set
	across all US states. Which states have similar crime profiles?
</Callout>

### Import Libraries

```python
import numpy as np
import pandas as pd

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

import matplotlib.pyplot as plt
%matplotlib inline
```



### Import Data Set and have a look

Let us have a look at the data.

```python
df = pd.read_csv("data_arrest.csv", delimiter=",", index_col=0)

df.head()
```



<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={600}
	height={251}
	caption={""}
/>

Rescaling data is important when using K means. This is because the different inputs can have different scales. Hence if not rescaled some variables will have a greater influence than others.

Here we use the normal `MinMaxScaler()` which rescales the data from 0 to 1 for each input variable.

```python
from sklearn.preprocessing import MinMaxScaler

minmax = MinMaxScaler()

hat_df = crime_rates_standardized = pd.DataFrame(minmax.fit_transform(df))
```


### Clustering with K Means

Here we now fit a K Means model, and we try out different values of K (the number of clusters). In each case we can calculate the inertia (the total distance from the data to the centroid of the cluster it is assigned).

```python
from sklearn.cluster import KMeans

KMeans()
plt.figure(figsize=(8, 6))
inertia = []
for i in range(1, 11):
    kmeans = KMeans(n_clusters = i, init = 'k-means++', n_init='auto',random_state = 123).fit(hat_df)

    inertia.append(kmeans.inertia_) # criterion based on which K-means clustering works
```



We can then plot the 'elbow' curve and see a good choice of number of clusters

```python
plt.plot(range(1, 11), wcss)
plt.title('The Elbow Method')
plt.xlabel('Number of clusters')
plt.ylabel('Inertia')
plt.show()
```



<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={680}
	height={542}
	caption={""}
/>

Looking at the elbow curve we see that K = 4 seems a good choice. So let us fit that model again, and look at the results.

```python
model = KMeans(n_clusters = 4, init = 'k-means++', n_init = 'auto', random_state = 42)

clusters = model.fit_predict(hat_df)

scatter = plt.scatter(df['Murder'], df['Assault'], c=clusters)

plt.xlabel('Murders Rates')
plt.ylabel('Assaults Rates')
plt.show()
```



There seems like a clustering which divided by the amount of crime. We see a clear correlation between murder rates and assault rates (no really surprise here). The clustering is clear visible along this relationship.

Now let plot this data set on a map. We do this using plotly which you can install using

`pip install plotly-express`

First we need to convert state names into the abbrevation of two letters, and then add them to the data set along with the assigned cluster numbers.

```python
import plotly.express as px # You can install this using `pip install plotly-express`

us_state_to_abbrev = {
    "Alabama": "AL",
    "Alaska": "AK",
    "Arizona": "AZ",
    "Arkansas": "AR",
    "California": "CA",
    "Colorado": "CO",
    "Connecticut": "CT",
    "Delaware": "DE",
    "Florida": "FL",
    "Georgia": "GA",
    "Hawaii": "HI",
    "Idaho": "ID",
    "Illinois": "IL",
    "Indiana": "IN",
    "Iowa": "IA",
    "Kansas": "KS",
    "Kentucky": "KY",
    "Louisiana": "LA",
    "Maine": "ME",
    "Maryland": "MD",
    "Massachusetts": "MA",
    "Michigan": "MI",
    "Minnesota": "MN",
    "Mississippi": "MS",
    "Missouri": "MO",
    "Montana": "MT",
    "Nebraska": "NE",
    "Nevada": "NV",
    "New Hampshire": "NH",
    "New Jersey": "NJ",
    "New Mexico": "NM",
    "New York": "NY",
    "North Carolina": "NC",
    "North Dakota": "ND",
    "Ohio": "OH",
    "Oklahoma": "OK",
    "Oregon": "OR",
    "Pennsylvania": "PA",
    "Rhode Island": "RI",
    "South Carolina": "SC",
    "South Dakota": "SD",
    "Tennessee": "TN",
    "Texas": "TX",
    "Utah": "UT",
    "Vermont": "VT",
    "Virginia": "VA",
    "Washington": "WA",
    "West Virginia": "WV",
    "Wisconsin": "WI",
    "Wyoming": "WY",
    "District of Columbia": "DC",
    "American Samoa": "AS",
    "Guam": "GU",
    "Northern Mariana Islands": "MP",
    "Puerto Rico": "PR",
    "United States Minor Outlying Islands": "UM",
    "U.S. Virgin Islands": "VI",
}

abbrev_state = []
for state in df.index:
    abbrev_state.append(us_state_to_abbrev[state])

df['state_code'] = abbrev_state
df['clusters'] = clusters

df.head()
```


<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={800}
	height={227}
	caption={""}
/>

We can now plot this map using `plotly-express`

```python
fig = px.choropleth(df,
                    locations='state_code',
                    locationmode="USA-states",
                    scope="usa",
                    color='clusters',
                    color_continuous_scale="Viridis_r",
                    )
fig.show()
```



<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={627}
	caption={""}
/>


Trying out Clustering with K-Means

In this section, you will not only learn about the components of the machine learning workflow but also how they relate to each other and how to apply them in practice. We'll cover topics such as data preprocessing, feature engineering and selection, model selection and evaluation, and finally deployment of models in production. By the end of this section, you will have a complete understanding of the machine learning workflow.
Understanding the Machine Learning Workflow

Whilst different machine learning algorithms and tasks may require different data, questions, and deployments, it is important to remain flexible in your approach. However, as a newcomer to machine learning, it can be helpful to have a clear idea of the workflow so you know "what is next?"

Below is a simple diagram which outlines a typical machine learning workflow or pipeline, and one that we follow regularly through out this course.

- **Data preparation**: The first step in a machine learning workflow is data preparation. Data from the real world can often be noisy, incomplete, or in different formats and must be cleaned in order for it to be suitable for use with a machine learning algorithm. Although not always glamorous, data preparation is an essential part of any AI project and having good processes in place to store and clean data will make you a more effective AI practitioner.

- **Feature Representation**: An important aspect of machine learning is finding the most efficient way to represent data for a particular task. For example, when trying to identify pictures of cats, it would take a huge amount of data if we were to analyze the image pixel-by-pixel. A better approach would be to use unsupervised machine learning algorithms to detect salient features (such as pointy ears) which are common to all cats. Feature representation is a crucial part of the machine learning workflow.

- **Model Selection**: In practice, it is not often the case that you know exactly what method to use on a dataset. Although there are some broad principles around which methods might work best for certain problems, in reality many different types of models and structures must be tested before making a final decision. This process is referred to as model selection.

- **Training Process**: The training process is where the calculations take place. During this phase, models are adjusted to fit the training data and performance is evaluated. This process takes advantage of advanced optimization techniques, but luckily there are many accessible tools that can be used to leverage these methods without needing to understand their inner workings.

- **Validation**: The final part of the traditional machine learning workflow involves validating the model's performance. This is done by setting aside a portion of the training data and testing how well the model can generalize to this unseen data. Depending on the application, it's important to take a close look at your model during this step and identify any areas where improvements can be made. This will help ensure that your model is ready to be deployed with confidence.

This isn't a linear process. At any stage, building a machine learning model requires you to loop back. Try a different model, tune parameters to adjust training, collect more data or explore different approach to representing inputs.

In this section we take a close look at each of these 5 parts, focusing particularly on training, loss functions and training curves.


<Callout type="success" heading={null}>
	In this explainer, we delve into the machine learning process. We'll focus on
	the pre-training steps involved in building a good machine learning model: Data
	Preparation, Data Representation and Model Selection. We aim to give you an
	overview of each step so that you can understand what to expect when creating
	your own machine learning models. We'll use examples in later sections and
	courses to illustrate each step.
</Callout>

This explainer is really a talk, so below we now summarise the key takeaway points.

## Data Preparation for Machine Learning

**Data preparation** is a huge part of any machine learning project, and at digiLab we estimate it can take up to 60% of a project's time if you're using new data. This makes it essential for us to develop efficient data workflows that allow us to quickly and accurately clean, organize, and use relevant data during our machine learning projects. This is the only way to ensure we can reliably and quickly make use of data in our models.

In this explainer, we summarise different types of data issues and talk about high-level strategies for resolving them. These include

- **Non-alignment of Data** - This is really common in multi-channel time series data. The step involves syncing data to the same timestamps and naturally involves a process called interpolation.

- **Missing or Partial Data** - A really common problem, I've seen anything from blanks, "-99999" to "gone home" in my datasets. How do we deal with this in a sensible way? In a way this can be a machine learning task itself, so here we are thinking about

- **Noisy Data or Outliers** - When is data noisy, rather than an outlier? Any real-world data is noisy. Lots of techniques available the best known are filtering-based methods.

**Imbalanced Data** - This is a really common problem in classification tasks when looking at rare events, for example, pictures of skin cancer or buildings falling down. These causes are a really big problem, we talk through techniques for handling this by both over and under-sampling.

## Starting with the end in mind

It's important to remember that building good data needs to be done right from the start of the project. Many issues related to data preparation can be resolved if you plan ahead. It also pays off to think about how your data will eventually be used in a machine learning model, and plan accordingly. Finally, it's useful to store more information than just the data itself, such as meta-data about how it was collected. Following the FAIR principles of Findability, Accessibility, Interoperability and Reuseability can help you create a data management plan that will serve you well in the future. The Turing Way is a great resource to get started with understanding these principles - here is the link Turing Way

## Data Representation

When designing a machine learning model, it is essential to consider carefully which features of the data should be included and which should be left out. With large datasets, it can be tempting to allow the algorithm to do all of the work; however, this approach can be costly and inefficient. A concept known as the "curse of dimensionality" indicates that the data requirements expand exponentially with the size of the input space. Unsupervised learning techniques are often applied to reduce the dimension of the input, whilst preserving as much information as possible. These techniques form an integral part of developing a successful machine learning workflow.

## Model Selection

Model selection requires trying a variety of approaches in order to select the best model for a particular dataset, task and desired outcome. It's important to understand the different options available in order to develop an effective machine learning workflow. twinLab is one platform that can assist with this process, by ranking models based on their score (which takes into account both lack of fit and model complexity). This helps to ensure that the most suitable model is selected for the job.


Data Preparation, Representation and Model Selection

## Machine Learning Workflow : Loss Functions

<Callout type="success" heading={null}>
In this explainer we define what a loss function is and how it is a central part of training a supervised learning algorithm. In particular, we will cover

- The key concept and properties of a loss function
- Examples of common loss functions for regression and classification tasks.

</Callout>

So looking at a supervised learning algorithm, we have our normal labelled dataset

$$
X := \{({\bf x}_0, y_0), ({\bf x}_1, y_1), \ldots, ({\bf x}_N, y_N)\}
$$

The aim of the game is use an algorithm to approximate the mapping from ${\bf x}$ to $y$, given this set of example / training set. The hope then is that the predictions of the approximation then generalised to unseen examples.

So let us say our trained supervised learning algorithm makes the predictions $\tilde y({\bf x})$, then when accessing the quality of the model we need some measure of how well it fits known examples.

<Callout type="info" heading={null}>
	A **loss function** is a mathematical function used to measure how well a
	machine learning model is performing on a given task. It calculates the
	difference between the predicted output of the model and the actual output
	(i.e., the target variable) for a particular input data point.
</Callout>

There are various choices of loss function, each of which depend on the task at hand, but some broad properties we require of a loss function

1. **Non-negativity** [Essential]: The loss function should always output non-negative values, this is an essential requirement. This is because a "perfect" is one that recovers the data exactly, which has a loss of zero, nothing can improve on this as a prediction.

2. **Differentiable** [Desired] : Since many algorithms are trained using gradient based optimisation methods (the most common is stochastic gradient), then differentiability means that optimisation strategies work effectively.

## Loss functions for Regression Tasks

First we consider the most common loss functions for regression based tasks. I.e. tasks for which the output or target variable is continuous.

### Mean Squared Error (MSE) Loss

This is the most commonly used loss function for regression. It measures the average squared difference between the predicted and actual values. The formula for MSE loss is:

$$
\mathcal L = \frac{1}{N} \sum_{j=1}^N \Big(y_j - \tilde y({\bf x}_{j})\Big)^2
$$

for $y_j$ true outputs, $\tilde y({\bf x}_{j})$ and predicted outputs.



### Mean Absolute Error (MAE) Loss

This loss function measures the absolute difference between the predicted and actual values. The formula for MAE loss is:

$$
\mathcal L = \frac{1}{N} \sum_{j=1}^N |y_j - \tilde y({\bf x}_{j})|
$$

where y is the true output, f(x) is the predicted output, and n is the number of samples.

### Log-Cosh Loss

This loss function is a smooth approximation of the MAE loss that is also less sensitive to outliers than MSE loss. The formula for Log-Cosh loss is:

$$
\mathcal L = \frac{1}{N} \sum_{i=1}^N \log \Big(\cosh\Big[y_j - \tilde y({\bf x}_j)\Big]\Big)
$$

where $\cosh(\cdot)$ is the hyperbolic cosine function.

## Loss functions for Classification Tasks

Second we consider the most common loss functions for classification based tasks. I.e. tasks for which the output or target variable is discrete label.

### Categorical Cross-Entropy Loss

This is a loss function used for multi-class classification tasks where the output is a probability distribution over multiple classes. It measures the difference between the predicted probability distribution and the true class labels.

The formula for categorical cross-entropy loss for $C$ label classification problem over $N$ samples

$$
\mathcal L = \sum_{j=1}^N\left( - \sum_{i=1}^C p_{ij}\log(\tilde{p}_{ij})\right).
$$

Here if $p_{ij}$ is the true probability for class $i$ of sample $j$, i.e. $p_{ij} = 1$

$$
p_{ij} =
\begin{cases}
1 \quad \text{if} \quad y_j = i \\
0 \quad \text{otherwise}
\end{cases}
$$

Where as $\tilde{p}_{ij}$ is the predicted probability of classifying sample with input ${\bf x}_j$ into class $i$, which comes out our model.

So looking at the simplest case for a binary classification class, i.e two labels either $0$ or $1$, then if the true probability of classifying as classification $0$ is $p_{0j}$ then the probability of classifying as a $1$ is

$$
p_{1j} = 1 - p_{0j},
$$

hence the loss is

$$
\mathcal L = - \sum_{j=1}^N \Big[p_{0j}\log(\tilde{p}_{0j}) + (1 - p_{0j})\log(1 - \tilde{p}_{0j}) \Big]
$$

## Hinge Loss

This is a function used for binary classification tasks, where the output is a scalar rather than a probability. It measures the difference between the predicted scalar value and the true binary label, and penalizes the model more heavily for incorrect predictions that are farther from the decision boundary.

In this binary classification, we have to change our labels to either $-1$ or $1$. This is an important step, otherwise you will get the odd results!

The formular for the hinged loss is

$$
\mathcal L = \sum_{j=1}^N\Big[ \max \Big(0, 1 - y_j \cdot \tilde y({\bf x}_j)\Big)\Big]
$$

So let's unpick this a little.

Suppose that the true label is $y_j = -1$. If the prediction from the model is then $\tilde y = -1 + \alpha$, where $\alpha$ is between $0$ and $2$. Then we see that the loss is the $\max(0, \alpha) = \alpha$. So the loss increases to a maximum of $2$ the further it is from the correct decision.

The same happens with when the true label is $y_j=1$. So let's assume the model predicts $\tilde y_j = 1 - \alpha$ again where $\alpha$ is between $0$ and $2$, we will see that the loss is $\max (0, 1 - (1 - \alpha)) = \alpha$. Again in this case the contribution to the overal loss increases to a maximum of $2$.

<Callout type="info" heading={null}>
	**Hinge loss** is commonly used in support vector machines (SVMs), a popular
	classification algorithm. SVMs aim to find the hyperplane that maximizes the
	margin between the two classes. Hinge loss is used as the loss function in SVMs
	because it encourages the model to find the decision boundary with the maximum
	margin.
</Callout>

## Concluding Remarks

A loss is a measure of the quality of a predictive model over either a training or test set. There are a number of choices, which depend on the task at hand, for example if the task is a regression or classification problem. Smoothness of the loss function, i.e. no jumps or sharp corners is preferred, since most ML algorithms are trained with gradient based methods.


Loss Functions

<Callout type="success" heading={null}>
In this explainer we go through the key concept of training. We look at this in two stages, training of the parameters of a model and also in selecting / tuning of hyper-parameters.

You should leave with a clear idea of that happens under-the-hood when you call `.fit(X,y)` in sklearn, and understand some of the choices you can make to fit a better machine learning model, or make the training process easier through good selection of hyper parameters.

</Callout>

Training a Machine Learning model really centres around using methods in optimisation. Training ML models often involves minimisation of a loss or objective function, or maximising a quantity (could be a reward or a variance).

We define two sets of parameters:

- **Parameters** are the values that a model learns during training in order to make predictions on new data. These values are determined by the algorithm and are specific to the model architecture and dataset being used. Examples of parameters include weights in a neural network or coefficients in a linear regression model. Parameters are updated during training to minimize the difference between the model's predictions and the actual output. A model with parameters is a "parameteric model"!

- **Hyperparameters**, on the other hand, are settings that are not learnt by the model during training. Instead, they are set by the user prior to training and are used to control the learning process. Examples of hyperparameters include the learning rate, the number of hidden layers in a neural network, and the regularization strength (all of which you will come across at some point). Hyper-parameters can have a significant impact on the model's performance, and finding the optimal values often requires experimentation and tuning. Optimisation are used (often different ones) to perform tuning of hyperparameters.

## Optimisation of Parameterised Models

So when we think of optimisation of parameter models we can think of the predicts being described by some parameterised function
$$\tilde y = f_{{\bf w}}({\bf x}),$$
here the dependence on parameters is noted by ${\bf w}$. So for example, these could be the coefficients in a linear regression model, or they could be weights within a neural network.

Once we set these parameters the mapping from inputs to outputs is well defined.

Here let's think about training of a supervised learning algorithm. So we have a training set which are example pairs of inputs and outputs

$$
X := \{ ({\bf x}_1, y_1), ({\bf x}_2, y_2), \ldots, ({\bf x}_N, y_N) \}
$$

As discussed (_explainer on loss functions_) we can describe how well a model fits this training set by calculating a loss function, so for example a simple loss for a regression problem might be

$$
\mathcal L({\bf w}) = \frac{1}{N}\sum_{i=1}^N \left(y_i -  f_{{\bf w}}({\bf x}_i)\right)^2
$$

the average mean square error over the training set. Training the model here, means finding the weights ${\bf w}$ which minimise the loss function, i.e. minimising the error.

Let's draw a picture

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={460}
	height={363}
	caption={""}
/>

This sketch shows a plot of a loss function with respect to weights. Here we have a simple single parameter problem. The idea is we want to find the value of $w$ which gives the local minma. We use optimisation strategies to do this. We can see for complex loss functions (i.e. complex models and data) we can easily get "trapped" in local minima, so this problem isn't easy.

For this task we can use a whole suite of optimisation techniques. But the most commonly used are gradient based methods, which we will look at.

### Gradient Descent

So I like to think of minimisation problems as an landscape, where I am trying to walk (or run) to the lowest point in a mountain range. Gradient descent is trying to do this too. It basically iteratively updates the weights ${\bf w}$ to move down the hill (the loss function) by a little bit, and then repeats

So to be a bit more mathematical, down hill (or gradient descent) means against the slope. The slope or gradient of a function is denoted by $\nabla \mathcal L$ (aka grad $\mathcal L$), and is calculated by differentiating the loss function with respect to the weights ${\bf w}$. So we see that gradient descent is written like

$$
{\bf w}^{(k+1)} = {\bf w}^{(k)} - \eta \nabla_{{\bf w}}\mathcal L
$$

Here the parameter $\eta$ is called the step size or learning rate, and therefore determines how big a step I take at each iterations. Here is a sketch of what is going on!

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={560}
	height={449}
	caption={""}
/>

In this example, at $w^(i)$ the gradient of the loss is negative, therefore the $-\eta\nabla\mathcal L$ is positive, stepping towards the minima.

The learning rate is hyper-parameter of my method, and needs to chosen so the optimisation strategy converges.

Too small a **learning rate**, and the solution converges very slowly towards a local minima. A too large learning rate and gradient descent can become unstable, diverging away from a solution.

We can illustrate this with a simple little sketch

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={950}
	height={317}
	caption={""}
/>

Finding a good learning rate can be challenging. There are more sophisiticated methods. We can talk more about this in **hyper-parameter** tuning.

### Stochastic Gradient

We see that the central cost in gradient descent is calculating the derivative of the loss

$$
\nabla_{\bf w} \mathcal L = \nabla_{\bf w}\left[ \frac{1}{N}\sum_{i=1}^N \left(y_i -  f({\bf x}_i)\right)^2\right]
= \frac{2}{N}\sum_{i=1}^N \left(y_i -  f({\bf x}_i)\right)\nabla_{{\bf w}} f({\bf x}_i)
$$

We can reduce the cost of this by only calculating an approximation of the gradient, by randomly sampling a subset of the training data. Reducing the computation to a loop over $M \ll N$ training points

We then call this approximate Gradient $\tilde{\nabla}_{{\bf w}}\mathcal L$, but the update to the weights is the same as before

$$
{\bf w}^{(k+1)} = {\bf w}^{(k)} + \eta \tilde{\nabla}_{{\bf w}}\mathcal L
$$

This process recalled **Stochastic Gradient**, there are various versions of this, but this is the core approach, and used widely in machine learning. Particularly when datasets are very large. This is very typical in Neural Network applications, since due to the huge number of parameters there is also huge data requirements.

## Optimisation of Hyper-Parameters or should we saying "Tuning".

Hyper-parameters are parameters which control the model defition or the training process. They are not learnt directly from data, and are set before the "training" process and greatly effect the training process and overall model performance.

Examples of this will include

- Learning Rate
- Dropout Rate (particularly to deep learning)
- Regularisation Parameters
- Parameters defining model structure, e.g. number of hidden layers, clusters, trees and layers.
- High level model parameters, for example kernel parameters for a Gaussian Process.

Let's look at took three different options

**Grid Search** - very simple way of finding "optimal" hyperparameters. Train models for each of the parameters over a grid of values, down select the best. The challenge he is it requires the building of many machine learning models, if expensive to train each then this can be really challenging computational. For simple models it is a really nice simple approach which can be implemented very quickly.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={382}
	caption={""}
/>

**Hyperband** - Hyperband is a very simple but effective idea. It initial start training many models with different hyper-parameters. After one a period, the models are rank in terms of performance, the worse are 50% and killed, and computational resource is doubled down on the best hyper-parameter values. This is also known as successive halfing.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img5.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={421}
	caption={""}
/>

**Bayesian Optimisation** - Is a more complex approach by which a surrogate model for the model performance over the space of hyper-parameters is built. This allows for the minimum can be estimated, and additional hyper-parameter can be sampled. The approach is much more targeted than Grid Search, and providing the number of hyperparameters is not too large then can be extremely effective. Implementation of these approach, which using Gaussian Processes as a surrogate, are more sophisticated to implement and require use of packages like [twinLab](https://www.twinlab.ai) for example.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-optimisation-hyper-parameter-tuning/img6.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={500}
	height={1005}
	caption={""}
/>


Training, Optimisation & Hyper-parameter Tuning

# Exploring if Your Model Is Any Good - Training Curves

<Callout type="success" heading={null}>
In this explainer we look at how we validate our machine learning models once they have trained. The is an essential part of any machine learning workflow. We will cover

1. The basic approarch to estimating a training and testing loss.
2. Extension of this to k-fold cross validation
3. Understanding what are and how to look for good and bad model in training / learning curves.
4. Understand what data leakage is, and how you might observe it at validation stage.

</Callout>

Validation is a key fundamental step in the machine learning workflow. The concepts of how you might do it are quite simple, however it requires some honest detective work to really be confident in you models ability to generalise. Remember this our ultimate goal in machine learning.

Let us first look at the most basic step, which is to estimate a training and testing loss.

## Estimating Training & Testing Loss - Test / Train Split

There are a few exceptitions in machine learning (aren't there always), but here we a primarily talking about supervised learning algorithms. This is when you have a dataset of example pairs of inputs and outputs together, i.e. $({\bf x}_j, y_j)$. As you will see in many of our supervised learning examples, prior to training a model we will perform a splitting of our data. A percentage of the total data is retained for testing (a typical default might be 20%).

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={400}
	height={86}
	caption={""}
/>

This ring-fenced testing data will not be seen by the algorithm at training. How well the machine learning model does at on this testing set gives an indication of how well a model generalises.

To evaluate **"how good?"** we need to evaluate the average loss function over training and testing data, which we might call $\mathcal L_{test}$ and $\mathcal L_{tr}$

### So how do we do this in `scikit-learn`?

In most ML libraries you will find a test / train split functionality. In `scikit-learn`, you can perform a test train split using the train_test_split function from the `sklearn.model_selection ` module. This function randomly splits a dataset into training and testing sets, which can be used for machine learning model training and evaluation, respectively.

Here is a bit of sample code of how to use train_test_split to split a dataset into 80% training and 20% testing data:

```Python
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
```

Here, `X` and `y` are the input features and target variable, respectively. `test_size=0.2` specifies that we want to allocate 20% of the data for testing, and `random_state=42` ensures that the split is reproducible.

The resulting `X_train`, `X_test`, `y_train`, and y_test arrays can then be used for training and evaluating machine learning models.

## K-Fold Cross-Validation

K-fold cross-validation is a technique used in machine learning to evaluate the performance of a model on a limited amount of data. The main reason why you would use k-fold cross-validation is to get an estimate of the model's performance that is more reliable than simply splitting the data into a training and a test set.

This is particular the case when you have limited amount of data, you might not be able to afford to set aside a portion of it for a test set. In this case, k-fold cross-validation allows you to make the most of the data you have by using all of it to evaluate the model.

So what do you do? You split you data into $k$ folds. This means that $1/k$ of the data is partitioned for training. You will then train $k$ models, cycling though using $1/k$ partition as the testing data, the rest as the training data.

Best shown by a picture. Here we show a $4$-fold cross valiation, a differen $25%$ of the data is used in each fold. We can then calculate the loss (error) in each fold, avereage the results.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={740}
	height={455}
	caption={""}
/>

Importantly we get better estimate of performance of a model. By repeating the training and testing process multiple times, k-fold cross-validation provides a more stable estimate of the model's performance than a single split into training and test sets. This is particularly important when the performance metric you're interested in is sensitive to the specific samples in the test set, which is particularly the case when we have limited data.

### So, again, how do we do this in `scikit-learn`?

In `scikit-learn`, you can perform k-fold cross-validation using the `KFold` function from the sklearn.model_selection module. This function splits the dataset into k equal-sized folds, and trains and tests the model k times, using a different fold for testing each time, and the remaining folds for training.

Here's an example of how to use KFold to perform 5-fold cross-validation:

```Python
from sklearn.model_selection import KFold
kf = KFold(n_splits=4, shuffle=True, random_state=42)
for train_index, test_index in kf.split(X):
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    # train + evaluate training + test loss
```

Here, `X` and `y` are the input features and target variable, respectively. `n_splits=4` specifies that we want to perform 4-fold cross-validation, and `shuffle=True` specifies that the data should be shuffled before splitting.

The `KFold` object `kf` returns an iterator that generates the indices of the training and testing data for each fold. The `train_index` and `test_index` arrays can be used to extract the training and testing data for each fold, and then train and evaluate the model using these data.

<Callout type="info" heading={null}>
	We will see more examples of this when we start demonstrating various
	supervised learning algorithms across the course.
</Callout>

## Validation at Training Loss Curve

So with many machine learning models, models are trained iterative. We have looked at this when talking about optimisation methods. A step of training is called an **epoch**. So when we are training a model, we will monitor both the training and testing loss as a function of number of epochs.

This leads to a typical training curve, as below. Here we show, what a training curve for "good model" might look like.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={430}
	height={253}
	caption={""}
/>

As we train the model, the blue curve shows the training loss reducing. This is to be expected, since we are updating the parameters of a model to achieve this. With this, for a model which is also learning to generalise, we expected a reduction in the testing loss. This should naturally lag behind, the training loss. If both reduce, as shown above, the this indicates a training has been successful.

<Callout type="info" heading={null}>
	If training loss does not decrease with epochs the first step is to try
	reducing your learning rate in your optimisation step or try different start
	guesses.
</Callout>

When this doesn't happen then this gives us strong indications of what is happening to our model.

First let's look at the case where the both the training and test loss do no reduce much, leaving a large residual error which can be removed by additional training. Like the following training loss curve.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={410}
	height={264}
	caption={""}
/>

A training loss curve like this indicates that the model is "underfitting". This will eb covered in other areas of the course, yet underfitting is when a model doesn't have sufficient flexibility to be able to express the variations in the data. A simple example would be if you used a straight line to model the tidal heights in a day. The best it could model would be the average. Since we can't fit even the training data well, we see this large residual in the error, which doesn't improve as we increasing training. Solution is here is to go back to your model, and add more features / flexibility.

Secondly, is the opposite of this effect, overfitting. Here the model has too much flexibility. This is way it can reproduce the training data well, but is not constrained sufficiently that it generalises well. We show a typical overfitting model next to our training curve to give the idea.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img5.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={375}
	height={249}
	caption={""}
/>

Thirdly, is an odd one. Here the model looks like his has trained really well. Both the training and testing data have come right down, as we increase the number of training epochs. The problem is it is too good. Since the testing data is matching the performance of the training data, in some cases doing better.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/training-curves/img6.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={730}
	height={523}
	caption={""}
/>

This typical occurs when we have data leakage. This means the testing data and the training data aren't independent. Therefore there is information in the training data which explicitly helps with our testing set, rather beyond the ability of the algorithm to generalise.

- Good examples of this is when we have time series data. If we do a random test train split of time series data, we have have data in both the testing and training set which is strongly correlated. To build a test train set for time series data we must leave sufficient space so that there is little or no correlation between the training and testing sets. As we show here.

- Another exampe is called "group leakage" - so a classic example was where Andrew Ng's research group had 100k x-rays of 30k patients, meaning approximately 3 images per patient. The algorithm used random splitting instead of ensuring that all images of a patient was in the same split. Hence the model partially memorized the patients instead of learning to recognize pneumonia in chest x-rays

## Concluding Remarks

Validation is a key fundamental step in the machine learning workflow. The concepts of how you might do it are quite simple, however it requires some honest detective work to really be confident that your model (1) is trained and (2) can generalise to unseen data.

We have presented a few ideas on how you can do this. Other techniques can be picked up in the worked examples through the digiLab courses. This is because evalating a good model, takes experience.

Remember, generalisation is our ultimate goal in when building a machine learning algorithm.


Exploring If Models are Any Good - Training Curves

Once you have trained a model, and there is comfort that it has "fitted well" I would say the best thing to look at is understanding how the ML model is performing on a task. How good a machine learning model depends on the particular applied questions you are being asked. For example, if it is a recommender system for recommending the color of underpants or a recommender system for air traffic control, the bounds around good are understandable very different.

A first port of call when looking at how good your model is are some simple plots. There are various plots you can look at. Here are a few, which are widely used in classification and regression problems.

## Confusion Matrix - Multiclass classification

First one up is the **confusion matrix**. A confusion matrix is a table that is often used to evaluate the performance of a classification model.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/is-my-model-any-good-validation-plots/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={370}
	height={342}
	caption={""}
/>

The table contains information about the model's predictions on a set of test data, as compared to the true labels for that data. Here is the informaiton on a confusion matrix:

- The **confusion matrix** is usually represented as a square matrix, with rows corresponding to the true class labels and columns corresponding to the predicted class labels.

- The sum of all the numbers in the table equals the total number of test samples.

- The overall **accuracy** of the model can be determined by the sum of the diagonal (all correct predicts) divided by the total number of test samples.

- There is no reason for the **confusion** matrix to be symmetric.

We can also evaluate metrics for a particular "LABEL". So let's define the four cases we might have

1. **True Positive (TP):** This represents the number of cases where the actual class is "LABEL", and the model correctly predicts it as "LABEL".

2. **False Positive (FP):** This represents the number of cases where the actual class not the "LABEL", but the model incorrectly predicts the "LABEL".

3. **True Negative (TN):** This represents the number of cases where the actual class is not the "LABEL", and the model correctly predicts it not the "LABEL".

4. **False Negative (FN):** This represents the number of cases where the actual class is the "LABEL", but the model incorrectly predicts it as not the "LABEL".

Once we have each of these classes we can calculate various metrics of the algorithms ability to classify "LABEL". Let's show these in a reduce confusion matrix.

Here is a picture which unpicks each of these

<Image
	src={
		"/images/courses/foundations-in-machine-learning/is-my-model-any-good-validation-plots/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={420}
	height={350}
	caption={""}
/>

### ROC and Area under ROC - Binary Classifiers

The ROC (Receiver Operating Characteristic) curve is a graphical representation of the performance of a binary classification algorithm. It is used to evaluate the trade-off between the true positive rate (TPR) and the false positive rate (FPR) for different classification thresholds.

In the context of binary classification, the ROC curve plots the true positive rate (TPR) on the y-axis against the false positive rate (FPR) on the x-axis, for different threshold values. The TPR, also known as sensitivity, is the proportion of actual positive cases that are correctly identified as positive by the model. The FPR, also known as the fall-out, is the proportion of actual negative cases that are incorrectly identified as positive by the model.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/is-my-model-any-good-validation-plots/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={390}
	height={310}
	caption={""}
/>

An ideal classifier would have a ROC curve that passes through the upper left corner of the plot, where the TPR is 1 (i.e., all positive cases are correctly classified), and the FPR is 0 (i.e., no negative cases are incorrectly classified as positive). In contrast, a random classifier would have a ROC curve that follows the diagonal line of the plot.

The area under the ROC curve (AUC) is a commonly used metric for evaluating the overall performance of a binary classification algorithm. AUC ranges between 0 and 1, with a value of 1 indicating a perfect classifier, and a value of 0.5 indicating a random classifier. An AUC value of less than 0.5 indicates that the classifier is worse than random, and an AUC value of greater than 0.5 indicates that the classifier is better than random.

Overall, the ROC curve is a useful tool for evaluating the performance of a binary classification algorithm, particularly in situations where the cost of false positives and false negatives may differ.

### True Vs. Predicted Plot - Regression

Visualising the performance of regression model can be quite simple. The natural plot is a simple plot of true value ($y$) and predicted values ($\hat y$) over the testing set.

So something like this.

<Image
	src={
		"/images/courses/foundations-in-machine-learning/is-my-model-any-good-validation-plots/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={300}
	height={264}
	caption={""}
/>

In this case, with the "perfect" model we would see all text points line up on the diagonal where $y = \hat{y}$.

Variation from this line represents contributions to the loss function

$$
\ell_j = (y_j - \hat{y}_j)^2
$$

If there are clear outliers, it is often to identify and consider those particular cases.


Getting Started in Machine Learning

Course Description

Section 1: Starting Blocks - An Introduction to Machine Learning

Section 2: We're Off - Understanding the Machine Learning Workflow

Lessons

1. Machine Learning 101 : Why, What and How?

2. Section Overview

3. Supervised Vs Unsupervised Learning - Explainer

4. K-Nearest Neighbour Explainer

5. K-Nearest Neighbour Walkthrough

6. K-Means Explainer

7. Trying out Clustering with K-Means

8. Section Overview

9. Data Preparation, Representation and Model Selection

10. Loss Functions

11. Training, Optimisation & Hyper-parameter Tuning

12. Exploring If Models are Any Good - Training Curves

13. Is My Model any Good - Validation Plots