Here, we'll start talking about how we formulateВ preciselyВ what we want PCA to do.For the problem of dimensionality reduction the most commonly used algorithm is PCA.Principle Component Analysis (PCA): Problem Formulation Helps show the two main dimensions of variation in a way that's easy to understand.It's up to you to asses what of the features can be grouped to form summary features, and how best to do that (feature scaling is probably important).So despite having 50 features, there may be two 'dimensions' of information, with features associated with each of those dimensions.and y axis may be the per-person well being/economic activity.may find horizontal axis corresponds toВ overallВ country size/economic activity Typically you don't generally ascribe meaning to the new features (so we have to determine what these summary values mean).Come up with a different feature representation (z values) which summarize these features.Using dimensionality reduction, instead of each country being represented by a 50-dimensional featureВ vector.How can we understand this data better?.Collect a large data set about many facts of a country around the world.

Good for explaining something to someone if you can 'show' it in the data.

Dimensionality reduction helps us do this, see data in aВ helpful.

Often helps to developВ algorithmsВ if we can understand our data better.

Dimensionality reduction can improve how we display information in a tractable manner for human consumption.

It's hard toВ visualizeВ highly dimensional data.

In reality we'd normally try and doВ 1000DВ ->В 100D.

So we've now reduced our 3D vector to a 2D vector.

we loose the data in the z-dimension of our 'shallow box' (NB 'z-dimensions' here refers to the dimension relative to the box (i.e it's depth) and NOT the zВ dimensionВ of the axis we've got drawn above) but because the box is shallow it's OK to lose this.

Because they're all in thisВ relativeВ shallow area, we can basically ignore one of theВ dimension, so we draw two new lines (z1В andВ z2) along the x and y planes of the box, and plot the locations in that box.

In the diagram below, imagine all our data points are sitting 'inside' the blue tray (has aВ dark blueВ exterior face and a light blue inside).

Either way, the plane is a small, aВ constant 3D space.

This is sort of hard to explain in 2D graphics, but that plane may beВ alignedВ with one of the axis.

The loss above comes from the rounding error in the measurement, however.

Gives lossy compression, but an acceptable loss (probably).

Allows us to half the amount of storage.

Now we can represent x 1В as a 1D number (ZВ dimension).

So before x 1В was a 2D feature vector (X and Y dimensions).

TakeВ exactВ example and record position on that line.

What does dimensionality reductionВ mean?.

This correlation can be combined into a single attribute calledВ aptitudeВ (for example).

These features may be highly correlated.

Helicopter flying - do a survey of pilots (x1В = skill,В x2В = pilot enjoyment).

Often generates redundant data (especially if you don't control data collection).

Data redundancy can happen when different teams are workingВ independently.

Example above isn't a perfect straight line because of round-off error.

Redundant data set - different units for same attribute.

Can you 'simply' your data set in a rational and useful way?.

So you've collected many features - maybe more than you need.

Why should we look at dimensionality reduction?.

Start talking about a second type of unsupervised learning problem - dimensionalityВ reduction.

Piazza pca column with no deviation