6.3 Non-metric MDS

In this chapter we have focused on classical MDS, which looks for points that have Euclidean distances close to the desired distances, where we measured close by minimizing the sum of squared errors (cf Equation (6.1)).

There is a much wider class of multidimensional scaling methods that do not use Euclidean distances, and do not judge the approximation quality using squared errors. These MDS methods can sometimes produce sets of coordinates that are more useful than classical MDS. However, it is only for classical MDS that there is a closed form solution. More general MDS methods rely upon numerical optimization to find a solution to the optimization problem. If you are interested, take a look at the R commands isoMDS, Shepard, and sammon from the MASS R package. Sammon mapping, for example, puts a greater emphasis on preserving smaller distances.

More generally still, there has been a lot of work on non-linear dimension reduction methods that try to find a low dimensional manifold upon which the data lie. Have a look here for some amazing visualizations and results using the MNIST dataset. These methods are not always easy to work with, and can require detailed user supervision in order to work well.