The Spider：一个“面向对象”的机器学习Matlab工具箱 | 丕子

The Spider

It's a library of objects in Matlab

It is meant to handle (reasonably) large unsupervised, supervised or semi-supervised machine learning problems.

Aims to become a complete research/analysis toolbox: Includes training, testing, model selection, statistical tests, ...

(Some more visualization tools would also be nice ...but not implemented yet)

Plugging objects together: e.g perform cross validation on the following system: greedy backward feature selection on a fast base algorithm, training on those features with an SVM for each output in a one-against-the-rest multi-class system, choosing all hyperparameters with a model selection method.

他在工具箱中提供的接口都是模仿者对象的思想，任何方法和数据都看成对象，例如:用一个SVM对象去训练一个Data数据对象：

X=rand(50)-0.5; Y=sign(sum(X,2)); d=data(X,Y) % make simple data

[res alg]= train(svm,d) % train a support vector machine

and that's it - the svm is trained! To test it on more data:

[res]= test(alg,d)

这里面对象有多少呢？看下面的列表，包括算法模型和数据：

Basic library objects.data+Storing input data and output resultsdata_global+Implementation of data object that limits memory overheadalgorithm+Generic algorithm object group+Groups sets of objects together (algorithms or data)loss+Evaluates loss functions get_mean+Takes mean loss over groups of algschain+Builds chains of objects: output of one to input of anotherparam+To train and test different hyperparameters of an objectcv+Cross validation using objects given datakernel+Evaluates and caches kernel functionsdistance+Evaluates and caches distance functionsStatistical Tests objects.wilcoxon-Wilcoxon test of statistical significance of resultscorrt_test-Corrected resampled t-test - for dependent trialsDataset objects.spiral+Spiral dataset generator. toy+Generator of dataset with only a few relevant featurestoy2d+Simple 2d Gaussian problem generatortoyreg+Linear Regression with o outputs and n inputsPre-Processing objectsnormalize+Simple normalization of data map+General user specified mapping function of dataDensity Estimation objects.parzen+Parzen's windows kernel density estimatorindep-Density estimator which assumes feature independencebayes+Classifer based on density estimation for each classgauss+Normal distribution density estimatorPattern Recognition objects.svm+Support Vector Machine (svm) c45-C4.5 for binary or multi-classknn+k-nearest neighbours platt-Conditional Probability estimation for margin classifiersmksvm-Multi-Kernel LP-SVM anorm-Minimize the a-norm in alpha space using kernelslgcz-Local and Global Consistent Learnerbagging+Bagging Classifier adaboost+ADABoost method hmm-Hidden Markov Model loom-Leave One Out Machine l1-Minimize l1 norm of w for a linear separatorkde-Kernel Dependency Estimation: general input/output machinedualperceptron-Kernel Perceptron ord_reg_perceptron-Ordinal Regression Perceptron (Shen et al.)splitting_perceptron-Splitting Perceptron (Shen et al.)budget_perceptron-Sparse, online Pereceptron (Crammer et al.)randomforest-Random Forest Decision Trees WEKA-Requiredj48-J48 Decision Trees for binary WEKA-RequiredMulti-Class and Multi-label objects.one_vs_rest+Voting method of one against the rest (also for multi-label)one_vs_one+Voting method of one against onemc_svm-Multi-class Support Vector Machine by J.Westonc45-C4.5 for binary or multi-classknn+k-nearest neighbours Feature Selection objects.feat_sel+Generic object for feature selection + classifierr2w2_sel-SVM Bound-based feature selectionrfe+Recursive Feature Elimination (also for the non-linear case)l0-Dual zero-norm minimization (Weston, Elisseeff)fsv-Primal zero-norm based feature selection (Mangasarian)fisher-Fisher criterion feature selectionmars-selection algorithm of Friedman (greedy selection)clustub-Multi-class feature selection using spectral clusteringmutinf-Mutual Information for feature selection.Regression objects.svr+Support Vector Regression gproc+Gaussian Process Regression relvm_r-Relevance vector machine multi_rr+(possibly multi-dimensional) ridge regressionmrs-Multivariate Regression via Stiefel Constraintsknn+k-nearest neighbours multi_reg+meta method for independent multiple output regressionkmp-kernel matching pursuit kpls-kernel partial least squares lms-least mean squared regression [now obselete due to multi_rr]rbfnet-Radial Basis Function Network (with moving centers)reptree-Reduced Error Pruning Tree WEKA-Requiredreg_jkm-Structure Output Learning using Joint Kernel MethodModel Selection objects.gridsel+select parameters from a grid of valuesr2w2_sel-Selecting SVM parameters by generalization boundbayessel+Bayessian parameter selection Unsupervised objects.one_class_svm+One class SVM kmeans+K means clustering kvq+Kernel Vector Quantization kpca+Kernel Principal Components Analysisppca-Probabilistic Principal Component Analysisnmf-Non-negative Matrix factorizationspectral-Spectral clustering mrank-Manifold ranking ppca-Probabilistic PCA Reduced Set and Pre-Image objects.pmg_mds-Calculate Pre-Images based on multi-dimensional scalingpmg_rr-Calculate Pre-Images based on learning and ridge regressionrsc_burges-Bottom Up Reduced Set; calculates reduced set based on gradient descentrsc_fp-Bottom Up Reduced Set; calculates reduced set for rbf with fixed-point iteration schemesrsc_mds-Top Down Reduced Set; calculates reduced set with multi-dimensional scalingrsc_learn-Top Down Reduced Set; calculates reduced set with ridge regressionrss_l1-Reduced Set Selection via L1 penalizationrss_l0-Reduced Set Selection via L0 penalizationrss_mp+Reduced Set Selection via matching pursuit