Algorithms have at least one input and one output. All algorithm endpoints are organized in groups. Groups are used by the platform to indicate which inputs and outputs are synchronized together. The first group is automatically synchronized with the channel defined by the block in which the algorithm is deployed.
| Endpoint Name | Data Format | Nature |
|---|---|---|
| features | system/array_2d_floats/1 | Input |
| client_id | system/uint64/1 | Input |
| subspace | chichan/a-collection-of-linear_machines/1 | Output |
Parameters allow users to change the configuration of an algorithm when scheduling an experiment
| Name | Description | Type | Default | Range/Choices |
|---|---|---|---|---|
| percentage-of-pca-energy | float32 | 0.949999988079071 | ||
| number-of-lda-components | uint32 | 30 |
xxxxxxxxxximport bobimport numpyclass Algorithm: def __init__(self): self.percentage_of_pca_energy = 0.98 self.number_of_lda_components = 30 self.data = {} def setup(self, parameters): self.percentage_of_pca_energy = parameters.get('percentage-of-pca-energy', self.percentage_of_pca_energy) self.number_of_lda_components = parameters.get('number-of-lda-components', self.number_of_lda_components) return True def _project_data_for_lda(self, machine,i): tdata = [] for client_id, client_files in self.data.iteritems(): # at least two files per client are required! if len(client_files) < 2: # "Skipping client since the number of client files is only %d" %len(client_files) continue tdata.append(numpy.vstack([machine(feature) for feature in client_files[i]])) return tdata def _perform_pca(self, pca_machine, training_set): # "Perform PCA on data" data = [] for client_features in training_set: data.append(numpy.vstack([machine(feature) for feature in client_features])) return data def process(self, inputs, outputs): features = inputs['features'].data.value c_id = inputs["client_id"].data.value machines=[] if c_id in self.data.keys(): for idx in xrange(features.shape[0]) : self.data[c_id][idx].append(features[idx]) else: feature_dict={} for idx in xrange(features.shape[0]) : feature_dict[idx]=[features[idx]] self.data[c_id] = feature_dict if not(inputs.hasMoreData()): # numpy.lib.asarray_chkfinite(self.data[self.data.keys()[0]][self.data[self.data.keys()[0]].keys()[0]][0])# assert(len(self.data[self.data.keys()[0]][self.data[self.data.keys()[0]].keys()[0]][0])==0,'data[c_id][0]==0')# assert(len(self.data[self.data.keys()[0]][self.data[self.data.keys()[0]].keys()[0]][0])>=59,'data[c_id][0]>=59')# assert(len(self.data[self.data.keys()[0]][self.data[self.data.keys()[0]].keys()[0]][0])<=59,'data[c_id][0]>=59') lda_trainer = bob.trainer.FisherLDATrainer() trainer = bob.trainer.PCATrainer() for i in self.data[self.data.keys()[0]].keys(): # PCA data_pca = numpy.vstack([self.data[j_id][i] for j_id in self.data.keys()])# if data_pca.shape[1]==59:# print cc pca_machine, eigen_values = trainer.train(data_pca) del data_pca # Reduce memory usage #remove zeros non_zeros_idx=len(numpy.where(eigen_values>0)[0]) n_eigen_values=eigen_values.copy() n_eigen_values.resize(non_zeros_idx) assert(eigen_values == n_eigen_values,'eigen_values == n_eigen_values') cum_eigen_values=numpy.cumsum(n_eigen_values/sum(n_eigen_values)) number_of_pca_components=numpy.where(cum_eigen_values >self.percentage_of_pca_energy)[0] if number_of_pca_components[0] <2 : raise ValueError('number_of_pca_components_%d[0]:%d'%(i,number_of_pca_components[0])) pca_machine.resize(pca_machine.shape[0], (number_of_pca_components[0]+1)) # outputs data numpy.lib.asarray_chkfinite(pca_machine.weights) # LDA data_lda = self._project_data_for_lda(pca_machine,i) lda_machine, lda_variances = lda_trainer.train(data_lda) del data_lda # Reduce memory usage if lda_machine.shape[1]>lda_machine.shape[0] : lda_machine.resize(lda_machine.shape[0], lda_machine.shape[0]) if lda_machine.shape[1]>int(self.number_of_lda_components) : lda_machine.resize(lda_machine.shape[0], int(self.number_of_lda_components)) if numpy.isnan(lda_machine.weights).any() or numpy.isinf(lda_machine.weights).any(): raise ValueError([lda_variances,number_of_LDA_components,number_of_pca_components[0]+1,lda_machine.shape, len(number_of_LDA_components)]) pcalda_subspace = numpy.dot(pca_machine.weights,lda_machine.weights) if numpy.isnan(pcalda_subspace).any() or numpy.isinf(pcalda_subspace).any(): raise ValueError(pcalda_subspace.shape) machines.append({ 'input_subtract': pca_machine.input_subtract, 'input_divide': pca_machine.input_divide, 'weights': pcalda_subspace, 'biases': lda_machine.biases }) # outputs data outputs["subspace"].write({"machine_array": machines}) print machines[0] print machines[-1] return TrueThe code for this algorithm in Python
The ruler at 80 columns indicate suggested POSIX line breaks (for readability).
The editor will automatically enlarge to accomodate the entirety of your input
Use keyboard shortcuts for search/replace and faster editing. For example, use Ctrl-F (PC) or Cmd-F (Mac) to search through this box
This algorithm performs principal component analysis (PCA) [PCA] on a given dataset using the singular value decomposition (SVD) [SVD], followed by linear discriminant analysis (LDA) [LDA].
This implementation relies on the `Bob <http://www.idiap.ch/software/bob>`_ library.
The inputs are:
The outputs are subspace_pca and subspace_lda for the PCA and LDA transformation, respectively.
| [SVD] | http://en.wikipedia.org/wiki/Singular_value_decomposition |
| [PCA] | http://en.wikipedia.org/wiki/Principal_component_analysis |
| [LDA] | http://en.wikipedia.org/wiki/Linear_discriminant_analysis |
| Updated | Name | Databases/Protocols | Analyzers | |||
|---|---|---|---|---|---|---|
| smarcel/chichan/full_pre_mlbphs_projection/2/mobio-f_TT_MLBPH_PCA98_LDA300_postperf-iso | mobio/2@female | tutorial/eerhter_postperf_iso/1 | ||||
| smarcel/chichan/full_pre_mlbphs_projection/2/mobio-m_TT_MLBPH_PCA98_LDA300_postperf-iso | mobio/2@male | tutorial/eerhter_postperf_iso/1 | ||||
| chichan/chichan/full_pre_mlbphs_projection/2/Prep_MLBPH_XM2VTS_LDA | xm2vts/1@darkened-lp1,xm2vts/1@lp1 | tutorial/eerhter_postperf/1 | ||||
| chichan/chichan/full_pre_mlbphs_projection/2/Prep_MLBPH_XM2VTS_no_uniform_p98LDA | xm2vts/1@darkened-lp1,xm2vts/1@lp1 | tutorial/eerhter_postperf/1 |
This table shows the number of times this algorithm has been successfully run using the given environment. Note this does not provide sufficient information to evaluate if the algorithm will run when submitted to different conditions.