Large/small

What is the number of parameters to be minimized? Let $ k$ be the number of parameters. Memory usage of an algorithm will be some function $ f(k).$ If this function is growing rapidly for a certain algorithm, that algorithm will cease to be useful for ''large'' problems, since memory will be exhausted. Of course, ''large'' is a relative term that increases over time as memory becomes cheaper. For ''small'' problems, the speed of convergence of the algorithm will be of primary importance, since memory resources will not be a bottleneck.



Søren Hauberg 2008-04-29