A sparse, anti-fragile universe.....

This article is being cross-posted [1] from my micro-blog, Tumbld Thoughts.

Not a particularly sparse universe, by the way. But a home of sparse variables? Perhaps.

And now, dispatches from an antifragile universe [2]. Nassim Nicholas Taleb has a guest article in Wired (entitled "Beware the Big Errors of Big Data") in which he warns against the use of sparse variables [3] in so-called high-throughput datasets.

NOTES:

[1] also re-posted to the Mathematica group on Tumblr.

[2] N.N. Taleb  "Antifragile: things that gain from disorder". RSA lecture here. I've yet to decide whether Taleb is the Rasputin or the Tesla of analytics. Regardless, here is a video from the New York Public Library speaker series featuring Taleb discussing the concept of anti-fragility with cognitive psychologist Daniel Kahneman.

[3] variables that have a high signal-to-noise ratio. Or, why sometimes Watson gives ridiculous answers. Generally speaking, the more variables you have in an analysis, the greater the number of spurious correlations you must contend with. This is quite similar to Bellman's "curse of dimensionality".