Luck or Skill?

I have been re-reading Michael Mauboussin's "Untangling skill and Luck" (pdf) over the past week.

One of my favorite parts is where he discusses the relative composition of luck and skill in a particular activity.

There’s a simple and elegant test of whether there is skill in an activity: ask whether you can lose on purpose. If you can’t lose on purpose, or if it’s really hard, luck likely dominates that activity. If it’s easy to lose on purpose, skill is more important.

The quote is attributed to Annie Duke's 2007 testimony on behalf of Poker Player's alliance. Although, I did not find the exact source, the testimony is itself a fascinating read. At one point she tries to distance poker from other forms of gambling:

There is critical distinction between poker and other forms of “gambling” which is the skill level involved to succeed at the game. I cannot stress this point enough: in poker it is better to be skillful than lucky. I ask anyone in this hearing room to name for me the top five professional roulette players in the world or the number one lottery picker in America. It is just not possible (my apologies to one obvious candidate, Congressman Sensenbrenner). We can however have a real discussion about the top five professional poker players, just like we can have a discussion about the top five professional golfers.

Gilbert Strang: Video Lectures

Gilbert Strang is an amazing teacher.

This summer, I've been watching some of his lecture videos on Linear Algebra, and Computational Science and Engineering, at MIT OpenCourseWare.

Although, they are not technologically fancy or gimmicky, they provide a superb introduction.

Here are short-cut links to:

1. Linear Algebra: I was still an undergrad when these lectures were filmed. As a matter of fact, I was taking a linear algebra course, at around the same time.
 
2. Computational Science and Engineering: Given my current academic home, this is a nice introductory series.

Numerical Differentiation using Mathematica

I am teaching myself Mathematica over summer, and coded up a simple module that computes numerical differentiation formulas automatically. Essentially, the module enables one to compute arbitrary differentiation rules like those explained here.

This module spits out the different formulae for a m-point approximation to the n-th derivative, and the leading error term.

Here is the (updated) module:

AppDeriv[m_, n_] := Module[{points, IP, nthDeriv, e1},
  points = Table[{x0 + i h, Subscript[f, i]}, {i, 0, m - 1}];
  IP = InterpolatingPolynomial[points, x];
  nthDeriv = D[IP, {x, n}];
  e1 = (1/Factorial[m])*
    D[Apply[Times, (x - Table[{x0 + i h}, {i, 0, m - 1}])], {x, n}];
  Formula =
   TableForm[
    Table[{Subsuperscript[f, i, n], nthDeriv /. x -> (x0 + i h),
       Superscript[f, m] e1 /. x -> (x0 + i h)}, {i, 0, m - 1}] //
     Simplify ]]

This seems to work alright for me (click to enlarge).

Test-taking success and "real" success

Interesting article in the New York Magazine: Paper Tigers: What happens to all the Asian-American overachievers when the test-taking ends.

From the article:

Entrance to Stuyvesant, one of the most competitive public high schools in the country, is determined solely by performance on a test: The top 3.7 percent of all New York City students who take the Specialized High Schools Admissions Test hoping to go to Stuyvesant are accepted. There are no set-asides for the underprivileged or, conversely, for alumni or other privileged groups. There is no formula to encourage “diversity” or any nebulous concept of “well-­roundedness” or “character.” Here we have something like pure meritocracy. This is what it looks like: Asian-­Americans, who make up 12.6 percent of New York City, make up 72 percent of the high school. 

This year, 569 Asian-Americans scored high enough to earn a slot at Stuyvesant, along with 179 whites, 13 Hispanics, and 12 blacks. Such dramatic overrepresentation, and what it may be read to imply about the intelligence of different groups of New Yorkers, has a way of making people uneasy. But intrinsic intelligence, of course, is precisely what Asians don’t believe in. They believe—and have ­proved—that the constant practice of test-taking will improve the scores of whoever commits to it. All throughout Flushing, as well as in Bayside, one can find “cram schools,” or storefront academies, that drill students in test preparation after school, on weekends, and during summer break. “Learning math is not about learning math,” an instructor at one called Ivy Prep was quoted in the New York Times as saying. “It’s about weightlifting. You are pumping the iron of math.” Mao puts it more specifically: “You learn quite simply to nail any standardized test you take.

...

And yet the numbers tell a different story. According to a recent study, Asian-­Americans represent roughly 5 percent of the population but only 0.3 percent of corporate officers, less than 1 percent of corporate board members, and around 2 percent of college presidents. There are nine Asian-American CEOs in the Fortune 500. In specific fields where Asian-Americans are heavily represented, there is a similar asymmetry. A third of all software engineers in Silicon Valley are Asian, and yet they make up only 6 percent of board members and about 10 percent of corporate officers of the Bay Area’s 25 largest companies. At the National Institutes of Health, where 21.5 percent of tenure-track scientists are Asians, only 4.7 percent of the lab or branch directors are, according to a study conducted in 2005.

...

“The loudest duck gets shot” is a Chinese proverb. “The nail that sticks out gets hammered down” is a Japanese one. Its Western correlative: “The squeaky wheel gets the grease.”

Diane Ravitch

I briefly heard Diane Ravitch on NPR, during a recent commute.

She commented (I paraphrase) that standardized measuring/testing using was intended as a diagnostic tool. You find out what is lacking, and you take constructive corrective action. Unfortunately it has morphed into an punitive assessment tool, which seeks to punish a negative diagnosis.

Here is a link to a bunch of interesting articles by her. In one of them (Ravitch answers Gates):

Gates: "Does she think all those ‘dropout factories’ are lonely?"
Ravitch: "This may come as a surprise to Bill Gates, but the schools he refers to as "dropout factories" enroll large numbers of high-need students. Many of them don't speak or read English; many of them enter high school three and four grade levels behind. He assumes the schools created the problems the students have; but in many cases, the schools he calls "dropout factories" are filled with heroic teachers and administrators trying their best to help kids who have massive learning problems.

"Unless someone from the district or the state actually goes into the schools and does a diagnostic evaluation, it is unfair to stigmatize the schools with the largest numbers of students who are English-language learners, special-education, and far behind in their learning. That's like saying that an oncologist is not as good a doctor as a dermatologist because so many of his patients die. Mr. Gates, first establish the risk factor before throwing around the labels and closing down schools."

Republican Presidential Debate

Last night, I had trouble going to bed, and ended up watching almost the entire Republican presidential primary debate. While many supposed heavy-weights were absent, the actual debate was an interesting presentation of the potential diversity in Republican thought.

There was Tim Pawlenty, a front runner; Herman Cain, the African-American pizza-guy; Rick Santorum, the social conservative; and Ron Paul and Gary Johnson, the libertarians.

Not unsurprisingly, I found Ron Paul  and Gary Johnson the most interesting candidates. As a theorist, I find their uniform small government stance on both fiscal and social issues, ummm, how do I put it, more self-consistent?.

For example, when asked about gay marriage or prostitution, Ron Paul said a small federal government had no business regulating social aspects of life. Governments should not tell people how to live. I wonder how such positions will play with Republican primary voters.

How to convert color EPS figures to grayscale?

Some journals still will not let you use color figures without extracting a ransom.

This is a problem because other than print versions of journal articles, all other fora (presentations, webpages, posters, online journal articles) in which figures are presented, are color-agnostic.

In fact, there it is often desirable to present things in color.

I recently found a GPLed perl tool pscol which lets you quickly make grayscale versions of color EPS figures. You can easily weave it into a shell script to make grayscale versions of an entire directory of color EPS figures, very quickly.

The usage is very simple:

pscol [flags] infilename outfilename
Allowed flags are:

-h print this message and exit.
-gray convert RGB colorscale to grayscale.
-0gray convert RGB colorscale to grayscale (simple).
-cmyk convert RGB to CMYK.

For simple files the results are decent, for example:

becomes