Smooth Transition Between Functions

 Stitching together two functions is sometimes required as a way to transition from one dependence to another. The following schematic describes the idea pictorially:

Two different approaches are considered in this PDF (or this Jupyter Notebook).

Snip Math

Mathpix Snip looks like an amazing tool.

You take a screenshot of some math and get it rendered in LaTeX.

The process as illustrated on their website:

It is available for download on all major OS.

The Beauty of Calculus

Here is a wonderful talk by Steve Strogatz at Yale.

It is based, in part, on his most recent book "Infinite Powers".

An Interactive Introduction to Fourier Transforms

Here is a great introduction to Fourier Transforms.

The focus is on developing a intuitive understanding for the decomposition of a signal into it component waves. You can draw different signals and "listen" to the contribution of the individual frequencies.

A good complement to this resource is this video from Grant Sanderson (3Blue1Brown).

There are numerous mathematical definitions of Fourier transforms. John D. Cook put together an exhaustive list, including pointers on how to move between the different definitions.

Euler and Graph Theory

I have been enjoying Marcus du Sautoy's fine podcast of famous mathematicians for BBC4. 

Yesterday, I listened to the Leonhard Euler episode. While I always knew Euler was one of the top mathematicians of all time, his contributions are truly remarkable.

The podcast talks about how he solved the seven bridges of Konigsberg problem by inventing graph theory, and proving its first theorem. I looked at that theorem as it applies to a "kids game" in a previous blog.

Complex Numbers: Part Deux

I was pointed to this excellent series on complex numbers from Welch labs, following my last post on complex numbers. It in the 3Blue1Brown mold, with just the right dose of insight and animation. The complex number series starts with basic ideas, and ends with a discussion of Riemann surfaces.

I also came across an interesting way of proving exp(ix) = cos x + i sin x (@fermatslibrary), which I feel compelled to share, since we are already talking about complex numbers.

Let \(f(x) = e^{-ix} (\cos x + i \sin x)\).

The derivative of this function is \[f'(x) = e^{-ix} (i\cos x - i \sin x) - i e^{-ix} (\cos x + i \sin x) = 0.\] Since \(f'(x) = 0\), the function is a constant.

Also f(0) = 1, which implies f(x) = 1.

Thus, \(e^{ix} = \cos x + i \sin x\).

PS: One of my students told me last week about the new podcast (Ben, Ben, and Blue) that Grant Sanderson (of 3Blue1Brown) hosts on math, computer science and education. It is delightful.

Complex Numbers

1. What really are complex numbers?

2. The joy of slightly fishy proofs.

3. This discussion on MathOverflow

History of the Logarithm of Negative Numbers

Not too long ago, I did a blog post on how matlab and python have different responses to logarithms of negative numbers.

It turns out that the history of the logarithm of negative numbers is truly fascinating, and had Leibnitz, Bernoulli, Euler, and the other greats embroiled. Take a look at this article (pdf) by Deepak Bal.

Here is the abstract:

In 1712, Gottfried Leibniz and John Bernoulli I engaged in a friendly correspondence concerning the logarithms of negative numbers. The publication of this correspondence in 1745 sparked an interest in the mathematical community on the topic. In this paper I will discuss the evolution of the logarithmic concept in the time leading up to their discussion. I will then give a synopsis of the correspondence followed by a description of a paper by Leonhard Euler in which he addresses the issue.

Linear Algebra Insights

3Blue1Brown has a wonderful new series on the "Essence of Linear Algebra", where he visually explores fundamental topics in linear algebra.

Worth a watch!

Kelly Criterion

Suppose someone offers you the following deal: you put down a certain amount of money. A fair coin is tossed.  Heads - you triple your money; tails - you lose the money wagered.

Loss Aversion

Q1: Do you take the deal?
A1: You probably should; the expected payout is 0.5 * $2 + 0.5 * (-$1) = $1 > 0.

Q2: Suppose you  have $1 million dollars of life savings. Do you go all in?
A2: You probably shouldn't. There is a 50% chance your life-savings will be wiped out! The downside doesn't justify the upside.

Payout based on fraction wagered assuming starting amount is $1, for a single bet

Now say the terms of the deal are extended.

Q3: The deal will be offered, say, once every month for the rest of your life. What fraction of your savings "f" would you wager?

If \(f \approx 0\), you don't play the game or bet small. You are probably not making the most of the opportunity offered. If \(f \approx 1\), you go all in. A single loss will wipe you off entirely.

The intutive answer seems to be somewhere between 0  and 1.

Further let us suppose that this "fraction" is a constant over all the (monthly) bets.

Kelly Criterion

The Kelly criterion informs the size of the bet, given the payouts and probabilities of possible outcomes. From the wikipedia article, if

• \(f^*\) is the fraction to bet;
• \(b\) is the net earnings per dollar wagered, if you win;
• \(q\) is the probability of winning;
• \(1-q\) is the probability of losing,

then the optimal fraction to bet is \[f^{*} = \frac{bq - (1-q)}{b} = \frac{q(b + 1) - 1}{b}.\] This fraction may be thought of as: \[f^{*} = \frac{\text{expected payout}}{\text{payout if you win}}\]

We can derive this formula later. Here, it is easy to see that, f* = (2*0.5 - 0.5)/(2) = 0.25.

Let's simulate a series of such bets. Suppose we start with an amount \(a_0\). After \(n\) bets, we are left with \(a_n = a_0 R_1 R_2 ... R_n\), or \[a_n = a_0 \prod_{i=1}^{n} R_i,\] where \(R_i\) is amount we are left with, for each available dollar, after bet \(i\).

Starting with $1, bet f*=0.25 of bankroll every bet for 100 bets.

The best one can do is to win all bets, leading to a payout of \[a_{n} = a_0 (1 + bf)^n.\]

The worst one can do is to lose all bets, leading to a payout of \[a_{n} = a_0 * (1-f)^n.\] Notice that in theory you never go bankrupt if \(f < 1\), since you are never wagering all your assets. However, in practice, there is usually a limit, since fractions of a penny may be unacceptable wagers.

Simulations

One series is anecdote. Let's collect some data, by repeating the experiment 10 times at different values of f = 0.25, 0.05 (smaller than optimal), and 0.6 (larger than optimal).

f=0.25. Black and blue lines are arithmetic and geometric means, respectively.

You notice that the arithmetic mean (the thick black line) is overly optimistic, and is heavily influenced by the lucky few outliers. The thick blue line is geometric average which seems more "reasonable". Geometric means weigh positive outliers less strongly.

To see this important concept in a more salient way, assume you took a series of 10 bets, and went all in (f = 1). Everything else about the problem is unchanged. Unless you win every round (probability of \(0.5^{10} = 1/1024\)) and end up with \(3^{10}\) dollars, you will end up with zero (probability = 1023/1024). However, the mean payout \(\langle a_n \rangle\) = 57.7 dollars, which obscures reality since the geometric mean and the median are essentially 0.

This is like playing a lottery.

If we use a smaller \(f\), we trade return for smaller variance. We are being more conservative when we do this.

Let's pick \(f = 0.05 < f^*\) to demonstrate this. For ease of comparison, I've kept the scale of the plot the same as the previous one.

f = 0.05

Now \(f = 0.6 > f^*\), shows how the downside can catch up with us, if we get too risk-on. Lot's of disasters.

f=0.6

This post has gone on too long, so I'll discuss the derivation of the criterion in a following post.

Proof Without Words

I was trying to explain the meaning and greater logic of integration by parts (beyond the mechanics) to someone today, and found this beautiful "proof without words" (PDF file).

Wikipedia reproduces this beautiful visual argument in its article on integration by parts. I like it, because the argument is more than a proof, it provides deep insight which a "normal" mathematical proof may not provide.

Upon googling, I found this nice related thread on Math Overflow, and this pdf (senior project?) document.

Mathy Drawing

Found this two drawing programs via MathMunch

1. Recursive Drawing

Recursive Drawing is an exploration of user interface ideas towards the development of a spatially-oriented programming environment.

2. Silk:  Beautifully done. With its haunting background score, the drawing experience is almost meditative.

Linear regression and logarithms don't mix?

You have a power-law model \(y=a_0 x^{a_1}\), and a bunch of experimental data points \[\begin{bmatrix}x_1 & y_1 \\ x_2 & y_2 \\ \vdots & \vdots \\ x_n & y_n\end{bmatrix}.\]
You want to estimate \(a_0\) and \(a_1\), it is tempting to take the logarithm of both sides \(\log y = \log a_0 + a_1 \log x\), and perform linear regression on suitably transformed experimental data \[\begin{bmatrix}\log x_1 & \log y_1 \\ \log x_2 & \log y_2 \\ \vdots & \vdots \\ \log x_n & \log y_n\end{bmatrix}.\]

Beware! You may get something different from what you expect! And your answers might not mean much.

Perhaps, you should be doing maximum likelihood instead. Here is a nice tutorial (pdf) on it.

PS: Part of the motivation for this post was this.

Subfunctions in Octave/Matlab

Consider the contents of the following Matlab program "mainFunction.m".

%

% A function that integrates the function

% given by the routine subFunction(x)

% over the range a and b

%

function I = mainFunction(a,b)
   a = -5;

   b = 5;

   I = quad('subFunction', a, b)

%  keyboard  % uncomment if needed

endfunction
 

%

% some nontrivial function

%

function y=subFunction(x)

   if( x > 0)

      y = x^2;

   else

      y = x;

   endif

endfunction

The function to be integrated, subFunction(x), can be saved as a separate file called subFunction.m, or, as in this case, it can be included in the same file as the main function. There are advantages to both these techniques.

If I store functions in their own individual files, I can call them from any other function. However, I can very easily end up with a lot of files. This may not necessarily be a bad thing, but it can be a challenge beyond a point.

If I use the subfunctions (as above), then the subfunctions themselves remain invisible outside the file in which they are contained. They may be accessed by the main function and other subfunctions within the same file.

Unfortunately, you cannot use "subfunctions" with script files. A useful workaround is to wrap a function declaration around your script, and use the command keyboard (as shown in the commented line in the example above) to transfer control out.

From here you can examine all the variables local to the "script".

When you are done, say dbquit, or return.

Nonsingular and Nondefective Random Matrices

In a previous post, I posed the folllowing question:

Consider again, a random square n by n matrix A, whose entries are restricted to the set of integers {-p, -p+1, ..., 0, ... p-1, p}. Each of the 2p+1 values are equally probable.

• What is the probability that this matrix is nonsingular?
• What is the probability that this matrix is nondefective?

For n = 3 and 4, using a simple Monte Carlo method, this is what I get. As p increases, we approach the continuous distribution of entries asymptotically.

Blue lines are for n=3, and green for n=4. Triangles and squares denote the probability that a random matrix is nonsingular, and nondefective, respectively

As we can see, both singular and defective matrices are extremely rare for large p. Between the two, defectiveness is a rarer feature.

Invertability of Random Matrices

Wikipedia says:

Singular matrices are rare in the sense that if you pick a random square matrix over a continuous uniform distribution on its entries, it will almost surely not be singular.

This means that if you use generate a random matrix (using matlab's rand function for example) the resulting matrix is likely to be nonsingular.

How likely? That depends on the size of the matrix.

To look at it slightly more quantitatively consider the well-known problem:

Consider an n by n square matrix A, whose entries are, with equal probability, either 0 or 1. What is the probability that this matrix is invertible (or nonsingular)?

Let us consider a slightly generalized version of the problem.

Consider again, a random square n by n matrix A, whose entries are restricted to the set of integers {-p, -p+1, ..., 0, ... p-1, p}. Each of the 2p+1 values are equally probable.

Thus, if p = 1, then the matrix A has entries {-1, 0, 1}.

• What is the probability that this matrix is nonsingular?
• What is the probability that this matrix is nondefective?

Mathematica Integral Fun

Consider the evaluation of the integral of 1/r between -1 and 1, in Mathematica.

• The function f(r) = 1/r has a discontinuity at r = 0.
• The indefinite integral of 1/r is simply log(r), which is real for positive "r", and is complex for -ve "r", and goes to negative infinity as r approaches zero.
• One might be tempted to think that getting definite integrals is easy if one knows the indefinite integral
• Not so fast. Discontinuties can complicate things!

Let us go back to the integral of f(r) between -1 and 1. From the symmetry of the figure it may be apparent that f(r) is an odd function, and the area under the curve (or the integral) should go to zero.

If we try to use Mathematica to integrate it with Integrate[f,{r,-1,1}], it complains that the integral does not converge.

We say, hmmm. Why don't we simply try to substitute the limits in the indefinite integral:

 

The answer makes sense since Log[1] = 0, and Log[-1] = i * pi, but is clearly not the correct answer?

The reason for this is that the First Fundamental Theorem of Calculus requires the antiderivative to be continuous over the range of integration, and from the second plot above, it is clear that this condition is violated.

So how do we figure this thing out? We could isolate the discontinuity by integrating close to zero (using "epsilon"), and perhaps take the limit later.

This gives the expected answer of zero.

The Next Best Mate

The so-called "Sultan's Dowry Problem" (a.k.a "Beauty Pageant Problem", "Secretary Problem" etc.) is sometimes used as a model for searching a mate. It is a nice model for decision-making under a certain type of uncertainty.

According to the wikipedia entry:

The basic form of the problem is the following. Imagine an administrator willing to hire the best secretary out of N rankable applicants for a position. The applicants are interviewed one-by-one in random order. A decision about each particular applicant is to be taken immediately after the interview. Once rejected, an applicant cannot be recalled. During the interview, the administrator can rank the applicant among all applicants interviewed so far, but is unaware of the quality of yet unseen applicants.

There is an elegant solution to the problem, when the objective of the game is to maximize the probability that the candidate chosen has the highest quality. We assume quality can be reduced to a simple number. The so called "1/e" or "37%" solution to this problem involves letting the first 37% of the candidates pass, remembering the quality Q of the best candidate from this set. Thereafter, the first candidate whose quality exceeds Q is chosen.

Todd argues that the way humans search for their mates is very different from this optimal solution. He points out that the 37% rule finds the best solution more often than any other algorithm, 37% of the time. However, what happens during the 63% of the times is not very flattering:

For instance, if applied to a set of 100 dowries ranging from 1 to 100, the 37% rule returns an average value of about 82, that is, the mean of all dowries chosen by this rule. Only 67% of the individuals selected by this rule lie in the top 10% of the population, while 8% fall in the bottom 25%. And it takes the 37% rule an average of 74 tests of potential mates that is, double the 37 that must be checked before selection can begin before a mate is chosen.

This probably explains why normal people do not apply this strategy. It turns out that normal people tend to use a much smaller "screening" period. It turns out that the length of the screening period is dependent on your appetite for risk.

If you are fixated on maximizing the probablity of ending up with the best candidate, then the 37% rule works fine. But if you are a risk minimizer - if you would rather protect your downside - while accepting anybody in the top 10% for example, the optimal screening period is much shorter - closer to 10%.

I first heard about this problem more than a decade ago, and have been fascinated by it ever since. I used this problem to create a programming assignment in the class I am currently teaching.

Math Links

1. Two nice math videos (H/T Wild About Math)

2. The Cold Hit Problem (or are fingerprints unique?)

Interesting Calculus Sites

1. Step-by-step Calculus from Wolfram Alpha: I wrote about Wolfram Alpha before. It apparently can show intermediate steps, which could be helpful for someone learning calculus.

Or perhaps, it may cause more harm, I don't know.

2. MyCalculus: I took it for a short test drive, and it seems to work fine. To see intermediate steps, it seems like you have to buy stuff, though.