ChapterZero

Seldom is it that one of my friends asks me a math question that I have or can readily find a good answer to. One of those rare occasions was yesterday, when one of my friends in the EE department asked me what’s the deal with eigenvectors and eigenvalues? After all, we learn about them in engineering math, but then it seems like we don’t use them for anything. I suspect that most EE majors feel the mystification about the use of eigensystems.

Well, the useful thing about eigensystems are they (in most situations) let you diagonalize linear transformations. Recall a linear transformation , where is a finite-dimensional vector space, is diagonalizable if where is the matrix whose columns are the eigenvectors of and is the diagonal matrix with the associated eigenvalues. Diagonalizability is useful because it gives us a quick and easy way to calculate the value where is an arbitrary vector in : first we calculate the coefficients of relative to the system of eigenvectors, which are a basis, using , then we multiply these by scalars, by calculating , and transform this back to coefficients in terms of the standard basis, using . The quick part comes about if we agree to express every vector in terms of its coefficients relative to the basis of eigenvectors, then to calculate , all you do is multiply by the diagonal matrix , a very fast calculation. Another useful feature of diagonalizable transforms is that we need only know how they operate on their eigenvalues to calculate how they operate on all vectors in the space.

As a practical example of a diagonalizable transform, consider the example of the boost (the transformation between two inertial frames with constant relative velocity ) in special relativity; one way of determining the form of the boost is by realizing that its eigenvectors (the unit vectors of the light cone– since the light cone is preserved under any such change of frames) form an orthonormal basis for . If you specify or determine how much it scales the eigenvectors (i.e. the eigenvalues), you’ve calculated the boost! Calculating the boost this way allows you to effectively ignore the complicated nature of the boost (e.g. it is a hyperbolic rotation), and just focus on its simpler actions on a fixed set of vectors.

The same ideas extend to infinite-dimensional vector spaces (Hilbert spaces), such as spaces of functions. I’m less clear about the specific mathematics involved here, since matrices probably no longer apply, but it is clear that the eigensystem and diagonalizablility concepts have just as important applications here. Take the usefulness of Fourier series, for example. The only thing useful about F.S. is that they diagonalize linear operators: if you express a periodic function in terms of a sum of sinusoids, and apply a linear operator, you get back a sum of sinusoids of the same frequencies, with different weights. That is, the F.S. expansion process acts like the matrix above, and the summation process acts like the matrix , while the linear operator ‘applied in the frequency domain’ acts like a diagonal matrix, in that it simply scales the and at a given frequency. This process works because the eigenvectors of linear operators are the sinusoids.

There is a similar process going on with the Fourier Transform, but I think that doesn’t really diagonalize .

I’m writing an essay on Stanislas Dehaene’s paper ‘Precis of The Number Sense’, a synopsis of his longer book ‘The Number Sense: How the Mind Creates Mathematics.’ His position is that the number sense, our ability to understand and manipulate numerical quantities, represents a biologically determined category of knowledge. He argues pretty convincingly for this, using four main arguments: 1) the presence of the evolutionary precursors of number sense in animals, 2) early arithmetic competence in infants, independent of their other capabilities, 3) homologies between adult human, infant human, and animals’ number processing abilities, and 4) the presence of a dedicated neural substrate for number processing.

There is a comparatively short section at the end of the paper where he addresses some questions in the philosophy of mathematics: why is mathematics so effective at describing the world? and what causes mathematics geniuses?. In the first case, he says mathematics beyond the biological level of numerical discrimination, which was supplied by physical evolution, is in a constant state of cultural evolution, where inefficient, inelegant, and useless mathematics is discarded as we go. In the second case, he says mathematics geniuses are the result of dedication and training— I’m not sure I agree with this: what about Ramanujuan, who had no training, yet managed to reproduce and surpass contemporary mathematics, all on his own? (Incidentally, he dispells the famous myth about Ramanujuan’s death-bed comment to Hardy).

I never thought I would enjoy statistics, but since I’m taking a course on stochastic processes this semester, and I know just about nothing about statistics, I started doing some reading. I checked out the classic– and deservedly so– ‘Probabiliy, Random Variables, and Stochastic Processes’ by Papoulis, et. al. I haven’t had time to read anything more than the first five chapters, but that’s enough to let me know how interesting this material is. And looking at the title of the following chapters is making me salivate. Too bad that someone put in a request for it, so I have to return it to the library by the 18th.

I suspect part of the reason I disliked statistics in the past is the way that the reasoning and non-trivial calculations that motivate certain definitions get cut out completely, and the class is reduced to plug and chug. Like, where did the statistic come from? I don’t know, but I can– or rather, I could when I was taking the engineering stats class– tell you how to use it in this formula to calculate this… A lot of the beauty I’m seeing now arises from the struggle to manage the complexity of the calculations involved, which leads to some clever definitions and ideas. To be as vague as possible

At the same time I’m taking the stochastic processes course from an engineering perspective, I’m also taking a math course in measure theory, which despite the fact that it is not aimed at a measure theoretic interpretation of probability theory, is giving me some neat ideas and insights on the stuff we’re just to memorize for the engineering course. E.g., why the cdf of a random variable is always right continuous.

We haven’t covered anything spectacular in class yet; so far, the most interesting things we’ve done are integral calculations of the sort you probably only see in probability theory, and using the characteristic function and moment generating function. Hopefully by the end of the semester, I’ll have a good idea of what Markov chains are, and how to use them in applications. And, some statistical signal processing tools in my kit

I’ve been working on the cardiac imaging digital contrast agent project that Bernhard, Papadakis’ last postdoc, was carrying before he got a professorship at the University of Waterloo. Since he was, by all accounts, a genius, and maybe thought that what his code does, and how to use it in conjunction with microView is perfectly obvious… that, and the fact that I’m not a genius, and don’t even understand the paper detailing the algorithm his code implements, has left me spinning my wheels.

It looks like– and here I may be being overly optimistic– Papadakis may be flying Bernhard back to Houston to explain his work to us. This is a very good thing: although I’ve spent much more time on it than I thought I would have to, I still don’t know how to reproduce the results he had. I’ve started reading more on wavelets again, in the probably vain hope that by the time he gets back I’ll have a more solid understanding of the algorithm. The funny thing is, I don’t really need to know how the code works, so much as the logistics of how to choose reference regions from the data input, and processing the data output appropriately.

I had a ‘discussion’ with my dad and my uncle about my future as a mathematician this Sunday, which mostly consisted of my father trying to convince me that I’d be dirt poor and unhappy— instead I should continue as an EE, and get a PE! The motivation behind this, is IMO, his bitterness at being dependent upon someone else for his living; in that sense, there’s no arguing that getting a PE is probably a better course than getting a Math PhD, although having a PhD does not preclude starting your own business (e.g. Waterloo Maple, Wolfram, Thinking Machines, etc.)— in fact, I fantasize about doing that some day, producing high end tools, currently of unknown purpose, for technical and academic professionals. From that angle, I won’t try to argue the point.

But it pissed me off that he didn’t take the time to do the research on what a career mathematician can expect in terms of employment and income. So, I tracked down the AMS employment survey I saw in an edition of the Notices: http://www.ams.org/employment/surveyreports.html. Let’s see if we can have a more reasoned discussion now.

I dislike the fact that I’ve spent more time learning about Fourier series/analysis and related subjects than any other similarly coherent subfield of math, and yet I know almost none of the relevant equations and theorems off the top of my head. Either I’m missing a constant factor somewhere in an equation, or I don’t remember anything except the gist of a theorem. I suppose this is because I am a lazy bum, who for the most part abhors memorization.

Today, I spent sometime reconstructing the DFT, and some more time recovering an intuitive basis for it. Generally, you don’t learn this stuff at the undergraduate engineering level– you’re just handed the equations–, and you don’t spend enough time on the DFT in a typical undergrad. math course to get to the point where you’d see this, so I thought I’d give a nice framework for the DFT. This is really only useful for engineering types, because math types would find these observations obvious:
The DFT of a vector is the vector where

I like to think of the DFT in dimensions as an automorphism on . Note that the ordinary Fourier transform is not an automorphism, because not every integrable function is the Fourier transform of another integrable function. The useful thing about the DFT is that it allows every vector to be expressed in terms of a basis of frequency vectors, in the same way that Fourier series allow a large class of functions to be expressed in terms of a basis of sines and cosines. In fact, the concept of periodic extensions, useful in extending the idea of Fourier series to apply to non-periodic functions, is at work with the DFT. In the DFT, we consider a vector to represent the infinite sequence of period where one period consists of the coefficients of :

This allows us to consider and to be -periodic functions, which makes life easier sometimes.

Right now, I’m downloading Yafray and Blender— they seem like the best freely available combination for messing around with 3d graphics generation. I don’t really have too much interest in the conventional “photorealistic” renderings, but I’ll admit they are more than slightly impressive on that front. What’s more, the downloads weigh in at less than 7Mb total! I intend to get familiar with them, then figure out how to modify Yafray to support alternate lighting and rendering models. Specifically, I’m thinking of toonish and painterly rendering. Not that I have much use for them, but it seems like yet another cool project idea. Also, Blender seems Python scriptable, so if I get any ideas on the geometry side, maybe I could hack it up in there. For instance, there’s a Python plugin for Blender that assists in the design of humanoid characters. If the scripting engine is up to that, I should be able to decimate polygons and mess around with implicit surfaces.