omenti

paths past 269

Just a quick post as I've been without net, electricity and heat for the past week.  I finally have the modern conveniences and it happens to be election day, so a quick post.  There are a few other posts that should come in the next week as I had a lot of time to think.

 

Today votes are cast for the election of the President of the United States.  Technically voters don't directly elect the president, but rather 538 electors.¹  To be elected President a candidate needs 270 or more electoral votes.

 

There are dozens of polls all with some noise.  Some are designed and executed better than others, some have built-in biases and most tend to move around during a campaign.  The press tends to focus on them and take them as the odds that a candidate might win or lose the race.

 

Recently Nate Silver, author of the 538 blog, has been criticized for suggesting that President Obama currently has a better than 90% chance of being re-elected despite the likelihood that the election might be very close - possibly a one percent or less popular vote margin for the winner.  This strikes some as entirely daft - how could the President have such a good chance in such a "tight" race?

 

It turns out this is a nice example of a statistical approach.  There is no inconsistency with a close election and a relatively high probability of a certain result - in this case the result required for a win is the accumulation of 270 or more electoral votes.

 

Rather than rely on one or a small number of national polls, the trick is to take a group of state polls and calculate a total probability that a candidate achieves 270 or more in a combination of ways.  Silver has a model that has some assumptions about the past accuracy and predictive properties of each of the individual polls as do others who use this approach. 

 

There are many possibilities to sort through - something over 2.3 quadrillion.  No one would try to be exact and some who use this meta analysis of polls look at a few thousand possibilities that are usually chosen to look at the important "battleground" states.²  

 

It should be noted that the various meta-analysis approaches are currently giving Obama anywhere between a 70% and 90% of keeping his job.  

 

We tend to have a notion that the popular vote is important, but to first order it isn't.  There is a weighting forced by the electoral college - the game that really matters - and districting.

 

It is easy to make some wrong assumptions about the real problem, add our own biases (and there are many with politics!) and slip into innumeracy.  I'm reminded of a section from John Paulos' book  Innumeracy

  

If from some stock market advisor you received in the mail for six weeks in a row correct predictions on a certain stock index and were asked to pay for the seventh such prediction, would you? Assume you really are interested in making an investment of some sort, and assume further that the question is being posed to you before the stock crash of October 19th, 1987. If you would be willing to pay for the seventh prediction (or even if you wouldn't), consider the following con game.

Some would-be advisor puts a logo on some fancy stationery and sends out 32,000 letters to potential investors in a stock index. The letters tell of his company's elaborate computer model, his financial expertise, and inside contacts. In 16,000 of these letters he predicts the index will rise, and in the other 16,000 he predicts a decline. No matter whether the index rises or falls, a followup letter is sent, but only to the 16,000 people who initially received a correct "prediction". To 8000 of them a rise is predicted for the next week, to the other 8000 a decline. Whatever happens now, 8000 people will have received two correct "predictions". Again to these 8000 people only, letters are sent concerning the index's performance the following week, 4000 predicting a rise, 4000 a decline. Whatever the outcome, 4000 people now have received three straight correct predictions.

This is iterated a few more times until 500 people have received six straight correct "predictions". These 500 people are now reminded of this and told that in order to continue to receive this valuable information for the seventh week they must each contribute $500. If they all pay, that's $250,000 for our advisor. If this is done knowingly and with intent to defraud, this is an illegal con game. Yet it's considered acceptable if it's done unknowingly by earnest, but ignorant publishers of stock newsletters, or by practitioners of quack medicine, or by television evangelists. There's always enough random success to justify almost anything to someone who wants to believe.

There is another quite different problem exemplified by these stock market forecasts and fanciful explanations of success. Since they're quite varied in format and often incomparable and very numerous, people can't act on all of them. The people who try their luck and don't fare well will generally be quiet about their experiences. But there'll always be some people who will do extremely well, and they will loudly swear to the efficacy of whatever system they've used. Other people will soon follow suit, and a fad will be born and thrive for a while despite its baselessness.

There is a strong, general tendency to filter out the bad and the failed and to focus on the good and the successful. Casinos encourage this tendency by making sure that every quarter that's won in a slot machine causes lights to blink and makes its own little tinkle in the metal tray. Seeing all the lights and hearing all the tinkles, it's not hard to get the impression that everyone's winning. Losses or failures are silent. The same applies to well-publicized stock market killings vs. relatively invisible stock market ruinations, and to the faithhealer who takes credit for any accidental improvement, but will deny responsibility if, for example, he ministers to a blind man who then becomes lame.

 

I'm a huge believer in basic statistics and probability courses in high school - it is at least as important as basic calculus for modern citizens to sort through information available and not be so inclined to take predictions on face value.  It is critically important to understand as much as possible about information and how it is filtered and manipulated.  There is real danger in considering the thing some call "data" as pristine and reliable.

 

Back to the election.  Many segments of the universe of voters are extremely stable and elections can be influenced by encouraging or suppressing some of them.  Black and Hispanic voters tend to vote for Democrats, so elections with poor turnout can pivot favorably for Republicans and the urge for that party to suppress votes has to be strong.  On the other hand the Democrats would want to focus on voter registration and get-out-the vote efforts in districts where a strong turnout can make a different.  The same can be said about fundamentalist Christians voting for Republicans and there are a huge number of segments of varying degrees of strength and reliability.  All of this gives enormous opportunities for trying to datamine and produce lists so these potential voters can be reached and potentially influenced.  

 

It will be interesting to see how the meta-analysis folks make out this time.  They usually do better than those who analyze (and mostly over-analyze) popular vote polls.

__________
¹ The number is based on the membership of the US Congress: 435 House members, 100 Senators, and 3 electors from the District of Columbia.  

² There are different approaches.  The Princeton Election Consortium uses a technique that looks at the probability of getting an exact number of votes.  It turns out you can do this by calculating a relatively simple polynomial expression.  To get a deeper look at a daily prediction from the Princeton group check this out.

 

 

 

 

don't even bother trying to comb a hairy banana and a gathering of themes

Or a hairy ball for that matter.. at least not if you want to perfectly comb them so their hair lies perfectly flat everywhere on their surface.

 

Imagine you have an unruly donut hair covered donut and want to comb it flat (I can't imagine eating a hair covered donut).  It turns out to be an easy task.  Now think about the problem of combing a hair covered ball.  It turns out you can't do it perfectly - the best you can do is to make the fur like flat (locally tangent to it's surface) at every point but one where you are left with a cowlick. 

 

\ A banana is really the same thing as a sphere topologically.  Imagine the banana was incredibly pliable - really really ripe - and the skin just as flexible.  You could push in bits here and pull out others there and change its shape into a sphere.¹

 

The hairy ball theorem has been proven.² Math, unlike science,  is an areas where solid black and white proofs are possible.

 

It turns out there is always at least one point on the planet where there is no wind.  This point is the eye of a cyclone or anticyclone (wind spirals around it - air can't flow into or out of it as it is static).  So there is, and has been, at least one cyclone everywhere on Earth ever since there was an atmosphere.  You can say the same about any body in the Universe with an atmosphere.

 

so if you want to take bets ...

 

And if you wonder what brought this on, the banana I had for breakfast this morning was on the verge of being too ripe and looked like it might be growing hair one of these days.  I also have an unruly cowlick today - put this together and some math has to come out.  Math is neat stuff!

 

While I'm far from being fashionable, some of you play it well.  It is really fascinating to think about as there is an ephemeral nature and personal style can be very artistic.  In the US personal style for many has been replaced by inexpensive, ephemeral uniforms.  Getting something cheap that looks like something that a vetted style setter wears has become more important than finding a personal style.  In theory there is much more choice in stores, but in reality this translates into a very narrow "look".  

 

Jheri and I have been re-thinking our views on mass customization.  The genesis of this was marrying some work I did for the Defense Department with the need of Jheri and some others to find clothing that fits.  The notion was that mass customization was becoming easier and fit was the most important component.  While fit still is the most important component there are clearly other drivers as well as roadblocks.  Recently Jheri pointed to a new book on overly-cheap apparel (halfway into this post).  The book is mostly about the over consumption of very low quality fashion and the dramatic change that has swept the industry.  We are now wondering if some changes we're seeing among the 20 to 30 year old demographic may be important sources of kindling for the ignition of real mass customization - forget things like fancy printing technologies which are still a long way off from large scale viability ...

 

Apparel is still largely handmade at this point and the cost structure is going to keep it that way for at least another decade or two.  A pair of jeans may have a few hours of hand labor in them, but if you can do that for two dimes an hour, why change? Major designers are now working for the high volume firms, but the quality of material and construction is poor.  The middle tier of quality and manufacture has mostly disappeared.  A few speciality makes have appeared that exploit this need - Lululemon is a good example.³  Their quality matches their pricing, but this is fairly rare - to first order even $300 dresses are very poorly made.  Many consumers no longer have the ability to judge construction and material quality.

 

The signal we are seeing among at the edge - mostly in places like San Francisco, NYC (especially Brooklyn and lower Manhattan!), Seattle, Minneapolis and San Diego may be the rejection of overconsumption and more value placed on owning a smaller number of higher quality items that can last.  These are also areas that tend to support the notion of a strong sense of personal fashion.  There is more of the diversity you see in some European cities and mall uniforms are distinctly clueless.

 

New small business is emerging to fill these needs and some people are becoming craftsmen and artisans.  Clothing repair and re-tailoring shops are exist in greater number and I even saw a few new cobbler shops in Brooklyn last week.   While it isn't clear that this is a trend that will spread, it should be pointed out that the traditional apparel industry control on media is much weaker with forces like Pinterest exploding in popularity and Etsy creating a new channel for accessories.  

 

It is very curious to note that those who ply vintage stores note that items at least thirty years old are worthy of salvage and modification - newer items become progressively more poorly made to the point where anything made in the past decade is so poorly made that it isn't worth bothering with.

 

There are some great opportunities for design tools and even manufacture automation that may come to the small scale shops before the large manufacturers.  The big guys (like Levis) have tried what they thought was mass customization a decade ago and have mostly abandoned it.  I think this is like Microsoft's decade old notion of the tablet - they simply lacked the chops to dive in deep and re-imagine that was really needed. TRVL is another example of a publication more properly reimagined for the iPad world - and a need to break beyond conventional tools (like InDesign and pdfs used by the publishing industry) and thinking.  They were trying to hammer their square peg into a round hole and, guess what... it didn't work very well.  

 

It is likely this won't be very large at first - a ten percent shift in the market within this age segment would be big in my mind, but very indicative of a major trend.  We may see the re-emergence of the middle tier of apparel quality and manufacturing along with people taking a more focused interest in what fashion is - they may become their own artists rather than mere consumers.⁴  Remember - these are the guys who see iPads and creative tools.  My generation tends to see them (and very improperly!) as consumption devices.

__________

¹ Try a little experiment:  The famous example would be to change a donut into a coffee mug - cookie dough will show the way and had a big advantage over modeling clay in that it is edible. This can make for a sweet introduction to topology:-)

² There are many proofs. Most are pretty complex requiring some deep math - usually homology theory is involved.  I could follow some in the day, but now I'm more rusty - you have to keep at this to be in good form.  I did see an elegant and simple proof by John Milnor when I was a student and have it in my notes.  Since at least one of you is a mathematician and several are physicists and serious computer scientists it is reasonable to show it.  Normally it would take at least an hour to typeset it in LaTeX and arrange images to embed in html, but here is a nice reproduction in a math blog. The warning is this is probably greek if you haven't done an college level math - so please ignore it and accept this a proven result, but if you had an advanced calculus course, you have the tools you need and it is a sweet little proof.

³ I've visited their shops a few times.  Once with Jheri trying to learn a bit and then out of curiosity when I was in Manhattan.  Several of the customers told me construction and material quality is much more important than the very careful marketing - it represents good value and is what makes them repeat customers.  The chain has established itself as one of the few mid-quality outlets in existance.  Sort of a sad thing when mid quality is the highest level most of us have access to - and many of us can't even get that.  A real opportunity space...

⁴ Distributed design and IP ownership needs tools that are probably closely linked with social media.  Improved mathematical models to convolve design and fit patterns need to be made along with improvements in user/designer feedback.  Automated and semi-automatic manufacturing tools.  More uniform measurement tools, improvements in user and designer feedback, user and designer education tools ...  the list goes on and on and the surface has only been scratched.  But it is important to note that this is an enormous industry - something like a half trillion a year in the US.  Even niches can be non-trivially large.

__________

Before getting on with the recipe, I'd like to make a recommendation.  I'm afraid my education is very narrow.  High school was ok, but in college I focused and spend most of my time in the physical sciences and math - placing out of other subjects where possible.  A really big mistake in retrospect.  Then on to grad school with a much more intense specialization.    

 

I do read a lot and am reasonably curious, but there isn't a lot of structure to this approach.  Recently I've been taking a few courses on iTunes U (there are other flavors of this sort of online learning). I'm not diving in deeply and doing the homeworks and all of the readings at this point, but am enjoying those with great lectures.  It is an excellent way to pass time on long boring trips and during exercise sessions.  Over a three week period, for example, I listened to all of the lectures in David Blight's excellent History 119 course at Yale: The Civil War and Reconstruction Era. Since then I've read a few of the books on his reading list.  Great stuff and highly recommended!

 

All of the iTunes U online material is free and some of the classes are extremely interesting.  It isn't a substitute for deeply focusing and doing all of the coursework and interacting with the professor, but it is so much better than TV or movies.

 

Here is a guide I use to hunt courses.  I'm sure there are others - if you know of good ones, let me know.  And a recommendation for me - an excellent short introduction to Quantum Mechanics from Oxford by James Binney.  He does an excellent job.  The warning is you really need to work out the problems to gain any understanding of the subject.  The class notes, recommended books and problem sets are here.

__________

Recipe section

 

Yesterday saw 97°F here and the urge for a homemade milkshake was strong as recently I had a rather poor excuse from what had been described as the best ice cream parlor in the area.

 

Homemade tahitian vanilla bean ice cream, homemade chocolate sauce + broken bits of Lindt 72% (added after the blending).  I used my antique Hamilton Beach milkshake maker from the mid 1960s - a very high speed spindle and careful attention is the trick for velvety textured, very thick shakes.  This one was - well - perfect (sigh)……

 

I won't go through this one exactly, but rather give a cheater's version that may be just as good for you.

 

 

Chocolate-Chocolate Chip Milkshake

Ingredients (for one or two servings depending on how much you like milkshakes)

° a pint of super premium vanilla ice cream - I like Häagen-Dazs Five Vanilla Bean as it is reasonably well crafted, widely available and is very simple with only five ingredients. Depending on where you are you may be able to find better and a really excellent homemade can be far better.

° about 120g whole milk (a half cup)

° a good chocolate topping - I've been making my own and can't make a recommendation.  I've seen many in stores.

° 20-40g dark chocolate chopped into small bits - I used Lindt 72% 

 

Technique 

° For a thick and velvety shake you really need a high speed spindle.  I use an ancient Hamilton Beach single spindle unit from the 1960s.  It's shaft runs at 17,000 rpm and should be treated with care.  For a consistency like many current ice cream parlors use a blender, and for a very thick shake find a study container for the mixture and use a table knife and large, heavy spoon and a LOT of elbow grease.

° cool the milk to near freezing and let the ice cream sit on your countertop for about 3 or 4 minutes before starting to soften it.

° Mix the milk, ice cream and chocolate syrup and blend using one of the techniques above.  I start with eight parts of ice cream for every part of milk - if you like them to be drinkable use a four to one mixture.  If you are using a pint of ice cream, try about 2 table spoons of chocolate syrup for a very light chocolate and increase as desired.  I like the lighter version.

° After blending mix in the chocolate chunks with a table knife.

° Pour into a chilled glass and, if you like, top with a bit of whipped cream and/or shaved chocolate.  I couldn't take it anymore and just had it as is.