omenti

baseball and machine learning

A few weeks ago two of us were talking about the perception of "now".  A fusion of sensory inputs, responses and thoughts takes place and a highly edited version is assigned in what we perceive as a smooth and linear flow of time.  Consider vision.  It turns out it takes us around three tenths of a second to put together a meaningful image from the time light hits the retina. First we note color, a bit later motion and finally form.  No big deal unless you have to react very quickly.  For most of us that isn't a big deal.  For athletes it is.

 

Baseball is useful as the time it takes for the ball to travel from pitcher to bat isn't much longer than the time it takes to process an image. But baseball "works."  To examine vision in baseball consider the great women's fastpitch softball player Jenny Finch.  A few years ago she broke what the best Major League Baseball hitters are capable of. 

 

Their problem is they don't know how to "read" her.  Beginning as little kids  they learned how to hit a ball getting progressively better over time.  Early on the time of flight for the ball was so slow that human vision and reflex speeds weren't the major issue.  With practice they began to learn how to read the pitcher.  Posture, arm movements and so on.  The shards of image they formed gave them information on the physics of the ball's flight path.  They were learning all of the patterns even though they thought  that they really sensed when the ball was released and when it hit their bat - after all, that's what their mind told them.  But what their mind put together wasn't accurate.  In reality they were linking reads and other early information with ball trajectories.

 

Jenny comes along with a pitch that is over ten meters per second slower than what they're used to, but the information they receive doesn't link reliably with ball trajectories.  They either try to rely on reflexes missing the ball entirely as it flies past before they can swing or they use a mistaken trajectory from their mistaken read and strike out. They could learn... give them a lot of time working with women's fastpitch experts and they'd finally build the appropriate library.  Ultimately some of them might even play the game well.

 

There's a striking similarity with machine learning.  You show a program a huge amount of data trying to be clever and you get a program that can quickly do great pattern recognition.  At least within a restricted domain.   The richness of training datasets is often poor.  Facial recognition programs might have respectable results for white people, but completely fail with people of color.    Such biases of omission are very common, but many other types of bias exist.  (again a rich and deep subject well beyond the scope of an hour). 

 

Machine learning can be useful but is only robust when the limits of where you can use it and the types of errors it makes are well-understood.  There are many areas where it's sort of good enough even though these limits aren't well established. Other areas are more critical.  We're a long way from it being generally useful in areas the tech press considers nearly solved -  self driving vehicles come to mind.

 

Back to the athlete.  Sports are played at many levels.  At an amateur level we play because it's fun.  Beyond that most of us are watching others.  In direct competition we usually prefer watching serious pros.  Beyond the beauty of watching human motion being executed so well there are those magical times where something extraordinary happens.  Athletes at the highest level develop a deep sense of the game and, in addition to being able to react faster than their vision and reflexes would suggest, are tracking and processing other channels of information.  In some games players develop a "field sense" .. they know where the other players are and how they're moving.   You see this dramatically in beach volleyball when played at the highest level and it is common in soccer and basketball.  This gives them the ability to do the impossible. These are very beautiful human moments beyond machine learning.   And it goes beyond sport ..  think of the human experts that just out of the ordinary when the time comes - like Captain Sully Sullenberger landed in the Hudson. 

 

We tend to think about athletes as having some physical gifts and a lot of training.  Certainly those are necessary, but there's an impressive mental component.  We don't say someone is good  at foreign languages because they have a dexterous tongue.

 

machine learning without a net .. from sports to smart toasters

And I don't mean Cylons.

 

Every now and again you notice a few trends coming together even though they hardly seem related at first glance.  Someone asked a question today that turned out to be something I've thought a lot about over the years - a collision between tiny machine learning, privacy and an ocean of tappable ambient information.

 

Twenty years ago some of us were playing with sound. After spending a lot of time analyzing it we wondered what areas existed beyond voice recognition and music. There was a small "name that tune" excursion where we tried to match sung, hummed or whistled bars of music to - well - name the tune.  We were trying to turn the sound into note patterns.  It sort of worked, but the the problem of building a notation library was too difficult at the time.  The easy way out would be a hammer and tongs approach - namely matching played music with a library of played music.  We filed a patent on it, but never pushed hard as we didn't see the use case. Oh well...

 

Next I tried to recognize misbehaving car sounds..  Auto mechanics have great stories of people trying to make broken car sounds.  Unfortunately the problem was not small enough and I shifted gears when a ventilation fan in my office started squealing.  

 

Big heating, ventilation, and air conditions systems mostly run with boring industrial sounds, but a good HVAC mechanic can recognize when something's about to fail.   They also tune these systems for efficiency by ear - for some systems the sweet spot isn't far from the point where problems happen and every installation behaves differently.   With my trust Sony DAT recorder I made recordings of these systems in various stages of failure.  Then some fancy pants signal processing and, lo and beware, you can detect and analyze failure modes.   There was probably a lot of interesting work that could be done, but my organization evaporated around that time. 

 

I was fascinated with low power computing a bit earlier than the sound work.  How little energy does it take to do a computation?  (you can make some interesting entropy arguments and get down to basic physics.)  More practically what could be done with very low powered sensors that didn't need batteries or wires?   The interest took several directions - all good as you slowly begin to learn.

 

Then about five years ago it struck me that the HVAC problem could be done more easily with machine learning.  I've expressed reservations about the misuse of machine learning and regard some of it to be not very different from phrenology, but potentially very useful where you can understand it and its use.¹  I supervised a senior thesis project that attacked the HVAC project using machine learning and a microphone.  It showed some potential, but for me the learning was finally realizing how computationally cheap machine learning can be at the low end.  I also ended up learning a fair amount about machine learning as the best way to learn is to do it yourself and then teach.

 

Given Bitcoin mining efficiency and the massive machine requirements at FaceBook and Google it isn't all together clear, but think about unconnected machine learning on your iPhone and perhaps your Apple Watch.  In conventional computing the CPU deals with a large number of instructions compared to the type of processor used in machine learning.  ML processing chips have a large number of tiny processors that work in parallel. Most ML is just matrix multiplication.  While there can be a lot of steps, each is really simple .. generally just addition and multiplication.  A properly designed ML element holds the numbers in the matrices its working memory cache.  It's much more efficient in time and energy use to keep what you're computing nearby rather than having to constantly read and write it to memory.  You should be able to do a specialized bit of silicon with enough ML horsepower to do some rather surprising tasks while using very little power.  And you can do a lot on a smartphone if some of it's silicon was designed to support ML.

 

I think Apple understands this at a rather deep level.  They're very careful about how to efficiently deal with graphics processors which is shorthand for saying they have optimized how to do huge volumes of very very simple computations.  The chips in the current iPhones are optimized for the task.  They've build machine learning machines you can hold in your hand and, in theory, run ML applications that don't connect to the net and divulge information.  The new measuring app on iOS 12 is an excellent example of a local machine learning deployment. I don't know if it is being done on the watch, but there's no reason why they couldn't. Perhaps the biggest design advantage Apple has these days is silicon. 

 

machine learning in your hand rather than the cloud.

 

I can think of a number of applications that use the sensors on the phone or watch -- from health, to motion analysis (think sports).. why you could imagine wearing a watch and using a couple of iPhones on tripods or held by friends to analyze your volleyball, gymnastic, or whatever moves.  A personal coach or a serious tool is you already have a coach.

 

Exciting stuff, but I suspect it goes much deeper.

 

A back of the envelope calculation suggests you can do useful ML with micropower.  I'm sometimes asked to come up with senior projects in engineering classes (dangerous as I'm not an engineer).  I think you could do the HVAC work with an accelerometer or microphone and simulate the type of compute power that, if done in silicon, could run for months on a watch battery.  Maybe some very simple chemical analysis.   Or recognize a very small vocabulary -- perhaps five words - to control your toaster,  thermostat or anything else.  You could use a key word to make it listen or shine a laser pointer on it.  You could also build a very simple image recognizer that would detect someone's gaze.  Look at your toaster and say medium please.  

 

Way safer than saying medium please to a Cylon.

 

A need for privacy may drive this even deeper and away from the Net.  You can sit down and go through a few dozen applications in an afternoon.  It seems very rich and very latent.    Apple knows how to do silicon and low power.

 

So many possibilities.. perhaps even my dream "big data" project of listening to the woods. 

__________

¹ I've written and talked a fair amount on problems.  It can get technical and I won't spend more time on them here, but it's a serious problem as surveillance systems transition to judgement systems.

 

raindrops keep fallin' ...

A few people sent links to the announcement of a "revolutionary" advance for solar cells - namely to let these large areas that would be unproductive in a rainstorm, be productive.   Numbers weren't mentioned, but it's worthwhile thinking about the problem.

 

The Sun heats evaporates water which rises up into the atmosphere and can condense and precipitate. A question is how much of the energy that went into evaporating the water can be recovered?  If rain could fall without resistance it would be a lot.. also walking outside in a storm could be a lethal activity.  The simple physics is the moisture in the sky has a considerable amount of potential energy relative to the ground it will fall on ..  as it falls the potential energy turns into kinetic energy .. you can work out some numbers if you like.¹  But that's not what happens.  First think about it.. a sanity check ..  are you feeling the force of rain when there's no wind?  What does that say about power density?

 

Raindrops run into air resistance and their terminal velocity is low..  fast ones are traveling about 8 meters per second - about 18 miles per hour.  You can throw a ball a lot faster than a raindrop.  So now the problem is simple.  You don't have to know how far it falls.   So here's a sample calculation I did in my head when I heard about this.  The kinetic energy for one centimeter of rain falling on a one square meter area is 320 joules.²  That isn't much ..  about 0.09 watt-hours per square meter.  One way to think about it.  If you had a 100 square meter array (large for a house!) you'd average 9 watts during a one hour storm that put down a centimeter of rain .. an inch of rain would get you into the 25 watt range ..  less as the converter won't be close to 100 percent efficient, but maybe enough to run a small LED lamp during a strong rainstorm.

 

 On my rowing machine I regularly convert power from my muscles into mechanical power at a flywheel averaging about 150 watts or so.   A good generator could convert about 90% of that into electricity.   A photovoltaic in cloudy Germany averages about 40 watts per square meter across the year (including night) - that includes the conversion efficiency.  Problem solved..  this is impractical for solving the general energy problem... no way would it average out periods where other forms of renewable power aren't working well.

 

So now for the more interesting question.  Why were people excited?  I think there may be several things going on.  

 

First it appears a group did make an advance in turning mechanical energy into electrical energy with piezoelectrics.  Traditionally this is very inefficient - one percent or so.  They've made an advance and it could be very useful for niche applications like low power electronics located where it is impractical to run power cables or replace batteries.  Aces for the team.

 

These announcements often fall into the hands of university or corporate PR people who don't understand what the invention or discovery is and don't ask questions or get sign off. This turns out to be a major problem even at name universities and often worse in the corporate world.  I don't blame the PR people as getting this right is a non-trivial problem.  The announcement is picked up by a tech writer who lacks a clear idea of the announcement, but thinks they understand.   They write something right out of the telephone game.   Meanwhile the latest version is picked up by social media or others who are too lazy to check things out.  The reader needs to learn what sources are trust-worthy and, if the work is important to them, figure out how to drill down.

 

There's a lot of interesting physics thinking about rain and even liquid water,  And water for energy storage can be inexpensive and enormously practical if you have the right location (a mountain lake for example).  Denmark cleverly uses Norway as a storage battery for wind energy to even out the load.  This may become less practical for the Danes as Norway is seeing a larger demand for electricity at night recharging electric cars so it will come down to economics.

 

__________

 ¹ gravitational potential energy mgh becomes kinetic energy mv²/2

² E = 1/2 mv²   where the mass of the water is density times volume.  The density is very close to 1000 kg per cubic meter.

 

the next step in home audio?

 Still cold from the UPS truck the Apple HomePod woke up when I plugged it in and asked my iPhone for a dance.  They spun about and chatted in their local network for a minute or two before Siri introduced herself.  It was completely unexpected surprise from a great friend who  shares the love of music.  A day into the journey I have a few thoughts.

 

I'm a poor musician, but find it an important part of my life.  Grad school was at Stony Brook.  Besides an excellent physics department one of my reasons for doing grad school at Stony Brook was its proximity to music in  New York City..  That wasn't quite enough, so two of us founded a concert series at the university that thrives to this day.  At Bell Labs it became apparent that music might be a significant driver for Internet use and I found myself walking several paths.  One was sound field reconstruction.

 

Sound field reconstruction is just the fancy way of  recreating the sound where it was recorded or engineered in your home.  In theory you record the acoustic signatures of the recording studio and your living room and create a mathematical operation that makes your living room sound like the recording space.  In practice it's approximately hard.  Sound waves are fairly large and get reflected, absorbed and transmitted by and through everything in your room.  Making matters worse the sound field at different spots in the room is usually very different and things in the room move between and during listening sessions.  We made serious progress, but it took exotic microphone and speaker arrays as well as state of the art (for the time) computing to make it sort of work.,

 

One detail is centrally important.  Imagine sitting before a simple speaker cone.  Most of the sound comes straight at you with the volume falling off as you get away from the speaker's central axis.  Speakers have very characteristic patterns showing how loud they are depending on the angle you're sitting at as well as the frequency of the sound they're producing.  If you have two stereo speakers there is usually a sweet spot where the sound reproduction is optimal.  In this area the sound field can be fairly good and you can close your eyes and visualize where the musicians are.

 

Imagine an array of speakers, each with its own amplifier, all controlled by a computer, you can control the phase and amplitude (loudness) of each speaker.  The waves they send out interfere with each other as they expand outwards.   By controlling the phase and loudness (amplitude) of each speaker you can make them interfere in such a way that they create a new wave that can be aimed.  The little animation shows an array of four speakers in a row, but it can be any number and they don't have to be in a line.  The HomePod uses seven small speakers arranged in a ring.  

(you may have to click on the animation to start it)

 

The HomePod also has a ring shaped array of six microphones.  They listen to the room (sample it) at a fairly high rate and decide what parts of the music, from its stereo signature, need to go where and how it needs be modified to interact with the room and everything in it.  An Apple A8 processor, the same two billion transistor processor used in the iPhone 6 and 6 Plus, handles the computational chores.  This is much more power than we had back in the day.  Although I don't know exactly what they're working on, several people I know who are real experts at this have been at Apple for years.

 

 

The HomePod also has a computer controlled single woofer with its own amplifier mounted vertically to handle the low notes.  

 

I'll leave the specsmanship, mechanical and most of the UX discussion to others and just talk about what it sounded like.  I'm not golden eared, but rather a serious listener.   I focus on the music and let it take me where it pleases. 

 

I listened to works across several genres with the HomePod placed at about ear height in a small acoustically complex room.¹   For very small groups - soloists and piano quartets for example - the sound stage was excellent ..  about as good as my stereo pair of conventional speakers (about $3,000 in speakers).  The HomePod didn't match the reference speakers when I was sitting at the sweet spot, but was better when I moved around the room.  That is astounding. 

 

It was creating stereo -a  term has nothing to do with a pair of anything, but rather three dimensional.  Moving  beams of audio around it was painting a sound field in my living room. 

 

As I listened to larger ensembles I found a less accurate sound field.  That said it was still comparable to my reference pair when I was away from  my very small sweet spot - its effective sweet spot was larger.   Note for some pieces of music a good $500 pair of speakers would do a better job if properly installed, but  would love to see what a pair of HomePods placed about two meters apart would do.  Apple has promised the feature will be available later this year.  I've already earmarked the funds for immediately buying a second unit then.  I suspect a pair of HomePods will blow my reference pair away when it comes to sound stage placement.   That will be even more true for those who haven't carefully adjusted their current speakers with test recordings and a spectrum analyzer. 

 

You can control it from an iPhone, iPad or Mac, but Siri is the intended interface.  I've had poor luck with Siri on my iPhone, but here it works much better.  It's also much more robust to interference.  Microphones beamform too and it's clearly listening to you doing its own cocktail party trick.   That said it, and every other interface I've seen are just wrong when it comes to classical music.

 

Comparing it to the networked speakers from Amazon and Google is silly.  Their speakers are terrible and useful only as acoustic wallpaper you listen to in the background.  They're really microphones in your home listening to you assuming you agree to Google's  or Amazon's value proposition (I don't).  Privacy is vastly different - I'll go with Apple's differential privacy rather than Google's "you are the product" flavor.   But if music is secondary and you want a digital assistant with a speech interface, Google and Amazon are the ticket.  Apple is going after the other audio companies along with making its platform stickier in the process

 

I suspect the speaker will be like the Apple TV - fairly stable hardware with the bulk of improvements coming through software updates.   My list of UI and UX issues is growing, but I suspect this will become part of my life. 

 

And a final caveat - speakers involve personal taste.  YMMV

 __________

¹ I'm very familiar with these pieces.  With one exception  I've heard them live and I've even been involved in the recording process on two. 

Ecstacy  Anonymous 4 and Mike Marshall

Us  Regina Spektor 

Seven Swans Al Petteway

The Heart Asks for Pleasure First  Ahn Trio

Black Horse and the Cherry Tree   K.T. Turnstall

The Littlest Birds  The Be Good Tanyas

Submarines The Lumineers

Pipeline (electric and acoustic versions) The Duo Tones

An Spailpin Fanach Connie Dover

The Heroic Conditions of the Universe Pt 7 After the Storm Alexandre Desplat

Turn it Around Lucius

Jolene Mindy Smith's version

Vespro della Beate Vergine: Domine ad Adiuvandum a 6 Monteverdi

Shchedrik Kitka

Our Town Iris DeMent

Sun Will Set Zöe Keating

Don Giovanni Madamina, IL Catalogo and Questo Mozart  (sort of a sign of politics)

Canon a due Violincelli in D Major Gabrielli

Cello Sonata in D Major Op 69 Beethoven

Suite No 6 in D Major  JS Bach (Yo-Yo Ma)

   

what does dim mean and a most unusual job

You're outside on a dark and starry night wondering about those stars.  Is someone out there? You take your flashlight out and point it at the sky.  Your message in a bottle tossed into the cosmos.   I'm guessing most of you did it when you were a kid.  The problem is it doesn't work.  The light you send skyward is scattered or absorbed before it gets out of the atmosphere. At least your imagination was fired - what would it take,  There are people who point lasers beams skyward  and that leads to one of the most unusual job openings I've ever seen.  But more on that later,  there's another interesting question to ask first.

 

Our eyes are amazing at what they do.  We get reasonably detailed color images along with less detailed, but much more sensitive night vision.  We usually think about owls as having some of the best night vision eyes.  They're pretty good, but some frogs can detect a single photon of light.  So image this.  You're a frog watching a light that glows with an unwavering brightness.  If you were a space frog it might be the Sun.   You measure the brightness of the light and then start a journey outwards.  Twice as far from the source the light appears a quarter as bright.  Three times as far and it's a ninth as bright.  One hundred times and it's one ten thousandth it's original brightness.  The measured brightness at your distance goes as 1/r² where r is the distance.  The inverse square law.¹ You continue on and and the light gets dimmer and dimmer.  How did can it get and what would it look like?  I posed the question to a class of premed students I was teaching physics to..

 

The light gets dimmer to a point - the single photon point.  Rather than diminishing further it begins to flash at that brightness. .  At first are flashes most of the time, but travel further and there an increasing number fleshless regions.  Add up the energy you receive - the number of photons over some unit of time.  The frog is  bored and haven't croaked yet, so it plots the energy over some amount of time .. just the number of photons  (flashes) it sees in some length of time.  The curve is still the inverse square law.  

 

Light, as it happens, comes in chucks.  In physicsy terms it's quantized and that chunk is a photon.²

 

When first encountered the paper on the sensitivity of frog's eyes I was blown away.  For perspective a 60 watt equivalent light bulb emit's about ten billion photons a second.   The back of the envelope suggests the Sun would start to look like a blinking single photon about 1000 light years out - the range of our deep space frog. 

 

If you're sending information you have to worry about how quickly you can flash the light to begin with.  If your receiver is seeing a bright light or radio signal (radio is just longer wavelength photons), you can send information at a fast clip.  If it's very dim things get slow.  Even with huge exotic antenna arrays the science from the New Horizons spacecraft that flew by Pluto had to be recorded and sent back over a period of months,  NASA and JPL have been experimenting with deep space laser communication systems. The idea here is a laser beam can be made to not spread out much and will appear bright,  HDTV from Mars orbit can be done.   As long as you don't have to go through the atmosphere.  

 

We've almost circled  back to the beginning - the people who shoot laser beams skyward.  But the atmosphere absorbs and scatters so much.  What's going on?  

 

You can point a very bright, highly culminated beam of laser light to the sky.  Some small lasers are bright enough to travel through a few miles of atmosphere and still be focused and bright enough to temporally blind a pilot.  In fact it's a federal crime to shine a laser at an airplane.

 

There are people who shine extremely powerful lasers skyward legally.  The atmosphere can be thought of as a sea of small regions about the size of your hand. They have slightly different optical characteristics than their neighbors.  Their turbulent movement is what causes stars to twinkle. They represent an enormous limiting factor to the resolution of telescopes with mirrors larger than a half meter in diameter.

 

The trick is to look at a reference star near the object you're studying.  You know where it is so you image it, look at it's new position, do a bit of math and change the geometry of your optics in real time to adapt.  Simple in principle, but difficult in practice, it has revolutionized optical and infrared astronomy at a half dozen of the largest observatories in the world.

 

One of the problems is you need a fairly bright reference star and for about 99.99% of interesting astronomy there aren't any suitable stars that are close enough to use.   Astronomers get around that by making their own reference stars.  The standard technique is to use a very powerful pumped dye laser to create a yellow beam with a wavelength of 589nm..  

 

The laser is pulsed on for a short time - microseconds - and the beam soars skyward.  For the first 30 kilometers there's enough atmosphere to scatter most of the light so it's a visible yellow line. At about 95 km above the Earth it encounters a layer of sodium atoms - leftovers from constant bombardment of our atmosphere with micrometeorites.  589nm is a resonant frequency of atomic sodium so they get excited and then spontaneously de-excite emitting yellow light in all directions.  A little artificial guide star shines during the pulse.  Some of it's light makes it back to the telescope through nearly the same path light from the astronomy target has to travel.  Everything you need to compute your mirror changes.  All of this happens about a thousand times a second.  The process works so well that the largest 'scopes on Earth now resolve better than the Hubble Space Telescope, although it still has other advantages.

 

That upward beam is really bright.  It's only on for about a tenth of a percent of the time, but it could blind anyone on an airplane who happened to be in the way.   You get around this with one of the most exotic jobs I know of - a night watch for the 21st century.

__________
¹ The way to think about it is to recognize light travels in a straight line.  Image two imaginary spheres - one is  one unit of length away from from a glowing sphere and the other two units.  If you pick a region with any area on the first sphere (say 10 square inches) it covers a fraction of the first sphere that is four times larger than the fraction covered by the same area on the second sphere because the area of the spheres goes as the square of their radius 4*Π*r²

² In your physics classes you probably learned light can be a particle and a wave.  You can think of the photon as the particle of light.  Well..  and this would take too much to get into ...  the particle/wave duality model is an artifact about how people saw quantum mechanics in 20s and 30s.   I'll just state what we now know ... everything's just a wave.  But unless you are deep into physics it's probably safest to think about particle wave duality.

 __________

 

Recipe Corner

 

Really easy and just right for the cooler weather

 

White Bean Toast 

 

Ingredients

 

° extra virgin olive oil

° 6 slices of crusty whole grain bread (or whatever constitutes excellent toast for you)

° 1 clove garlic, minced

° 1 tsp chopped fresh rosemary

°  a 15oz can of no salt cannellini beans drained and rinsed

° 2 tbl lemon juice

°  salt

° 2 tsp chopped parsley

° bit of red pepper flakes

 

Technique

 

° oven to 425°F.  Use 1 tbl of EVOO and brush one side of each piece of toast.  Arrange face up on a sheet and toast on top rack until brown and crisp.

° heat 2 tbl of EVOO in a skillet over medium heat.  Add garlic and rosemary when the oil is hot and cook for a minute and remove from heat.

° add beans, lemon juice and salt and mash some of the beans a bit to give a variety of textures and some whole beans.

° spread over the toast, drizzle on some good EVOO, sprinkle on parsley and red pepper flakes.

 

much more artificial than intelligent

Look at the little group of numbers on the right.  They were handwritten and aren't exactly neat and tidy.   Take the first four on the third line.  Our brain says 1 , 7, 4 , 2.   We might bring it together as 1742 and some of you might think of a year - perhaps the year Handel's Messiah was first performed.  And we do it with a brain composed of nearly 90 billion neurons each with up to ten thousand connections to other neurons.  All of this  uses just 20 watts of power and weighs about a kilogram and a half.

 

The brain's neurons are simple computational objects.  There are input 'wires' called dendrites that connect to the cell body and outputs called axons.  Signals arrive on the dendrites and the neuron sums them.  If they exceed some critical value the neuron "fires", sending a signal out through the axons to other neurons.  There's a lot of detail to the nature of the wiring and how it comes to be, but for now think of the neurons as objects that sum inputs and fire if a critical value is exceeded...  more abstractly they're just mathematical functions.

 

The reason for starting off with some numbers is one of the basic programs - the 'hello world' if you will - for machine learning is to build a number recognizer.  A very large set of handwritten and printed numbers was scanned by a 28 by 28 pixel sensor giving a 784 pixel image.  Each of these pixels has a value depending on the brightness of the number it corresponds to.  For the sake of argument say it ranges from 0.00 for black to 1.00 for bright white.  A very crisply written 1 would be a line of dark values in a sea of mostly white. The edges would have intermediate grey values.  Less crispy numbers have lot of grey regions.  

 

To make our neural network (call it a NN) each one of these numbers is put into a corresponding neuron. The number in the neuron is called the activation.  You can think of the neuron as doing something when it's activation is a high number.

 

These 784 neurons are  going to be the first layer of our NN.  We'll have 784 neurons on one side representing the brightness of each pixel of the image.  On the far side is a small layer of 10 neurons on the other side the numbers 0 through 9.  Each of these contains the NN's best guess for how much the scanned number is one of the numbers.  For example you might have 0.02, 0.14, 0.38, 0.02, 0.98, 0.22, 0.13, 0.22, 0.33, 0.07 .. the guess  that it is the number 1 is 0.02, 2 is 0.14 and so on.  In this case 0.98 stands out, so NN is jumping up and down saying 'it's 5".

 

In between are other layers .. often called 'hidden layers.'  Rather than presenting the math that describes the network and waving our hands saying "it's all in the training", let's go in more deeply and see what's going on.  The diagram shows a very simple NN intended to recognize numbers with 784 input neurons on the left (it only shows a few... use your imagination), a single hidden larger of 15 neurons and an output layer with 10 neurons..   

 

Several years ago I wrote my first machine learning program to recognize numbers from this data set.  I used two hidden layers with 20 neurons each.  Their number and sizes are not hard and fast choices - there's a lot of room for experimentation and frankly a lot of this is more art rather than engineering.

 

Consider of the numbers from the pixels in the first layer.  These activations will determine the activations in the first hidden layer which in turn determine the activations in the second hidden layer which determine those in the output layer.  A pattern in the first determines a pattern in the second which determines a pattern in the third and the final pattern tells us the guesses for each number.

 

I like to think about the hidden layers doing something rather than just saying they're a black box.  It may be something very foreign to us, but in this case it is possible that one might, say, recognize components like edges and the next may be larger components like lines or roundish structures (a 9 is a line and a roundish structure, a 7 has two lines...).  It probably isn't doing this, but it's reasonable to think of a NN breaking down problems into layers of abstraction.  Trying to understand what's going on in these hidden layers is very difficult in large problems - perhaps impossible and that becomes an important issue when machine learning is used for purposes that impact us.¹  

 

At this point the post will diverge a bit.  I'll go into some but not detail that some folks may find useful.  I'll highlight it in blue.  Feel free to skip it if you're only interested in a high level view.  Up to now we've established a neural network is just a function that takes input on one side and spits out an answer on the other.  In this case the input is the brightness values of a 28 by 28 pixel scan of numbers and the output is the NN's best guess for each digit, from 0 to 9, of how likely the input was that number. 

 

Consider the simple, but artificial,  example where the second layer is trying to look for a some kind of feature of the number - say certain types of line segments.  Each of the 784 neurons is connected each of the 20 neurons in the second layer.    Each value in the left most layer is a pixel value.  As an exercise to make things clear take out a sheet of paper and make a vertical column of circles labeling them a₁, a₂ , a₃, .... up to a₇₈₄.  (only do a few of them  ^... are your friends)  On the right draw another column with a couple of circles.  Connect each circle on the left with each circle on the right.  We multiply each of these activations by a weight - just a number - and find the weighted sum  of all of the pixels ..  labeling the n^th neuron as a_n and the n^th weight as w_n we get:  w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄.   In the example of a trying to find a little line segment imagine making all of the weights 0 except for those in the region right around the little segment.  Then taking the weighted sum just focuses on the values of the pixels we care about.  If we really wanted to force the issue we could add add a negative number called the bias to the sum to require the weighted average be large.  The bias surpasses other bits of signal in the vicinity of what we're interested in.  The weighted sum going into the neuron on the second row is now (w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄ - b),where b is the bias.  

 

We're getting closer.  This new activation for the pixel in the second layer can have quite a range. If the weights are between -1 and 1, it can be between -784 and +784 before adding the bias.  It is common to scrunch everything to a range between 0 and 1 using a sigmoid function, call it s(x).²  Now the scrunched weighted sum is   s(w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄ - b) for each of the 20 neurons in the second layer.  This new activation that goes into the neuron in the second column is the crude equivalent of a synapse in biology.

 

You'll note the naming convention I've used isn't good enough.  We're just looking at what goes into a single neuron of the second layer. In this example there are  19 more, each with a connection to each of the 784 neurons of the first row and each connection with it's own weight.  And there's another bias.  Do this and you have the first two rows.  Next you consider the connections between the second and third layers (the two hidden layers in this example).  After you finish that you have the connections between the third and final layer. And this is just a simple example.   Fortunately computers are really good at this kind of math and there's a natural way to easily organize this.  If you get into this but  you'll need to be on good speaking terms with linear algebra.³

 

Before getting into the "magic"of how these weights are found take a look at the complexity of this example.  There are (784 * 20) + (20 * 20) + (20 * 10) weights and 20 + 20 + 10  biases.  That's 16,300 knobs to adjust correctly - yikes!  Imagine what it's like for problems with columns millions of elements long.  This approach, and there are many variations,  did't take off until computers became powerful enough and we had interesting data - big data - to play with.   Although there are a lot of calculations, each is very simple.  You don't need a big Intel processor with it's complex instruction set.  GPU - graphics processing units - are arrays of thousands of tiny and fairly slow simple processors.  Each is something like ten or twenty times slower than a modern Intel CPU, but all of them can run in parallel..  you end up being faster by a factor of a thousand or so.  Complex machine learning problems have moved Nvidia beyond computer graphics.  It extends to smartphones - Apple's machine learning core on it's latest processors can make this happen in your phone rather than having to rely on the cloud. 

 

Onward to machine learning because we're not daft enough to adjust this by hand.  

 

What we want to do is come up with a recipe - an algorithm - that adjusts those 16,300 knobs so we're likely to get the right answer when we give the machine a scanned number.  For that we'll use some big data.. the enormous set of  scanned and labeled (what the scan really is:  7 for example)  28 by 28 pixel numbers that's in the public domain.⁴   The data is used to train the network.  You start out running some training data, check what the results are, make some adjustments to the weights and biases and keep repeating the process until the result is good enough.  Then you test your NN by giving it another set of labeled training data it hasn't seen so you can see if it's still doing a good job. The hope is the layered structure generalizes what the recognizer does beyond the training data.  This is critical and whoever implements a NN needs to be aware of the domain(s) where it's valid.

 

So each neuron is connected to all of the neurons in the previous layer and it's activation is the weighted sum of all of the all of those neurons.  You might think of the weights as the strength of each connection and the bias is an indication if that neuron tends to be active or inactive.   To start off you set all of the weights and biases completely randomly.  It will give terrible results on the first set of training data. 

 

To make it learn you define a cost function.  We know what the number should be:  4 for example.  Now look at the differences between each final digit and what the machine guessed.  Consider the square of the difference between what you expect and what you get.  If the original number was 4 we expect the output list to behave like this:  0 to have the value 0, 1 to be 0, 2 to be 0, 3 to be 0, 4 to be 1, 5 to be 0,...

 

The first column is the number in question, the second column is the expected result of a perfect NN, the third is the square of the difference of the computed expectation and the expected result (the second column) which are shown in the forth column.

0  0        0.3       (0.3 - 0)²    0.09        

1  0        0.2       (0.2 - 0⁾²    0.04

2  0        0.1       (0.1 - 0)²    0.01

3  0        0.8       (0.8 -0)²    0.64

4  1        0.3       (0.3 - 1)²   0.49

5  0        0.7       (0.7 - 0)²    0.49

6  0        0.3       (0.3 - 0)²    0.09

7  0        0.8       (0.8 - 0)²    0.64

8  0        0.5       (0.5 -0)²    0.25

9  0        0.0       (0.0 - 0)²   0.00

 

The cost is the sum of the squares of the differences... in this case  2.74. Ideally we would get 0.00 for the cost...  

 

The cost is small when the image is accurately recognized, large otherwise. .  Now look at the average of all of the costs over all of the training data...  that gives a measure of how good our NN is for that set of data.  A single number for each set of settings.

 

The trick is just calculus.  Our AI is much more artificial than intelligent.  I'll just state a method exists to find out how to turn the knobs and when to stop.  I'll use a small bit of  blue so be ready to skip if you don't know about gradients, 

 

Imagine the case where we just have one variable.  There would be some kind of curve for the weight relating that weight to the average cost.  If it was very simple, and it's not likely to be, you could just find the minimum by taking the derivative and be done with it.  A better tactic is to start somewhere on the curve and figure out which direction to move to lower the output by measuring the slope .. move left if the slope is positive, right if it's negative. You'll end up at a local minimum.  Now imagine the function with many inputs - so you have many dimensions.  Now you're looking for the direction and steepness of steepest descent ..  just the negative of the gradient of the function.

 

So compute the gradient of the function, take a small step downhill and repeat over and over until you find a local minimum.  So put all of the 16,300 weights and biases into a tall column vector. The negative gradient of the cost function of that vector tells which corrections to all of those numbers gives the most rapid decrease of the cost function.  Minimizing it means better average performance across all of those training samples.

 

A useful way of thinking about the negative gradient of the input vector is each element tells us if a weight or bias should be increased or decreased and how important it is.  so one that starts out as 0.02, -1.29, 0.33 ...  would say w₁ should decrease a little and it is relatively unimportant, w₂ should increase a lot and it is important, w₃ should decrease a somewhat.

 

This setting of the weights and biases is called backpropagation.  We're just minimizing the cost function.  It's important that this function is smooth so we can find local minimums...  that's why the activation numbers of the neurons have a smooth range rather than something more discrete like ones and zeros.  Oddly enough this is where the brain is more digital than the neural network.

 

Playing with this example I was able to recognize a set of number images I hadn't trained the NN on at about 94% accuracy.  There are more sophisticated approaches.  It is important to stress that the innards get very messy and difficult to understand.  That's an unavoidable bug as it is very easy to fool ourselves.  There's bias in the building of the neural network, bias in the training data and bias on it's application.  It is great for some applications, but when it's used in science (I have some experience with it in astrophysics), you use it as a last resort and try very hard to keep the models simple and understandable.  The opposite of what happens in machine learning on social networks. 

 

I think they're a fantastic way to approach certain classes of otherwise intractable problems  and potentially misleading and even dangerous in other domains.

 __________

¹ Like associating aspects of our life and the games we play with our likelihood of us being a good engineer or a neo-Nazi.  These biases, some on purpose and others by accident are extremely important and worth taking up separately.  Machine learning tends to be good at identifying things very similar to known sets.  There be dragons and caution is required.  

² For example s(x) = 1/(1 + e^-x).   There are a variety of functions of this class.  This one is known as the logistic function - you pick what's appropriate for the problem at hand.   In this case large negative numbers go to 0 and large positive numbers go to 1.  

³ Organize the input activations as a column vector and the weights as a matrix where the row corresponds to the connections between one layer and a particular neuron in the next layer.

 ⁴ the MNIST Database from LeCun, Cortes, and Burges.

__________

 

The first good brussels sprouts are out.

 

Roasted Brussels Sprouts with Mustard

 

Ingredients 

° 1 pound brussels sprouts halved

° 6 tbl olive oil

° kosher salt

° 1 tbl dijon mustard

° 1 tbl coarse grain mustard 

° 1 tbl cider vinegar

 

Technique

° oven to 425°F

° put brussels sprouts on a baking sheet.  drizzle on half of the olive oil and some salt and stir them (I use my hands).  Roast on middle rack for about 35 - 45 min turning them a few times until softened and browned.

° whisk the mustards, olive oil and vinegar into a dressing and add a bit of salt to taste.

° put the baked brussels sprouts in a bowl and toss with the dressing.

 

 

 

G, A, F, ↓F, C

just kidding .. well .. sort of..

 

About a month ago I noticed my left shin and foot had swollen enormously.  There are a few nasty possibilities and a close friend died from one in December, so it was off to the emergency care unit.  It turned out to be something easy to deal with, but the exams were interesting. I have this fascination for instrumentation and had a nice conversation with the ultrasound operator.  He didn't know what one of the settings on his machine was doing.  The name was a giveaway so we talked.  In return he gave me a detailed tour of the innards of my legs (good and bad one) and a few other places too.  The interface was aimed at experts, but very interesting.  He had a rich menu of information to display on the screen - often images and graphs. He could also choose to represent some of then acoustically over speakers or earphones.    He said he almost always used the visual displays along with simultaneous audio as combining the channels was much more informative than either alone.   It took me back to my first big particle physics experiment.

 

 

At a conceptual level particle physics is easy - smash particles together and sort through the debris.  The problem is the easy stuff's been done.  More exotic interactions are rare and happen at energies high enough that most of the interactions are complicated mixes of well studied results.  All of this happens at a furious rate much higher than is encountered anywhere else - there is no way to save all of the data you're measuring. You need to be able to throw away what isn't interesting while keeping the tiny bit that is.  When I started we kept one in a few thousand events.  Now some of the larger experiments are more like one in a million.   This is done with with triggers.

 

Triggers are logical groupings of signals that give you an early hint, think a hundred nanoseconds or so, that something is candidate for further study.  Conceptually it's something like this  electron over a certain energy hits this part of your detector, this other part of the detector is completely quiet, and another particle  is coming out at nearly a right angle to everything else. It may or may not be real - that's for further study, but  it happens it's worth saving.  The trigger fires and you save it.  We had about a dozen triggers.  It was often convenient to look at an event soon it took place to learn about your experiment and possibly even find a hint of something interesting.   That's where the title of the post comes in.

 

A few years before this experiment  I had seen Close Encounters of the Third Kind.  A meh as a film, but the music caught my attention.    The aliens used music to communicate.  To signal them, scientists sent played five notes:  G, A, F, F (an octave down) and C. 

 

five tones

 

When the spacecraft arrives they used a variation on the theme and the aliens respond.  Then something of a jam session takes place.   

 

Designing my triggers it struck me that some had cluster groupings in the detector  that could be expressed as a unique three note cord.  I knew about synthesizer music (Switched on Bach and Walter/Wendy Carlos) and sort of borrowed a dozen frequency generators from the undergrad teaching lab (they had a lot of them).  A bit of tuning and I had a full musical scale.  Some of the most important triggers were F major (F-A-C), C major (C-E-G), and D major (D-F#-A) .. there were a couple of minor chords.  It worked and proved itself.  It was my gateway drug -  I've been interested in sound as data and as a computer-human interface ever since.  Data sonification can be a nice way to handle human data overload. We seem to be wired for it (some of us more than others), but modern interfaces rarely use it. 

 

Then there's listening...

 

Imagine a manager stepping onto a railing overlooking the production floor in a textile mill or some other 19^th century. He (sadly always a he) would look around, listen and maybe even smell.  Machines slightly out of spec would stand out ..  a sour note in the harmony.  Moving forward most car mechanics use sound as a diagnostic tool and even listen to customer renditions to narrow the problem space.  I use it to sort out issues with the washing machine and dryer - usually as a diagnostic tool when something breaks, but I'm beginning to use it as an indicator of future failure.   

 

Consider mechanical devices that may be difficult to fully incorporate into data analysis programs.  A feature of mechanical devices is the units of the same model are unique.  An xyx -102 refrigerator in your home probably has somewhat different operating characteristics than the same model in your neighbor's home.  Monitoring such devices and feeding using the information for optimal efficiency and lifetime can be difficult - especially on older existing hardware.  A good example are chillers used in building-scale cooling.  Their efficiency improves with increasing load - often dramatically until you reach a point where the unit shakes and is in danger of failing.  Each chiller is different.  A human operator usually is in charge of several and learns about their individual quirks.  A lot of valuable information - often hundreds of thousands of dollars worth - is locked up in a few local experts.  An expert from a different building would have a learning curve.  Bringing information out of the the human is a major challenge.  You have to be clever. You spend a lot of time with chillers.

 

A neat trick is to study the acoustic signature of chillers as they approach and go through their point of maximum efficiency.  They begin to emit a banging sound that changes to a screeching noise.  They shake on their mounts.  If you can begin to detect this you can back them off a bit and stay near their maximum efficiency while prolonging their life.  The easiest way is a microphone or a vibration sensor (which is all a microphone is).  It is pretty simple to digitize this and move it into a computer than can offer feedback to the machine control.¹  It can also feed into a machine learning program so you can begin to characterize all of your chillers prolonging their lives and predicting when they will fail so maintenance takes the place of failure.  

 

Getting data about the world around us can take a wide variety of sensors, but shortcuts are emerging.  Cameras output images, but add a computer and they can do quite a bit more: image recognition, crop analysis, temperature (infrared cameras) ... the list is dramatically expanding.   We mostly neglect sound, but microphones are a simple and often inexpensive sensor that with a computer can do much more.  Data streams from these powerful computational enhanced sensors can be integrated with other data streams for real machine learning.  Opportunistic data hunting.

 

I have a few of personal interests along these lines.  One is the acoustic ecosystem signature of ecosystems.  The local woods have coywolves.  I've put digital recorders out there just to capture their howls, but it is clear the background sounds might be important.  Flora and fauna have their own signatures spanning large frequency and time domains.  This is probably near ground floor fun for amateur scientists.  I can even imagine it being useful in many sports.

 

Now think about other senses - there are many more than five.

__________

¹ An Arduino and a piezoelectric buzzer/speaker used as a microphone is probably overkill for an application like this. 

 

 

of secondary importance

Perhaps something useful..  a quick cheat sheet on the care and feeding of the rechargeable batteries in your devices.

 

For historical reasons rechargeables are known as secondary batteries and one non-rechargeables are primaries.  We may complain about constantly having to charge everything, but you'd have to spend a few bucks a day to power your smartphone on primary batteries ..  you'd end up sending the bunny about as much money as you send Apple or Samsung.

 

I'm not going into any depth on the various battery families and where they're appropriate. Lithium-ion refers to a very broad category with hundreds of types of batteries, while nickel-cadmium, nickel-metal hydride and lead-acid are much more specific terms.  I'm only going to talk about lithium-ion and lead-acid as they're the most common types most of us use daily.   

 

lithium-ion tips

 

Don't swing the charge too much if you can avoid it.  The innards of the battery expand and contract during charge and discharge  by as much as ten percent.  This results in cracking that lowers storage capacity  Most smartphone and laptop batteries are rated to retain at least 80% of their original capacity after about 500 charge/discharge cycles.  This generally assumes a usage pattern where you rarely completely charge the battery and rarely discharge it to under 20 or 30%.    If you try to stop charging around 80% and never drop below 30%, you should be able to double the battery's service life.  Conversely dropping to under 10% capacity most of the time followed by fully charging can drop the service life to 100 cycles or less.

The best rate to charge a lithium-ion battery depends on its state of charge and varies throughout the charging period, battery history, and a few other things.  Fortunately you don't have to worry about this any more as most name-brand phones and laptops have smart chargers that measure and sometimes even chat with the batteries.  Stay away from third-party chargers.  Not only are you likely to shorten battery life, some have proven to be serious fire hazards.  

Temperature is important.  Below about 40°F capacity drops dramatically.  Fortunately there isn't any damage unless they get really cold.   Heat is another matter.   Anything over about 95°F (35°C) will begin to cause permanent damage.  Not much at first -- constantly keep your phone in your pocket against your body and you may see a 20 or 30 percent drop in cycle life - but a few hours at 110°F (43°C) can kill a battery.   You don't need to live in Phoenix to get to this type of temperature in the Summer - just leave your device in your car on a sunny day.  If you're spending time in hot weather and carry your phone in a bag, it makes sense to put it near a cold pack.  Note that warranties do not cover batteries for excursions beyond rated temperatures and some devices track that.

And if you have to store your device for a long period - a month or more - store it in a cool place (below room temperature if possible, but not in the 'fridge) with a half charge.

so ...

don't overcharge

don't swing them too much

keep them cool - I tell people to treat handle like good chocolate (except don't eat them)

use the charger recommended by the device's manufacturer

 

lead-acid tips 

 

This  is the battery under the hood of almost every car with an internal combustion engine.

Keep them charged.  Your car usually does a good job when the engine is running.  Some of the discharged materialis transformed to a new state from which it can't be recharged. There is a  failure mechanism known as sulfation where some of the discharged material is transformed to a material that can't be recharged.   This takes place slowly over the life of a battery, but can be accelerated to a matter of hours if you allow the battery to fully discharge.   I remember my father accidentally leaving the lights on during the day only to come out to a dead battery eight hours later.  There is a small discharge that runs some of your car's resting electronics, but it isn't a major drain.

If you're away from your car for more than two weeks you should have someone start it up and run the engine for at least fifteen minutes.  If it is a few months you should follow instructions for long term storage (this also applies to other parts of the car).  When I'm away for extended periods I disconnect the battery to halt the small drain from the car's electronics. 

 In the cold weather it is much more difficult to start a car - the oil is more viscous and the starter may take so much energy that abnormally large amounts of sulfation take place.  Also the amount of power a battery can deliver drops.  Keeping a car above freezing for starting can make a big difference.  In cold Northern climates it is common to keep the oil warm by heating the engine block with a 1000 to 3000 watt heating element.  My father called this approach his "poor man's garage".

If you're really careful with the charging cycles and temperature a lead-acid battery can easily go 30 years.  This isn't possible in the more compromised environment your car has to live with.  Figure on replacing the battery in warm climates every six years and every five to six in colder regions.  If you manage to completely drain the battery - even if it is new - just replacing it makes sense.

 

Batteries are getting better, but other than cost it is a slow game.  Don't expect dramatic improvements in the next five years at a minimum.

 

it's not easy being green

The lovely little cherry in front of our place says Spring is finally here.  It's beginning to leaf and currently is generating a bit north of a watt per square meter.  In a few weeks it will  throttle up to max cherry tree power.

 

Photosynthesis generates about three times as much energy per year as humanity uses. All of our food and most of the energy we purchase comes from it, but we usually don't walk by a tree and think - oh what a lovely oak and such a sunny day.  It's probably up to three or four hundred watts at the moment.¹   Photosynthesis never had to be extremely efficient, and it never evolved to anything close to the theoretical maximum.²   SciFi films sometimes show green skinned aliens who do photosynthesis, but it would be barely enough to charge their phones - if you were close to nude you'd be lucky to present a square meter  If you spent your time outside every day  might average about 25 watt-hours a day -- maybe a watt of power.  But throw on clothes and go inside and it falls apart quickly.  Anything that moves around and has a brain can't easily harvest power passively -- storage key to our existence.

 

For an idea of scale you need about a hundred watts of power to run your body - that's close to 2,000 nutrition calories a day.  If you could plug into a tree with perfect efficiency, you'd need a good sized one to keep going.  We do this by eating green plants or things that green plants eat, but you begin to get an idea of scale. 

 

Systems of all flavors and change radically with scale and purpose.   A car is a wasteful means of moving a person around ... payload to gross vehicle weight is often under five percent while something like a bicycle  is usually above eighty five percent.  Since most of us are unwilling and unable to spend more than an hour a day commuting and we like traveling by ourselves,  a car gives convenient range and energy costs have been low enough to justify enormous growth of the transportation mode.    

 

Now everyone is talking about autonomous vehicles.  Many assume we'll see something like Teslas or other conventional electric vehicles with the appropriate sort of computation.  I've written about such things - get in touch or check out Horace Dediu and Jim Zelmer's excellent Asymcar podcast.  Moving people in cities doesn't scale with payload to GVW ratios below a third or so.  Rethinking the vehicle is essential and there are reasons to think something like it could happen. 

 

I've done some work looking at single and two passenger vehicles with GVWs under 500 kg - sometimes much smaller.  If you keep the speed under about 40 km/h you're fine for most cities with good crash survivability and less expensive autonomy. Electrics and regenerative braking are a must in this regime, but aerodynamics are unimportant. Parking is a much smaller issue and navigation is easier.  Hybrid navigation schemes involving buried cables can happen.  With modest engineering thirty plus year life cycles outside of batteries and tires is possible, so you would probably wouldn't own one.   Not that this would happen, but it is probably much more practical than the Uber/autonomous schemes.   There are still some rather large social, data ownership, security and privacy challenges and none of this is going to happen overnight - the timescale of fleet replacement is about an order of magnitude longer than smartphones.  Along the way we're likely to see much of what we have now with much more computation for safety.  

 

Rather than write a book on the subject I'll stop here, but if you're interested there are reasons why the future could be different from the popular vision - in fact historically it almost always is.

___________

¹ There are a couple of ways of getting at this.  The quickest is to consider the surface area of the tree covered by leaves - the two dimensional cross section normal to the solar direction is a good first approximation, multiply that by the solar power per square meter at the time.  On a clear day at altitude with the Sun perfectly overhead it's about a kilowatt per square meter, but usually it's 500 watts per square meter or so in the Summer on a clear day at mid latitudes...  averaging for clouds and night and solar scatter gives you 150 to 300 watts per square meter in mid latitudes... about 240 world average.   Now plug in the photosynthetic efficiency of the tree..  generally 0.2 to 0.5 percent for deciduous trees.

Another way is to look at the biomass tree produces in a year and recognize the metabolic needs of a tree aren't huge.  It isn't moving around or thinking. 

° the average solar energy per day for this part of the country for April through September is about 4 kWh/m² per day and trees are in leaf about 220 days a year.  

° an average mature oak is about 50 feet tall.  I approximate the leaf area by a disk with a radius of 6 meters or an area of about 100 m²

this → 88,000 kWh for the oak per leaf season

° a 50 foot oak has a dry mass of about a ton - I'll call it a metric tonne.  It produces about 100 kg of new dry wood a year

° dry wood has an energy content of about 16 MJ/kg or about 4.4 kWh/kg

this → 440 kWh of energy from the dry wood and an efficiency of 440/88,000 or 0.5%

440 kWh/leaf season → 440kWh/(220 d * 24h/d) or a bit over 80 watts average

² Caltech is working on that one

 

___________

Vegan Meringue

 

I'm not kidding...  I tested the claim that you can substitute the brine from chickpeas.  Here's what worked .. better is probably possible.  I've had a couple of edible disasters and a couple of successes .. the trick on the oven door is important!

 

Ingredients 

 

° 6 Tbl of brine from canned chick peas 

° 1/2 tsp cream of tartar

° 3/4 cup of fine sugar (the finer the better)

° bit of salt

° 1 tsp vanilla extract

 

Technique

 

° oven to 250°F

° in a mixing bowl wisk the brine and cream of tartar with an electric mixer for a few minutes until it foams.  Add the sugar a bit at a time and keep wisking, when the sugar is mixed in add a pinch of salt

° as peaks begin to form mix in the vanilla after an eternity - over ten minutes for me - stiff peaks formed. 

° pipe it to a cookie sheet and bake about 45 minutes.  You need to do the meringue trick of not opening the door - just turn off the heat - for an hour or so to let them dry.  

 

the dawn of programming to serious hidden tech

I love the music, but usually ignore the technical part of SXSW, but this year a few people reported progress in the Google's Project Jacquard initiative with Levi's.  It's an interesting project in a potentially rich area where most have been failing - weaving computation into clothing. 

  

At the beginning of the 19^th century Joseph Marie Jacquard invented a powered loom that could manufacture textiles complex patterns.  An amalgam of inventions going back nearly a century, the cleverness was a chain of cards. Punched cards laced together forming a sequence of instructions - a program if you will - that instructed the machine how to weave.  These sequences could be arbitrarily long.  Suddenly brocades and damasks could be made at scale. Jacquard's name became something like Xerox - a company, a machine and a process.  Like Xerox the term became generic.  

 

The complexity of patterns could be astonishing.  Jacquard was an inspiration to Charles Babbage who commissioned this portrait to be woven in silk on a Jacquard machine.  Jacquard is frequently mentioned as one of the earliest bits of computation.  An interesting symbol for Google's project.

 

Mixing local computation and fashion has been a tough area. Most companies have been trying to show off their tech paying little attention to use and practicality.  Whatever is done it must be useful at some level,  machine washable several hundred times, comfortable, easy to manufacture, inexpensive and aesthetically pleasing to some non-trivial segment.  Personal style is important and no one worries about what kind of ARM processor is up their sleeve.

 

The Google-Levi's collaboration aims to weave touch and gesture into a variety of textiles using modern industrial versions of Jacquard looms.  They're currently working with a Japanese firm that does 3x1 denim for Lanvin and Prada.  The reports suggest the wires are almost invisible and the material still feels like 3x1 denim.  Taps and gestures are sent to a little module that talks to your smartphone via Bluetooth.  (it has to be removed for washing) The jacket gives some tactile feedback with a little vibration and it also lights up in different colors.  It's a simple input and display.

 

The first product due out in the Fall is a $350 cycling jacket - a variation of their Trucker Commuter Jacket which currently aims at the twenty something urban cycling market.¹ It strikes me as an interesting interface research project (something I've been involved in at times), that is narrowly focused at this time.  Software would have to be tailored to a variety of needs - of course that is possible, but to get it right isn't trivial. Reports were that Google's software was incomplete and a bit rough.   I have to wonder if augmented reality from something like Apple's AirPods makes more sense for this niche.

 

It is interesting that the killer app here is the same as what many claim for the smartwatch - fitness.  These devices are really only useful as approximations of what your body is doing.  The relationship between work and weight is poorly understood and the wrist isn't necessarily the best place to measure work.  The measurements can be very useful if it is an encouragement, but they need to be taken with a grain of salt.  It's unlikely athletes will take a serious interest at this point.  

 

A more accurate fitness future may be suggested by Leomo's Type-R with it's small network of wearable sensors.  I think the future is much richer than just a watch and a phone.   A cluster of  appropriately located sensors using a local secure network to communicate with your smartphone.  A watch may well be part of this cluster.

 

But that is fitness.  Where is fashion - regular clothing - personal style?  I've been excited about developments in smart fabrics, both active and passive.  There's been little in the way of aesthetics.  Functionality will be addressed.  This is another post, but imagine if you could keep a person comfortable in a larger thermal envelope.    It doesn't make sense to heat and cool very large air spaces if you can keep people comfortable.  Comfortable, normal looking clothing that keeps a desk worker comfortable at 17° or 18° C in an office -- or at 28° or 29° C.  HVAC bills could be greatly reduced. Local humidity and UV exposure are also being tackled.  You can also make functional clothing that helps people with disabilities as well as the infirmities of age.  A long list of possibilities exist and the known unknowns and unknown unknowns lists are larger still.

 

This sort of work cries out for diverse collaborations. Google clearly has the idea by teaming with Levi's, but that's a baby step.²  I'd love to see some universities that are strong in materials get involved with clothing designers, engineers and others - I can think of some folks doing animation I'd like to see involved.

 

So cheers to the collaboration, but there is an enormous amount of blue sky and a $1.5 -$1.8T apparel market to think about.

 

And then there's the question of doing it with humanity.  The apparel industry has been involved in human right's abuises for at least two centuries.   

 

__________

¹ The promotion video

compare with the video for a similar Levi's commuter product that paints a different person  --  I don't know if this is by design or if accidental, but a very different message.

 

² The folklore is Levi's was not the first choice.  Experience in tech/design collaborations tells me they're at least as important as Google in this and possibly more so.