omenti

woodie

an Omenti mini-post

I was shocked and saddened to learn of Woodie Flowers' passing.

 

There's an interesting question you can ask about a school.  Who is the most important person?  Of course it depends on your point of view and what you think a school should be doing.  It might be an administer who has managed to turn the place around,  a remarkable alumni member who does the same, a Nobel Laurette perhaps? and for some it might be a coach who somehow energizes the place.  Is there someone who makes the place special?  Universities mean a lot of things to different people.  But if you asked me about MIT, I'd suggest Woodie Flowers.  He managed to light the spark and inspire more young engineers than you can imagine.  And in light of the current ethical problems ripping apart a school on campus, he was a beacon of ethics for decades.  

 

I was lucky enough to have spent time with him on a few occasions.  He was even more optimistic and even evangelistic than what comes over in this video:

 

 

Woodie was like this in many areas.  He was responsible for developing and promoting the concept of gracious professionalism..  from an email a few years ago:

 

The election has been painful. It reminds me that we have a responsibility to help students become thoughtful citizens. Rational thought blended with empathy seems too rare everywhere.

Personal experience leads me to believe our students would be receptive to hearing us address their obligations to society. Dean Kamen founded FIRST (For Inspiration and Recognition of Science and Technology) in 1989. A couple of years later, Dean and I co-founded the FIRST Robotics Competition (FRC) patterned after the creative design exercises I had been running for many years in the Mechanical Engineering design courses. The “2.70 Contest” was anchored by “gracious professionalism,” a term I coined to celebrate the wonderful behavior prevalent among students in that course. They competed like crazy, but treated one another with respect and took pride in helping and teaching one another.

At the first FRC Kickoff, I used one slide with the term “Gracious Professionalism.” Encouraged by Dean, at the second FRC Kickoff, I used six slides featuring the phrase. Since then, Gracious Professionalism™ has become a powerful part of the ethos of the whole FIRST community. FIRST competitions have grown and have millions of alumni. Scattered over 80 countries, young people have embraced the notion that competition and kindness are compatible. At MIT, 10% of our freshman classes are FIRST alumni.

As we make the transition to digitally-enhanced learning and machine-assisted professions, I believe “uniquely-human” will be a powerful differentiator. Professionals who understand the laws of the universe and also have an emphatic response to others will be the leaders. Those who think science is a la carte will have aspirations that Mother Nature will not tolerate.

MIT students will have an obvious advantage because of their understanding of what is possible. We need to make sure they are also well equipped to manifest creativity, leadership, good judgment, and ethical behavior.

I offer Gracious Professionalism as a convenient label. It blends rigorous adherence to the laws of the universe with the human qualities we hope to see as those laws are applied by our alumni. Giving back with conscience. Blending hard knowledge with soft feelings. Facts, feeling, and fairness. The FIRST community, which includes all ages, have embraced “GP” and actually compete to “out GP” one another.

How might Gracious Professionals have voted?

Woodie Flowers

 

I'm not an engineer, but am amazed what the good ones do.  This is a big loss for MIT, engineering and the world

 

canada's beautiful new $16 million camera

My Grandmother King gave me my first camera for my eighth birthday - a Kodak Brownie.  My parents had a camera, but I wasn't allowed to use it.  This was my camera.  I also received two rolls of black and white film and a voucher for developin.  Of course I did what most eight year olds would do - I took pictures of the sky, the side of the house, my sister, the dog - twelve exposures in all.  A week later the prints came back and you can guess how bad they were.  The second roll was a bit more better, but I never developed the artistic skill.  But I did I did become fascinated with devices that capture images and have designed and built a few over the years.  

 

Some of you are excellent photographers.  My friend Om happens to be just such an artist and I believe he is writing a book on the subject.  My sister Corinne is an artist who uses photography and computers to offer quirky commentary.  I'll be content with my simple snapshots and videos enjoying the art of others.  And I'll think about the process and tools.

 

The term camera conjures up something that focuses light on film or an electronic image sensor where an image is captured.  Still or moving images, but almost always visible light.  Currently the definition is being extended to computational photography - the processing that goes on in your iPhone for example.  I stretch the definition a bit.

 

For me a camera is something that captures some event or series of events in time.  It doesn't have to be visible light, but any form of electromagnetic radiation and or something a bit more exotic - like charged particles or the shaking of space-time.

 

I first came to this way of thinking on Sulphur Mountain near Banff, Alberta.  I was almost thirteen and had gone over to a curious little shack on one of the mountains twin peaks while my parents had lunch at an observation deck a half kilometer away.   The two young men in the shack turned out to be cosmic ray physicists.  They patiently explained in kid language how they captured muons.  What they were really doing is capturing and recording part of a particle shower that came from cosmic rays striking the atmosphere twenty or so kilometers up.  They'd run these recorded events through some calculations and  would be able to say something about the shower and from that perhaps something about the Universe.  After listening I said something like "you've built a cosmic ray camera.  That tape is your film and you turn it into something like a picture"  They nodded , smiled and showed me how an image could be abstract and still give a lot of information. That day would change my life. 

 

Since then I've been around my share of particle physics detectors - essentially cameras that capture the aftermath of violent particle collisions at places like CERN.  I've also used optical and radio telescopes that, typically after a lot of computation, produce real or abstract images of places and events back in time.  It still astounds me to look at parts of the Universe as it was twelve or thirteen billion years ago.  And for fun I  even designed a homemade digital camera back in the 80s. 

 

Cameras capture images that can be meaningful.  Images that can be artistic, historical or even offer evidence of Nature's workings.   Some of the techniques are familiar and straightforwards, others are sometimes deeply computational. And if you consider the eye-brain combination a camera, as many do, there are surprises  We usually think of an image falling on an imaging surface and being turned into a more of less faithful representation of the image.   In fact there is a lot of processing that goes on between the front of the retina and the brain.  What eventually turns into an image has very little in common with what fell on the retina.  Our brain stitches together information, often out of order and spread over several seconds, to construct an approximation of reality we think of as the real thing and a sense of "now" that is equally artificial.  The comic strip is a nice example of something that is temporally impossible that happens in your brain all the time.  It turns out temporally out of order processing often happens in computational photography.    And then there are many instruments you might not think of as a camera that are much closer to the classical definition.. things like MRI, ultrasound imaging, radio telescopes and the like.  

 

Back to this $16 million Canadian camera...  

 

CHIME is a radio telescope at the Dominion Radio Astrophysical Observatory near Caleden, British Columbia.  It consists of four 100 meter by 20 meter upward facing semi-cylinders that are sort of like snowboarding half-pipes.  Sprinkled up and down the foci the semi-cylinders are 1024 radio receivers that operate between 400 and 800 MHz .. covering a frequency range similar to that of UHF television - a frequency range was chosen to detect a signal hydrogen emits.¹  That's where the name  comes from - the Canadian Hydrogen Intensity Mapping Experiment.

 

An enormous amount of computation takes place in several steps after the radios receive the signal.  The first step sets the instrument apart.  You may have a wifi router that does beam forming.  Basically you use a small array of antennas and the properties of radio waves to construct a "beam" to the source.²   CHIME creates 1024 beams listening to over 16,000 frequencies (think of them as colors) all imaged 1000 times a second.  While beam-forming adds to the computational load it dramatically decreases construction costs over a mechanically aimed antenna and for this application is a very clever strategy.  The computing just behind this is equally clever.  Sixteen million dollars to get this kind of data over what will be several decades of life is a bargain. The electronic and computer designs are elegant and if you're into that sort of thing you should look into the details.

 

The hope was it could detect an unusual recently discovered event called a fast radio burster.   At first some suggested the short radio bursts  of a thousandth of a second or so might be signals from extraterrestrials as no one had a compelling physical explanation.  They're very rare and transient.  Before CHIME only one burster called FRB121102 has been seen to repeat sending out about 100 irregularly spaced bursts since its discover in 2012.   A few hypothesis have been made, but studying a single event can lead you astray.   Then CHIME switched on for three weeks last Summer and thirteen new events were detected even though it wasn't fully functional completely.  This is far and away the most sensitive camera in the world for studying these events.  It is also well suited for mapping - making images of - hydrogen clouds in a much younger Universe, studying a special class of neutron stars called magnetars, and answering a number of other astrophysical and cosmological questions.  There's the added bonus of looking at the sky with a new type of camera.  You tend to discover things you hadn't imagined.  

 

 Human beings have become remarkably adroit at extending and even going beyond their senses as we try to understand the world.  CHIME is easily one of the most beautiful cameras ever made.

__________

¹ The 21 centimeter line of hydrogen is a great signal for studying the distribution of hydrogen atoms in the Universe.  It would be far too short to be detected by the telescope, but distant hydrogen is red shifted down to this frequency range..  Since the emission line is very narrow and well known, you can calculate the redshirt and therefore the distance of the gas.

² Beam forming is done by changing the signals phase at each antenna in an array in such a way that constructive and destructive interference create a pointed beam. It works well for wavelengths not much different from the antenna size.  Wifi base stations aren't very accurate, but they don't have to be. You can also do it with sound leading to some interesting pranks.

 

baseball and machine learning

A few weeks ago two of us were talking about the perception of "now".  A fusion of sensory inputs, responses and thoughts takes place and a highly edited version is assigned in what we perceive as a smooth and linear flow of time.  Consider vision.  It turns out it takes us around three tenths of a second to put together a meaningful image from the time light hits the retina. First we note color, a bit later motion and finally form.  No big deal unless you have to react very quickly.  For most of us that isn't a big deal.  For athletes it is.

 

Baseball is useful as the time it takes for the ball to travel from pitcher to bat isn't much longer than the time it takes to process an image. But baseball "works."  To examine vision in baseball consider the great women's fastpitch softball player Jenny Finch.  A few years ago she broke what the best Major League Baseball hitters are capable of. 

 

Their problem is they don't know how to "read" her.  Beginning as little kids  they learned how to hit a ball getting progressively better over time.  Early on the time of flight for the ball was so slow that human vision and reflex speeds weren't the major issue.  With practice they began to learn how to read the pitcher.  Posture, arm movements and so on.  The shards of image they formed gave them information on the physics of the ball's flight path.  They were learning all of the patterns even though they thought  that they really sensed when the ball was released and when it hit their bat - after all, that's what their mind told them.  But what their mind put together wasn't accurate.  In reality they were linking reads and other early information with ball trajectories.

 

Jenny comes along with a pitch that is over ten meters per second slower than what they're used to, but the information they receive doesn't link reliably with ball trajectories.  They either try to rely on reflexes missing the ball entirely as it flies past before they can swing or they use a mistaken trajectory from their mistaken read and strike out. They could learn... give them a lot of time working with women's fastpitch experts and they'd finally build the appropriate library.  Ultimately some of them might even play the game well.

 

There's a striking similarity with machine learning.  You show a program a huge amount of data trying to be clever and you get a program that can quickly do great pattern recognition.  At least within a restricted domain.   The richness of training datasets is often poor.  Facial recognition programs might have respectable results for white people, but completely fail with people of color.    Such biases of omission are very common, but many other types of bias exist.  (again a rich and deep subject well beyond the scope of an hour). 

 

Machine learning can be useful but is only robust when the limits of where you can use it and the types of errors it makes are well-understood.  There are many areas where it's sort of good enough even though these limits aren't well established. Other areas are more critical.  We're a long way from it being generally useful in areas the tech press considers nearly solved -  self driving vehicles come to mind.

 

Back to the athlete.  Sports are played at many levels.  At an amateur level we play because it's fun.  Beyond that most of us are watching others.  In direct competition we usually prefer watching serious pros.  Beyond the beauty of watching human motion being executed so well there are those magical times where something extraordinary happens.  Athletes at the highest level develop a deep sense of the game and, in addition to being able to react faster than their vision and reflexes would suggest, are tracking and processing other channels of information.  In some games players develop a "field sense" .. they know where the other players are and how they're moving.   You see this dramatically in beach volleyball when played at the highest level and it is common in soccer and basketball.  This gives them the ability to do the impossible. These are very beautiful human moments beyond machine learning.   And it goes beyond sport ..  think of the human experts that just out of the ordinary when the time comes - like Captain Sully Sullenberger landed in the Hudson. 

 

We tend to think about athletes as having some physical gifts and a lot of training.  Certainly those are necessary, but there's an impressive mental component.  We don't say someone is good  at foreign languages because they have a dexterous tongue.

 

machine learning without a net .. from sports to smart toasters

And I don't mean Cylons.

 

Every now and again you notice a few trends coming together even though they hardly seem related at first glance.  Someone asked a question today that turned out to be something I've thought a lot about over the years - a collision between tiny machine learning, privacy and an ocean of tappable ambient information.

 

Twenty years ago some of us were playing with sound. After spending a lot of time analyzing it we wondered what areas existed beyond voice recognition and music. There was a small "name that tune" excursion where we tried to match sung, hummed or whistled bars of music to - well - name the tune.  We were trying to turn the sound into note patterns.  It sort of worked, but the the problem of building a notation library was too difficult at the time.  The easy way out would be a hammer and tongs approach - namely matching played music with a library of played music.  We filed a patent on it, but never pushed hard as we didn't see the use case. Oh well...

 

Next I tried to recognize misbehaving car sounds..  Auto mechanics have great stories of people trying to make broken car sounds.  Unfortunately the problem was not small enough and I shifted gears when a ventilation fan in my office started squealing.  

 

Big heating, ventilation, and air conditions systems mostly run with boring industrial sounds, but a good HVAC mechanic can recognize when something's about to fail.   They also tune these systems for efficiency by ear - for some systems the sweet spot isn't far from the point where problems happen and every installation behaves differently.   With my trust Sony DAT recorder I made recordings of these systems in various stages of failure.  Then some fancy pants signal processing and, lo and beware, you can detect and analyze failure modes.   There was probably a lot of interesting work that could be done, but my organization evaporated around that time. 

 

I was fascinated with low power computing a bit earlier than the sound work.  How little energy does it take to do a computation?  (you can make some interesting entropy arguments and get down to basic physics.)  More practically what could be done with very low powered sensors that didn't need batteries or wires?   The interest took several directions - all good as you slowly begin to learn.

 

Then about five years ago it struck me that the HVAC problem could be done more easily with machine learning.  I've expressed reservations about the misuse of machine learning and regard some of it to be not very different from phrenology, but potentially very useful where you can understand it and its use.¹  I supervised a senior thesis project that attacked the HVAC project using machine learning and a microphone.  It showed some potential, but for me the learning was finally realizing how computationally cheap machine learning can be at the low end.  I also ended up learning a fair amount about machine learning as the best way to learn is to do it yourself and then teach.

 

Given Bitcoin mining efficiency and the massive machine requirements at FaceBook and Google it isn't all together clear, but think about unconnected machine learning on your iPhone and perhaps your Apple Watch.  In conventional computing the CPU deals with a large number of instructions compared to the type of processor used in machine learning.  ML processing chips have a large number of tiny processors that work in parallel. Most ML is just matrix multiplication.  While there can be a lot of steps, each is really simple .. generally just addition and multiplication.  A properly designed ML element holds the numbers in the matrices its working memory cache.  It's much more efficient in time and energy use to keep what you're computing nearby rather than having to constantly read and write it to memory.  You should be able to do a specialized bit of silicon with enough ML horsepower to do some rather surprising tasks while using very little power.  And you can do a lot on a smartphone if some of it's silicon was designed to support ML.

 

I think Apple understands this at a rather deep level.  They're very careful about how to efficiently deal with graphics processors which is shorthand for saying they have optimized how to do huge volumes of very very simple computations.  The chips in the current iPhones are optimized for the task.  They've build machine learning machines you can hold in your hand and, in theory, run ML applications that don't connect to the net and divulge information.  The new measuring app on iOS 12 is an excellent example of a local machine learning deployment. I don't know if it is being done on the watch, but there's no reason why they couldn't. Perhaps the biggest design advantage Apple has these days is silicon. 

 

machine learning in your hand rather than the cloud.

 

I can think of a number of applications that use the sensors on the phone or watch -- from health, to motion analysis (think sports).. why you could imagine wearing a watch and using a couple of iPhones on tripods or held by friends to analyze your volleyball, gymnastic, or whatever moves.  A personal coach or a serious tool is you already have a coach.

 

Exciting stuff, but I suspect it goes much deeper.

 

A back of the envelope calculation suggests you can do useful ML with micropower.  I'm sometimes asked to come up with senior projects in engineering classes (dangerous as I'm not an engineer).  I think you could do the HVAC work with an accelerometer or microphone and simulate the type of compute power that, if done in silicon, could run for months on a watch battery.  Maybe some very simple chemical analysis.   Or recognize a very small vocabulary -- perhaps five words - to control your toaster,  thermostat or anything else.  You could use a key word to make it listen or shine a laser pointer on it.  You could also build a very simple image recognizer that would detect someone's gaze.  Look at your toaster and say medium please.  

 

Way safer than saying medium please to a Cylon.

 

A need for privacy may drive this even deeper and away from the Net.  You can sit down and go through a few dozen applications in an afternoon.  It seems very rich and very latent.    Apple knows how to do silicon and low power.

 

So many possibilities.. perhaps even my dream "big data" project of listening to the woods. 

__________

¹ I've written and talked a fair amount on problems.  It can get technical and I won't spend more time on them here, but it's a serious problem as surveillance systems transition to judgement systems.

 

raindrops keep fallin' ...

A few people sent links to the announcement of a "revolutionary" advance for solar cells - namely to let these large areas that would be unproductive in a rainstorm, be productive.   Numbers weren't mentioned, but it's worthwhile thinking about the problem.

 

The Sun heats evaporates water which rises up into the atmosphere and can condense and precipitate. A question is how much of the energy that went into evaporating the water can be recovered?  If rain could fall without resistance it would be a lot.. also walking outside in a storm could be a lethal activity.  The simple physics is the moisture in the sky has a considerable amount of potential energy relative to the ground it will fall on ..  as it falls the potential energy turns into kinetic energy .. you can work out some numbers if you like.¹  But that's not what happens.  First think about it.. a sanity check ..  are you feeling the force of rain when there's no wind?  What does that say about power density?

 

Raindrops run into air resistance and their terminal velocity is low..  fast ones are traveling about 8 meters per second - about 18 miles per hour.  You can throw a ball a lot faster than a raindrop.  So now the problem is simple.  You don't have to know how far it falls.   So here's a sample calculation I did in my head when I heard about this.  The kinetic energy for one centimeter of rain falling on a one square meter area is 320 joules.²  That isn't much ..  about 0.09 watt-hours per square meter.  One way to think about it.  If you had a 100 square meter array (large for a house!) you'd average 9 watts during a one hour storm that put down a centimeter of rain .. an inch of rain would get you into the 25 watt range ..  less as the converter won't be close to 100 percent efficient, but maybe enough to run a small LED lamp during a strong rainstorm.

 

 On my rowing machine I regularly convert power from my muscles into mechanical power at a flywheel averaging about 150 watts or so.   A good generator could convert about 90% of that into electricity.   A photovoltaic in cloudy Germany averages about 40 watts per square meter across the year (including night) - that includes the conversion efficiency.  Problem solved..  this is impractical for solving the general energy problem... no way would it average out periods where other forms of renewable power aren't working well.

 

So now for the more interesting question.  Why were people excited?  I think there may be several things going on.  

 

First it appears a group did make an advance in turning mechanical energy into electrical energy with piezoelectrics.  Traditionally this is very inefficient - one percent or so.  They've made an advance and it could be very useful for niche applications like low power electronics located where it is impractical to run power cables or replace batteries.  Aces for the team.

 

These announcements often fall into the hands of university or corporate PR people who don't understand what the invention or discovery is and don't ask questions or get sign off. This turns out to be a major problem even at name universities and often worse in the corporate world.  I don't blame the PR people as getting this right is a non-trivial problem.  The announcement is picked up by a tech writer who lacks a clear idea of the announcement, but thinks they understand.   They write something right out of the telephone game.   Meanwhile the latest version is picked up by social media or others who are too lazy to check things out.  The reader needs to learn what sources are trust-worthy and, if the work is important to them, figure out how to drill down.

 

There's a lot of interesting physics thinking about rain and even liquid water,  And water for energy storage can be inexpensive and enormously practical if you have the right location (a mountain lake for example).  Denmark cleverly uses Norway as a storage battery for wind energy to even out the load.  This may become less practical for the Danes as Norway is seeing a larger demand for electricity at night recharging electric cars so it will come down to economics.

 

__________

 ¹ gravitational potential energy mgh becomes kinetic energy mv²/2

² E = 1/2 mv²   where the mass of the water is density times volume.  The density is very close to 1000 kg per cubic meter.

 

the next step in home audio?

 Still cold from the UPS truck the Apple HomePod woke up when I plugged it in and asked my iPhone for a dance.  They spun about and chatted in their local network for a minute or two before Siri introduced herself.  It was completely unexpected surprise from a great friend who  shares the love of music.  A day into the journey I have a few thoughts.

 

I'm a poor musician, but find it an important part of my life.  Grad school was at Stony Brook.  Besides an excellent physics department one of my reasons for doing grad school at Stony Brook was its proximity to music in  New York City..  That wasn't quite enough, so two of us founded a concert series at the university that thrives to this day.  At Bell Labs it became apparent that music might be a significant driver for Internet use and I found myself walking several paths.  One was sound field reconstruction.

 

Sound field reconstruction is just the fancy way of  recreating the sound where it was recorded or engineered in your home.  In theory you record the acoustic signatures of the recording studio and your living room and create a mathematical operation that makes your living room sound like the recording space.  In practice it's approximately hard.  Sound waves are fairly large and get reflected, absorbed and transmitted by and through everything in your room.  Making matters worse the sound field at different spots in the room is usually very different and things in the room move between and during listening sessions.  We made serious progress, but it took exotic microphone and speaker arrays as well as state of the art (for the time) computing to make it sort of work.,

 

One detail is centrally important.  Imagine sitting before a simple speaker cone.  Most of the sound comes straight at you with the volume falling off as you get away from the speaker's central axis.  Speakers have very characteristic patterns showing how loud they are depending on the angle you're sitting at as well as the frequency of the sound they're producing.  If you have two stereo speakers there is usually a sweet spot where the sound reproduction is optimal.  In this area the sound field can be fairly good and you can close your eyes and visualize where the musicians are.

 

Imagine an array of speakers, each with its own amplifier, all controlled by a computer, you can control the phase and amplitude (loudness) of each speaker.  The waves they send out interfere with each other as they expand outwards.   By controlling the phase and loudness (amplitude) of each speaker you can make them interfere in such a way that they create a new wave that can be aimed.  The little animation shows an array of four speakers in a row, but it can be any number and they don't have to be in a line.  The HomePod uses seven small speakers arranged in a ring.  

(you may have to click on the animation to start it)

 

The HomePod also has a ring shaped array of six microphones.  They listen to the room (sample it) at a fairly high rate and decide what parts of the music, from its stereo signature, need to go where and how it needs be modified to interact with the room and everything in it.  An Apple A8 processor, the same two billion transistor processor used in the iPhone 6 and 6 Plus, handles the computational chores.  This is much more power than we had back in the day.  Although I don't know exactly what they're working on, several people I know who are real experts at this have been at Apple for years.

 

 

The HomePod also has a computer controlled single woofer with its own amplifier mounted vertically to handle the low notes.  

 

I'll leave the specsmanship, mechanical and most of the UX discussion to others and just talk about what it sounded like.  I'm not golden eared, but rather a serious listener.   I focus on the music and let it take me where it pleases. 

 

I listened to works across several genres with the HomePod placed at about ear height in a small acoustically complex room.¹   For very small groups - soloists and piano quartets for example - the sound stage was excellent ..  about as good as my stereo pair of conventional speakers (about $3,000 in speakers).  The HomePod didn't match the reference speakers when I was sitting at the sweet spot, but was better when I moved around the room.  That is astounding. 

 

It was creating stereo -a  term has nothing to do with a pair of anything, but rather three dimensional.  Moving  beams of audio around it was painting a sound field in my living room. 

 

As I listened to larger ensembles I found a less accurate sound field.  That said it was still comparable to my reference pair when I was away from  my very small sweet spot - its effective sweet spot was larger.   Note for some pieces of music a good $500 pair of speakers would do a better job if properly installed, but  would love to see what a pair of HomePods placed about two meters apart would do.  Apple has promised the feature will be available later this year.  I've already earmarked the funds for immediately buying a second unit then.  I suspect a pair of HomePods will blow my reference pair away when it comes to sound stage placement.   That will be even more true for those who haven't carefully adjusted their current speakers with test recordings and a spectrum analyzer. 

 

You can control it from an iPhone, iPad or Mac, but Siri is the intended interface.  I've had poor luck with Siri on my iPhone, but here it works much better.  It's also much more robust to interference.  Microphones beamform too and it's clearly listening to you doing its own cocktail party trick.   That said it, and every other interface I've seen are just wrong when it comes to classical music.

 

Comparing it to the networked speakers from Amazon and Google is silly.  Their speakers are terrible and useful only as acoustic wallpaper you listen to in the background.  They're really microphones in your home listening to you assuming you agree to Google's  or Amazon's value proposition (I don't).  Privacy is vastly different - I'll go with Apple's differential privacy rather than Google's "you are the product" flavor.   But if music is secondary and you want a digital assistant with a speech interface, Google and Amazon are the ticket.  Apple is going after the other audio companies along with making its platform stickier in the process

 

I suspect the speaker will be like the Apple TV - fairly stable hardware with the bulk of improvements coming through software updates.   My list of UI and UX issues is growing, but I suspect this will become part of my life. 

 

And a final caveat - speakers involve personal taste.  YMMV

 __________

¹ I'm very familiar with these pieces.  With one exception  I've heard them live and I've even been involved in the recording process on two. 

Ecstacy  Anonymous 4 and Mike Marshall

Us  Regina Spektor 

Seven Swans Al Petteway

The Heart Asks for Pleasure First  Ahn Trio

Black Horse and the Cherry Tree   K.T. Turnstall

The Littlest Birds  The Be Good Tanyas

Submarines The Lumineers

Pipeline (electric and acoustic versions) The Duo Tones

An Spailpin Fanach Connie Dover

The Heroic Conditions of the Universe Pt 7 After the Storm Alexandre Desplat

Turn it Around Lucius

Jolene Mindy Smith's version

Vespro della Beate Vergine: Domine ad Adiuvandum a 6 Monteverdi

Shchedrik Kitka

Our Town Iris DeMent

Sun Will Set Zöe Keating

Don Giovanni Madamina, IL Catalogo and Questo Mozart  (sort of a sign of politics)

Canon a due Violincelli in D Major Gabrielli

Cello Sonata in D Major Op 69 Beethoven

Suite No 6 in D Major  JS Bach (Yo-Yo Ma)

   

what does dim mean and a most unusual job

You're outside on a dark and starry night wondering about those stars.  Is someone out there? You take your flashlight out and point it at the sky.  Your message in a bottle tossed into the cosmos.   I'm guessing most of you did it when you were a kid.  The problem is it doesn't work.  The light you send skyward is scattered or absorbed before it gets out of the atmosphere. At least your imagination was fired - what would it take,  There are people who point lasers beams skyward  and that leads to one of the most unusual job openings I've ever seen.  But more on that later,  there's another interesting question to ask first.

 

Our eyes are amazing at what they do.  We get reasonably detailed color images along with less detailed, but much more sensitive night vision.  We usually think about owls as having some of the best night vision eyes.  They're pretty good, but some frogs can detect a single photon of light.  So image this.  You're a frog watching a light that glows with an unwavering brightness.  If you were a space frog it might be the Sun.   You measure the brightness of the light and then start a journey outwards.  Twice as far from the source the light appears a quarter as bright.  Three times as far and it's a ninth as bright.  One hundred times and it's one ten thousandth it's original brightness.  The measured brightness at your distance goes as 1/r² where r is the distance.  The inverse square law.¹ You continue on and and the light gets dimmer and dimmer.  How did can it get and what would it look like?  I posed the question to a class of premed students I was teaching physics to..

 

The light gets dimmer to a point - the single photon point.  Rather than diminishing further it begins to flash at that brightness. .  At first are flashes most of the time, but travel further and there an increasing number fleshless regions.  Add up the energy you receive - the number of photons over some unit of time.  The frog is  bored and haven't croaked yet, so it plots the energy over some amount of time .. just the number of photons  (flashes) it sees in some length of time.  The curve is still the inverse square law.  

 

Light, as it happens, comes in chucks.  In physicsy terms it's quantized and that chunk is a photon.²

 

When first encountered the paper on the sensitivity of frog's eyes I was blown away.  For perspective a 60 watt equivalent light bulb emit's about ten billion photons a second.   The back of the envelope suggests the Sun would start to look like a blinking single photon about 1000 light years out - the range of our deep space frog. 

 

If you're sending information you have to worry about how quickly you can flash the light to begin with.  If your receiver is seeing a bright light or radio signal (radio is just longer wavelength photons), you can send information at a fast clip.  If it's very dim things get slow.  Even with huge exotic antenna arrays the science from the New Horizons spacecraft that flew by Pluto had to be recorded and sent back over a period of months,  NASA and JPL have been experimenting with deep space laser communication systems. The idea here is a laser beam can be made to not spread out much and will appear bright,  HDTV from Mars orbit can be done.   As long as you don't have to go through the atmosphere.  

 

We've almost circled  back to the beginning - the people who shoot laser beams skyward.  But the atmosphere absorbs and scatters so much.  What's going on?  

 

You can point a very bright, highly culminated beam of laser light to the sky.  Some small lasers are bright enough to travel through a few miles of atmosphere and still be focused and bright enough to temporally blind a pilot.  In fact it's a federal crime to shine a laser at an airplane.

 

There are people who shine extremely powerful lasers skyward legally.  The atmosphere can be thought of as a sea of small regions about the size of your hand. They have slightly different optical characteristics than their neighbors.  Their turbulent movement is what causes stars to twinkle. They represent an enormous limiting factor to the resolution of telescopes with mirrors larger than a half meter in diameter.

 

The trick is to look at a reference star near the object you're studying.  You know where it is so you image it, look at it's new position, do a bit of math and change the geometry of your optics in real time to adapt.  Simple in principle, but difficult in practice, it has revolutionized optical and infrared astronomy at a half dozen of the largest observatories in the world.

 

One of the problems is you need a fairly bright reference star and for about 99.99% of interesting astronomy there aren't any suitable stars that are close enough to use.   Astronomers get around that by making their own reference stars.  The standard technique is to use a very powerful pumped dye laser to create a yellow beam with a wavelength of 589nm..  

 

The laser is pulsed on for a short time - microseconds - and the beam soars skyward.  For the first 30 kilometers there's enough atmosphere to scatter most of the light so it's a visible yellow line. At about 95 km above the Earth it encounters a layer of sodium atoms - leftovers from constant bombardment of our atmosphere with micrometeorites.  589nm is a resonant frequency of atomic sodium so they get excited and then spontaneously de-excite emitting yellow light in all directions.  A little artificial guide star shines during the pulse.  Some of it's light makes it back to the telescope through nearly the same path light from the astronomy target has to travel.  Everything you need to compute your mirror changes.  All of this happens about a thousand times a second.  The process works so well that the largest 'scopes on Earth now resolve better than the Hubble Space Telescope, although it still has other advantages.

 

That upward beam is really bright.  It's only on for about a tenth of a percent of the time, but it could blind anyone on an airplane who happened to be in the way.   You get around this with one of the most exotic jobs I know of - a night watch for the 21st century.

__________
¹ The way to think about it is to recognize light travels in a straight line.  Image two imaginary spheres - one is  one unit of length away from from a glowing sphere and the other two units.  If you pick a region with any area on the first sphere (say 10 square inches) it covers a fraction of the first sphere that is four times larger than the fraction covered by the same area on the second sphere because the area of the spheres goes as the square of their radius 4*Π*r²

² In your physics classes you probably learned light can be a particle and a wave.  You can think of the photon as the particle of light.  Well..  and this would take too much to get into ...  the particle/wave duality model is an artifact about how people saw quantum mechanics in 20s and 30s.   I'll just state what we now know ... everything's just a wave.  But unless you are deep into physics it's probably safest to think about particle wave duality.

 __________

 

Recipe Corner

 

Really easy and just right for the cooler weather

 

White Bean Toast 

 

Ingredients

 

° extra virgin olive oil

° 6 slices of crusty whole grain bread (or whatever constitutes excellent toast for you)

° 1 clove garlic, minced

° 1 tsp chopped fresh rosemary

°  a 15oz can of no salt cannellini beans drained and rinsed

° 2 tbl lemon juice

°  salt

° 2 tsp chopped parsley

° bit of red pepper flakes

 

Technique

 

° oven to 425°F.  Use 1 tbl of EVOO and brush one side of each piece of toast.  Arrange face up on a sheet and toast on top rack until brown and crisp.

° heat 2 tbl of EVOO in a skillet over medium heat.  Add garlic and rosemary when the oil is hot and cook for a minute and remove from heat.

° add beans, lemon juice and salt and mash some of the beans a bit to give a variety of textures and some whole beans.

° spread over the toast, drizzle on some good EVOO, sprinkle on parsley and red pepper flakes.

 

much more artificial than intelligent

Look at the little group of numbers on the right.  They were handwritten and aren't exactly neat and tidy.   Take the first four on the third line.  Our brain says 1 , 7, 4 , 2.   We might bring it together as 1742 and some of you might think of a year - perhaps the year Handel's Messiah was first performed.  And we do it with a brain composed of nearly 90 billion neurons each with up to ten thousand connections to other neurons.  All of this  uses just 20 watts of power and weighs about a kilogram and a half.

 

The brain's neurons are simple computational objects.  There are input 'wires' called dendrites that connect to the cell body and outputs called axons.  Signals arrive on the dendrites and the neuron sums them.  If they exceed some critical value the neuron "fires", sending a signal out through the axons to other neurons.  There's a lot of detail to the nature of the wiring and how it comes to be, but for now think of the neurons as objects that sum inputs and fire if a critical value is exceeded...  more abstractly they're just mathematical functions.

 

The reason for starting off with some numbers is one of the basic programs - the 'hello world' if you will - for machine learning is to build a number recognizer.  A very large set of handwritten and printed numbers was scanned by a 28 by 28 pixel sensor giving a 784 pixel image.  Each of these pixels has a value depending on the brightness of the number it corresponds to.  For the sake of argument say it ranges from 0.00 for black to 1.00 for bright white.  A very crisply written 1 would be a line of dark values in a sea of mostly white. The edges would have intermediate grey values.  Less crispy numbers have lot of grey regions.  

 

To make our neural network (call it a NN) each one of these numbers is put into a corresponding neuron. The number in the neuron is called the activation.  You can think of the neuron as doing something when it's activation is a high number.

 

These 784 neurons are  going to be the first layer of our NN.  We'll have 784 neurons on one side representing the brightness of each pixel of the image.  On the far side is a small layer of 10 neurons on the other side the numbers 0 through 9.  Each of these contains the NN's best guess for how much the scanned number is one of the numbers.  For example you might have 0.02, 0.14, 0.38, 0.02, 0.98, 0.22, 0.13, 0.22, 0.33, 0.07 .. the guess  that it is the number 1 is 0.02, 2 is 0.14 and so on.  In this case 0.98 stands out, so NN is jumping up and down saying 'it's 5".

 

In between are other layers .. often called 'hidden layers.'  Rather than presenting the math that describes the network and waving our hands saying "it's all in the training", let's go in more deeply and see what's going on.  The diagram shows a very simple NN intended to recognize numbers with 784 input neurons on the left (it only shows a few... use your imagination), a single hidden larger of 15 neurons and an output layer with 10 neurons..   

 

Several years ago I wrote my first machine learning program to recognize numbers from this data set.  I used two hidden layers with 20 neurons each.  Their number and sizes are not hard and fast choices - there's a lot of room for experimentation and frankly a lot of this is more art rather than engineering.

 

Consider of the numbers from the pixels in the first layer.  These activations will determine the activations in the first hidden layer which in turn determine the activations in the second hidden layer which determine those in the output layer.  A pattern in the first determines a pattern in the second which determines a pattern in the third and the final pattern tells us the guesses for each number.

 

I like to think about the hidden layers doing something rather than just saying they're a black box.  It may be something very foreign to us, but in this case it is possible that one might, say, recognize components like edges and the next may be larger components like lines or roundish structures (a 9 is a line and a roundish structure, a 7 has two lines...).  It probably isn't doing this, but it's reasonable to think of a NN breaking down problems into layers of abstraction.  Trying to understand what's going on in these hidden layers is very difficult in large problems - perhaps impossible and that becomes an important issue when machine learning is used for purposes that impact us.¹  

 

At this point the post will diverge a bit.  I'll go into some but not detail that some folks may find useful.  I'll highlight it in blue.  Feel free to skip it if you're only interested in a high level view.  Up to now we've established a neural network is just a function that takes input on one side and spits out an answer on the other.  In this case the input is the brightness values of a 28 by 28 pixel scan of numbers and the output is the NN's best guess for each digit, from 0 to 9, of how likely the input was that number. 

 

Consider the simple, but artificial,  example where the second layer is trying to look for a some kind of feature of the number - say certain types of line segments.  Each of the 784 neurons is connected each of the 20 neurons in the second layer.    Each value in the left most layer is a pixel value.  As an exercise to make things clear take out a sheet of paper and make a vertical column of circles labeling them a₁, a₂ , a₃, .... up to a₇₈₄.  (only do a few of them  ^... are your friends)  On the right draw another column with a couple of circles.  Connect each circle on the left with each circle on the right.  We multiply each of these activations by a weight - just a number - and find the weighted sum  of all of the pixels ..  labeling the n^th neuron as a_n and the n^th weight as w_n we get:  w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄.   In the example of a trying to find a little line segment imagine making all of the weights 0 except for those in the region right around the little segment.  Then taking the weighted sum just focuses on the values of the pixels we care about.  If we really wanted to force the issue we could add add a negative number called the bias to the sum to require the weighted average be large.  The bias surpasses other bits of signal in the vicinity of what we're interested in.  The weighted sum going into the neuron on the second row is now (w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄ - b),where b is the bias.  

 

We're getting closer.  This new activation for the pixel in the second layer can have quite a range. If the weights are between -1 and 1, it can be between -784 and +784 before adding the bias.  It is common to scrunch everything to a range between 0 and 1 using a sigmoid function, call it s(x).²  Now the scrunched weighted sum is   s(w₁a₁ + w₂a₂ + w₃a₃ + ... + w₇₈₄a₇₈₄ - b) for each of the 20 neurons in the second layer.  This new activation that goes into the neuron in the second column is the crude equivalent of a synapse in biology.

 

You'll note the naming convention I've used isn't good enough.  We're just looking at what goes into a single neuron of the second layer. In this example there are  19 more, each with a connection to each of the 784 neurons of the first row and each connection with it's own weight.  And there's another bias.  Do this and you have the first two rows.  Next you consider the connections between the second and third layers (the two hidden layers in this example).  After you finish that you have the connections between the third and final layer. And this is just a simple example.   Fortunately computers are really good at this kind of math and there's a natural way to easily organize this.  If you get into this but  you'll need to be on good speaking terms with linear algebra.³

 

Before getting into the "magic"of how these weights are found take a look at the complexity of this example.  There are (784 * 20) + (20 * 20) + (20 * 10) weights and 20 + 20 + 10  biases.  That's 16,300 knobs to adjust correctly - yikes!  Imagine what it's like for problems with columns millions of elements long.  This approach, and there are many variations,  did't take off until computers became powerful enough and we had interesting data - big data - to play with.   Although there are a lot of calculations, each is very simple.  You don't need a big Intel processor with it's complex instruction set.  GPU - graphics processing units - are arrays of thousands of tiny and fairly slow simple processors.  Each is something like ten or twenty times slower than a modern Intel CPU, but all of them can run in parallel..  you end up being faster by a factor of a thousand or so.  Complex machine learning problems have moved Nvidia beyond computer graphics.  It extends to smartphones - Apple's machine learning core on it's latest processors can make this happen in your phone rather than having to rely on the cloud. 

 

Onward to machine learning because we're not daft enough to adjust this by hand.  

 

What we want to do is come up with a recipe - an algorithm - that adjusts those 16,300 knobs so we're likely to get the right answer when we give the machine a scanned number.  For that we'll use some big data.. the enormous set of  scanned and labeled (what the scan really is:  7 for example)  28 by 28 pixel numbers that's in the public domain.⁴   The data is used to train the network.  You start out running some training data, check what the results are, make some adjustments to the weights and biases and keep repeating the process until the result is good enough.  Then you test your NN by giving it another set of labeled training data it hasn't seen so you can see if it's still doing a good job. The hope is the layered structure generalizes what the recognizer does beyond the training data.  This is critical and whoever implements a NN needs to be aware of the domain(s) where it's valid.

 

So each neuron is connected to all of the neurons in the previous layer and it's activation is the weighted sum of all of the all of those neurons.  You might think of the weights as the strength of each connection and the bias is an indication if that neuron tends to be active or inactive.   To start off you set all of the weights and biases completely randomly.  It will give terrible results on the first set of training data. 

 

To make it learn you define a cost function.  We know what the number should be:  4 for example.  Now look at the differences between each final digit and what the machine guessed.  Consider the square of the difference between what you expect and what you get.  If the original number was 4 we expect the output list to behave like this:  0 to have the value 0, 1 to be 0, 2 to be 0, 3 to be 0, 4 to be 1, 5 to be 0,...

 

The first column is the number in question, the second column is the expected result of a perfect NN, the third is the square of the difference of the computed expectation and the expected result (the second column) which are shown in the forth column.

0  0        0.3       (0.3 - 0)²    0.09        

1  0        0.2       (0.2 - 0⁾²    0.04

2  0        0.1       (0.1 - 0)²    0.01

3  0        0.8       (0.8 -0)²    0.64

4  1        0.3       (0.3 - 1)²   0.49

5  0        0.7       (0.7 - 0)²    0.49

6  0        0.3       (0.3 - 0)²    0.09

7  0        0.8       (0.8 - 0)²    0.64

8  0        0.5       (0.5 -0)²    0.25

9  0        0.0       (0.0 - 0)²   0.00

 

The cost is the sum of the squares of the differences... in this case  2.74. Ideally we would get 0.00 for the cost...  

 

The cost is small when the image is accurately recognized, large otherwise. .  Now look at the average of all of the costs over all of the training data...  that gives a measure of how good our NN is for that set of data.  A single number for each set of settings.

 

The trick is just calculus.  Our AI is much more artificial than intelligent.  I'll just state a method exists to find out how to turn the knobs and when to stop.  I'll use a small bit of  blue so be ready to skip if you don't know about gradients, 

 

Imagine the case where we just have one variable.  There would be some kind of curve for the weight relating that weight to the average cost.  If it was very simple, and it's not likely to be, you could just find the minimum by taking the derivative and be done with it.  A better tactic is to start somewhere on the curve and figure out which direction to move to lower the output by measuring the slope .. move left if the slope is positive, right if it's negative. You'll end up at a local minimum.  Now imagine the function with many inputs - so you have many dimensions.  Now you're looking for the direction and steepness of steepest descent ..  just the negative of the gradient of the function.

 

So compute the gradient of the function, take a small step downhill and repeat over and over until you find a local minimum.  So put all of the 16,300 weights and biases into a tall column vector. The negative gradient of the cost function of that vector tells which corrections to all of those numbers gives the most rapid decrease of the cost function.  Minimizing it means better average performance across all of those training samples.

 

A useful way of thinking about the negative gradient of the input vector is each element tells us if a weight or bias should be increased or decreased and how important it is.  so one that starts out as 0.02, -1.29, 0.33 ...  would say w₁ should decrease a little and it is relatively unimportant, w₂ should increase a lot and it is important, w₃ should decrease a somewhat.

 

This setting of the weights and biases is called backpropagation.  We're just minimizing the cost function.  It's important that this function is smooth so we can find local minimums...  that's why the activation numbers of the neurons have a smooth range rather than something more discrete like ones and zeros.  Oddly enough this is where the brain is more digital than the neural network.

 

Playing with this example I was able to recognize a set of number images I hadn't trained the NN on at about 94% accuracy.  There are more sophisticated approaches.  It is important to stress that the innards get very messy and difficult to understand.  That's an unavoidable bug as it is very easy to fool ourselves.  There's bias in the building of the neural network, bias in the training data and bias on it's application.  It is great for some applications, but when it's used in science (I have some experience with it in astrophysics), you use it as a last resort and try very hard to keep the models simple and understandable.  The opposite of what happens in machine learning on social networks. 

 

I think they're a fantastic way to approach certain classes of otherwise intractable problems  and potentially misleading and even dangerous in other domains.

 __________

¹ Like associating aspects of our life and the games we play with our likelihood of us being a good engineer or a neo-Nazi.  These biases, some on purpose and others by accident are extremely important and worth taking up separately.  Machine learning tends to be good at identifying things very similar to known sets.  There be dragons and caution is required.  

² For example s(x) = 1/(1 + e^-x).   There are a variety of functions of this class.  This one is known as the logistic function - you pick what's appropriate for the problem at hand.   In this case large negative numbers go to 0 and large positive numbers go to 1.  

³ Organize the input activations as a column vector and the weights as a matrix where the row corresponds to the connections between one layer and a particular neuron in the next layer.

 ⁴ the MNIST Database from LeCun, Cortes, and Burges.

__________

 

The first good brussels sprouts are out.

 

Roasted Brussels Sprouts with Mustard

 

Ingredients 

° 1 pound brussels sprouts halved

° 6 tbl olive oil

° kosher salt

° 1 tbl dijon mustard

° 1 tbl coarse grain mustard 

° 1 tbl cider vinegar

 

Technique

° oven to 425°F

° put brussels sprouts on a baking sheet.  drizzle on half of the olive oil and some salt and stir them (I use my hands).  Roast on middle rack for about 35 - 45 min turning them a few times until softened and browned.

° whisk the mustards, olive oil and vinegar into a dressing and add a bit of salt to taste.

° put the baked brussels sprouts in a bowl and toss with the dressing.

 

 

 

G, A, F, ↓F, C

just kidding .. well .. sort of..

 

About a month ago I noticed my left shin and foot had swollen enormously.  There are a few nasty possibilities and a close friend died from one in December, so it was off to the emergency care unit.  It turned out to be something easy to deal with, but the exams were interesting. I have this fascination for instrumentation and had a nice conversation with the ultrasound operator.  He didn't know what one of the settings on his machine was doing.  The name was a giveaway so we talked.  In return he gave me a detailed tour of the innards of my legs (good and bad one) and a few other places too.  The interface was aimed at experts, but very interesting.  He had a rich menu of information to display on the screen - often images and graphs. He could also choose to represent some of then acoustically over speakers or earphones.    He said he almost always used the visual displays along with simultaneous audio as combining the channels was much more informative than either alone.   It took me back to my first big particle physics experiment.

 

 

At a conceptual level particle physics is easy - smash particles together and sort through the debris.  The problem is the easy stuff's been done.  More exotic interactions are rare and happen at energies high enough that most of the interactions are complicated mixes of well studied results.  All of this happens at a furious rate much higher than is encountered anywhere else - there is no way to save all of the data you're measuring. You need to be able to throw away what isn't interesting while keeping the tiny bit that is.  When I started we kept one in a few thousand events.  Now some of the larger experiments are more like one in a million.   This is done with with triggers.

 

Triggers are logical groupings of signals that give you an early hint, think a hundred nanoseconds or so, that something is candidate for further study.  Conceptually it's something like this  electron over a certain energy hits this part of your detector, this other part of the detector is completely quiet, and another particle  is coming out at nearly a right angle to everything else. It may or may not be real - that's for further study, but  it happens it's worth saving.  The trigger fires and you save it.  We had about a dozen triggers.  It was often convenient to look at an event soon it took place to learn about your experiment and possibly even find a hint of something interesting.   That's where the title of the post comes in.

 

A few years before this experiment  I had seen Close Encounters of the Third Kind.  A meh as a film, but the music caught my attention.    The aliens used music to communicate.  To signal them, scientists sent played five notes:  G, A, F, F (an octave down) and C. 

 

five tones

 

When the spacecraft arrives they used a variation on the theme and the aliens respond.  Then something of a jam session takes place.   

 

Designing my triggers it struck me that some had cluster groupings in the detector  that could be expressed as a unique three note cord.  I knew about synthesizer music (Switched on Bach and Walter/Wendy Carlos) and sort of borrowed a dozen frequency generators from the undergrad teaching lab (they had a lot of them).  A bit of tuning and I had a full musical scale.  Some of the most important triggers were F major (F-A-C), C major (C-E-G), and D major (D-F#-A) .. there were a couple of minor chords.  It worked and proved itself.  It was my gateway drug -  I've been interested in sound as data and as a computer-human interface ever since.  Data sonification can be a nice way to handle human data overload. We seem to be wired for it (some of us more than others), but modern interfaces rarely use it. 

 

Then there's listening...

 

Imagine a manager stepping onto a railing overlooking the production floor in a textile mill or some other 19^th century. He (sadly always a he) would look around, listen and maybe even smell.  Machines slightly out of spec would stand out ..  a sour note in the harmony.  Moving forward most car mechanics use sound as a diagnostic tool and even listen to customer renditions to narrow the problem space.  I use it to sort out issues with the washing machine and dryer - usually as a diagnostic tool when something breaks, but I'm beginning to use it as an indicator of future failure.   

 

Consider mechanical devices that may be difficult to fully incorporate into data analysis programs.  A feature of mechanical devices is the units of the same model are unique.  An xyx -102 refrigerator in your home probably has somewhat different operating characteristics than the same model in your neighbor's home.  Monitoring such devices and feeding using the information for optimal efficiency and lifetime can be difficult - especially on older existing hardware.  A good example are chillers used in building-scale cooling.  Their efficiency improves with increasing load - often dramatically until you reach a point where the unit shakes and is in danger of failing.  Each chiller is different.  A human operator usually is in charge of several and learns about their individual quirks.  A lot of valuable information - often hundreds of thousands of dollars worth - is locked up in a few local experts.  An expert from a different building would have a learning curve.  Bringing information out of the the human is a major challenge.  You have to be clever. You spend a lot of time with chillers.

 

A neat trick is to study the acoustic signature of chillers as they approach and go through their point of maximum efficiency.  They begin to emit a banging sound that changes to a screeching noise.  They shake on their mounts.  If you can begin to detect this you can back them off a bit and stay near their maximum efficiency while prolonging their life.  The easiest way is a microphone or a vibration sensor (which is all a microphone is).  It is pretty simple to digitize this and move it into a computer than can offer feedback to the machine control.¹  It can also feed into a machine learning program so you can begin to characterize all of your chillers prolonging their lives and predicting when they will fail so maintenance takes the place of failure.  

 

Getting data about the world around us can take a wide variety of sensors, but shortcuts are emerging.  Cameras output images, but add a computer and they can do quite a bit more: image recognition, crop analysis, temperature (infrared cameras) ... the list is dramatically expanding.   We mostly neglect sound, but microphones are a simple and often inexpensive sensor that with a computer can do much more.  Data streams from these powerful computational enhanced sensors can be integrated with other data streams for real machine learning.  Opportunistic data hunting.

 

I have a few of personal interests along these lines.  One is the acoustic ecosystem signature of ecosystems.  The local woods have coywolves.  I've put digital recorders out there just to capture their howls, but it is clear the background sounds might be important.  Flora and fauna have their own signatures spanning large frequency and time domains.  This is probably near ground floor fun for amateur scientists.  I can even imagine it being useful in many sports.

 

Now think about other senses - there are many more than five.

__________

¹ An Arduino and a piezoelectric buzzer/speaker used as a microphone is probably overkill for an application like this. 

 

 

of secondary importance

Perhaps something useful..  a quick cheat sheet on the care and feeding of the rechargeable batteries in your devices.

 

For historical reasons rechargeables are known as secondary batteries and one non-rechargeables are primaries.  We may complain about constantly having to charge everything, but you'd have to spend a few bucks a day to power your smartphone on primary batteries ..  you'd end up sending the bunny about as much money as you send Apple or Samsung.

 

I'm not going into any depth on the various battery families and where they're appropriate. Lithium-ion refers to a very broad category with hundreds of types of batteries, while nickel-cadmium, nickel-metal hydride and lead-acid are much more specific terms.  I'm only going to talk about lithium-ion and lead-acid as they're the most common types most of us use daily.   

 

lithium-ion tips

 

Don't swing the charge too much if you can avoid it.  The innards of the battery expand and contract during charge and discharge  by as much as ten percent.  This results in cracking that lowers storage capacity  Most smartphone and laptop batteries are rated to retain at least 80% of their original capacity after about 500 charge/discharge cycles.  This generally assumes a usage pattern where you rarely completely charge the battery and rarely discharge it to under 20 or 30%.    If you try to stop charging around 80% and never drop below 30%, you should be able to double the battery's service life.  Conversely dropping to under 10% capacity most of the time followed by fully charging can drop the service life to 100 cycles or less.

The best rate to charge a lithium-ion battery depends on its state of charge and varies throughout the charging period, battery history, and a few other things.  Fortunately you don't have to worry about this any more as most name-brand phones and laptops have smart chargers that measure and sometimes even chat with the batteries.  Stay away from third-party chargers.  Not only are you likely to shorten battery life, some have proven to be serious fire hazards.  

Temperature is important.  Below about 40°F capacity drops dramatically.  Fortunately there isn't any damage unless they get really cold.   Heat is another matter.   Anything over about 95°F (35°C) will begin to cause permanent damage.  Not much at first -- constantly keep your phone in your pocket against your body and you may see a 20 or 30 percent drop in cycle life - but a few hours at 110°F (43°C) can kill a battery.   You don't need to live in Phoenix to get to this type of temperature in the Summer - just leave your device in your car on a sunny day.  If you're spending time in hot weather and carry your phone in a bag, it makes sense to put it near a cold pack.  Note that warranties do not cover batteries for excursions beyond rated temperatures and some devices track that.

And if you have to store your device for a long period - a month or more - store it in a cool place (below room temperature if possible, but not in the 'fridge) with a half charge.

so ...

don't overcharge

don't swing them too much

keep them cool - I tell people to treat handle like good chocolate (except don't eat them)

use the charger recommended by the device's manufacturer

 

lead-acid tips 

 

This  is the battery under the hood of almost every car with an internal combustion engine.

Keep them charged.  Your car usually does a good job when the engine is running.  Some of the discharged materialis transformed to a new state from which it can't be recharged. There is a  failure mechanism known as sulfation where some of the discharged material is transformed to a material that can't be recharged.   This takes place slowly over the life of a battery, but can be accelerated to a matter of hours if you allow the battery to fully discharge.   I remember my father accidentally leaving the lights on during the day only to come out to a dead battery eight hours later.  There is a small discharge that runs some of your car's resting electronics, but it isn't a major drain.

If you're away from your car for more than two weeks you should have someone start it up and run the engine for at least fifteen minutes.  If it is a few months you should follow instructions for long term storage (this also applies to other parts of the car).  When I'm away for extended periods I disconnect the battery to halt the small drain from the car's electronics. 

 In the cold weather it is much more difficult to start a car - the oil is more viscous and the starter may take so much energy that abnormally large amounts of sulfation take place.  Also the amount of power a battery can deliver drops.  Keeping a car above freezing for starting can make a big difference.  In cold Northern climates it is common to keep the oil warm by heating the engine block with a 1000 to 3000 watt heating element.  My father called this approach his "poor man's garage".

If you're really careful with the charging cycles and temperature a lead-acid battery can easily go 30 years.  This isn't possible in the more compromised environment your car has to live with.  Figure on replacing the battery in warm climates every six years and every five to six in colder regions.  If you manage to completely drain the battery - even if it is new - just replacing it makes sense.

 

Batteries are getting better, but other than cost it is a slow game.  Don't expect dramatic improvements in the next five years at a minimum.