Sergey Mikhanov  

Steve Reich, Jan van Eyck and human perception (June 17, 2015)

It’s currently my holidays, and I have no plans to blog about anything serious during this time. So I’ll write about music and arts instead.

I recently went to a concert to hear Steve Reich’s one long piece, called Drumming, performed by a New York musical collective So Percussion. Reich’s work wasn’t previously unfamiliar for me, but I’ve never heard it live before and only had some surface knowledge about him being a minimalist-academic and an important figure in NYC musical avantgarde of the sixties. For me “minimalism” in this context meant that his work is about getting rid of decorative elements in music — such as melody — in an attempt to get to the music’s essential bare bones. Whereas “academism” was about keeping the musical semantics aligned with the academic tradition, i.e. that even this stripped bare music has certain expressive qualities that cannot be substituted with something else, like silence, for example.

Drumming essentially consists of short repeating percussion patterns, gradually shifting relative to each other and slowly mutating by either adding a beat (substituting the rest with the beat) or removing a beat. And that’s all.

When performed though, Drumming exhibits much more complicated effect on listener that it may seem from this description. First of all, although the tempo of the piece is not changing — and this is meant in the strictest sense, as the entire structure of composition depends on the rigid rhythm, — two repeating patterns with a small phase difference between them effectively sound slower compared to the same patterns later, when their phase shift reaches about half of the phase. This makes the music sound slow and then gradually get certain “merry galloping” quality, very unexpected from a simple piece like this.

Second interesting effect was the “roar” that echoing percussion establish in the large concert hall — this one is completely absent in the recordings I previously heard. It sounds like a constant standing sound wave formed by precussion sounds being reflected by the auditoriums walls. It sometimes “swallows” simple vocalizations in Drumming or tricks your listening into thinking you actually hear some sounds when there’s nothing to produce them.

Have a look for yourself (the video below only shows a part of a first movement, the entire piece lasts more than an hour):

Both described aspects of Drumming are interesting to me because they deal with our perception of music. Reich knew that the listener’s brain tends to group close beats together into one, and although musicians are instructed to play as strictly to the rhythm as they can, your listening perception just can’t help itself and assigns subjective qualities to shifting patterns, like “jumpy” or “restrained”. Note that musicians in other styles are fully aware of this effect. In funk and soul, slight off-tempo beats are used to give the music special groove, that pulls the listener ahead. Reich’s rhythms are restricted a very formal scheme, that looks cold and austere and certainly has no groove, but it also opens an extra dimension of the piece’s richness when off-tempo beats appear and disappear.

If this all sounds like a useless theoretical exercise to you, here’s another example of using mechanisms of perception from a different field.

Arnolfini Portrait - Jan van Eyck

This is Jan van Eyck’s Arnolfini Portrait, dating from 15th century. This masterpiece warrants a much longer description of its qualities, but I’ll focus on very few of them here. Van Eyck’s goal with this and other paintings was to reach the ultimate levels of authenticity, i.e. he wanted to make his paintings looks exactly like the reality they depict. Now, we, humans, can only get an image of reality through our sensory devices, like eyes (or ears). Does Arnolfini Portrait, being as detailed as it is, triggers exactly the same reaction in our eyes as the scene where it was painted?

To a modern viewer it’s obvious that it doesn’t. Despite putting immense number of details into the painting and strictly adhering to the rules of perspective (only have a look at the spherical mirror!), Van Eyck lacked some understanding of visual perception that was only established much later, with the invention of photography and understanding, among other things, what “depth of focus” is. Have a look at a much later painting by Gerhard Richter, called Betty.

Gerhard Richter - Betty

This is not a photo, it’s a painting, which certainly looks more “real”. Note that the human eye doesn’t have a depth of focus the way photographic camera does, and the “blurring” in our perception of reality is not due to some optical effect, it’s a property of our visual cortex.

So what are the similarities between Reich and Richter in their respective fields? In the case of Gerhard Richter, he advanced our understanding of painting to a new level by reading into the visual properties of the photographs. Steve Reich advanced our understanding of music by restricting his practice of composing to some formal rules. In both cases, it took a genius to see beyond the medium and into perception mechanisms for it.

Open-plan offices, creativity and programming (June 8, 2015)

This is a very important text for me. If my reasoning is even remotely correct, this is going to have a very significant influence on how I — and probably many other people — think about their career in programming in the future.

Let’s look at some headlines:

  • Google got it wrong: the open-office trend is destroying the workplace (Washington Post)
  • Open-plan offices makes employees less productive, less happy and more likely to get sick (Quartz)
  • The open-office trap (The New Yorker)
  • You’re not alone: most people hate open offices (Fast Company)

Plus thousands of blog posts and comments in online communities. There seem to be a consensus: overwhelming majority of people dislike open-plan offices, but are left with no choice. I’m among those who put “good office” close to the very top of my list of priorities when looking for a new job and yet, of my fourteen years in the industry I spent ten in open-plan offices due to the complete absence of alternatives.

Why is it so? Mutiple studies (check any of the links above) confirm that employees in the open-plan offices are more stressed, less productive, take more sick leave and, above all, less creative. The entire idea of being intellectually creative at work is very comforting for us, programmers. Simply being creative is one thing; being creative and being paid for it is completely different. A very famous movie about hackers, “Triumph of the Nerds”, captures this vibe right in its title: this triumph is over the the business people, the dumb MBAs, the managers who demand weekly TPS reports, it’s the victory of the smart underdog.

So surely, with all that creativity hindered by the noise and distractions of the open-plan, there should be a niche for a company that puts smart hackers in comfortable working condition and reaps rewards coming from their increased productivity, right?

Except that it’s not so. One of the studies I found, estimates the average office area occupied by a knowledge worker in an American office with private rooms to be 37.16 m², which is almost three times more than in the open-plan office: 13.94 m². Think about it: the same office building with the same rent can sit almost three times as many people. No CFO would sign-off a private office policy after seeing this difference.

What about creativity and productivity? In most companies programmers require much less of it than they think. The software industry is incredibly commoditized nowadays and the business people are well aware of it. Nobody expects programmers to be creative; merely not being too bad is good enough. Note I’m not saying that those “business people” are evil or incompetent. You can replace “business people” with “free market” with the same result: nobody expects creativity from a person working on a web marketplace for vacation rentals, a taxi ordering app, or supply chain management system. With my anecdotal experience, I estimate the proportion of programmers involved in non-commoditized software development — the tiny innovative core of our industry — to be around 0.1%. This is not as low as it seems. If this estimation is correct, Apple has 50-60 developers engaged in truly creative software and hardware development, Google has 35-40, Facebook around 10. Them, plus other members of their immediate teams (say, ten times as many; those who do gruntwork only partly and have some potential in them) are responsible for most of the technology we see and use today. I am sure they can have any office they like.

The rest of us are not as creative as we like to think and our bosses know it:

A few years ago, I was working at a company where they had just purchased an old shoe making factory and were renovating it into office space so they could have room to grow. In each spot where a person had sat for 8-12 hours hunched over in front of a sewing machine, I was told to install a workstation and run network and electric cable down from the ceiling where the sewing machines had been hooked up.

At the end the project, a co-worker of mine that had the foresight to take a picture of the old setup took one of our finished work and compared the results. The room looked a lot cleaner, and didn’t have the smell of oil and leather anymore, but in the photo, it looked like a factory that you’d see in the early part of the 20th century by its layout. The lone difference being instead of looking like it would make shoes, it looked like it’d make code. They both looked like you’d have the same amount of privacy (aka none), and unless you bought headphones, the same amount of silence for concentrating on your task(s). […]

I see a cleaner, digital sweatshop, a modern version of an age that many thought we had left decades ago. It hasn’t really left, it’s just had the cleaning crew in and been given a few runs through the marketing machine to make what was once undesirable “sauve and sexy!”.

— from Reddit

On reading scientific articles (June 1, 2015)

I have created a shared folder with Computer Science articles I read, understood and liked. At the moment, there are two items in there, but will be more over time. It’s not a surprise that both of them deal with the question of engineering a system as large as a programming language infrastructure and runtime and staying sane — this is a question that occupies lots of my thinking lately (I don’t know the answer yet, but promise not to withhold my findings). Here they are:

  • P. Li, A. Tolmach, S. Marlow, and S. P. Jones, “Lightweight Concurrency Primitives for GHC”, in Proceedings of the ACM SIGPLAN workshop on Haskell, 2007;
  • K. Wang, Y. Lin, S. M. Blackburn, M. Norrish, and A. L. Hosking, “Draining the Swamp: Micro Virtual Machines as Solid Foundation for Language Development”, in 1st Summit on Advances in Programming Languages (SNAPL 2015), 2015.

This does not mean, of course, that I only read two articles in my life. It’s just that these two I understand much better than the others and I see how they can immediately be useful in my programming practice. Understanding is a key point here: I remember that during my PhD time my approach to scientific articles was utterly wrong. Here are few rules I learned over time:

  • Reading and understanding abstract is a prerequisite to even attempting to read an article. There’s a video of one of the Don Knuth’s lectures where he famously answers a question on performance of some tree-based algorithm from the audience and says something in the vein: “I remember seeing an article describing recent advances about this algorithm, but authors didn’t describe performance limits in the abstract, so not sure.” Abstract absolutely must contain each and every key important moment of the entire article.
  • One consequence of the above is that there must be articles out there describing important things, but not doing well in their abstract and being completely overlooked because of that. My attitude towards that is sad acceptance: it is simply not possible to assume the best about every article and spend time reading it anyways when article’s abstract is not clear or not exhaustive.
  • At the university I read articles trying to get an answer to what-questions: “what kind of new things people work on?” The better question, the way it seems to me now, is how-question: “how do people do those things they do?” The crucial difference between the two is that the second one implies at least a brief familiarity with the field for which the article is written. What-questions, I now think, are better answered on other media, like textbooks.

Couple of good videos I watched lately — 2 (May 26, 2015)

Big Data is (at least) Three Different Problems. Michael Stonebraker, the most recent recipient of ACM Turing Award, in his usual charismatic manner addresses the most common real (as opposed to imaginary) problems arising when dealing with large amounts of data. Tons of valuable insights about modern databases.

Facebook’s iOS Architecture. Facebook people talk about implementing Facebook app on iOS. You can clearly tell that classical MVC approach is terribly outdated when using it to build apps of the complexity you see in 21st century. It’s not that mobile development is inherently difficult, it’s just gets entangled and messy very quickly as the apps grow.

The world’s most complicated software (May 19, 2015)

A typical software developer in a company possessing some level of technical sofistication routinely switches between abstraction levels during a working day. He or she may go from reasoning about product structure on a web page level to the intricacies of file allocation in their database system. Those dealing with some sort of message processing can switch from a byte-level layout of the protocol messages to a more general view of interconnected queues within the system — you get the drift. From the very early days of being in the profession, programmers are told that abstraction is the key to fighting complexity natural to all software. It’s only after spending few years in the profession, some may discover that few domains are surprisingly resistant to abstraction alone. Without the only tool to fight complexity, developers are left to accept the difficulty of the field as a given. I’ll give an example of a truly difficult problem.

Meet the calendar, the world’s most complicated software.

I’m talking about a product like Microsoft Outlook (actually, its calendaring part and the server). On a first sight, there’s nothing special about it, but if you try to think about it, complexity starts manifesting itself from a very basic level. The interaction protocol between participants trying to agree on a meeting is surprisigly hard to get right. For example:

  • When someone receives a meeting invitation, should it be shown to him if a meeting has already passed? How do we detect this (note that we should take time zones in consideration, including the cases when participant is not in his default time zone)?
  • When someone proposes a new time for a meeting, but it can’t be sent because the participant was offline should we try re-sending it when participant goes online? What about if the meeting time has already passed? What if not, but his proposed time did?
  • What about the time zone changes? If the participant changed a zone, should we reschedule his events? What if all participants changed the time zone? What would be the time zone of the newly proposed events? Of the changes into existing events?
  • Should we, once the participant goes online, notify him about the changes to the events that were not accepted by the participant because he was offline? What about the already passed ones?

And so on with an added inherent complexity of dealing with time zones. Calendar is a classical distributed system with participants being people within the same organizations using it simultaneously. Participants can be offline for extended periods of time; they must find consensus on timing of the group events using some reasonably robust protocol; they move around. You probably noted already that part of the problem is the difficulty in specifying it correctly — you’ll have lots of fuzzy and vague sentences with “except” in your specification, rendering almost all your abstraction skills useless. The difficulty of developing a calendar suite differs from the difficulty of your typical job ad’s “hard and interesting problem” in the same way as your morning 2 km jog differs from doing an Iron Man.

If you made a note to yourself never to work on calendar suites, here’s the second most complicated software in the world: library dependency manager. It does not have to deal with people as participants, but is just as full of fuzzy specs: how to handle conflicting (or broken) transitive dependencies, non-mandatory ones, source vs. binary, etc.

Would you want to work on a calendar suite or a dependency manager?