Sergey Mikhanov  

On the difficulty of thinking big (April 7, 2014)

Yesterday it suddenly struck me what I think is so special about the most successful startup founders.

Let me first say that I haven’t met any of them in person yet. The part of London startup scene that I’m exposed to is not your classic Silicon Valley breed of Stanford and CalTech grads, ex-Googlers and friends of Marc Andreessen. People here build taxi ordering systems and video curation apps. They may be fine pieces of engineering, but this is not the thinking and scale that wins big.

That scale and thinking is, I think, is what differentiates successfull founders. Not alone, though, but combined with the capacity to execute.

Mentioning any of those qualities as differentiators is not unusual at all. Paul Graham wrote about “thinking big”, just as well as Fred Wilson and countless others. Calling founders “execution machines” is also pretty common. What I find most amazing is that those qualities are almost mutually exclusive, and hence so rare. Thinking big, as the authors above mention, is truly difficult.

There is a group that is inherently good at it — children. Here’s, for example, a project from Elif Bilgin, the last year’s global winner of Google Science Fair in her age category. Just think about it: a sixteen year old proposes a way to make plastics from banana peel. She most likely does not know about the political and economical intricacies of the chemical and agricultural industries, about how hard it is to bring a technology like this to a mass market, and about mind-blowingly complicated relationship between different parts of the Earth’s ecosystem. That’s great: all this does not restrain her imagination. You truly believe that you can change the world when you’re so young. This is thinking really big.

If you think you can do this too, try. It’s not that easy to let your imagination go. For every little project detail, your brain does a micro feasibility check, without you even noticing. The wilder your dreams are, the stronger is the pull to the ground.

It’s still possible to imagine a project like being born in the depths of, say, BASF, and given a green light. Then a person responsible for that would drown in the details. Local legislation, partner companies, manufacturing processes, keeping the eco groups at bay, etc. A really big checklist, meticulously followed may help. This is the startup’s execution part that may be less glamorous, but is always absolutely necessary.

Successful founders, I think, are those who can do both thinking big while keeping the execution complexity at bay. Their micro feasibility checks turn into a gut feeling about growing markets. Their execution capacity is big enough for so many variables involved in building of the startup. This is what makes them special.

A love letter to Java (March 31, 2014)

I used to say that when it comes to comparing Ruby and Python, my personal opinion is that Ruby is worse. I absolutely was wrong. To misquote Stalin, they are both worse.

It’s about a month since I switched to working in Python. The project consists of few layers: an infrastructure to control ffmpeg-based video transcoding, elastic cloud running on AWS that hosts those transcoders, and a web front-end to control all that. The more I deal with it, the more I think fondly of the time when I’ve worked on similarly-structured projects in Java.

Don’t get me wrong: what comes below is not a criticism of Python as a language or an ecosystem, it’s more of “what a pity that a great platform like Java gets so little love from hackers.”

For example, a typical setup for a Python web app is nginx, WSGI and some framework (we use Bottle). Now, a typical setup for Java web app is a container (say, Tomcat) and a framework. In Java, when I need to debug my web application, I go to $TOMCAT_HOME/log directory and find all the logging output there. The logging libraries everyone loves to hate, log4j and slf4j, do an enormous amount of heavy lifting to allow a fine-grained control of the logging output not only of your program, but of all the libraries you depend on, plus the container itself. Where do logs from nginx go? Oh, it’s /var/log, whereas WSGI and your app normally log to some different place. Want to reconcile them? Good luck with that.

The famous slogan of Java, “Write once, run everywhere” was a foundation for a mindset that Sun used to enforce. Dependency on the external native libraries was considered a bad taste and a last resort. For example, JDBC drivers were split into four types, from least portable type 1 to pure-Java type 4. Python language is portable, but Python libraries are not — most of them are just wrappers around native binaries, and when Pythonistas say “native”, they mean “Linux”. It took me about an hour to run into a library that our project needs and that does not work on my favourite operating system. That’s pyinotify, a wrapper around a Linux kernel function called inotify, used to watch for changes in the file system. I do understand that it’s hard to develop a library like this that will be portable, but developers of Java managed to.

The situation with native libraries is made even more complex by the fact that Python processes themselves are rarely long-running. They are normally spawned by more stable processes, in our case crond and nginx. Controlling native libraries with LD_LIBRARY_PATH gets even more painful. How do those processes inherit environment variables? How to debug it?

You may ask why I mention Ruby here at all. I think that Ruby and Python together did a very big thing to programming. They made it popular. In parallel with that, they made it seem easy. But it’s not easy, it’s very hard. Programming only seems easy when you focus on some particular detail and ignore the rest, because in most cases it’s the communication with the rest of the world that makes software systems complicated and hard. And as much as everyone likes to say that Java is for people who like to build factories of factories, you have to give Sun credit. They didn’t ignore details they didn’t like.

What a great language.

Couple of good videos I watched lately (March 27, 2014)

C10M: Defending The Internet At Scale. This is a surprisingly deep video about how to program very big machines with huge RAM and lots of CPU cores under Linux disguised as an introduction video to imaginary problem for web developers. You didn’t know how poll or select scale when machine characteristics grow? You can learn about it from the video.

Cloud Haskell. This is a video about how to design a distributed system disguised as a tutorial for a specific library. Really useful to watch even if you don’t practice Haskell. It covers lots of dimensions of a distributed system, such as process execution and distribution, instance allocation and configuration and the way to unify them in a coherent library.

Objective: Get Real (March 23, 2014)

This post is a reply to Jason Brennan’s post “Objective: Next.” Jason reiterates that programming environment for the iOS is very limited in both language and standard library capabilities, outlines some concrete issues he sees as problematic and calls to liberate programming from textual style. My opinion on that is that this is (a) impossible in the near future, and (b) harmful.

There was a lot said and written about how iOS, Objective-C and Xcode programming ecosystem is, basically, inadequate. The language still maintains most of the basics inherited from early eighties despite all later advances in programming languages design, thus causing frustration of programmers familiar with more friendly and open programming environments. But it’s hard to argue with popularity. Mobile is very attractive for even the best hackers (and businesses), and all those very smart people try to find their ways around the limitations of the technology.

In itself, this is not so much of a problem. Much more problematic is the situation when smart people start extrapolating day-to-day problems with their very narrow, closed and restrictive environment to a programming profession as a whole. Bret Victor quoted in the post (absolutely out of context, in my opinion) probably due to his experience and breadth of worldview manages to avoid this mistake. A common vein in his work has nothing to do with programming per se, but addresses abstractions of any kind: those of mathematics, electrical engineering, animation, etc. His recurring point is that computers can help us expressing our thoughts in more coherent and effective manner, thus being better tools for thought.

One may argue here that developing software is just another type of intellectual activity and therefore can benefit from using better abstractions in the process of programming. I mostly agree with this point, but it’s clear that intellectual leverage offered by any abstractions of this sort would only be valid in a very limited context. To underline this limitation, I’ll use this example. Some of my UI designer friends see very little special in Bret’s presentations and videos. For them, it boils down to the “immediacy of response”, a concept know to everyone in the UI world. It postulates that the result of each action should be immediately obvious for the user, even if the outcome of the requested action is delayed. This is why LED lights in control buttons of your washing machine turn on immediately after you press them, or a “Logging on” spinner is shown after you type your username and password on a web site.

When taken in a simple context of washing machines, this approach makes a lot of sense. It can even be expanded to some basics of programming. There are tools that show you contextual information about the code and the project: see touch operations with code in Codea, for example, or the way left-hand side ruler works in some tools from JetBrains. The daily minutae of iOS development, like updating a font size or not seeing the color when it’s typed in hexadecimal all fall into this category (funny enough, the developer-designer interaction was quoted as a problematic point, but team work is one of the reasons why programming languages are textual.)

It’s scaling those abstractions to the world outside of iOS that will break them. Unifying the process of development of a website, an OS kernel, a real-time software that handles 911 calls, and an iPhone app is a task bordering on impossible.

As I said above, you can’t argue with popularity. It’s normal to ask for better tooling, like Jason does, despite talking about avoiding instrumental thinking, but given the sheer number of young programmers starting their careers as developers on the iOS platform, it’s very dangerous to redefine programming on iOS in such a narrow sense. We don’t want all those young programmers becoming incapable of development anywhere outside of Xcode, or scared of moving past Xcode.

Because it’s when you see how wide the whole spectrum of the programming profession is, you can get truly scared.

Logging is the new commenting (July 3, 2013)

Back in 1999, when I was in the university, I was studying the C programming language — just like every other CS major in the world. In our little programs, there were header files and implementation files and everyone in my class, myself included, learned quite early that the number of files in your project will grow twice as fast as you add what our mentor was calling “a new compilation unit.”

Someone in the class — unfortunately, not me — pointed at this obvious design “flaw.” Really, why do you have to duplicate your effort? All the information compiler needs about your program is in the implementation file already. I have only learned later that this is the point where most of the programmers start their affairs with more concise programming languages. But the lesson of that day was clear — duplication is bad.

One semester later everyone in class got their copy of 3rd edition of Bjarne Stroustrup’s “C++ programming language”. One quote I learned almost by heart. It was — this time, fortunately — not related to C++ itself (it’s from the chapter 6.4 Comments and Indentation):

Comments can be misused in ways that seriously affect the readability of a program. The compiler does not understand the contents of a comment, so it has no way of ensuring that a comment:

  1. is meaningful,
  2. describes the program, and
  3. is up to date.

Most programs contain comments that are incomrehensible, ambiguous, and just plain wrong. [...] If something can be stated in the language itself, it should be, and not just mentioned in a comment.

As much as I dislike C++, the writing style of its author is excellent.

For the next ten years or so of my programming career I was barely leaving any comments in the code I wrote. Of course I did understand that comments could potentially be useful in the future for anyone reading the code (myself included) but writing code seemed easy and no explaination was needed. Throughout the same years I discovered the usefulness of logging; it was so helpful that I stopped using debugger completely. Here’s an example snippet from a deliberately incomplete recent program. Note the amount of logging statements:

- (NSArray *)unify:(GAClause *)query with:(NSDictionary *)base dynamicContext:(NSArray *)dynamicContext {
    DLog(@"Unifying query %@, dynamic context is %@", query, dynamicContext);
    
    NSMutableArray *result = [NSMutableArray array];

    GATuple *selfUnified = [query selfUnify:nil dynamicContext:dynamicContext];

    if (selfUnified) {
        DLog(@"Query self-unified, got tuple: %@ ", selfUnified);

        NSMutableArray *tuples = [NSMutableArray array];
        for (GATuple *t in dynamicContext) {
            [t merge:selfUnified];
            
            [tuples addObject:t];
        }
        
        DLog(@"Merged self-unification result with dynamic context: %@", tuples);
        
        return tuples;
    } else {
        for (GAClause *left in [base allKeys]) {
            GAClause *right = [base objectForKey:left];

            GATuple *tuple = [self unifySingle:query with:left dynamicContext:result];

            DLog(@"Bindings after unifying: %@ (clauses were %@ and %@)", tuple, query, left);

            if (!tuple)
                continue;

            [result addObject:tuple];

            DLog(@"Accumulated result: %@", result);

            if (![right isEqual:[GAConstant kTRUE]]) {

Objective-C methods [selfUnify:dynamicContext:] and [unifySingle:with:dynamicContext:] generate even more logging inside. Over a course of years, once I have produced and worked with some significant number of programs that generate meaningful logs, I noticed that reading logging statements in the code help greatly in understanding what the program is about — even without running it. Really, try mentally replacing all logging above with comments:

- (NSArray *)unify:(GAClause *)query with:(NSDictionary *)base dynamicContext:(NSArray *)dynamicContext {
    // Unifying query with dynamic context
    
    NSMutableArray *result = [NSMutableArray array];

    GATuple *selfUnified = [query selfUnify:nil dynamicContext:dynamicContext];

    if (selfUnified) {
        // If we reached here, query is self-unifyable

        NSMutableArray *tuples = [NSMutableArray array];
        for (GATuple *t in dynamicContext) {
            [t merge:selfUnified];
            
            [tuples addObject:t];
        }
        
        // Merged self-unification result with dynamic context
        
        return tuples;
    } else {
        for (GAClause *left in [base allKeys]) {
            GAClause *right = [base objectForKey:left];

            GATuple *tuple = [self unifySingle:query with:left dynamicContext:result];

            // Finished unifying, variable bindings now created

            if (!tuple)
                continue;

            [result addObject:tuple];

            // At this point we have partly accumulated result

            if (![right isEqual:[GAConstant kTRUE]]) {

Once you’ve got the logging, the need in having lots of comments in code reduces dramatically. You don’t need to duplicate your effort. You even can get rid of the downsides described by Bjarne Stroustrup, because someone running a program and reading its logs will make sure the output is meaningful, descriptive and current. Logging therefore becomes a better commenting system, helpful both during runtime and when figuring out what a particular piece of code does. Given the flexibility and robustness of modern logging frameworks available in any language, there are very few reasons to use lots of comments. Just log more.