I guess there are. Without having a big desire to fall back to the original topic of my blog, I want to point to a recent issue we had with our biggest SLEE-based project. This is the first project where we are using big branchy trees of SBBs in our application and the first one that showed such a bad results during the initial performance tests. What is happening? Why it is so slow? After several nights and weekends spent staring at the CPU profiler and thread dumps and a help from our vendor, the root cause of the problem became clear.

When the event flow is being delivered to the SBBs in the same tree, event router thread does not have any choice except for locking the whole tree. After all, there’s a SLEE transaction ongoing and the state of the SBB tree should be kept consistent. Other event router threads at that moment are waiting for the tree to be freed, even if they try to deliver the event to another SBB in the tree. More blocking time, higher latency, lower performance.

Interesting is that JAIN SLEE just like any similar event-driven framework tries to isolate the developer from threading almost completely. But it’s clear that a developer can’t even diagnose a problem like ours without resorting to knowledge about thread states. So personally I have no doubts about the necessity of this knowledge. And I think our playful age of Mongrel and Cucumber gives more opportunities to learn low-level things with less pain. Ruby folks seem to discover fibers lately (green threads whose execution is controlled programmatically) so there will be a lucky few who will master the art of its scheduling. I myself had a chance to play with the Java emulator of x86 PC — seriously, having the possibility to compile your C program, take the binary, and see how Processor.processProtectedModeInterrupts() is executed in your Java emulator is astonishing.

I’m excited to read about other people thinking the same.

The reader might be familiar with the David’s philosophy of doing business: it emphasizes profits (they are more important than market share, investments or anything else) and simplicity (minimalistic approach to products is the way the company could stay focused on delivering it and being frugal.) He sees the way chosen for 37signals as universal. When he evangelizing it, he is fastidious in his choice of words.

While it’s easy to fall for his “bad boy in the startups world” charisma, the interview gives a stark contrast between David’s tendency for overgeneralization and genuine experience of Jason. David is thirty and his interview host, Jason Calacanis, is ten years older. When placed together, they reveal the most attractive traits of each other’s personalities.

What I have enjoyed most in this video:

• How masterly Jason leads the discussion of some sorts of business models to its culmination, and how he ends this with a strong counter example like “Oracle might disagree”
• A discussion of work ethics. David: I always take Friday off. I prefer to work smarter, not harder. Jason: do you know what kind of schedule Steve Jobs enforced in Apple in the early days of PCs?
• How Jason constantly repeats “interesting” on almost all David’s remarks, and after somebody tweets about that in the middle of the show, and Jason reads the tweet aloud, he never repeats this word again. This is what I call a brilliant speaker.

This line is a signal to scroll up and click Play. Enjoy.

Among other insightful things, the comments for that post brought a link to Who Needs an Architect?, a Martin Fowler’s article. I can’t resist the temptation to cite Ralph Johnson mentioned by Martin in the text

One of the differences between building architecture and software architecture is that a lot of decisions about a building are hard to change. It is hard to go back and change your basement, though it is possible.

There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change.

Martin writes a bit earlier referencing the talk by economist Enrico Zaninotto

One aspect I found particularly interesting was his comment that irreversibility was one of the prime drivers of complexity. He saw agile methods, in manufacturing and software development, as a shift that seeks to contain complexity by reducing irreversibility — as opposed to tackling other complexity drivers. I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs.

Most of the efforts behind the software architecture activity are focused on removing the need for it. I can’t be fascinated more about the extent how this fits into my vision.

On the first glance this seems wrong. There are dozens of open positions in software companies bearing this title. A quick search on Amazon for “software architecture” reveals thousands of titles. But first 50 results or so are dealing with the software modeling using some higher-level description language, which is usually UML, and sometimes more exotic choices are presented like Z Notation (anyone?). There’s even a book luring the described hacker into buying it with its title: Software Design: From Programming to Architecture (it is about UML modeling, too). The only exception is the useless collection of developers’ anecdotes named 97 Things Every Software Architect Should Know (carefully cached by Google here), but I tend to think that this is the exception which supports the rule.

But wait, shouldn’t software architecture be an activity which involves dealing with software itself and not the models of it? Book authors do not think so. I guess the reason for that is the fundamental dichotomy between programmers in different companies. A former colleague of mine wrote recently a piece on that:

From what I see, a typical Java-centered company is either a J2EE enterprise or a place where they care about util.concurrent. Amusingly, this seems to be a very clear dichotomy. In both cases, there is a pile of (de facto) standard APIs and frameworks which take considerable time to master. And people who gravitate to one of the two types tend to be rather apprehensive if not ignorant of the other

“J2EE enterprises” love “outsourcing-style interview questions”:

[…] an endless stream of extremely low-level technical questions on particular technologies and APIs. RDBMS isolation levels, the servlet lifecycle, the methods on EJB home/remote interfaces, JSP scopes. A design pattern for them is always one from the GoF book or Sun J2EE patterns guidelines. Characteristically, they do not care about your design skills or code quality and really like J2EE-style

For the recruiters or book writers “software architecture” is neither an activity nor a way of thinking, it is a way of determining the job title. In their reality “software architect” is the person squeezing 3rd party code into a conceptual software model. He clearly belongs to the J2EE side of the dichotomy though might not use J2EE at all. It is a person who values dependency on someone else’s software in favor of developing it. This sort of dependency may be good, but for the demanding subject domains this is a double-edged sword (Paul Graham covers this issue with his usual bias towards Lisp in one of his essays). The reason behind all that is simple: “architect” does not write code. Mr Joe Hacker, is this the career path you were dreaming about?

A curious reader might ask: why do you proudly claim to hold this title then? Because I believe in software architecture as a way of thinking. I see this as a capacity to deal with the huge problems and significant amounts of code nip and tuck with the in-house development team. I do even know a single good book on the subject: Eric S. Raymond’s The Art of Unix Programming. Even is you never will hack Unix, this is a very valuable book to read.

What about career path then? Well, I can’t give any personal advice here except for the “avoid advertised ’software architect’ positions”. The best company will allow you to keep writing code while expanding your area of influence wider and wider into the company products at the same time. This is what really software architecture is about.

For those who hasn’t seen the film: Malik, a 19-year old Arab is sentenced for six years in prison. He starts as a dogsbody for the Corsican mafia boss, then becomes his deputy, strengthens his own influence outside of the prison, initiates the clash between Corsican and Arab crime groups, and when Arabs take power becomes a leader of theirs.

Leaving apart the dubious “romance” of the criminal world, this sounds like the story of successful career. It involves significant amount of luck (the episode with the deer), quick wit, gut feeling for what takes priority at any moment, ability to infuence and negotiate, and stamina (when I see Malik following Corsican’s advice to be a “good prisoner” after Malik was taken under his protection, I can’t help but imagine junior lawyers in the firm doing the most boring due diligence tasks in the course of their first three years.) Huge number of people commit themselves to similar but lawful path in the everyday life. Sure, careerists do not risk their lives, but the stakes are high anyway — after talking to some successful New York traders, I guess I do understand what “high” means here. When looking at the story under this angle, I wonder why there are so few movies around covering someone’s career in the making? Sure, we have Pollack’s The Firm and Soderbergh’s Erin Brockovich and some other pieces where the door of personal office is opened for the protagonist to his biggest surprise, but unlike A Prophet those are not the stories of steel being tempered.

Someone might say that a story like this would not be captivating? Doubt that. What was in Jérôme Kerviel’s head when he entered his office at Société Générale every morning? Did the Gap’s manager who issued abortion policies for their labor force in Bangladesh get promoted? They all build their career. Unlike Malik, they join an “invisible graveyard”, but please filmmakers tell us about those who has survived.

ABCD (Self-portrait) by Raoul Hausmann (1923-1924)

If you’re subscribed to this blog’s feed expect more entries of general interest soon; “100% telecom” mood has been left behind.

Just like the last year, the conference has attracted participants from top telecom companies and standardizing organizations from all over the world. Interesting enough, the main focus of the conference has shifted. Instead of the walled garden, as some critics nicknamed the IMS architecture, most of the discussions were about open APIs of different sorts. From Deutsche Telekom’s Developer Garden (this very interesting and ambitious project aiming to provide paid access to DT’s infrastructure for developers has been rolled out in just around a year) to BONDI (attempt to standardize device API to give developers unified access to the device’s resources): different projects showed a great amount of interest in acquiring third-party developers. This trend clearly shows that the future of the telcos’ NGNs and the set of services they are to provide will be determined by community of developers or by ecosystem of startups supported by telco, and not by a telco itself.

Not only the openness was the hot topic. Over the years FOKUS shows the ability to respond to the industry needs in a very flexible manner. For example this year among the technical demos was the full-blown IaaS/PaaS solution. The virtual slices hosting FOKUS’ OpenIMS were moved around, resized, and reconfigured on the fly. As IMS itself (I will cite Prof. Magedanz again) became “vintage in regard to real research challenges, although there are still a lot of open deployment and integration issues”, the mentioned FOKUS solutions are capable of providing the base for necessary “openness” and “flexibility” of the future telecom.

One of the interesting discussions sparkled after a presentation on BONDI, a device API which allow mobile applications of the future access device capabilities in a standardized way. The question being discussed was “Why do we need a way to restrict the application’s access in any way, be this AppStore review process or standardized API policies?“ Indeed, web and desktop applications ecosystem is able to survive and flourish without any external intervention and in majority of cases it’s the end user who (successfully) chooses good applications over malicious ones.

The very interesting answer being born in the lobby discussions included two major players in the AppStore game: telcos, who play directly, and government, who plays indirectly. Telecom industry is huge when compared to the internet industry. Just look at Google’s 2008 net income of $4,2 Bn, compare it with AT&T’s $10,5 Bn (data is from Wikipedia) and remember that Google is one of a kind, whereas companies comparable with AT&T exist on every continent. Telecom industry is also highly regulated, so the biggest fear for the telcos is the government intervention. I can easily imagine the situation where government closes GPS for public use after some criminal has stolen the victim’s GPS location and was able to commit a crime. For telcos this means complete loss of the revenue from location-based services, and to avoid that they in advance force the control over assets that may potentially compromise the end user’s security, hence the whole AppStore approval process story.

Note that none of similar regulations exist in internet, because there’s no easy way to enforce the control over it. This also explains why Android Market is review-free and Palm App Catalog is not (I consider those two as AppStore’s competitors): Google, unlike Palm, does not manufacture any phones subsidized by telcos, and will not suffer should government intervention happen.

Up to now market shows that AppStore protected by approval process could still be a success. Looking at the situation under this angle I may only guess whether developers will ever have a non-restricted access to AppStore.

NSBR divides companies participating in any stage of the software development and getting profit from that in four broad categories (examples are mine):

• Companies manufacturing hardware and included software like operating systems (Microsoft, Apple, Palm and some less known ones)
• Companies manufacturing software for internal use (most of the banks, insurance companies, etc)
• Companies manufacturing software to be sold to the end users (Microsoft, Adobe, Autodesk and others)
• Companies manufacturing custom software for someone else (any outsourcing company falls into this category)

What is very interesting about this division (note that NSBR was written in 1995) is that it completely unaware about the software service market — it hasn’t existed yet. You cannot place Google in any of the categories, just as you could not do this for zillions of small profitable companies flourishing in the Internet. This fact was so surprising for me because Brooks made a lot of predicions in NSB in 1975 of which almost all became true but even in 1995 he could not foresee this huge niche. The obvious observation that commodization of Internet lead to overwhelming results, just as commodization of computers did the same earlier become striking when you see that with the angle of the software engineer with the experience of Brooks’.

I recently asked in the programmer’s community whether there is any post-MMM influential book on the process of software development and it seems like it is yet to be written. Probably, will have to re-read Mythical Man-Month next year.