Open Source

How open source can solve Silicon Valley's engineering crisis

Silicon Valley may think itself the center of the technology universe, but 76% of open-source development happens elsewhere, a rich talent pool for engineer-hungry startups.

Open source

Silicon Valley may think itself the center of the universe, but when it comes to open source, it can only muster a third-place finish. According to an analysis of top GitHub contributors, both Europe and the rest of the United States develop more open-source software than Silicon Valley. While this may not be surprising given Europe's long-standing affection for open source, it is a reminder that much of the best development talent doesn't live along Highway 101 and probably never will.

Who contributes the most code?

The Wall Street Journal's Christopher Mims recently took Silicon Valley venture capitalists to task for funding the wrong kinds of startups -- you know, advertising funded startups that eschew "basic research and development that transforms lives."

Had he looked at GitHub, however, Mims may have noticed a different picture.

While open-source developers can be as faddish as the next person, one of the cardinal tenets of open source is that it encourages developers to "scratch their own itches." In other words, solve pressing problems that they may have, regardless of venture funding.

In this way, open source gave us Linux (operating system), Hadoop (big data analytics), Lucene (search), JBoss (application server), Drupal (web content management), MySQL (database), Day (web content management), nginx (web serving) and more, many of which projects emerged from Europe (not surprising given the wealth of innovation coming from Europe today).

While VCs piled on in later years to fuel these projects, they started out as an open-source answer to an "itch."

Itches being itches, they aren't geography-dependent. It isn't a perfect proxy for all open-source development, but a quick analysis of GitHub's top-250 contributors suggests an interesting demographic mix for the overall ecosystem:

  • Silicon Valley - 24%
  • Asia-Pacific - 14%
  • Europe - 31%
  • Rest of USA (somewhat even split between NYC/DC, Seattle, and Middle America) - 25%
  • Brazil - 3%
  • Canada - 3%

As for other demographic information, I couldn't get any recent data. However, I doubt that open source has changed much from its original demographics: middle-aged developers with several years of professional experience. Given its rise, this has likely only increased.

Language of choice?

As for programming language preferences among the most active GitHub contributors, JavaScript reigns supreme, claiming 38% of contributed code. The full rankings are as follows:

  • JavaScript - 38%
  • Ruby - 16%
  • Python - 10%
  • PHP - 8%
  • C - 4%

Everything else is 2% or lower, including Perl, Objective-C, Haskell, and others.

Hiring the best developers

Given how tight the job market is for engineers in Silicon Valley, maybe it's time for Silicon Valley employers to go remote. Given that 76% of open-source developers likely do not live along Highway 101, and the relative ease of evaluating performance through code review and online interactions, why not build out remote teams?

For some, this is anathema. With few exceptions, for example, Facebook and other Silicon Valley tech giants require employees to work in the same office in the interest of speeding development. Yes, studies suggest that co-located engineering teams can be more productive, with less time needed to coordinate resources and make decisions.

But co-location also means you hire from a much narrower talent pool. In fact, former MySQL and ZenDesk executive Zack Urlocker told me that it is "much easier to hire when you have distributed teams."

Given the prevalence of open source, many engineers already know how to operate within distributed development teams, even if their employers don't currently function that way. To ease Silicon Valley's talent crunch, it's time to look to the open-source community. Do you agree? Share your opinion in the discussion thread below.

About

Matt Asay is a veteran technology columnist who has written for CNET, ReadWrite, and other tech media. In his day job, he is the vice president of business development and marketing at MongoDB. He was previously chief operating officer at Canonical, ...

5 comments
Programming VB6
Programming VB6

Microsoft won't even reply to a letter asking for VB6 - the Visual Basic 6 programming language - to be open sourced.

anil_g
anil_g

Is there any reason to think that Microsoft employs the top talent?

jk2001
jk2001

Or,  you can move to Redmond Washington and hire up talent from Microsoft, who are laying off 18,000 people in the next year.  That, and the numerous contractors they're using are going to be out of work.  Think about it - some of the top programming talent in the country, now available.

jaywink
jaywink

What is the source for the language stats? That doesn't look like anything I've seen. Ruby that high?


This seems a bit more realistic TBH: http://langpop.corger.nl/

Edit: sorry, it is of course "...among the most active GitHub contributors".

martinwastaken
martinwastaken

Yeah I agree. Using agile processes, it's not that hard to integrate distributed team members while keeping time and quality in check. I mean, once a project is kicked off, developers work better on their own in an environment they can just focus and get things done. We do this all the time using SCRUM. Your point about the talent pool is a big one, for me it makes much more sense to go the distributed route and find talent in the open source community. And like you said, it's pretty easy, because GitHub shows you more than an interview ever could. Within minutes you can identify a developer that works well in a team and writes good code. It's a brave new world.

Editor's Picks