Why your organisation needs to embrace working in the open-source ecosystem

For any modern technology company, consuming open-source software isn’t optional anymore: it’s a mandatory requirement for your organisation.

Consuming open-source is only the start of the journey for many companies. Whether by design or necessity, organisations find themselves adapting from being simply open-source consumers to being full-fledged producers, contributors and maintainers, eventually working in public, following ‘open by default and ‘upstream first’ principles.

But organisational transformation is never easy. Codethink has worked with many clients as they embark on this journey of change, and based on our experience, we believe the most viable option is to fully embrace open-source culture and operate as part of the open-source ecosystem.

This article intends to outline some of the things we’ve learnt from this experience, about the benefits of working in the open, and some mistakes to avoid.

The open development model is simply better

Firstly, let’s touch on why open-source is so ubiquitous in modern technology. In my heart, I’d like to believe that it’s down to the fact that companies care deeply about the utility of free software, viewing it as a shared public resource and a force for a common, greater good. But sadly, that’s probably not the case.

Open-source is so wildly popular because the development model is so successful. Nine times out of ten, it’s simply the better way to develop software in all measurable aspects: faster development cycles, fewer defects, more robust and versatile result, to name but a few. Lots have been written on this already, and I won’t go into it too much here. Still, these merits essentially stem from the collective intellect of an unlimited number of developers able to contribute and review the source code (consider Linus’s law: ‘given enough eyeballs, all bugs are shallow’), in combination with progressive strategies for controlling changes

Prior to open-source, most proprietary software followed a traditional design and development approach: a team of experts worked behind entirely closed doors to develop and eventually release their product. The result of this in today’s market would be a severe duplication of effort and re-inventing the wheel due to the ‘Not Invented Here’ mentality. In contrast, let’s consider the Automotive industry, where the world’s vehicle manufacturers work collaboratively in the open via the AGL project. The project provides an infotainment based platform for all auto-makers to build on. It gives them 70-80% of the starting point for a production system and allows them to focus on innovating and customising the last 20-30% to differentiate themselves, rather than investing wasted effort to develop the same core each time in silo.

This is not to say that open-source is an ‘all-or-nothing’ proposition. Organisations do well to identify which of their software is close to their strategic core and perhaps cannot be shared; the rest can be considered as supporting services and will likely benefit from the open-source approach.

The lesson: gain an appreciation of why open-source is so popular and consider the ethos behind the development model as part of your culture.

Don’t wait until it starts raining to fix a hole in the roof

At the beginning, consuming open-source is expensive, as organisations have to accept changes: learning how to integrate new dependencies, modify features for specific use cases and apply bug fixes. After this, organisations can realise the medium-term benefits: reduced costs from removing licence fees and greater freedom to control inputs.

For the time being, consuming open-source seems like a solid business decision. Actively contributing to open-source and operating within the ecosystem may not seem necessary. It can certainly be hard to appreciate the return on investment doing so. After all, it takes time and expertise to be done correctly.

Eventually, though, the market demands innovation in the form of new features, and new security patches are likely also needed. Some backporting of the latest developments in open-source is possible, but again it feels expensive. The kernel and toolchain now feel a little dated. So, organisations realise that they must upgrade. Again this feels expensive but is absolutely necessary, and the benefits are clear and obvious.

But here is where the specific changes made by organisations matter. All the differentiation factors that provide the USP for the product have been applied locally: the code modifications for specific use-cases and bug-fixes mentioned earlier. They’re critical to your business, and they now must be forward-ported to the newly upgraded software. Because the organisation has carried these changes locally, only they can re-apply them, and the cost is immense. It’s a complex task and requires a specific expertise, often requiring outside help [side note: often, it’s in these exact situations where Codethink has been contracted to assist customers].

As a result, leaders at the organisation realise that they have less control over the inputs of their product than they thought and decide they must contribute features upstream in order to avoid this cost when the next upgrade is required.

The lesson: wherever possible, upstream features sooner rather than later, and avoid local patch carrying at all costs.

Make upstreaming part of your development process, not an afterthought

By this point, it is clear that contributing code in the open to the projects that are relied on is necessary. The action required now seems relatively straightforward: simply submit the required patches to those projects. But without proper planning, this activity will almost certainly fail. Many organisations fall into the trap of considering this as something to be done when time permits or assigning it to the wrong people who cannot focus on it due to other priorities.

Instead, upstreaming needs to be embedded as part of the development culture: with proper planning of which release cycles are the target for features and active engagement upstream. Teams should be built with the express intention of becoming key members of the upstream community, working towards eventually being able to drive the focus of the project. This will also lead to faster resolution of issues and shorter development cycles for the features that matter to your organisation. Only then will you be able to fully benefit from the innovation brought by the intellectual horsepower of the community.

This will also help with your hiring process: encouraging engineers to work in the open as part of their day job is a major point of desirability when hiring.

The lesson: plan properly how to invest in open-source projects that develop the key software you consume, and feel the benefit of the innovation from the community by being a driving force in the direction of the projects.

Keep up with the latest Codethink news

Subscribe to our newsletter to receive our articles about open source software in your inbox.

Related to the blog post:

• We are hiring: Software Engineers >>
• Making the tech industry more inclusive for women >>