Every engineer worth their salt has been in this situation: It’s late at night and they are alone, grumbling and grumpy, struggling to get something they’ve built up and running in production. The good ones take pride in overcoming these challenges. The bad ones shrug those challenges off, saying “It works on my machine,” and leave the hard part to someone else. In my opinion, it’s a necessary stage in any engineer’s career.
The battle scars that remain from these experiences lead to a deep respect for the singular truth of software development. It doesn’t matter how the code you write runs on your machine. It only matters how it performs for your users in production.
This fundamental truth has led to a revolution in the software industry. Industry leaders like Amazon, Microsoft and Google have all embraced an approach to building and deploying software called DevOps. DocuWare has embraced these principles as well.
But what is DevOps? Dev is an abbreviation of Development and involves the people who design and create the software. Ops is an abbreviation of Operations and involves the IT staff such as system and database administrators and network engineers who keep the software running properly. DevOps, in turn, represents a set of philosophies that emphasize bringing development and operations together to work more closely.
Within my industry, some sneer at the term DevOps saying that it is little more than a buzzword. I think this is because DevOps doesn’t provide a simple checklist of steps to follow. It’s centered around guiding principles that point you in the right direction and leave the “what” and “how” up to you. The core principles were actually developed as part of the lean manufacturing movement in the ‘90s. They weren’t aimed at software development at all. The principles of DevOps can be used to help any business.
DevOps is guided by The Three Ways. I’ll explain what each of the ways is and how they can help you.
The First Way: Systems Thinking
Let me illustrate the First Way by using the software business as an example. Historically, software developers, QA, and operations folks worked in silos and rarely interacted. We developers would build features and functionality within our group. When the software was deemed ready for testing, we’d hand if off to the next group: The Quality Assurance (QA) team. They test it, report on defects and approve it for release. Then the software is passed to the next group: Operations. They set up and manage the operations of the system.
The handoffs between the groups were often problematic (a lot of work to deploy the code and a lot of information to pass along with it). So, we tended to avoid doing them too often. We’d wait until we had a lot of stuff “finished” and then hand it off to QA. The same goes for the handoff to operations. What we have in production now is working so let’s wait until it makes sense to deploy a big update, right?
The First Way is about tearing down those silos. Bring everyone involved together and use their knowledge to develop a true understanding of the entire system. View your enterprise as a classic value chain focusing energy on defining how your system creates value for your customers.
This heightened awareness results in being able to answer questions like:
- Where is the constraint in my system? (i.e. the bottleneck that limits how quickly work can pass through it)
- How long does it take for a work item to complete its lifecycle? In our case, that’s the time that it takes for an idea from the product team to become a deployed feature our customers can use.
- What are the major obstacles that prevent you from achieving business efficiencies? How can workflow automation improve these processes?
Using these principles, if there’s a difficult aspect of your business, attack it relentlessly. In the IT business, deployments can be a source of pain (remember the aforementioned battle scars? I’ve got plenty of them). DocuWare now uses automated deployments to reduce human error and simplify the process of getting the features we create deployed to both the QA and production environments. Because we have embraced DevOps, we can deploy new code several times a day.
To improve your processes, find your weaknesses and turn them into strengths. As a rule, if it hurts, you should strive to do it more often, not to avoid it.
By applying the principles of the First Way you will:
- Never pass a known defect to a work center downstream. Rework is very costly.
- Never let an optimization to a part of the process degrade the overall process. The key is to optimize the flow from the beginning of the process to the end.
- Always look to speed up the flow of work.
- Always seek a profound understanding of your system.
- Avoid leaving a lot of work in process. Rather, constantly push it through to completion.
The Second Way: Feedback Loops
Once you’ve conquered the First Way, you’ll have learned where your constraints and what parts of your processes are painful. Now you’re ready to make improvements. But how will you know if your improvement works?
The answer is to create feedback loops to monitor and measure the effects of the changes you make. If you make a change but can’t really see the effects of that change for several months, course corrections become incredibly difficult. Your goal should be to shorten and amplify feedback loops. Companies like Amazon experiment with pricing and incentives in small targeted groups and gather the results of those experiments immediately to understand how well they worked. The feedback provides them the business intelligence they need to understand the market they’re serving. In software, our teams monitor the health and behavior of our products after we make changes. Do we see errors? Did the software get faster or slower? We automate testing to determine if the changes we make are safe to deploy rather than wait for the results of a QA handover to let us know.
Combining the First and Second Ways provides the business agility that you need to respond to your customers’ needs.
If you understand your value chain, you can use that understanding to improve your processes and measure the effects of those improvements, then you’ve put your business in an excellent competitive position.
The Third Way: Adopt Continuous Learning
The Third Way involves shifting the culture of your organization. It’s about battling the mentality of “we’ve always done it that way.” It is comfortable to let a process become deeply ingrained through repetition. Over time, however, it will hold your business back as your competitors innovate around you.
Instead you should work to create a culture that:
- Encourages continual experimentation. This requires taking risks and learning from success or failure.
- Uses repetition and practice to develop mastery.
The key is not to eliminate mistakes, but to learn to fail gracefully. In software, we call this the Mean Time to Recover (MTTR). So, in the case that one of my engineers makes a mistake and our code falls flat on its face in production, how long does it take to recover? At DocuWare, the answer is measured in minutes, and we are constantly working to improve its speed.
Learning and embracing the principles of DevOps has radically changed how I do my job as compared to when I started in this industry. It’s opened my eyes to its applicability to any process. I hope I’ve shown how it can help you look at your own business in a new way.
