DevOps Roadmap 2023
In this article you will learn what tools you need to learn to become a devops engineer and what are the tasks and responsibilities of a devops engineer.
First you need to understand that there are two main parts when creating an application. The development part where software developers program the application and test it. And operations part where application is deployed and maintained on a server.
Devops is a link between the two now this is a bit too abstract so let’s dive into the details to really understand the devops tasks. And which tools are needed to carry out these tasks.
Concepts of Software Development – DevOps
It all starts with the application. Developers team will program an application with any technology stack different programming languages build tools etc. And they will of course have a code repository to work on the code in a team. One of the most popular ones today is git.
Now you as a devops engineer will not be programming the application. But you need to understand the concepts of how developers work which git workflow they’re using. Also how the application is configured to talk to other services or databases as well as concepts of automated testing and so on.
Operating System & Linux Basics
Now that application needs to be deployed on a server so that eventually users can access it. That’s why we’re developing it so we need some kind of an infrastructure on-premise servers or cloud servers.
And these servers need to be created and configured to run our application. Again you as a devops engineer may be responsible for preparing the infrastructure to run the application. And since most of the servers where applications are running are linux servers.
You need knowledge of linux and you need to be comfortable using command line interface. Because you will be doing most of the stuff on the server using command line interface. So knowing basic linux commands installing different tools and software on servers. Understanding linux file system basics of how to administer a server. How to ssh into the server and so on.
Networking & Security – DevOps
You also need to know basics of networking and security for example to configure firewalls to secure the application. But also open some ports to make application accessible from outside. As well as understand how ip addresses ports and dns works.
However to draw a line here between it operations and devops. You don’t have to have advanced super operating system or networking and security skills.
And be able to administer the servers from start to finish. There are own professions like network and system administrators security engineers and so on. That really specialize in one of these areas.
So your job is to understand the concepts and know all this to the extent. That you’re able to prepare the server to run your application. But not to completely take over managing the servers and whole infrastructure.
Containers – Docker
Nowadays as containers have become the new standard. You will probably be running your application as containers on a server. This means you need to generally understand concepts of virtualization and containers. And also be able to manage containerized applications on a server.
One of the most popular container technologies today is docker. So you definitely need to learn it.
Continuous Integration & Continuous Deployment (CI/CD)
Now we have developers who are creating new features and bug fixes on one side. And we have infrastructure or servers which are managed and configured to run this application.
The question now is how to get these features and bug fixes from development team to the servers. To make it available to the end users. So how do we release the new application versions basically.
And that’s where the main tasks and responsibilities of devops comes in. With devops the question is not just how we do this in any possible way. But how we do this continuously and in an efficient fast and automated way.
Build Automation & CI/CD Pipeline
So first of all when the feature or bug fix is done. e need to run the tests. And package the application as an artifact. Like jar file or zip etc so that we can deploy it. That’s where build tools and package manager tools come in.
Some of the examples are Maven and Gradle for java applications for example npm for javascript applications and so on.
So you need to understand how this process of packaging testing applications work. As i mentioned containers are being adopted by more and more companies as a new standard. So you will probably be building docker images from your application.
As a next step this image must be saved somewhere right in an image repository. So docker artifact repository on nexus or dockerhub etc will be used here.
So you need to understand how to create and manage artifact repositories as well. And of course you don’t want to do any of this manually. Instead you want one pipeline that does all of these in sequential steps.
Jenkins – DevOps
So you need build automation. And one of the most popular build automation tools is jenkins. Ofcourse you need to connect this pipeline with git repository to get the code. So this is part of continuous integration process. Where code changes from the code repository get continuously tested.
And you want to deploy that new feature or bug fix to the server. After it’s tested built and packaged. Which is part of continuous deployment process. Where code changes get deployed continuously on a deployment server.
And there could be some additional steps in this pipeline like sending notification to team about the pipeline state. Or handling failed deployment etc.
But this flow represents the core of the CI/CD pipeline. And the CI/CD pipeline happens to be at the heart of the devops tasks and responsibilities.
So as a devops engineer you should be able to configure the complete CI/CD pipeline for your application. And that pipeline should be continuous. That’s why the unofficial logo of devops is an infinite cycle. Because the application improvement is infinite. New features and bug fixes get added all the time that need to be deployed.
Cloud Providers & IaaS
Now let’s go back to the infrastructure where our application is running. Nowadays many companies are using virtual infrastructure on the cloud instead of creating and managing their own physical infrastructure.
These are infrastructure as a service platforms like AWS, Google-Cloud, Microsoft-Azure and Linux etc.
One obvious reason for that is to save costs of setting up your own infrastructure. But these platforms also manage a lot of stuff for you. Making it much easier to manage your infrastructure there so for example using a UI. You can create your network configure firewalls route tables and all parts of your infrastructure through services and features. That these platforms provide however many of these features and services are platform specific.
So you need to learn them to manage infrastructure there so if your applications will run on AWS. You need to learn the AWS and its services. Now AWS is pretty complex but again you don’t have to learn all the services that it offers. You just need to know those concepts and services. That you need to deploy and run your specific application on the AWS infrastructure.
Container Orchestration – Kubernetes
Now our application will run as a container right. Because we’re building docker images and containers need to be managed. For smaller applications docker compose or docker swarm is enough to manage them.
But if you have a lot more containers like in case of big MicroServices. You need a more powerful container orchestration tool to do the job.
Most popular of which is kubernetes so you need to understand how kubernetes works. And be able to administer and manage the cluster as well as deploy applications in it.
Monitoring
Now when you have all these maybe thousands of containers running in kubernetes on hundreds of servers. How do you track performance of your individual applications. Or whether everything runs successfully. Whether your infrastructure has any problems. And what’s more important how do you know in real time if your users are experiencing any problems.
One of your responsibilities as a devops engineer may be to set up monitoring for your running application. The underlying kubernetes cluster and the servers on which the cluster is running.
So you need to know a monitoring tool like Prometheus or Nagios or etc source like cfix
Infrastructure Provisioning & Configuration Management (IaC)
Now let’s say this is our production environment. Well in your project you will of course need development and testing or staging environments. As well to properly test your application before deploying it to the production.
So you need that same deployment environment multiple times. Creating and maintaining that infrastructure for one environment already takes a lot of time and is very error prone. So we don’t want to do it manually three times. As i said before we want to automate as much as possible. So how do we automate this process.
Creating the infrastructure as well as configuring it to run your application. And then deploying your application on that configured infrastructure. Can be done using a combination of two types of infrastructure as code tools.
Infrastructure provisioning tool like terraform for example. And configuration management tool like Ansible or chef etc.
So you as a devops engineer should know one of these tools to make your own work more efficient. As well as make your environments more transparent. So you know exactly in which state it is and easy to replicate and easy to recover.
Scripting Language
In addition since you are closely working with developers and system administrators. To also automate some of the tasks for them. You would most probably need to write scripts maybe small applications to automate tasks.
Like doing backups, system monitoring tasks, cron jobs, network management and so on. In order to be able to do that you need to know a scripting language. This could be an operating system specific scripting language like bash or powershell. Or what’s even more demanded a more powerful and flexible language like python, ruby or golink. Which are also operating system independent.
Again here you just need to learn one of these languages. And python without a doubt is the most popular and demanded one in today’s devops space. Easy to learn easy to read and very flexible.
Python has libraries for most of the databases operating system tasks as well as for different cloud platforms.
Version Control – GIT – DevOps
Now with this automation tools and languages you write all of these automation logic as code. Like creating managing configuring infrastructure that’s why the name infrastructure is code.
Now how do you manage your code just like the application code. You manage this also using version control like git. So as a devops engineer you also need to learn git
So at this point you may be thinking how many of these tools do i need to learn. Do i need to learn multiple tools in each category also which ones should i learn. Because there are so many of them.
Well you should learn one tool in each category one that’s the most popular and most widely used.
Because once you understand the concepts well building on that knowledge and using an alternative tool will be much easier. If for example you need to use another tool in your company or project. So as you see these are a lot of technologies that you need to learn as a devops engineer.
If you want to learn more then please visit our blog page.
