Under the hood of a software development process

Here in The7bits, we are developing custom software solutions for a wide range of clients. Modern software development is not a simple process. We should follow a lot of practices and processes for an excellent and reliable result. The complexity of modern software products is increasing dramatically for the last decade. It means that the role of customer education is also increasing.

What do I mean when say “customer education”? I am talking about a complex of activities like meetings with clients and preparing information about different challenges in software development.

Let me give an example. Today we are the eyewitnesses of a significant change in the global economy. Many traditional offline sectors of the economy are changing dramatically. Just look on Uber or AirBnb. Moreover, now we see like many non-technical entrepreneurs with expertise and experience in different real sectors of economy tries to start their businesses with the help of modern technologies. If you go to AngelList, you can see many startups like that. In logistics, delivery, medicine, transportation, agriculture, food and other sectors. We are also a part of that world, the last year we are working on Crane-Locator startup and tries to change a sector of project cargos. I think that it is a fantastic experience for us. The vast opportunity of modern days is a possibility for non-technical entrepreneurs to create a simple prototype of their products almost without coding with a very minimal volume of technical knowledge. Services like Squarespace or Shopify could help to create a pure “zero version” product straightforward and fast. However, very often an idea of a startup requires a custom software development. Moreover, price of making a wrong decision is very high for a young startup with (as it often is) limited funding. In such situation an entrepreneur has few options:

  1. Hire a developer or few developers on services like UpWork on a per-hour basis.
  2. Hire a developer or a team of developers for full-time work.
  3. Outsource a whole development process to the agency.
  4. Outsource a dedicated development team to the company which provides such kind of services.

Each of these options has pros and cons. However, let me give advice. Do not think about a team of developers first. Think about technical management. The best situation is when one of the co-founders of a startup could play a role of Chief Technology Officer (CTO). It is a guy who makes decisions about project technology stack, development processes (who, what, how and why should do), a set of necessary team roles, development strategy, release schedule and bunch of other things. For technology startups, it is a crucial role because of it almost impossible to build a good and reliable product without such guy. If an entrepreneur does not have a co-founder like that he or she has two options:

  1. Hire CTO. Alternatively, find yet another co-founder in this role.
  2. Outsource CTO role for some period to the agency.

The first option is not easy. Hiring such kind of person is a strict process. Because life or death of a startup depends on the person, you hire. In general, the best choice in such situation is choosing the second option. Outsourcing a whole product development process including a technical management role is an excellent choice for developing the first version of product or MVP (Minimum Viable Product). If your project is successful, you can build a full-time dedicated development team for your project for increasing reliability of your business later.

However, why technical management is so important? Some people (and I saw people like that) think that software development is a straightforward process. Just find a developer and ask him or her to write an application which does something. The typical idea looks like that:

Office whiteboard

It is a real photo of our whiteboard in the office. Believe me that if you take an average developer and ask him to implement this idea without any involvement in the process the result cannot be right. And here I am talking not only about the result on the screen of the developer’s laptop. The next list of questions are significant:

  1. Where to store project’s source codes?
  2. Who has access to source codes?
  3. How to follow changes in source codes?
  4. How to organize work with source codes
  5. What to do if we found a critical bug when a developer is working on a significant feature and cannot implement a fix without breaking a whole product?
  6. How often to update source codes on the remote storage?
  7. How to launch a project for the public?
  8. Where to host a project? Will we use dedicated or virtual servers or cloud platform (like Heroku)?
  9. If we use a dedicated or virtual server, who can administrate the server?
  10. What the strategy for scalability? Do we need horizontal scalability (when an application can scale to many servers) or vertical scalability (when we can scale computational resources with a fixed amount of servers) is enough for us?
  11. How to deploy source codes to a production server (or servers)?
  12. Do we require a staging server?
  13. Do we require automated tests?
  14. Where will automated tests run?
  15. Do we need a continuous integration?
  16. And many other questions about monitoring of server resources, web app monitoring, errors tracking, debugging services and many related ones.

A complete list of such questions could be huge. Moreover, if you need a developer, not for the development of any simple script or doing a little experiment you should know answers for each of these questions. If you do not have an answer on few of such questions, you must understand a risk. Moreover, here you should understand that if you found a developer for a per-hour basis on the service like UpWork the answers for such questions is your competence, it is not developer’s competence.

Let us explain a software development process in details a little bit. First of all, we should make some decisions about project’s infrastructure. It is a significant thing because web project must be accessible online. Moreover, it is much more critical for lean development practices because we should update public version continuously for getting feedback for modifying the product. So this is a cyclic process. It means that infrastructure is a necessary part of the development process.

Write code - deploy - get a feedback process

The infrastructure should provide a solution for the next problems:

  1. Production environment
  2. Staging environment
  3. Server orchestration
  4. Error collection
  5. Server monitoring
  6. Application monitoring
  7. Version control and source code storage
  8. Continuous integration (automated tests running, reporting and deployment)

Each project requires its infrastructure due to the available budget and project features. The next picture is a result of one of our brainstorms where we planned a start of a new project.

AWW board brainstorm

In this particular case, our customer had an insufficient budget, and we built infrastructure by following a lean startup methodology. The result was like that:

  1. Bitbucket for version control (Git) and storing the source codes because it has reasonable prices for work with private repositories and has enough functionality.
  2. DigitalOcean as a hosting provider.
  3. Self-hosted open source version of Sentry for collecting the application errors.
  4. Our custom developed server orchestration platform Grape which provides a unique web-based tool for operating the Docker containers.
  5. Pingdom for check the application health and load time.
  6. NewRelic for server and application monitoring.
  7. DroneIO as continuous integration service for running automated tests and deployment.
  8. Mailgun as a mail delivery service.
  9. Amazon S3 as storage for daily incremental backups.

Such kinds of decision are significant because now we have a construction site for our project. Of course, different starting conditions produce different decisions. Also, you need a high-class specialist for that. The next piece of decisions is a list of rules and standards which every developer in the team must follow. There are two pieces:

  1. Technologies.
  2. Internal standards of coding and deployment.

The first decision should be a list of accepted technologies. For example:

  1. Python 3.5 as a programming language.
  2. Django 1.9 as a web framework.
  3. Ubuntu Linux Server 14.04 as server operating system.
  4. PostgreSQL 9.4 as database.
  5. Redis as key-value storage.
  6. Memcached as a distributed cache system.

The list could be increased later during the process of development, but each item should be accepted by a lead developer or CTO because technology stack must be stable and maintainable, there is no place for surprises.

The second piece of development rules is internal standards. For example:

  1. Each line of code must follow PEP-8 rules.
  2. Branch with a name “master” contains a stable version of a project. The source codes of a version which are currently in development must be in “develop” branch. Each experiment or feature implementation must previously use separated branches.
  3. Each commit must be reviewed by another developer from the team.
  4. Each new feature must be covered by automated tests.
  5. And so on.

It is an insufficient list, just for example. However, the availability of such rules is also a very critical part of the development process.

Why are infrastructure and rules so essential? Because it is a skeleton of the future engineering department of your business. It is a canvas which allows to hire required specialists and grow your team. If you do not have rules, you have chaos. The sad truth is that there are much more people who can code than people who can write rules and build a team. Such kind of job is technical management.

As you see the software development is a complicated process which includes different kinds of activity. Of course, it should not stop you on the way of implementing your business ideas. Do not fear to work on your startups. The main advice which we would give you is doing the right things right.