The Legacy Pain of Late-Stage Software Startups and How to Cope With It

Today there is a large number of software startups, though only few of them become giants. We could name a lot of reasons why most startups that seemed promising experience slowdown in growth or even die. However, I would say that the most painful reason for the failure of many software startups is their inability to develop their software systems. They may start successfully, have ambitious and promising plans for further development, and nevertheless fail. Below, I want to describe this problem in detail and show how it can be solved. All my ideas and conclusions are based on the experience I have gained as CEO of a software development company.

Due to the well-known fact that software startups grow faster than those in other sectors, it may seem strange that many of them face such an issue. Nevertheless, I have met many problematic startups that were unable to keep up the pace of software development. And that was not because of an inefficient development team or inappropriate technologies; though these aspects do also matter, I would say that what matters most is the software itself—that is, its architecture.

Why Software Architecture Matters

To start with, I would like to touch on the evolution of software architecture. This was perfectly illustrated by Benoit Hediard:

First, a monolithic architecture (now often called spaghetti) was used. Therein, an application is built as a single unit; new features are added to the core software, so that all its functionality is tied together. With each feature added to the software, its codebase becomes more complex; therefore, any changes to the software require a great deal of testing. This complexity makes further development slow down.

Later, a multi-layer architecture was preferred for enterprise software systems. Usually, a three-layered architecture is implied: a client-side user interface, a database, and a server-side application. In this approach, each layer is a monolith; thus, adding new functionality requires deployment of a new version of a single layer, but not the entire system. However, as the system functionality expands, layers—especially the server-side application—become more complex. Developers feel frustration with this complexity because even minor changes require the huge codebase block to be rebuilt and redeployed.

This led to a new approach to software development—building layers as suites of services—which is called a service-oriented architecture (SOA). Each service has a defined interface that can be used to perform its tasks; this interface definition hides details of the service’s implementation. Thus, all services are interoperable and can be deployed independently. Moreover, they are independent of the application and programming language.

As SOA is evolving, services are divided into microservices with a range of sizes. Microservices should be as fractional as reasonably possible, considering all risks and spendings.

Though this view may change in the future, today I consider microservice architecture as the most appropriate for enterprise software development.

How Software Architecture Relates to Late-Stage Software Startups

A lot of startups started developing their software years ago, when the vision of software architecture was quite different, and software scalability and maintenance were not matters of concern for developers. Today, having built a rather complex codebase, such companies have trouble expanding the software functionality. The following challenges appear:

• Any changes made in code lead to a great deal of testing. Thus, the development is significantly protracted.
• The creators of the software are the only ones able to deal with the code.
• It is impossible to parallelize development of different functional pieces because of the tangled codebase. Therefore, it is difficult to increase the size of the team, which means that the development may not be expedited.

Even if a startup intends to develop software using service architecture, when they conceive the software it is almost never possible to predict how successful the software will be, which functionality will be the most popular, and what extra functionality will be required as the software develops. This is why services are often extensive, and over time the software becomes complex and its further development slows down.

How to Deal with Complex Software to Expand its Functionality

Let’s assume that a startup has a development team that created software according to the startup’s business goals. The software has a layered architecture, though it is rather monolithic. It is integrated with databases and a number of external systems.

As the business grows, new functionality is required. However, adding this functionality to the monolith is very challenging. Moreover, if several new functions are required, the team will not be able to deal with the tasks even being increased, because adding changes to the monolith leads to redeployment of the entire software system again and again.

In this situation, the following actions are possible:

1. A new functionality should be built as a microservice (or microservices) that communicates with the system using an API. A developer or a group of developers may be hired for this work.
2. A small part of the monolith is split out into a microservice that communicates with the system and new functionality by means of API. This work can be done either by the core development team or new developers.

Step by step, more and more pieces of the monolith codebase may be split out into microservices. Thus, the size of the monolith software is reduced. All new functionality is developed as microservices. Later, as the functionality grows, services may become large enough to then be divided into several microservices; each of them may be developed by a separate developer, if required.

In this case, a number of new developers may be hired. The work of microservices they develop does not influence the core block and other microservices. The new developers don’t have access to the core software, which is vital to understand when referencing security issues. The core development team does not need to be increased, though it remains responsible for the core block, which may not be assigned to new developers.

As a result, development of new functionality is parallelized and the pace of development accelerates.

Because the design that occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important for effective design.

“Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization’s communication structure.” This is the famous Conway’s Law, which has been confirmed by the experience of companies such as Netflix, Amazon, and many others. Due to the development team structure, which consists of multiple small teams, which are loosely coupled and often distributed, the outcome of the development is software consisting of a number of services that can change, evolve, and be created and removed separately from one another. The more autonomous each team, and the better the communication established between these teams, the more efficient software development becomes. As a result, the software system becomes flexible, and changes may be delivered to production faster.

Reasons for Using Microservices for Reengineering Legacy Software Systems

As a rule, startups create new software by means of a small development team. If the business is successful, the software grows, the team increases, and at some stage the software architecture does not allow for rapid expansion of functionality. In this case, reengineering the software using microservices must be done in a timely manner. The software architecture should evolve together with the system and team growth.

Using microservices brings the following advantages:

• The system does not need to be rebuilt from scratch. New functionality should be added as microservices; parts of the monolith architecture may be gradually split out into microservices with the same functionality.
• Microservices communicate using APIs. Microservices are independent of each other and of the core application. Thus, technologies used for their development do not influence other parts of the software; in each case, the best technology may be used.
• Because microservices are independent of each other, their development, testing, and deployment take less time and may be implemented independently.
• Development may be parallelized and developers may work on a number of microservices independently. The development team may be increased; it can be divided into several teams or each developer can work separately, even in different time zones. Thus, the development team may be distributed.
• The security of the main software application may not be affected because only the core team has access to it; microservices interact with it using APIs.
• The system can be easily maintained and quickly evolve.
• The system becomes more scalable. Under load increase, the entire application does not need to run on several servers; instead, one or a few services may be scaled out using fewer resources and less infrastructure.

The microservices approach brings even more benefits; it is crucial, nevertheless, to plan its implementation and consider all possible risks and effects.

What Risks are Involved When Using Microservices for Reengineering?

Today, using microservices is on trend. Many specialists write about the benefits of using microservices; however, few of them write about risks and disadvantages. But these do exist, and this fact should be definitely considered.

First of all, when using microservices instead of a monolith or multi-layered architecture, the software infrastructure increases significantly and new challenges appear, such as:

• The infrastructure should run smoothly, but without over-architecting.
• The system performance should be continually monitored so as not to reduce.
• As a service may fail, the system should be designed to tolerate such failure so that this doesn’t affect users.
• Not only should microservices be tested, but their integration as well.
• The stack of different technologies, which is selected as the best choice for each microservice, should be maintained.

When software reengineering is needed, two scenarios are possible:

1. A new software system is created from scratch based on a stack of modern technologies. The new system has the same or extended functionality; microservices are used instead of the monolithic architecture. As the system is implemented, it can be deployed to production in place of the legacy system.
   Pros and cons:
   The new system is implemented and tested before being part of production; thus, the system within production is always operable. However, while the new software is being developed, users have to work with the legacy system, which has no new features and might suffer from poor performance due to aged technologies.
2. The legacy system is continuously decomposed into microservices; new features are developed as microservices as well. Once a new microservice is implemented and tested, it can be deployed into production.
   Pros and cons:
   The system is being enhanced gradually, so users can benefit from new features sooner. However, when split out into a microservice, the particular functionality should be located everywhere in the codebase and replaced with a link to the new microservice. This step threatens to create bugs and even crash the system.

From our Experience

In the past, we have used both scenarios. When reengineering an online marketing platform, we took the following steps:

1. Create a new system architecture;
2. Improve data structure;
3. Select technology stack appropriate to new data structure and system architecture.
4. Implement the renewed system, test it, and deploy it into production.

So, the process was first thoroughly planned, possible risks were assessed, and we then executed actions to mitigate those risks.

Another scenario was used to reengineer an enterprise knowledge-management system. Rapid deployment of a new functionality into production was required; thus, the following actions were done:

• Upgrade the source code language version;
• Gradually refactor the monolith application into microservices;
• Optimize the logic of working with the database;
• Conduct system maintenance and further development using microservices.

To achieve a safer reengineering process and avoid the risk of failure in production, we created a local replication of the system and used it as the groundwork for all changes made.

When providing software reengineering under the second scenario, the main challenge is to replace the particular functionality with links to the new microservice. For example: a new microservice has been developed that sends mass emails to a selected group. In addition to the functionality that existed in the monolith application, some new features have been added. To use this microservice, developers need to find every place in the system where sending emails is executed, remove these blocks of coding from the codebase, and link these places to the microservice. After this work has been conducted, the whole system should be tested to check that removing this part of code hasn’t affected other parts of the software. If one of the blocks has been missed and the new microservice has not been linked there, the software will suffer from inadequate mailing functionality in that particular place.

Other Signs that Reengineering is Required

There are a number of signs that make it obvious that system reengineering is urgently required. In the cases below, reengineering is needed if the problem cannot be solved quickly:

• Regarding users:
  • The system suffers from poor performance or clumsy navigation;
  • The system is unable to serve some or all expected users’ needs;
  • The client application is not Web-oriented—there is a desktop version or a Java applet instead.
• Regarding business:
  • The system is not coordinated with the company’s business processes;
  • Adapting the system to new market conditions appears to be too time-consuming and expensive.
• Regarding technologies:
  • The system runs on an older platform that is no longer supported (e.g., an old version of the Java EE Application Server, such as JBoss AS 5.x, or a desktop Java application such as AWT or Swing);
  • The system was created using aged technologies, for example EJB 2.1, JSP, or Apache Struts 1.x. In this case, security issues may appear and adding new functionality may be complicated.
• Regarding data:
  • If an inefficient data model leads to a large number of queries being required to obtain data, then the data structure should be optimized, which may lead to a need to reengineer the entire system.
  • The system may use an RDBMS that doesn’t suit the business tasks; for example, an SQL database is not a good choice for a social networks—a graph-oriented database should be used instead.

When talking about a legacy software that requires enhancements or expansions to functionality, the following signs show that it cannot be done without reengineering the entire system:

• Nobody knows how the system works;
• Nobody knows how to improve the code;
• There is a lack of developers in the market who are able to maintain the system;
• The system cannot be easily migrated to new hardware;
• The system performance does not improve even after upgrading hardware;
• The system has poor scalability and its performance can only be improved by upgrading hardware;
• It is difficult and expensive to maintain, improve, and expand functionality;
• The system has experienced a lack of regular maintenance and updates;
• When being developed and deployed into production, the system has experienced a lack of testing;
• The system is vulnerable due to a lack of security patches applied;
• Integration with third-party systems is difficult due to different technologies;
• The entire system is a monolithic functionality with spaghetti code;
• The system code suffers from redundancy (various blocks of code solve the same tasks).

So, What Should be Done to Reengineer a Legacy Software?

It is impossible to specify exact steps of reengineering—they definitely depend on a particular system. However, there are some mandatory actions that can promote success in reengineering your legacy software. They are as follows:

1. Start by analyzing the software architecture, source code, and data structure. Prepare a list of required improvements and ways to implement them.
2. Analyze risks and spendings needed to develop new functionality or a new system and replacing the legacy system, or parts of it.
3. Prioritize the tasks and draw up a plan of action to reengineer the system.
4. Analyze the technologies available and select the best prospect. Take into consideration that the new system should work stably for at least 5–10 years.
5. As the plan is being implemented, it is useful to include tools for metrics and profiling; they enable monitoring of actual performance and stability indicators of the system.
6. Before deploying renewed software, or especially parts of the legacy software, test the system at all levels to avoid failure in production.
7. If you created new software from scratch, take into account that migrating clients to the new system may be even more labor-intensive than migrating data. Thus, your migration plan should be customized for each client.

These are general steps. Obviously they don’t reflect the process in detail; however, it is impossible to write a one-size-fits-all plan because each case differs in terms of existing software, data, and requirements for the new system. Having expertise in reengineering legacy systems becomes a great advantage, especially when analyzing risks and selecting technologies. This is why I always suggest consulting with software vendors that have considerable expertise in reengineering.