The most annoying problems with microservices architecture

A diagram that displays the relations a microservice has with other microservices

Table of Contents

The number one problem with the microservice architecture is to maintain knowledge about all the microservices and integrations between different systems. Colleagues comes and goes. Some people are organized, some are not.

Where and how should we document the different integrations?

Should we document integrations in the source code? How do we describe how integrations are working? How do we know the documentation is up to date? How do we document scheduled integration jobs?

How can we “zoom out” to see all the microservices we have? Where should we put the documentation on them so they are always updated and found?

We’ve been helping hundreds of companies and this problem is true to all organizations. Not just the organizations that has adopted the microservice architecture.

The SaaS market is exploding and what was once a fun tool to try has now become business critical. The company data is scattered around both internal systems and among cloud services.

By listening to our customers we decided to come up with a solution for all major problems that microservices brings. We are using ourselves for all internal integrations within Copyl’s different modules.

A perfect solution for the most annoying problems

This diagram displays the relationships one application in a microservice has with the rest of the services. It’s automatically updated from the API and the cloud Integration Platform in Copyl.

All relations a microservice has with other services are displayed as a diagram in Copyl.
The screenshot above shows the API for the Contract Management App in the Copyl Microservice Management solution.

The API has Triggers and Actions. Actions are automatically generated from the OpenAPI documentation. Triggers are first defined in Copyl and then implemented as small code snippets in the APIs.

All the Triggers are displayed but only the Actions that are used in any Integrations.

So whenever a contract is deleted, it triggers an Integration that starts an Action in the Budget application. The action deletes any budget rows that were associated with the contract.

When a user, contact or company is deleted, the Action that removes counterparts to a contract is started from other integrations. By marking the application as public, other Copyl customers can use it in their integrations.

How to integrate applications

This conceptual diagram shows how environment, microservices/systems, variables, integrations, data watchers, triggers and actions are working together in Copyl.

In short; integrations are made up of two things; Triggers and Actions. A Trigger starts the integration and the Action is what that is being done during the integration job.

Defining your microservices and their apps

The Microservice overview shows the current status of each microservice. The status comes from the logging from the source code. You can select which environment and what time frame you want to view status from.

Click on the microservice to view the log from the source code.

Similar events are grouped together. The Service info displays information about the team, the documentation of the microservice, connected contracts etc.
Event logging details
See the line number where the error was raised and detailed stack info in the message. Comment on the event and create tasks directly from it.

Each Microservice have one or more apps, that is usually represented by an API.

Triggers on an App

The trigger tab on an Integration App in Copyl Integration Platform
In the tab Triggers you find all defined triggers on the app and when they were triggered last time. If the Trigger hasn’t reported any events yet you’ll see that and what Actions that are starting the Trigger.

How to define a Trigger

When you define a new Trigger in Copyl, you’ll decide what data that the Trigger should bring to the integrations. In this case below, the trigger is attaching the newly created Contract so the integration can use all properties from it. You can add as many properties you want.

Trigger details like how many starts it has had per month etc.
The chart is displaying how many times the Trigger has been fired per month.

After you have defined a Trigger you can download the code to implement the trigger in your source code. All the zero:s will have real values for you.
Download automatically generated source code to implement the trigger in your API
You can download the source code for c#, javascript, python and java. Put this code where the trigger is started, e.g., in the delete, insert, update parts of the API.

Integrations overview

This startpage for the Integration Platform lists all integrations, scheduled jobs and variables you can use in the integrations. You can view integration logs and start integrations manually from here.

It’s very easy to deploy and manage all your integrations with any cloud based applications in Copyl Integration Platform.

10 useful integration requirements

Copyl Integration Platform fully supports advanced integration requirements like:

  1. Multi-step integrations that continues according to success/error rules.
  2. Dynamic data used in different steps in an integration.
  3. Scheduled integration jobs and maintenance jobs.
  4. Watch datasets and start a Trigger when data has changed.
  5. Centralized event logging and notifications/alarms.
  6. Support transactions covering multiple services.
  7. Supports HTTP REST with JWT authentication and API keys.
  8. Forwards the user context (On-Behalf-Of).
  9. Automatic defining of Integration Apps from Swagger/OpenAPI documentation.
  10. Multiple environment support so you have different integrations for production, development and testing.

Multi-step integrations with dynamic data

If you have a multistep integration you sometimes want to use data returned from the one step in another step. Copyl supports that by having tags like this; {{step2.out.contractId}} in the parameters to any action in any step of the integration. You can also use incoming parameters like this; {{}} in any of the integration steps.

Scheduled jobs that can be started again and again

With all the apps and their defined Actions we can easially setup scheduled jobs that does some management of data. It can move data from microservices to data warehouses. Scheduled jobs can be manually started if something needs to run again.

Watch for data changes and start a Trigger when data has changed

If you have legacy systems that doesn’t report events you can simulate that by monitoring the data. We call this function a Data Watcher and it checks for changes on a regular basis.

Example of an Data Watcher in Copyl Integration Platform
This Data Watcher starts the “getWeatherForNY” action on the app OpenWeatherAPI. If data has changed since last check, the trigger is fired together with the new data. If that trigger is used in any integrations, the integrations will start. We call it a hybrid integration were API and data are integrated.
You can also view the last data the data watcher is comparing with.

Centralized logging for all microservices

Another pain with microservice architecture is the debugging when something goes wrong. By collecting all service’s logging you can see what’s happning inside each app. You will come to better conclusions when you have the full picture. Use Open Telemetry in your logging to add extra information about the infrastructure your microservices are running on.

Automatic defining of all the Actions an App has

By letting Copyl read your OpenAPI/Swagger file for the API, you will get all methods defined automatically. All the documentation and data models are also imported.

Information Models are automatically updated

When Copyl reads the OpenAPI definition, it saves all classes separatly. We can view and update them as Information Models in Copyl’s Microservice Management solution. You can also use the code generator tool to get the source code for the classes.

The picture above shows how to keep track of classes / tables that contains PII (Personally Identifiable Information). You can group classes how you like, grouping them by team etc.

We are all obligated to stay up to date with the new regulations concerning privacy. 

Copyl Integration Platform to the rescue

Please contact us if you have problems overviewing how different services/systems are integrated.

Copyl Microservice Management is an integration solution offered as a software as a service (SaaS). It’s tightly integrated with the Copyl Microservice Management software. The integration solution helps you connect applications and synchronize data from a wide variety of data sources and pre built connectors.

Copyls Integration Platform as a service (iPaas) is perfect for api integration between different saas applications. It runs in the cloud but could also be installed on-prem if needed.

The platform can handle millions of requests per second due to the Event-Driven Architecture.

We also help companies with onboarding and implementation of Copyl.

How much does Copyl Microservice Management cost?

The microservice management product has different levels depending on functionality, number of events, storage time and SLA. We recommend you start with the forever-free version. It has some limitations on the number of integrations, storage time and some features are not included. But you will understand how Copyl can make a really positive change for your business.

How do I get started?

The onboarding process is still manual when it comes to the Microservice Management software. We want to learn as much as possible from our customers. The process usually looks like this:

  1. You send an email to [email protected] telling us you want a demo of the Microservice Management service. Please suggest some time slots for a 20 minute meeting.
  2. We meet in a digital demo where we will answer any questions you have.
  3. After the demo you decide if you want to set up a 14-days free trial. We will guide you on how to setup your environments, microservices, APIs and define triggers. We will help you set up your first integrations.

This process will help you understand the full potential of having access to an advanced integration platform with microservice management capabilities.


Do You Want To Boost Your Business?

Drop us a line and keep in touch

Copyl CRM for service companies