Kotlin

Running integration tests locally with Docker can be challenging when fixed ports are unavailable due to conflicts. This issue is compounded in shared CI environments where multiple workers are in use. However, using testcontainers can help overcome these obstacles by enabling the startup of Docker containers that listen on random ports.

In this post I will show how to build simple data-enriching pipeline using Kotlin coroutines. I will use Channels and Actors abstractions provided by kotlinx-coroutines.

In Actors model “actors” are the universal primitives of concurrent computation. In response to a message that it receives, an actor can: make local decisions, create more actors, send more messages, and determine how to respond to the next message received. Actors may modify their own private state, but can only affect each other through messages (avoiding the need for any locks).

Let’s start with high-level definition of the pipeline:

(👤Producer) -✉️→ 📬(👤Enricher) -✉️→ 📬(👤Updater)
           RawData            RichData

• Pipeline will have Producer Actor, which will get some raw data from database or some mock data and will send it to pipeline for enrichment.
• Then Enricher Actor will handle raw data object and add some attributes to it.
• Finally, Updater Actor will store enriched data to the database.

For the sake of simplicity, let’s implement squaring function: we will take integers as raw data and will enrich them by squaring.

Let’s define our data model: data class RawData(val value: Int)

Enriched data will be represented by data class RichData:

Following Actors model, we will use Kotlin Actors to represent processing units in a pipeline and Channels to communicate with Actors.

In Kotlin actors are implemented as part of kotlinx-coroutines library:

An actor is an entity made up of a combination of a coroutine, the state that is confined and encapsulated into this coroutine, and a channel to communicate with other coroutines. A simple actor can be written as a function, but an actor with a complex state is better suited for a class.

Defining error format is important part of REST API design.

Spring-Boot and Spring Security provide pretty nice error handling for RESTful APIs out of the box. Although it has to be documented, especially when contract-first approach to API design is used.

It is good idea to follow some common format for error responses. But OAuth2 specification and Spring Boot format may not satisfy those requirements.