Communication between modules: calls, in-process events, the outbox and one schema per module

Introduction#

So far in the series I’ve pulled apart the axes (monorepo, monolith, microservices), drawn boundaries with strategic DDD and made CI enforce them. What’s missing is what happens between them.

In a modular monolith, it’s tempting to think communication is trivial: it’s all one process, so just call the method. But every direct call is a dependency, and every dependency is something you’ll have to undo if you ever extract the module. On the other hand, turning everything into events creates a system nobody can debug.

This post shows both styles, when to use each and how not to lose events, with Spring Modulith in Java and with smarthome-hub’s outbox in Python.

Prerequisites#

Concept#

Call or publish?#

An example from the school platform: when enrolling a student, the enrollment module needs to know whether the class has an open seat. That’s a call: without the answer, it can’t continue. Once the enrollment is confirmed, pedagogical tracking, communication and tuition need to know the enrollment happened. That’s an event: enrollment shouldn’t know about any of them.

The module names in the code are in Portuguese, as in the repository: matricula is enrollment, turmas is classes, pedagogico is pedagogical tracking and mensalidades is tuition.

Notice the direction of the arrows in the dependency graph: matricula → turmas through the call (reservarVaga reserves a seat), and pedagogico → matricula and mensalidades → matricula through the event contract (MatriculaConfirmada, enrollment confirmed). Enrollment doesn’t know its listeners.

The lost event problem#

An in-process event looks simple:

1. Save the enrollment.
2. Publish MatriculaConfirmada.
3. The listeners react.

Now think about what can go wrong. If the process dies between the commit and the delivery, the event is lost. If the listener runs inside the transaction and fails, it takes down the enrollment, which had nothing to do with it. If it runs after the commit and fails, nobody finds out.

The solution is the usual one in distributed systems, and it applies inside a single process too: record the intent to publish in the same transaction as the data, and deliver later, with retries.

Data: one schema per module#

If the tuition module does a JOIN on the enrollment table, the code boundary is useless: the coupling just moved somewhere else. The rule is:

• Each module owns a schema, and only that module reads and writes there.
• No foreign keys across schemas. A reference to another module is just an id.
• If it needs data from another module, it either asks through the facade or keeps its own copy fed by events.

The upside is that extracting a module becomes moving a schema. The cost is that referential integrity between modules becomes your responsibility.

Hands-on#

Java: events with Spring Modulith#

git clone https://github.com/dev-danilocordeiro/monolito-modular
cd monolito-modular/java
git checkout passo-0          # this post's state; main is already at the end of the next post
docker compose up -d postgres

dependencies {
    implementation(platform("org.springframework.modulith:spring-modulith-bom:2.1.1"))
    implementation("org.springframework.modulith:spring-modulith-starter-core")
    implementation("org.springframework.modulith:spring-modulith-starter-jdbc") // registro de publicações
    testImplementation("org.springframework.modulith:spring-modulith-starter-test")
}

The project uses Spring Boot 4.1.1 with Spring Modulith 2.1.1; starter-jdbc provides the publication registry, and the full build.gradle.kts, with JDBC, PostgreSQL and Flyway, is in the repository.

The synchronous call goes through the turmas module’s public facade. Publishing uses Spring’s regular ApplicationEventPublisher, inside the transaction:

package escola.matricula;

import escola.matricula.internal.MatriculaRepository;
import escola.turmas.TurmasService;
import org.springframework.context.ApplicationEventPublisher;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class MatriculaService {

    private final TurmasService turmas;
    private final MatriculaRepository matriculas;
    private final ApplicationEventPublisher eventos;

    MatriculaService(TurmasService turmas, MatriculaRepository matriculas, ApplicationEventPublisher eventos) {
        this.turmas = turmas;
        this.matriculas = matriculas;
        this.eventos = eventos;
    }

    @Transactional
    public Matricula matricular(NovaMatricula pedido) {
        turmas.reservarVaga(pedido.turmaId());                 // chamada: precisa da resposta
        var matricula = matriculas.save(Matricula.confirmada(pedido));
        eventos.publishEvent(new MatriculaConfirmada(          // evento: ninguém espera
            matricula.estudanteId(), pedido.turmaId(), pedido.anoLetivo(), pedido.nomeSocial()));
        return matricula;
    }
}

The two comments mark the split: the call needs the answer, and nobody waits for the event.

package escola.matricula;

import java.util.UUID;

public record MatriculaConfirmada(UUID estudanteId, UUID turmaId, int anoLetivo, String nomeSocial) {}

And the listener, in another module:

package escola.pedagogico;

import escola.matricula.MatriculaConfirmada;
import org.springframework.modulith.events.ApplicationModuleListener;
import org.springframework.stereotype.Component;

@Component
class AoConfirmarMatricula {

    private final Aprendizes aprendizes;

    AoConfirmarMatricula(Aprendizes aprendizes) {
        this.aprendizes = aprendizes;
    }

    @ApplicationModuleListener
    void on(MatriculaConfirmada evento) {
        aprendizes.criarSeNaoExistir(evento.estudanteId(), evento.nomeSocial());
    }
}

@ApplicationModuleListener bundles three things: it runs after the commit, asynchronously and in a new transaction. If pedagogical tracking fails, the enrollment is already saved and isn’t rolled back.

And the event isn’t lost because, with starter-jdbc, Spring Modulith writes each publication to a table (event_publication) in the same transaction as the enrollment, one row per listener. The row is only marked as completed when the listener finishes successfully.

spring.modulith.events.jdbc.schema-initialization.enabled=true
spring.modulith.events.republish-outstanding-events-on-restart=true

If the process dies halfway, incomplete publications are resent when it comes back. To reprocess without restarting, there’s the IncompleteEventPublications interface:

package escola.infra;

import java.time.Duration;
import org.springframework.modulith.events.IncompleteEventPublications;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

@Component
class ReprocessaEventos {

    private final IncompleteEventPublications pendentes;

    ReprocessaEventos(IncompleteEventPublications pendentes) {
        this.pendentes = pendentes;
    }

    @Scheduled(fixedDelay = 60_000)
    void reprocessar() {
        pendentes.resubmitIncompletePublicationsOlderThan(Duration.ofMinutes(5));
    }
}

@Scheduled only fires with @EnableScheduling on some configuration; Spring Boot doesn’t turn scheduling on by itself. Without it, the class starts up and never runs.

Leaving the process without changing the code

When an event needs to go out to another system, Spring Modulith publishes it to the broker without changing the publisher. With the spring-modulith-events-kafka dependency, you just annotate the event:

import org.springframework.modulith.events.Externalized;

@Externalized("escola.matriculas.confirmadas")
public record MatriculaConfirmada(UUID estudanteId, UUID turmaId, int anoLetivo, String nomeSocial) {}

In Spring Boot 4, Kafka support became a separate module, so spring-boot-starter-kafka comes in too. Spring Modulith already configures JSON serialization. In the repository, this annotation lands on the passo-2 (“step 2”) tag, together with the second step of the next post.

Externalization goes through the same publication registry, so it gets the same no-lost-events guarantee. It counts as one more listener: from then on, each MatriculaConfirmada creates two rows in event_publication, one for pedagogical tracking and one for Kafka. That will matter in the next post.

Python: the hand-written outbox in smarthome-hub#

Without a framework doing it for you, the pattern becomes explicit. In smarthome-hub, a command to a device has to be saved to the database and published to the MQTT broker. ADR 0009 describes the dilemma:

• Publish, then commit: if the commit fails, the device acts on a command the hub never recorded (a lock opens with nothing in the audit log).
• Commit, then publish: if the process dies in between, the command is saved and never sent.

The way out is the outbox: the command row and the message row go into the same transaction, and a relay in the worker publishes later.

The heart of the relay is a claim query. Simplified:

SELECT id, topic, qos, payload, headers
FROM outbox.messages
WHERE published_at IS NULL
  AND next_attempt_at <= now()
ORDER BY id
LIMIT 100
FOR UPDATE SKIP LOCKED;

FOR UPDATE SKIP LOCKED lets several workers run side by side without publishing the same row twice: each one takes a batch the others aren’t holding. A row is only marked as published after the broker’s PUBACK, inside the same transaction that claimed it.

Two details from the ADR that apply to any outbox:

• Waking up without aggressive polling. A trigger fires NOTIFY outbox, which Postgres delivers on commit, so the relay publishes within milliseconds. A 1-second poll covers lost notifications.
• Keeping order while the broker is down. If a publish fails, the relay backs off exponentially and stops the batch, instead of skipping to the next row.

Testing#

In Java, Spring Modulith supports per-module tests. By default (BootstrapMode.STANDALONE), @ApplicationModuleTest starts only the module under test; beans from other modules have to be mocks. Since enrollment really calls turmas here, the test asks for DIRECT_DEPENDENCIES, which also starts the direct dependencies. Scenario waits for the event:

@ApplicationModuleTest(mode = BootstrapMode.DIRECT_DEPENDENCIES)
@Import(TestcontainersConfiguration.class)   // PostgreSQL de verdade, nada de H2
class MatriculaTest {

    static final UUID TURMA_COM_VAGA = UUID.randomUUID();

    @Autowired MatriculaService service;
    @Autowired JdbcClient jdbc;

    @BeforeEach
    void abreTurmaComVaga() {
        jdbc.sql("""
                INSERT INTO turmas.turmas (id, ano_letivo, capacidade) VALUES (:id, 2027, 1000)
                ON CONFLICT (id) DO NOTHING
                """)
            .param("id", TURMA_COM_VAGA)
            .update();
    }

    @Test
    void publicaMatriculaConfirmada(Scenario scenario) {
        var pedido = new NovaMatricula(UUID.randomUUID(), TURMA_COM_VAGA, 2027, "Ana");

        scenario.stimulate(() -> service.matricular(pedido))
            .andWaitForEventOfType(MatriculaConfirmada.class)
            .matching(ev -> ev.estudanteId().equals(pedido.estudanteId()))
            .toArrive();
    }
}

The test runs against a real PostgreSQL through Testcontainers, not H2.

./gradlew test

The repository has three more tests worth a look: one showing the seat coming back when the enrollment fails (the reservation and the save are in the same transaction), one checking that each listener’s publication is marked as completed, and the one with simultaneous deliveries to the pedagogical listener.

In smarthome-hub, the outbox integration tests cover the hard cases with a real database and broker (Testcontainers): rollback, ordering, relays side by side without double publishing (the test fails without SKIP LOCKED) and the broker going down with backoff.

make test-it

Pitfalls and trade-offs#

Summary#

Conclusion#

Inside a single process, the temptation is to treat communication as a detail. But the decisions you make here (call or event, shared schema or not, persisted event or not) are exactly the ones that decide whether extracting a module will cost a week or a quarter.

In the last post in the series, I do that extraction: when it’s worth it, when it isn’t, and how to do it step by step without a big bang.

Code on GitHub#