It was a Tuesday morning in production. An order had just been placed, the database row was committed, but the RabbitMQ message never arrived. The downstream service never processed it. The order sat there, forever in a “pending” state, and nobody knew why.
Sound familiar? If you’ve worked on event-driven microservices long enough, you’ve probably hit this wall. The root cause is what we call the dual-write problem, and today I want to walk you through the pattern that solves it cleanly: the Transactional Outbox Pattern.
The Problem: Two Writes, No Guarantee
In a typical microservice, when something meaningful happens — say, an order is placed — you want to do two things:
- Save the order to your database.
- Publish an
OrderPlacedevent to your message broker.
Seems straightforward. But here’s the thing: these two operations are not atomic. If your app crashes between the DB commit and the broker publish, or if the broker is temporarily unavailable, you end up in an inconsistent state.
// ⚠️ Naïve approach — DO NOT do this in production
public void placeOrder(Order order) {
orderRepository.save(order); // ✅ DB write succeeds
rabbitTemplate.convertAndSend( // 💥 Broker down? Message lost forever
"orders.exchange",
"order.placed",
new OrderPlacedEvent(order.getId())
);
}Here’s what can go wrong:
SCENARIO 1 — App crashes after DB commit, before broker publish:
[DB] ✅ Order saved
[Broker] ❌ Message never sent
→ Order exists in DB, downstream service is unaware
SCENARIO 2 — Broker unavailable at publish time:
[DB] ✅ Order saved
[Broker] ❌ Connection refused
→ Same result: silent failure
SCENARIO 3 — App publishes first, then DB write fails:
[Broker] ✅ Message sent
[DB] ❌ Rollback
→ Ghost event: downstream processes an order that doesn't existThe Solution: Write Once, Publish Reliably
The Transactional Outbox Pattern solves this elegantly. Instead of writing to the broker directly, you write the event to a dedicated outbox table in the same database transaction as your business data. Then, a separate relay process reads from that table and publishes to the broker — and only marks the event as published after receiving a broker acknowledgment.
Here’s the flow:
Step 1 — Atomic write (same DB transaction):
┌─────────────────────────────────┐
│ DB Transaction │
│ │
│ INSERT INTO orders (...) │
│ INSERT INTO outbox_events (...) │
│ │
│ → COMMIT │
└─────────────────────────────────┘
Step 2 — Relay process (separate thread/scheduler):
┌───────────────────────────────────────────────────────┐
│ SELECT * FROM outbox_events WHERE is_published = false│
│ │ │
│ ▼ │
│ rabbitTemplate.convertAndSend(exchange, routingKey) │
│ │ │
│ ▼ │
│ ConfirmCallback ACK received? │
│ │ │
│ ├─ YES → UPDATE is_published = true │
│ └─ NO → Leave it, retry on next run │
└───────────────────────────────────────────────────────┘The magic here is that your business logic and your event record live or die together. If the transaction fails, neither the order nor the outbox entry is saved. And since the relay handles publishing separately, a temporary broker outage just means a short delay — not data loss.
Let’s Build It — An Order Service Example
Let’s make this concrete. We’ll build a simple Order Service using Java 21, Spring Boot, JPA, and RabbitMQ.
1. The Outbox Table
CREATE TABLE outbox_events (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
event_type VARCHAR(100) NOT NULL,
payload TEXT NOT NULL,
exchange VARCHAR(255) NOT NULL,
routing_key VARCHAR(255) NOT NULL,
is_published BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);Simple, flat, and stateless. Each row is an event waiting to be delivered. No foreign keys — the outbox is self-contained.
2. The OutboxEvent Entity
@Entity
@Table(name = "outbox_events")
public class OutboxEvent {
@Id
private UUID id = UUID.randomUUID();
@Column(name = "event_type", nullable = false)
private String eventType;
@Column(nullable = false, columnDefinition = "TEXT")
private String payload;
@Column(nullable = false)
private String exchange;
@Column(name = "routing_key", nullable = false)
private String routingKey;
@Column(name = "is_published", nullable = false)
private boolean published = false;
@Column(name = "created_at")
private LocalDateTime createdAt = LocalDateTime.now();
// Getters and setters omitted for brevity
}3. Atomic Write in the Order Service
This is the key part. Both the order and the outbox entry are saved within the same transaction:
@Service
@RequiredArgsConstructor
public class OrderService {
private final OrderRepository orderRepository;
private final OutboxEventRepository outboxEventRepository;
private final ObjectMapper objectMapper;
@Transactional
public Order placeOrder(CreateOrderRequest request) throws JsonProcessingException {
// 1. Save the business data
Order order = new Order(request.customerId(), request.items());
orderRepository.save(order);
// 2. Record the event in the outbox — same transaction
OutboxEvent event = new OutboxEvent();
event.setEventType("OrderPlaced");
event.setPayload(objectMapper.writeValueAsString(new OrderPlacedEvent(order.getId())));
event.setExchange("orders.exchange");
event.setRoutingKey("order.placed");
outboxEventRepository.save(event);
return order;
// If anything throws here, BOTH writes are rolled back. Clean state guaranteed.
}
}4. The Outbox Relay
A scheduled component periodically picks up unpublished events and forwards them to RabbitMQ:
@Component
@RequiredArgsConstructor
public class OutboxRelay {
private final OutboxEventRepository outboxEventRepository;
private final RabbitTemplate rabbitTemplate;
@Scheduled(fixedDelay = 5000) // runs every 5 seconds
@Transactional
public void relay() {
List<OutboxEvent> pending = outboxEventRepository.findByPublishedFalse();
for (OutboxEvent event : pending) {
CorrelationData correlation = new CorrelationData(event.getId().toString());
rabbitTemplate.convertAndSend(
event.getExchange(),
event.getRoutingKey(),
event.getPayload(),
correlation
);
// Do NOT mark as published here — wait for broker ACK in the confirm callback
}
}
}5. Publisher Confirms — Closing the Loop
This is the piece most tutorials skip. We use publisher confirms (RabbitMQ’s per-message ACK mechanism) to know with certainty that the broker received and accepted the message. Only then do we mark the event as published:
@Component
@RequiredArgsConstructor
@Slf4j
public class OutboxConfirmCallback implements RabbitTemplate.ConfirmCallback {
private final OutboxEventRepository outboxEventRepository;
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
if (correlationData == null) return;
UUID eventId = UUID.fromString(correlationData.getId());
if (ack) {
// Broker confirmed — safe to mark as published
outboxEventRepository.findById(eventId).ifPresent(event -> {
event.setPublished(true);
outboxEventRepository.save(event);
});
} else {
// NACK — log it, the relay will retry on the next scheduled run
log.warn("Broker NACK for event {}: {}", eventId, cause);
}
}
}To wire this up, configure your RabbitTemplate bean and enable publisher confirms in your properties:
@Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory,
OutboxConfirmCallback confirmCallback) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setConfirmCallback(confirmCallback);
return template;
}# application.yml
spring:
rabbitmq:
publisher-confirm-type: correlated
publisher-returns: trueWhat Can Still Go Wrong?
The outbox pattern gives you at-least-once delivery — not exactly-once. Here’s what that means in practice:
┌─────────────────────────────────┬──────────────────────┬────────────────────────────────────┐
│ Failure Scenario │ What Happens? │ Mitigation │
├─────────────────────────────────┼──────────────────────┼────────────────────────────────────┤
│ App crashes before relay runs │ Event stays pending │ Relay picks it up on next run │
│ Broker down during relay │ Send fails, no ACK │ Event stays pending, retried later │
│ App crashes after send, no ACK │ May re-send (dup) │ Make consumers idempotent │
│ DB unavailable for ACK update │ May re-send (dup) │ Idempotency keys on consumer side │
│ Multiple relay instances │ Concurrent publishes │ Use distributed lock (ShedLock) │
└─────────────────────────────────┴──────────────────────┴────────────────────────────────────┘The duplicate message case is the trickiest. The relay may occasionally re-send an event that was already delivered (if the ACK was received by the broker but the DB update failed before completing). This is why your downstream consumers should handle messages idempotently — using the event UUID as a deduplication key is usually enough.
The Full Picture
┌─────────────────────────────────────────────────────────────────┐
│ Order Service │
│ │
│ POST /orders │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────┐ │
│ │ DB Transaction │ │
│ │ INSERT orders ──┐ │ │
│ │ ├─ COMMIT ──────────────────► │ │
│ │ INSERT outbox ──┘ │ │
│ └────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ Relay (every 5s) │ │
│ │ SELECT * FROM outbox WHERE published = false │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ rabbitTemplate.convertAndSend(...) ───────────┼──┐ │
│ │ RabbitMQ │ │ │
│ │ ConfirmCallback: ◄────────────────────┼──┘ ACK │
│ │ ACK → UPDATE published = true │ │
│ │ NACK → retry next run │ │
│ └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘A Few Production Tips
- Use UUID correlation IDs. Generate the outbox row ID upfront and pass it as
CorrelationDatato the relay — this lets you precisely match broker ACKs to database rows. - Add a
created_atcolumn. It makes debugging and alerting straightforward. You can monitor for events older than X minutes that are still unpublished — a clear signal something is stuck. - ShedLock for clustered deployments. If you run multiple app instances, protect the relay with a distributed lock so only one instance runs it at a time. ShedLock integrates cleanly with Spring’s
@Scheduled. - Clean up published events. Add a periodic cleanup job that deletes rows where
is_published = true AND created_at < NOW() - INTERVAL '7 days'. The outbox table is a buffer, not a permanent event log. - Monitor your outbox depth. An alert on
SELECT COUNT(*) FROM outbox_events WHERE is_published = false AND created_at < NOW() - INTERVAL '15 minutes'is your canary in the coal mine.
Wrapping Up
I’ve been using this pattern in a Java 21 / Spring Boot / RabbitMQ / PostgreSQL production system handling real-time financial events, and it’s been rock-solid. The elegance of it is that there’s no magic — just a well-placed database row and a disciplined relay process. No Kafka, no Debezium, no CDC pipelines required (though those are entirely valid alternatives for different scale requirements).
If you’re building event-driven microservices and haven’t adopted the outbox pattern yet, I’d strongly encourage you to give it a shot. Your future self — staring at a production incident at 2am — will thank you.
Have questions or a different approach you’ve used in production? Drop a comment below — I’m always happy to geek out about distributed systems patterns.