Event sourcing in .NET: what happens when you stop overwriting your data

"Who changed this price, and when?" is a question that, in a system built the normal way, has no answer. Not because the old value was lost by accident: because it was covered on purpose. The UPDATE statement wrote one hundred and twenty where one hundred and thirty used to be, and one hundred and thirty no longer exists anywhere. There is a way of writing data in which that question always has an answer, and it is not keeping a separate log next to it: it is called event sourcing.

This article is not another watered down translation of Martin Fowler's page. You get the database schema, C# code that compiles, the arithmetic of how much an event store actually weighs, and the three cases in which I would not use this pattern. Above all you get a real system: the content factory of the application I am using right now to write this article is built exactly this way, and the editorial plan of the blog you are reading has been running on it since August.

I have been building software for twenty six years, almost always on systems where the data matters more than the code: business systems, industrial software, platforms somebody has to defend in front of a client or an auditor. I have seen event sourcing solve problems nothing else could solve, and I have seen it make life harder for teams that did not need it. So both sides are here, and the second one is the part nobody writes because writing it requires having put the pattern into production.

One clarification straight away, because half the arguments on this topic start there: event sourcing is not an audit log. A log is added to the system and can drift away from it, because it is one extra write somebody can forget. Here the events are the system: there is no other place where the data lives, and therefore no way for the two to disagree.

What event sourcing is: you store the facts, not the state they produced

Your bank does not store your balance. It stores the transactions. The balance you see in online banking is the sum of those transactions, computed at the moment you open the page.

That sounds like an accounting detail and it is in fact the complete definition of event sourcing. If you store the facts instead of the state, you can always derive the state whenever you need it. And because all the facts remain, you do not only derive the current state: you derive every intermediate one too, without having done anything special to preserve them.

Try turning the question around. Why does a bank not simply store a number and update it? Because the number on its own is indefensible. If a customer balance goes from one thousand to seven hundred and the only thing the system knows is seven hundred, there is no way to say whether three hundred was withdrawn, whether somebody mistyped, or whether a calculation went wrong. With transactions, every figure has a line that explains it and a signature that attributes it.

Event sourcing takes that idea, which accounting has had for five hundred years, and applies it to any single part of a software system. The price of a product. The status of a case. The configuration of a plant. The card of a blog article.

What actually changes in the database

In a normal system a change is a rewritten row. The product price is one hundred and twenty, and that is the whole of the available truth. If somebody asks what it was in March, the honest answer is that nobody knows.

In an event sourced system that same change is an added row. The products table, in the sense you know it, is no longer written by hand by anybody: there is an events table holding, in order, "created at one hundred", "raised to one hundred and thirty", "corrected to one hundred and twenty", each with who did it, when, and if you ask for it, why. The current price is what comes out after reading all three.

Everything else follows from that tiny difference: the full history without history tables, the comparison between any two moments without copies, concurrency control almost for free, and the ability to answer questions nobody anticipated when the system was written. Because the facts remain, and on facts that remain you can ask new questions.

The three words you need, and no more

The vocabulary around this pattern is full of terms that frighten without adding anything. Three are enough.

An event is a fact that already happened, written in the past tense: "price changed", not "change price". The difference is not stylistic. A fact that happened is not up for debate and cannot be undone: if you got it wrong, you add the fact of the correction, and both remain. It is exactly what an accountant does with a credit note.

An aggregate is the thing the event talks about: that product, that case, that card. It is the unit inside which the order of events matters and outside which it does not. The term comes from domain driven design, and it is the only borrowing you need from there.

A projection is a read table built by replaying the events. It exists so you can run normal queries without replaying history every time. It is a convenience, not the truth: you can throw it away and rebuild it, and it is the part most people only really understand the first time they do throw it away and notice that nothing happened.

Event sourcing and CQRS are two different things and can be used apart

They are named together so often that many people treat them as a single package. They are not, and confusing them is the fastest route to a wrong decision about both.

CQRS says one thing only: the operations that write and the ones that read may use different models. It says nothing about how data is stored. You can do CQRS with a perfectly ordinary relational database updated with UPDATE statements, and in fact most people doing CQRS do exactly that.

Event sourcing says one other thing only: state is not stored, facts are. It says nothing about how you organise reads. You can rebuild the state and serve it from the very same model you write with, no separation involved.

That said, they are often found together for a practical reason: if you only store events, sooner or later you need a table to run normal queries against, and that table is effectively a separate read model. In other words you arrive at CQRS starting from event sourcing, not the other way round. The direction matters, because it explains why most projects that "do CQRS" need no event sourcing at all, while almost everyone doing event sourcing ends up with two models.

And it is not event driven architecture either, nor the outbox pattern

It is worth closing the other two overlaps as well, because in meetings they get mixed up constantly and lead to wrong decisions in both directions.

Event driven architecture is about how different parts of a system talk to each other: one component announces that something happened and others react, without calling each other directly. It is a question of communication between services. Event sourcing is a question of how a single component keeps its own data internally. You can have either without the other: a monolith storing everything as events and sending nobody a message is pure event sourcing with no event driven architecture in it; a dozen services exchanging messages on a queue while each one runs its own UPDATE statements is the exact opposite.

The confusion comes from the word "event", which means different things in the two contexts. In event sourcing an event is data stored forever, whose shape is your own business and which nobody outside sees. In event driven architecture it is a message published outwards, whose shape becomes a contract with other teams and can therefore no longer be changed at will. Confusing the two leads to the most expensive mistake of all: publishing your internal events outwards, and finding your own database structure frozen because other systems now depend on it. If your system does both, keep two separate families of events, even at the cost of writing a piece of code that translates between them.

The outbox pattern solves a different problem again: how to send a message out and store data in, without risking one happening and the other not. You write the message into a table in the same transaction as the data, and a separate process ships it. It is a reliable delivery pattern, and if you already have event sourcing that table is often unnecessary, because the event store itself can act as the queue for the shipping process. But the two are chosen for different reasons: the outbox when the problem is delivery, event sourcing when the problem is memory.

Event sourcing in C#: the event store, the reducer and the projection

There are three parts, and in a system built properly they fit into a few hundred lines. What follows is C# that compiles, cut down to the essential fields so it fits in an article: the real system has a few more columns and error handling, not more concepts.

1. The store of facts

A single table, insert only. No UPDATE, no DELETE: not out of discipline, but because no use case requires them.

CREATE TABLE Events (
    Id          UNIQUEIDENTIFIER NOT NULL PRIMARY KEY,
    Aggregate   VARCHAR(100)     NOT NULL,   -- what the fact is about
    Number      INT              NOT NULL,   -- 1, 2, 3... within the aggregate
    Type        VARCHAR(100)     NOT NULL,   -- article.price.changed
    Payload     NVARCHAR(MAX)    NOT NULL,   -- the fact itself, as JSON
    Author      NVARCHAR(200)    NULL,       -- who
    Reason      NVARCHAR(500)    NULL,       -- why, when stated
    WhenUtc     DATETIME2        NOT NULL,
    CONSTRAINT UQ_Events_Aggregate_Number UNIQUE (Aggregate, Number)
);

CREATE INDEX IX_Events_Aggregate_Number ON Events (Aggregate, Number);

Two lines of this schema are worth more than all the others, and they are the last two.

The unique constraint on aggregate plus number is what stops two concurrent changes from overwriting each other. I come back to it shortly, because it is the piece most articles on this subject do not mention at all.

The index on the same two columns is why rebuilding a state costs milliseconds even when the table holds millions of rows. The read you always perform is "give me the events of this aggregate in order of number", and with that index the database touches nothing you do not need.

The Reason field looks like a nice to have and is not. It is where whoever makes a change writes down why, and in a system where decisions matter it is worth more than half the rest: six months later the real question is almost never "what changed", it is "why".

2. The reducer

A function that takes a state and an event and returns the next state. Nothing else: no database access, no service calls, no clock. It must be pure, and the reason is practical: the same function has to be used both when writing and when rebuilding history.

public sealed record Event(
    Guid Id, string Aggregate, int Number, string Type,
    string Payload, string? Author, string? Reason, DateTime WhenUtc);

public sealed record ArticleState(
    string Aggregate, string Title, decimal Price,
    string Status, int Version, DateTime UpdatedUtc)
{
    public static ArticleState Empty(string aggregate) =>
        new(aggregate, "", 0m, "none", 0, default);
}

public static class Reducer
{
    public static ArticleState Apply(ArticleState state, Event evt)
    {
        ArticleState next;

        switch (evt.Type)
        {
            case "article.created":
                var created = Read<Created>(evt);
                next = state with { Title = created.Title, Price = created.Price, Status = "draft" };
                break;

            case "article.price.changed":
                next = state with { Price = Read<PriceChanged>(evt).After };
                break;

            case "article.published":
                next = state with { Status = "published" };
                break;

            default:
                next = state;
                break;
        }

        return next with { Version = evt.Number, UpdatedUtc = evt.WhenUtc };
    }

    private static T Read<T>(Event evt) =>
        JsonSerializer.Deserialize<T>(evt.Payload)
        ?? throw new InvalidOperationException($"Event {evt.Number} cannot be read.");
}

Note the default branch: a state that does not recognise an event type ignores it rather than blowing up. You need that when you add new types later and old code still has to be able to replay history.

Note the last line too: the version of the state is the number of the last event applied. There is no separate counter to keep in sync, and that removes an entire class of bugs at the source.

The single most common mistake is having two functions: one that updates the table on write and one that rebuilds when reading history. The day they diverge, and they do, the system starts telling two different stories about the same value, and working out which of the two is wrong costs more than rewriting the lot. One function, used in both places: it is the most important rule in this article.

3. Writing and rebuilding

public async Task<int> Append(
    string aggregate, string type, object payload,
    string? author, string? reason, int? expectedVersion = null)
{
    var current = await db.Events
        .Where(e => e.Aggregate == aggregate)
        .MaxAsync(e => (int?)e.Number) ?? 0;

    if (expectedVersion is not null && expectedVersion != current)
        throw new VersionConflict(aggregate, current, expectedVersion.Value);

    var evt = new Event(
        Guid.NewGuid(), aggregate, current + 1, type,
        JsonSerializer.Serialize(payload), author, reason, DateTime.UtcNow);

    db.Events.Add(evt);
    await db.SaveChangesAsync();
    return evt.Number;
}

public async Task<ArticleState?> Rebuild(string aggregate, int? upTo = null)
{
    var events = await db.Events
        .Where(e => e.Aggregate == aggregate && (upTo == null || e.Number <= upTo))
        .OrderBy(e => e.Number)
        .ToListAsync();

    if (events.Count == 0) return null;

    var state = ArticleState.Empty(aggregate);
    foreach (var evt in events) state = Reducer.Apply(state, evt);
    return state;
}

The upTo parameter of Rebuild is four characters of code and it is the reason the next section of this article exists. Without it you get the current state. With it you get the state the aggregate had at that version. There is nothing else to write to get time travel: it is a consequence of the structure, not an added feature.

Concurrency control, in three lines

This is the point that justifies the change of approach on its own, and hardly anybody writes about it.

In a normal system, if Anna and Luca open the same record and save it thirty seconds apart, Anna's work disappears. No error, no warning: Luca's UPDATE wins and nobody will ever know there was anything else. It is the bug reported in business systems as "sometimes changes get lost" and never successfully reproduced.

Here, whoever writes declares which version they started from. If another one has arrived in the meantime, Append stops and returns a comprehensible error instead of erasing somebody else's work. And if the two writes arrive in the same instant, close enough that the in memory check lets both through, the unique constraint in the database acts as the final judge: two rows with the same aggregate and the same number cannot coexist, so the second one is rejected by the engine itself.

That is one table constraint and one parameter. In a traditional system the same guarantee needs a version column on every table, its handling in every UPDATE, and the discipline never to forget it. Here it is structural: you cannot forget it, because without the sequence number the pattern does not work at all.

4. The projection

After every write you rewrite one row in an ordinary table, with the columns your day to day queries need. It is the only part of the system that looks like what you were doing before, and indeed you keep running SELECT statements, sorts, filters and list pages against it exactly as you always have.

There is one rule to keep in mind: the projection is not the truth. If you lose it, you rebuild it by replaying the events. If you change your mind about the columns, you throw it away and rebuild it with the new ones, and the historical data is all there because you never threw that away. The system I describe in a moment has a command that rebuilds every projection from scratch, and it serves two purposes: structural migrations, and proving that the store of facts really is the source rather than a courtesy copy.

The benefit nobody talks about: comparing two versions comes for free

People presenting event sourcing talk about auditability, history and compliance. Those are true and they are the boring part. The gain that genuinely changes how you work is a different one, and it only becomes visible after production.

Because every past state can be rebuilt, all the functions that compare two moments do not need writing: they already exist.

Take a request that turns up in every project sooner or later: "I would like to see what changed in this record between March and today". In a normal system that is a project. You need a versions table, you need to decide when to take a snapshot, you need to manage the space those snapshots take, you need to write the field by field comparison, and you need to discover six months later that snapshots were taken on some changes and not on others.

Here the same request is: rebuild twice, stopping at two different points, and look at where the two pictures fail to match.

public async Task<IReadOnlyList<Difference>> Compare(string aggregate, int from, int to)
{
    var before = await Rebuild(aggregate, from);
    var after = await Rebuild(aggregate, to)
        ?? throw new InvalidOperationException($"Aggregate {aggregate} does not exist.");

    var differences = new List<Difference>();

    foreach (var field in typeof(ArticleState).GetProperties())
    {
        var valueBefore = before is null ? null : field.GetValue(before);
        var valueAfter = field.GetValue(after);
        if (Equals(valueBefore, valueAfter)) continue;
        differences.Add(new Difference(field.Name, valueBefore, valueAfter));
    }

    return differences;
}

Twenty lines, and there is not a single line of versioning code anywhere else in the system. No history table, no copies, no decision about when to take the picture. The comparison works between any two versions, including ones nobody expected to need when the code was written.

The real case: the content factory behind this blog

This article exists because a card in an application said it should be written. I use that application to decide what to write about on the blog: it holds the searches with their volume, the cost per click, the site's current position, the angle to use and the writing prompt. It is called the content factory and it lives inside the SEO monitor I built for my own site.

It is built exactly as described above. There is a forge_events table with the same columns as the schema above and the same unique constraint on aggregate plus number. There are two projections, one for the card and one for the languages, both rebuildable. There is a reducer that is a single function, used both when saving and when rebuilding. And there is the comparison written above.

The practical result, after some months of real use, is this. The card for this article reached its seventh version: it was born from research done on Search Console and Keyword Planner data, then its angle, section titles, supporting evidence, images and writing prompt were all changed. Every step has its author, which is sometimes a person and sometimes a model, its date and its stated reason. I can ask the application what changed between version two and version seven and get the list of fields with the before and the after.

I wrote no code to obtain this. It came from the structure. And here is the point worth taking away: the value of event sourcing is not measured on the features you planned, it is measured on the ones you did not. The day I wanted to know which cards had been changed by a model and which by me, the answer was already in the data, because the column recording in what capacity the author acted had been there since day one and nobody had ever overwritten it.

The next level: replaying history with changed logic

There is a consequence of all this that only becomes visible after a while, and it is the most powerful one. Because the state is not stored but computed, if you change the reducer you change the past as well.

Put like that it sounds like a threat, and it does need care. But it is the thing normal systems simply cannot do. Take a calculation rule: agent commissions, the priority score of a case, the classification of a customer into a band. In a traditional system that number was computed at the time and stored, and if a year later management asks "with the new rule, how would the last two years have gone?", the answer is that you would need the raw inputs from back then. Which are not there, because the result was stored instead.

With the facts preserved, that question is an execution: replay history with the new reducer and you get the numbers that would have come out. It is not an approximate simulation over aggregated data, it is the exact values computed on the very same inputs. I have watched arguments about commercial rule changes drag on for weeks for the single reason that nobody could say what the new rule would have cost.

There is one caution, and it belongs in the repository: the state rebuilt with today's reducer is not the number the system showed back then. They are two different things and in some contexts the difference is delicate. When you need to preserve the value shown at the time as well, for instance on a tax document that was issued, that value goes inside the event as a fact of its own: "invoice issued with taxable amount X". That way you have both, the number that was and the number that would be, and the distinction is explicit rather than hidden.

Event sourcing and the database: where events go and how much they weigh

This is the question that stops most projects before they start, and almost always because of a wrong answer: "you need a specialised database".

You do not. SQL Server is fine. PostgreSQL is fine. SQLite is fine, and I say so from experience, because the factory above runs on Cloudflare D1, which is SQLite. Databases built specifically for events do exist and they earn their place when volumes get serious or when you need live subscriptions downstream, but their absence is not a valid excuse for not starting. The table above, with its index, holds far more than most projects will ever see.

The numbers, with their conditions

Let us put figures on it, because that is the part you have to defend in a meeting.

A typical event, payload included, takes between 200 bytes and 2 KB. The range depends almost entirely on how large the piece of data the event carries is: a status change is a few dozen bytes, an event carrying a long text can reach a few KB. As an order of magnitude, a million events of 1 KB is a gigabyte, a quantity any database on ordinary hardware handles without noticing.

Rebuilding an aggregate with fifty events costs a few milliseconds if the index is there. It is dominated by the sequential index read, not by computation: the reducer is an in memory function and over fifty iterations it does not register.

The number that matters for decisions is events per aggregate, not the total in the table. A store with ten million events spread over a hundred thousand aggregates behaves beautifully, because each read touches a hundred. A store with a hundred thousand events over ten aggregates is a problem, because each read touches ten thousand.

Snapshots, and when to stop postponing them

A snapshot is a row saying "at version 500 the state was this", stored now and then, so you do not always have to start from the first event.

The right question is not whether to have them: it is when. And the answer is later than most people think. Below a few hundred events per aggregate they are unnecessary, and implementing them earlier means adding a part to maintain and another thing that can drift, to solve a problem you do not have. The practical threshold is simple: when rebuilding a single aggregate starts showing up in your response times, and not before.

When you do add them, the one thing to keep in mind is that a snapshot is a copy and can therefore be wrong. Mark it with the version it refers to and with the version of the reducer that produced it, so a change in the logic invalidates it rather than turning it into a silent lie. Which is why the "rebuild everything from scratch" command should be written on day one: it is the safety net that makes snapshots an optimisation rather than a risk.

Changing the shape of events

One day you will want to add a field to an event type, and the store will hold ten thousand old events without it. Events already written are not touched: that is the rule, and it has no exceptions.

There are two ways out and both are simple. First: the new field is optional and the reducer, when it does not find it, uses a sensible default. That covers the vast majority of cases. Second: when the change is so deep that the old event is no longer interpretable, you introduce a new type, say article.price.changed.v2, and the reducer handles both. That is a couple of extra lines in the switch and they stay there forever, which is the honest price of not having rewritten history.

If keeping code that handles old shapes feels distasteful, consider the alternative: systems where old data has been "migrated" and nobody can say with certainty what it said before the migration. Those have a cost too, it is just paid later and not by whoever made the decision.

The three cases where event sourcing is the wrong choice

This is the part missing from almost every article on the subject, and it is missing for a reason: you only write it after paying the bill at least once.

1. Reference data nobody will ever ask the history of

The table of provinces. The list of VAT rates. A supplier's contact details. A list of codes to which a row gets added now and then.

On this kind of data event sourcing is complexity paid for nothing. Nobody will ever ask who changed a postcode and why, and if one day somebody did, an ordinary change log would answer. Use an UPDATE.

The acid test is a single question, to be asked before writing any code: will anybody, one day, need to know what it was before? If the honest answer is no, you are done, and the honest answer is no far more often than an enthusiastic architect is willing to admit. In a healthy system the event sourced aggregates are few: two, three, five. If you have forty, you almost certainly applied the pattern to everything instead of to what deserved it.

2. The team has never seen it and nobody owns it

This is the most frequent cause of failure, and it is not a technical problem.

Event sourcing done badly is distinctly worse than CRUD done well. The things that go wrong are always the same ones: the reducer that becomes two different functions; somebody adding an UPDATE "just to fix a wrong value", after which the store is no longer the source; events written in the future tense, that is, commands dressed up as facts; the projection treated as the truth and therefore never rebuilt, until it turns out it has drifted and nobody knows since when.

None of these are beginner mistakes: they are all reasonable errors made by good developers who have never worked with this pattern. You need one person who owns it and a rule written in three lines in the repository, and if neither exists, the right answer is to wait. In the meantime, a sensibly layered architecture delivers ninety per cent of the value at a tenth of the risk.

3. There is an obligation to delete personal data

Here the conflict is real and deserves to be faced rather than waved away with "we will sort that out later".

GDPR grants the right to have personal data erased. Event sourcing rests on a store from which nothing is removed. If the events contain names, email addresses or other data attributable to a person, an erasure request has no comfortable answer: deleting events breaks the rebuild of everything that follows, and modifying them breaks the premise the system stands on.

The technique that works is called crypto shredding. Personal data inside the event is not written in the clear: it is encrypted with a different key per person, held in a separate table. When the erasure request arrives, the key is deleted. All the events stay where they are, the sequence numbers have no gaps, the rebuild keeps working, and the personal fields become permanently unreadable to everybody, including whoever wrote the system.

It is a sound solution and it must be said that it adds work: key management, encryption and decryption in the read path, classifying which fields are personal, and the fact that a state rebuilt after erasure will have holes downstream code must handle. That is an acceptable cost in a system where event sourcing genuinely earns its place. It is an absurd cost in a system that ended up there because of fashion.

The rule of thumb I use: personal data stays out of the events whenever possible. The event carries the person's identifier, their data lives in an ordinary table under ordinary rules, and the problem never arises.

Where to start without rewriting anything: one aggregate, the one that hurts most

If you got this far you probably have in mind a system that is already running, with years of data in it and people working on it every day. The good news is you do not have to convert it. In fact you should not even try.

Event sourcing coexists happily with the rest of the system, because it is a choice made per aggregate rather than per application. You can have one events table for prices and a hundred and twenty ordinary tables for everything else, and the two never notice each other. There is no big migration to plan, no database change, no rewrite.

How to choose the first one

There is one criterion and it is not technical: the entity people most often ask "who changed what, and when" about.

If you work on a business system it is almost always the price or the commercial terms. On a case management platform it is the case status. On a configurator it is the configuration. On an industrial system it is the recipe or plant parameter, where incidentally the sequence of events is already the primary data and the move is more natural than elsewhere. If you cannot name that entity within ten seconds, take it as a signal: perhaps this pattern is not for you, and that is perfectly fine.

The second criterion, subordinate to the first, is that the entity should be small. Few fields, few transitions. Do not take the order with its forty columns and its lines: take the order status. The first aggregate is there to learn on, and you learn better on something that fits on one screen.

The first three things to write, in this order

One. The events table with its unique constraint and its index, and the function that appends an event checking the expected version. That is the two pages of code from the previous section and they will not change again.

Two. The reducer, with the three or four event types you genuinely need. Write it as a pure function and put tests on it that never touch the database: given a state and a list of events, you expect this state. They will be the fastest and most stable tests in the whole project, because there is nothing to mock.

Three. The projection and the command that rebuilds it from scratch. Write the command straight away, even if you do not need it: it is what will let you change your mind about the columns without fear, and it is the proof, to you and to whoever looks at the system later, that the store of facts is the source.

What should not be written on day one: snapshots, a message bus, event subscriptions, a specialised database. They are all things you add when a number demands them, and before that they are just surface to maintain.

The point, in short

Event sourcing is not a better way of doing things: it is a different way, worth it on a small part of a system and not worth it on all the rest. You store the facts instead of the state, and derive the state by replaying them. There are three parts, they fit into a few hundred lines, and they run on any ordinary relational database.

What you get is not primarily compliance or the comfort of not losing data, although you get both. It is that an entire category of future requests stops being a project and becomes a query: what changed between two moments, who changed it, for what stated reason, and how things would have gone had we stopped earlier. Those are questions normal systems face one at a time, expensively, whenever somebody asks.

What you pay is discipline: one single function to apply events, never an UPDATE, the projection treated as a convenience rather than the truth, and personal data kept out of the events or encrypted with a key you can throw away. If nobody on the team owns those four rules, wait: CRUD done well beats event sourcing done badly, and that is not a joke.

And if you had to keep one sentence from the whole article, this one: event sourcing is not about not losing data. It is about not losing decisions.