{ "version": "https://jsonfeed.org/version/1.1", "title": "ByteAether", "language": "en", "home_page_url": "https://byteaether.github.io/", "feed_url": "https://byteaether.github.io/feed/feed.json", "description": "Decoding complexity to craft beautiful simplicity.", "authors": [ { "name": "Joonatan Uusväli" } ], "items": [ { "id": "https://byteaether.github.io/2026/byteaetherulid-v142-expands-the-net-ecosystem-with-aspnet-core-and-cli-support/", "url": "https://byteaether.github.io/2026/byteaetherulid-v142-expands-the-net-ecosystem-with-aspnet-core-and-cli-support/", "title": "ByteAether.Ulid v1.4.2 Expands the .NET Ecosystem with ASP.NET Core and CLI Support", "content_html": "

A production-grade identifier library requires tight integration across data persistence layers, API boundary definitions, and operational tooling. ByteAether.Ulid implements a high-performance, specification-compliant .NET type for the ULID standard, engineering strict monotonicity, throughput optimization, and deterministic ordering under high-concurrency workloads.

The ByteAether.Ulid v1.4.2 release extends the ecosystem beyond the core primitive. Alongside existing data-access integrations, the repository (available on GitHub) now provides two specialized packages targeting modern .NET architectures:

These abstractions eliminate the need to maintain bespoke serialization layers or conversion adapters on top of the primitive Ulid struct.

Architectural Overview: Moving Beyond the Core Type

The ByteAether.Ulid v1.4.0 release established native ORM and micro-ORM integrations across EF Core, LinqToDB, and Dapper. While this solved low-level database mapping challenges, application boundaries remained unoptimized.

Modern enterprise architectures expose two distinct integration boundaries:

  1. HTTP/RPC API Edge: REST endpoints, Minimal APIs, and MVC controllers demand fast route constraint evaluation, efficient model binding, zero-allocation JSON parsing, and schema generation for OpenAPI definitions.

  2. Operational Tooling: DevOps pipelines, database migration scripts, and debugging sessions require CLI utilities to parse timestamps, extract randomness components, and convert byte representations without writing scratch C# applications.

ByteAether.Ulid.AspNetCore and ByteAether.Ulid.Cli address these requirements while preserving the foundational guarantees of the core ByteAether.Ulid repository: zero-allocation parsing, high throughput, and monotonicity under bursty thread scheduling.

Source-First ASP.NET Core Integration

\"ByteAether.Ulid.AspNetCore

ByteAether.Ulid.AspNetCore introduces native framework bindings for ASP.NET Core pipelines:

The following implementation demonstrates Minimal API endpoint integration with custom route validation:

using ByteAether.Ulid;\nusing ByteAether.Ulid.AspNetCore;\n\nvar builder = WebApplication.CreateBuilder(args);\nbuilder.Services.AddUlidRouteConstraint();\n\nvar app = builder.Build();\napp.MapGet(\"/users/{id:ulid}\", (Ulid id) => Results.Ok(new UserResponse(id)));\napp.Run();\n\npublic record UserResponse(Ulid Id);

By handling parsing and model binding at the routing boundary, the framework prevents malformed identifier requests from executing pipeline middleware or reaching application handlers.

For API spec generation, registering the schema transformer ensures client code generators emit strongly typed identifier fields instead of generic string definitions.

Operational Tooling: CLI Generation and Inspection

\"ByteAether.Ulid.Cli

ByteAether.Ulid.Cli provides shell-level operations for inspecting and generating ULIDs.

Install the utility via the .NET SDK tool registry:

dotnet tool install -g ByteAether.Ulid.Cli

Command Usage and Scripting Capabilities

Generate a single string identifier:

ulid

Generate sequential identifiers in batches:

ulid --count 5

Parse and inspect internal structure (extracting Unix timestamp and high-precision randomness payload):

ulid 01AN4Z07BY79KA1307SR9X4MV3

Process standard input streams for log analysis or data transformation pipelines:

cat ulids.txt | ulid --json

Utility Features

System Impact and Adoption

The expanded ByteAether.Ulid ecosystem establishes a uniform model for identifier handling across the runtime lifecycle:

  1. Performance Overhead: The core library remains a memory-efficient, zero-allocation struct implementation.

  2. Persistence Integrity: Dedicated ORM providers maintain raw byte-ordering efficiency at the database index layer.

  3. API Edge Precision: ASP.NET Core extensions validate and serialize inputs without custom application layer code.

  4. Developer Experience: Shell utilities standardise operational introspection and test vector generation.

Package Installation

Add the API dependencies or install the global shell utility:

dotnet add package ByteAether.Ulid.AspNetCore\ndotnet tool install -g ByteAether.Ulid.Cli

Reference material, core source files, and benchmark suites are accessible in the official ByteAether.Ulid GitHub repository. Nuget packages are hosted on the public package index:

", "date_published": "2026-10-14T00:00:00Z" } , { "id": "https://byteaether.github.io/2026/streamlining-net-data-layers-with-byteaetherulid-v140/", "url": "https://byteaether.github.io/2026/streamlining-net-data-layers-with-byteaetherulid-v140/", "title": "Streamlining .NET Data Layers with ByteAether.Ulid v1.4.0", "content_html": "

The choice of data identifiers dictates database performance, indexing efficiency, and security vectors at scale. For high-volume distributed systems, legacy options present clear operational drawbacks. Standard sequential integer identifiers expose internal operational velocity and collapse under cross-region replication due to centralized state bottlenecks. Traditional UUIDv4 variants eliminate the synchronization bottleneck but introduce severe index fragmentation within B-tree structures, degrading disk I/O throughput as tables scale.

While modern alternatives like UUIDv7 attempt to combine temporal sortability with randomness, they introduce distinct operational compromises. The RFC 9562 standard leaves sub-millisecond monotonicity optional. The native .NET Guid provider sacrifices strict ordering to prioritize generation speed. This approach reintroduces out-of-order database writes and causes index page splits under heavy execution loops.

The Universally Unique Lexicographically Sortable Identifier (ULID) specification addresses these structural flaws by enforcing strict, deterministic monotonicity alongside lexicographical sortability. Earlier in our engineering roadmap, we introduced ByteAether.Ulid to provide a zero-allocation, lock-free, and fully specification-compliant ULID engine for .NET.

However, an identifier primitive is only as valuable as its connection to the data access layer. Previously, integrating the ByteAether.Ulid primitive into object-relational mappers (ORMs) and micro-ORMs required substantial boilerplate, custom type handlers, and specialized value converters. Developers were forced to manually map internal structs to database engines, introducing risks such as misconfigured mappings, broken index optimizations, or mismatched endianness formats.

With the release of ByteAether.Ulid v1.4.0, we are moving past raw primitive mechanics. In addition to core performance enhancements, this version introduces a suite of official companion packages: ByteAether.Ulid.EntityFrameworkCore, ByteAether.Ulid.linq2db, and ByteAether.Ulid.Dapper. These extensions remove manual plumbing, allowing senior engineers and software architects to deploy high-performance ULIDs across the entire data access lifecycle with minimal configuration.

A New Visual Identity for Our Libraries

Along with this ecosystem expansion, we are introducing a subtle rebranding for our package logos. Previously, we used the identical blue, black, and white ByteAether identity across all NuGet packages, making it difficult to visually distinguish different libraries at a glance in long package lists.

While the ByteAether retains the original classic identity, we have updated our library logos to incorporate a distinct color-coding system. Each individual library we release now features a unique color assigned to the bottom-right quadrant of our four-segment logo. For companion integration libraries, we transition this segment using a smooth, dual-color gradient—blending from the primary library's color into the target framework's signature hue.

Other active packages in our ecosystem, such as our WeakEvent and QueryLink libraries, will receive this visual update treatment alongside their next releases.

\"ByteAether\"ByteAether.Ulid\"ByteAether.Ulid.EntityFrameworkCore

Architectural Refresher: The Mechanics of ByteAether.Ulid

To understand why direct data access integration is critical, we must look at the internal trade-offs managed by the core library. A ULID is a 128-bit structure composed of a 48-bit Unix timestamp (millisecond resolution) followed by an 80-bit random component. When encoded to text, it yields a 26-character string using Crockford's Base32 alphabet. This ensures case-insensitive, URL-safe readability while maintaining perfect big-endian chronological sortability.

 0                   1                   2                   3\n 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1\n+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n|                      32-bit Timestamp                         |\n+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n|     16-bit Timestamp          |       16-bit Random           |\n+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n|                      32-bit Random                            |\n+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n|                      32-bit Random                            |\n+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

High-Concurrency and Monotonicity

When multiple identifiers are generated within the same millisecond, standard UUID engines or naive ULID implementations can suffer from two structural issues:

  1. Random-Part Overflow: Under high-frequency allocation loops, incrementing the 80-bit random suffix sequentially to preserve monotonicity can saturate the random bits. This causes an unhandled OverflowException, halting transaction processing. While a naive mathematical analysis suggests this collision probability is a negligible 1 in 280, the reality under high-concurrency workloads reveals greater volatility. As detailed in our previous analysis, "Prioritizing Reliability When Milliseconds Aren't Enough", the practical probability of encountering this overflow is significantly larger than intuition suggests.
  2. Enumeration Attack Vectors: Standard sequential increments create highly predictable tracking sequences, opening security gaps that allow bad actors to guess subsequent IDs.

ByteAether.Ulid handles these edge cases directly. Our monotonic engine uses a lock-free Compare-and-Exchange (CAS) loop to prevent thread contention. If the 80-bit random space reaches saturation within a millisecond burst, the generator automatically increments the timestamp component. This approach prevents runtime errors and preserves strict chronological sorting under extreme load.

To mitigate enumeration risks, developers can inject multi-byte entropy steps using GenerationOptions:

var options = new Ulid.GenerationOptions\n{\n\tMonotonicity = MonotonicityOptions.MonotonicRandom2Byte,\n\tInitialRandomSource = new CryptographicallySecureRandomProvider(),\n\tIncrementRandomSource = new PseudoRandomProvider()\n};\n\nUlid secureId = Ulid.New(options);

By configuring MonotonicRandom2Byte, the engine generates a 16-bit pseudo-random (IncrementRandomSource = new PseudoRandomProvider()) integer and adds it plus one to the suffix, resulting in a delta ranging from 1 to 65,536 on consecutive generations within the same millisecond.

Low-Level Performance Metrics

The core library relies on modern .NET memory primitives, explicit multi-targeting, and ahead-of-time (AOT) compilation compatibility to keep allocation overhead at zero. The metrics below represent a chosen subset of all benchmarks executed during development. The full, comprehensive test suite covering exhaustive operational states can be found directly in the GitHub repository at github.com/ByteAether/Ulid. Please note that these results reflect performance figures current at the time of writing this article:

| Type            | Method             | Mean        | Gen0   | Allocated |\n|---------------- |------------------- |------------:|-------:|----------:|\n| Generate        | ByteAetherUlid     |  40.5719 ns |      - |         - |\n| Generate        | NetUlid            | 162.1996 ns | 0.0095 |      80 B |\n| GenerateNonMono | ByteAetherUlidP    |  40.9386 ns |      - |         - |\n| GenerateNonMono | GuidV7             |  78.6568 ns |      - |         - |\n| FromString      | ByteAetherUlid     |  15.2724 ns |      - |         - |\n| FromString      | NUlid              |  64.0466 ns | 0.0086 |      72 B |\n| CompareTo       | ByteAetherUlid     |   1.3642 ns |      - |         - |\n| CompareTo       | Ulid (Cysharp)     |   6.7843 ns |      - |         - |\n

The engine achieves these sub-50 nanosecond generation speeds with zero managed heap allocations. This optimization prevents garbage collection pauses in hot paths like HTTP ingestion loops or message brokers.

The Database Storage Challenge: Endianness and Alignment

While memory-level optimizations provide significant benefits, they mean little if the identifier degrades database performance. Translating a 128-bit big-endian ULID structure into a relational database index requires careful planning around storage types and endianness alignment.

Relational database engines parse 128-bit identifiers differently depending on their internal page layouts:

The Microsoft SQL Server UniqueIdentifier Problem

Microsoft SQL Server handles the uniqueidentifier type using an unusual internal sorting hierarchy. Instead of sorting a 16-byte block from left to right, its indexing engine evaluates bytes out of order. Specifically, it prioritizes the final 6 bytes (bytes 10 through 15) as the primary sorting criteria.

Standard Big-Endian ULID Layout:\n[0-5: Timestamp] [6-15: Monotonic Entropy]\n\nMicrosoft SQL Server uniqueidentifier Storage Target:\n[0-9: Random Padding / Sub-Entropy] [10-15: Most Significant Timestamp Bytes]

If a standard big-endian ULID is mapped directly to an MSSQL uniqueidentifier column without alteration, the database will sort the values using the random entropy block rather than the timestamp. This breaks chronological order, transforms sequential insertions into random writes, triggers heavy index page splits, and degrades I/O performance.

To prevent this issue, the companion libraries include UlidStorageFormat.SqlServerGuid. When selected, the integration library rearranges the internal bytes before sending them to the database driver. It shifts the 48-bit timestamp to the end of the byte block, ensuring that SQL Server's internal sorting logic aligns perfectly with the temporal order of generation.

Entity Framework Core Integration

The ByteAether.Ulid.EntityFrameworkCore extension package simplifies type handling for Entity Framework Core architectures. Historically, using the library required developers to author and maintain their own custom value converters to bridge the Ulid primitive over to the database driver. While EF Core allowed applying those custom converters globally via convention hooks or explicitly per property, writing that boiler-plate mapping infrastructure fell entirely on the consumer. The companion package eliminates this operational friction by providing ready-made, highly optimized value converters out of the box alongside standard global registration hooks.

Installation

dotnet add package ByteAether.Ulid.EntityFrameworkCore

Global Model Conventions

To apply ULID mappings uniformly across an entire domain model, override the ConfigureConventions hook inside the active DbContext. This applies the selected storage layout to all Ulid and Nullable<Ulid> properties discovered by the model scanner.

using Microsoft.EntityFrameworkCore;\nusing ByteAether.Ulid.EntityFrameworkCore;\n\npublic sealed class ApplicationDbContext : DbContext\n{\n\tpublic DbSet<Order> Orders => Set<Order>();\n\tpublic DbSet<Customer> Customers => Set<Customer>();\n\n\tpublic ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)\n\t\t: base(options) { }\n\n\tprotected override void ConfigureConventions(ModelConfigurationBuilder configurationBuilder)\n\t{\n\t\t// Registers global mappings for both Ulid and Ulid? types.\n\t\t// Available strategies: String, Binary, Guid, and SqlServerGuid\n\t\tconfigurationBuilder.RegisterUlid(UlidStorageFormat.Binary);\n\t}\n}

Fine-Grained Property Mapping

If an application works with multiple legacy storage schemas, global conventions can be overridden. By using fluent API declarations inside OnModelCreating, architects can configure distinct storage strategies for specific tables or columns.

protected override void OnModelCreating(ModelBuilder modelBuilder)\n{\n\tbase.OnModelCreating(modelBuilder);\n\n\tmodelBuilder.Entity<Customer>(entity =>\n\t{\n\t\t// Persist as a 26-character human-readable string\n\t\tentity.Property(c => c.Id)\n\t\t\t  .HasConversion<UlidToStringConverter>();\n\t});\n\n\tmodelBuilder.Entity<Order>(entity =>\n\t{\n\t\t// Persist as a standard native GUID block\n\t\tentity.Property(o => o.Id)\n\t\t\t  .HasConversion<UlidToGuidConverter>();\n\t});\n\n\tmodelBuilder.Entity<ArchiveLog>(entity =>\n\t{\n\t\t// Re-order byte arrays specifically for SQL Server uniqueidentifier indexes\n\t\tentity.Property(a => a.Id)\n\t\t\t  .HasConversion<UlidToSqlServerGuidConverter>();\n\t});\n}

LinqToDB Micro-ORM Integration

For data layers built on the LinqToDB micro-ORM — mirroring the architectural layout explored in our Building an Enterprise Data Access Layer series — the ByteAether.Ulid.linq2db package introduces a streamlined integration route. While our original enterprise series relied on localized, embedded code snippets to coordinate LinqToDB data types prior to the v1.4.0 release, this extension replaces that manual plumbing by hooking directly into the DataOptions pipeline via a single, standardized configuration call.

Installation

dotnet add package ByteAether.Ulid.linq2db

Configuration Pipeline

Register the ULID conventions directly during the initialization of the DataOptions instance. This step configures the mapping schema before opening any database connections.

using LinqToDB;\nusing ByteAether.Ulid.LinqToDB;\n\npublic static class LinqToDbConfiguration\n{\n\tpublic static DataOptions BuildOptions(string connectionString)\n\t{\n\t\treturn new DataOptions()\n\t\t\t.UseSQLite()\n\t\t\t.UseConnectionString(connectionString)\n\t\t\t// Seamlessly registers global mapping engines for Ulid and Ulid?\n\t\t\t.RegisterUlid(UlidStorageFormat.Binary);\n\t}\n}

Once registered, standard LINQ expressions translate Ulid properties directly into parameterized SQL statements without requiring manual casting or intermediate transformations.

Dapper Integration

Dapper maps .NET types globally using a strict one-to-one relationship (Type -> TypeHandler). The ByteAether.Ulid.Dapper companion package hooks into Dapper's internal type handler cache to ensure clean, automated parameter serialization.

Installation

dotnet add package ByteAether.Ulid.Dapper

Initialization

Invoke DapperUlid.RegisterUlid() once during application startup (typically within Program.cs), prior to executing any database interactions.

using ByteAether.Ulid.Dapper;\n\npublic static class Program\n{\n\tpublic static void Main(string[] args)\n\t{\n\t\t// Apply the chosen binary storage mapping globally across Dapper operations\n\t\tDapperUlid.RegisterUlid(UlidStorageFormat.Binary);\n\n\t\t// Continue running standard host infrastructure...\n\t}\n}

Executing Queries

With the type handler registered, Dapper automatically processes Ulid fields during parameter execution and object mapping.

using Dapper;\nusing System.Data;\n\npublic sealed class AccountRepository\n{\n\tprivate readonly IDbConnection _dbConnection;\n\n\tpublic AccountRepository(IDbConnection dbConnection) => _dbConnection = dbConnection;\n\n\tpublic async Task<Account?> GetByIdAsync(Ulid accountId)\n\t{\n\t\tconst string sql = \"SELECT * FROM Accounts WHERE Id = @Id\";\n\n\t\t// The accountId argument is automatically serialized into a 16-byte block\n\t\treturn await _dbConnection.QueryFirstOrDefaultAsync<Account>(sql, new { Id = accountId });\n\t}\n}

Note: Because Dapper uses a global type handler registration system, applications must standardize on a single storage format per execution lifecycle.

High-Performance Time-Range Filtering

A major advantage of using ULIDs over traditional UUIDs is the embedded 48-bit millisecond timestamp. This allows developers to run fast time-range queries directly against the primary identifier index, eliminating the need for a secondary index on a CreatedAt column.

Using Ulid.MinAt() and Ulid.MaxAt(), applications can generate boundary identifiers for precise time windows.

public async Task<IReadOnlyList<Order>> GetOrdersFromPastWeekAsync(ApplicationDbContext context)\n{\n\tDateTimeOffset startBoundary = DateTimeOffset.UtcNow.AddDays(-7);\n\tDateTimeOffset endBoundary = DateTimeOffset.UtcNow;\n\n\t// Generate boundary tokens matching the precise timestamps\n\tUlid minId = Ulid.MinAt(startBoundary);\n\tUlid maxId = Ulid.MaxAt(endBoundary);\n\n\t// Queries translate to standard, highly optimized index scans:\n\t// WHERE Id >= @minId AND Id <= @maxId\n\treturn await context.Orders\n\t\t.Where(order => order.Id >= minId && order.Id <= maxId)\n\t\t.ToListAsync();\n}

Essential Production Requirements

When running time-range queries against database tables, keep these architectural requirements in mind:

  1. Storage Format Consistency: The storage strategy must preserve lexicographical sortability. UlidStorageFormat.Binary and UlidStorageFormat.String maintain correct chronological sorting across all platforms.
  2. Byte Order Mismatches & Engine Behavior: The success of range queries depends heavily on how a database engine and its ADO.NET provider manage GUID layouts. For instance, PostgreSQL's native uuid type handles .NET's Guid cleanly, allowing range queries to resolve correctly. Conversely, engines like SQLite typically persist GUID values as raw byte blocks, which causes immediate sorting and ordering issues due to the conflict between .NET's mixed-endian internal Guid representation and the database's expected sort behavior. Before deploying UlidStorageFormat.Guid, thoroughly verify how your specific provider maps and indexes the underlying bytes.
  3. SQL Server Index Routing: When working with Microsoft SQL Server, always use the SqlServerGuid storage format. This ensures that the generated values match SQL Server's internal uniqueidentifier indexing rules, allowing for fast, efficient index scans.

Conclusion

The release of ByteAether.Ulid v1.4.0 shifts the library's focus from a standalone high-performance primitive to a fully integrated ecosystem. By introducing official packages for EF Core, LinqToDB, and Dapper, it eliminates the manual boilerplate and database alignment issues that previously complicated production deployments.

Whether you are designing a high-throughput microservices architecture or managing heavy enterprise data models, these companion libraries handle the low-level serialization details for you. They ensure that your application benefits from zero-allocation, thread-safe monotonic identifiers while keeping your database indexes lean and highly performant.

For source code, complete benchmark datasets, and detailed documentation, visit the official repository at github.com/ByteAether/Ulid.

", "date_published": "2026-07-22T00:00:00Z" } , { "id": "https://byteaether.github.io/2026/the-architecture-of-obsession-why-byteaetherulid-is-the-last-identifier-library-you-will-ever-need/", "url": "https://byteaether.github.io/2026/the-architecture-of-obsession-why-byteaetherulid-is-the-last-identifier-library-you-will-ever-need/", "title": "The Architecture of Obsession: Why ByteAether.Ulid is the Last Identifier Library You Will Ever Need", "content_html": "

Most developers treat unique identifiers like plumbing. You want them to work, you want them to be out of sight, and you certainly do not want to spend your Friday nights thinking about them. For the average CRUD application, Guid.NewGuid() is a perfectly acceptable choice. However, if you are reading this, you are likely not an average developer. You are an architect or a senior engineer who understands that at scale, "good enough" is a ticking time bomb of index fragmentation and out-of-order writes.

I have spent an unreasonable amount of time obsessing over Universally Unique Lexicographically Sortable Identifiers (ULIDs). While the rest of the world moved on to other things, I stayed behind to build ByteAether.Ulid, a library that is, quite frankly, over-engineered to the point of insanity. But in a world of "lazy" implementations and optional monotonicity, perhaps a bit of insanity is exactly what your distributed system needs.

The Sortability: Why UUIDv7 Isn't the Hero We Expected

For a long time, the industry waited for a standardized, sortable replacement for the legacy GUID (UUIDv4). When UUIDv7 arrived via RFC 9562, it seemed like the answer. Even .NET introduced a native provider for it. However, if you dig into the implementation details (specifically how the native .NET provider implements the generation of UUIDv7), you find a compromise.

The specification for UUIDv7 makes monotonicity optional. This means that during a high-concurrency burst within the same millisecond, the native provider may sacrifice strict ordering to maintain performance. To a database architect, this is heresy. Out-of-order identifiers lead to page splits in B-Tree indexes, which leads to physical disk fragmentation and degraded insert performance.

ULIDs solve this by design. By mandating lexicographical sortability and strict monotonic increments, ULID ensures that your primary keys are always appended to the end of the index. ByteAether.Ulid doubles down on this by providing a robust, fully compliant .NET implementation that refuses to take the "lazy" path.

Eliminating the Overflow Exception

One of the darker corners of the ULID specification involves what happens when the 80-bit random component overflows during a monotonic increment within the same millisecond. Many libraries simply throw an OverflowException (as specification dictates), effectively crashing your ingestion pipeline because you were "too fast" for the generator. This isn't just a theoretical concern for high-volume systems as the initial random component is cryptographically generated and it could start near its maximum value, causing the very next increment to fail.

In ByteAether.Ulid, we have implemented a more elegant solution. When the random part overflows, the library automatically increments the timestamp component. This ensures that unique ULID generation continues without interruption while maintaining the sort order. This behavior ensures that your system remains resilient even under extreme throughput. For a deep dive into the engineering rationale and why this is more likely than you think, check out our dedicated article on Handling ULID Overflows Gracefully.

High-Performance, Lock-Free Concurrency

In high-concurrency environments, locks are the enemy of throughput. If every thread in your web farm has to wait for a global lock just to generate an ID, you have created a bottleneck.

ByteAether.Ulid utilizes a lock-free compare-and-exchange (CAS) approach for monotonic generation. This allows multiple threads to attempt to increment the state simultaneously without traditional locking overhead. If a collision occurs, the CAS operation retries until it succeeds, ensuring thread safety with maximum performance.

In terms of raw speed, ByteAether.Ulid stands as one of the most performant implementations available, often matching or exceeding the most optimized competitors in the .NET ecosystem while maintaining a zero-allocation footprint for standard generation. Detailed performance metrics and environment-specific results are maintained on our GitHub page.

Preventing Enumeration Attacks with Random Increments

Standard monotonic ULIDs can be predictable. If an attacker knows one ULID, they can often guess the next one by simply adding 1 to the random component. This is a significant security risk for public-facing APIs.

To mitigate this, I have implemented configurable random increments. Instead of adding a static 1 to the random component, you can configure the library to use a random increment ranging from 1 byte up to 4 bytes. This significantly increases the entropy between sequential IDs, making enumeration attacks computationally expensive while still preserving the lexicographical sortability required for your database indexes.

The Developer Experience: Frictionless Integration

A library should feel like a native part of your stack, not an obstacle. I've focused on ensuring that ByteAether.Ulid integrates seamlessly with modern .NET defaults while providing clear pathways for specialized tooling. The library includes native, built-in support for:

Beyond the standard runtime bindings, the integration model balances zero-dependency core architecture with ecosystem convenience:

Advanced Filtering with LINQ

One of the most powerful features of ULIDs is the ability to perform time-range queries without a separate CreatedAt column. Because the timestamp is embedded in the ID, you can generate boundary ULIDs:

var start = Ulid.MinAt(DateTimeOffset.UtcNow.AddDays(-7));\nvar end = Ulid.MaxAt(DateTimeOffset.UtcNow);\n\n// EF Core translates this into an efficient range comparison on the Primary Key\nvar reports = await context.Reports\n\t.Where(r => r.Id >= start && r.Id <= end)\n\t.ToListAsync();

This approach allows you to leverage the primary key index for high-performance temporal queries, reducing the need for additional composite indexes.

Embracing the Over-Engineered

The development of ByteAether.Ulid represents a commitment to refining edge cases, ensuring AOT (Ahead-of-Time) compilation compatibility, and hardening logic against the weirdest distributed system failures.

Is it overkill? Probably. Do you need a library that handles sub-millisecond random overflows via timestamp carry-over using lock-free CAS? Maybe not today. But when your system scales to millions of events per second and your database starts choking on index fragmentation, you will be glad someone was "insane" enough to build this.

Stop settling for "lazy" identifiers. Give your architecture the precision it deserves by checking out the project on GitHub.

", "date_published": "2026-04-02T00:00:00Z" } , { "id": "https://byteaether.github.io/2026/the-weight-of-decisions-solving-weighted-random-sorting-at-scale/", "url": "https://byteaether.github.io/2026/the-weight-of-decisions-solving-weighted-random-sorting-at-scale/", "title": "The Weight of Decisions: Solving Weighted Random Sorting at Scale", "content_html": "

Several years ago at Microsoft, I worked on a back-end service responsible for routing high volumes of traffic. The service functioned as a traffic dispatcher: we received an incoming request, determined the most appropriate downstream destination, and forwarded the payload.

Our routing decisions relied on a pool of destination services. To ensure reliability, we tracked ongoing statistics regarding the success rates of these partners. When a destination consistently processed requests successfully, we increased its traffic share. Conversely, if we observed frequent errors, we reduced its allocation.

The Requirement for Ordered Fail-over

Routing in a distributed environment requires more than just selecting a single destination. Downstream partners occasionally fail to process specific items due to transient internal errors or malformed data. Because of this, we needed a "fail-over" list for every request. If the primary choice failed, the system needed an immediate second and third option ready to go.

This meant our routing logic had to produce a fully sorted list of all available destination services. We would attempt to send the request to the first service in the list; if that service returned an error, we moved to the second, and continued until the request succeeded or we exhausted the list.

Balancing Traffic via Weighted Randomization

A simple deterministic sort based on quality scores would create a "starvation" problem. If Service A had the highest success rate, a standard sort would consistently place it at the top. This would result in Service A receiving 100% of the initial traffic, leaving services B and C with no data points. Without ongoing "exploration" traffic to lower-ranked services, we could never observe if their performance had recovered or improved.

To maintain fresh statistics across the entire pool, we needed a weighted random sort, effectively a weighted permutation. Consider a scenario with three services and the following target distributions:

The sorted list needed to respect these probabilities at every rank. For the first position, the chances are exactly as defined (50%, 30%, 20%). However, if Service A is selected for the first slot, the second slot must be filled by B or C based on their remaining relative weights. In this case, Service B should appear in the second position with a probability of 0.3 / (0.3 + 0.2) = 60%, while Service C takes the spot 40% of the time.

Implementation Logic

Both the initial and the improved versions of our code use the following data structure to represent a destination:

public class Item\n{\n\tpublic string Name { get; set; }\n\tpublic double Weight { get; set; }\n}

The Intuition and the Initial O(N2 log N) Approach

I suspected a mathematical shortcut existed to calculate a single value for each item that, when sorted, would result in this exact distribution. This thought surfaced repeatedly over the years and never truly gave me peace. I felt certain a single-pass solution was possible, yet the specific formula remained out of reach. At the time, we settled on an iterative approach that picked items one by one.

In C#, the implementation relied on a "weighted pick and remove" loop. The computational cost was actually O(N2 log N) in the version below because a full sort is performed inside a loop to pick just one item. While N = 10 renders this inefficiency invisible in production, the logic scales poorly for larger pools or high-throughput middleware.

public static List<T> InefficientWeightedRandomize<T>(IEnumerable<T> source)\n\twhere T : Item\n{\n\tvar remainingItems = source.ToList();\n\tvar result = new List<T>();\n\n\twhile (remainingItems.Any())\n\t{\n\t\t// Sort the source list by a random factor modified by weight\n\t\tvar nextItem = remainingItems\n\t\t\t.Select(item => new\n\t\t\t{\n\t\t\t\tItem = item,\n\t\t\t\tSortKey = Random.Shared.NextDouble() * item.Weight\n\t\t\t})\n\t\t\t.OrderByDescending(x => x.SortKey)\n\t\t\t.First();\n\n\t\tresult.Add(nextItem.Item);\n\t\tremainingItems.Remove(nextItem.Item);\n\t}\n\n\treturn result;\n}

The Efraimidis-Spirakis Algorithm

Recently, I came across the Efraimidis-Spirakis Weighted Random Sampling paper, which provides the mathematical proof for the "one-pass sort" I had originally envisioned. The paper demonstrates that calculating a key ki = ui1/wi (where u is a random value and w is the weight) and sorting by that key produces a mathematically perfect weighted random sample.

This technique is a specialized form of Reservoir Sampling. It allows us to process the list in O(N log N) time with a much cleaner implementation:

public static List<T> WeightedRandomize<T>(IEnumerable<T> source)\n\twhere T : Item\n{\n\treturn source\n\t\t.Select(item => new\n\t\t{\n\t\t\tItem = item,\n\t\t\tSortKey = Math.Pow(\n\t\t\t\tRandom.Shared.NextDouble(),\n\t\t\t\t1.0 / item.Weight\n\t\t\t)\n\t\t})\n\t\t.OrderByDescending(x => x.SortKey)\n\t\t.Select(x => x.Item)\n\t\t.ToList();\n}

Applications for Streaming Data

The Efraimidis-Spirakis algorithm offers an interesting property: it works on data streams without requiring the full list to be present at the start. Since each item's SortKey is calculated independently, we can assign keys as items arrive. Once the stream ends, a single pass through the data (or maintaining a min-heap for a "Top K" selection) yields the result.

For the team back at Microsoft still managing these services, this is a small but elegant improvement. While N = 10 doesn't demand high-performance algorithms, replacing the O(N2) loop with a proper weighted sort aligns the implementation with the intended mathematical model. I suspect there might even be an old research ticket regarding this specific efficiency improvement sitting in the backlog, likely created by me before I moved on. It might be time to finally close it.

", "date_published": "2026-01-07T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-user-auditing-and-series-wrap-up/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-user-auditing-and-series-wrap-up/", "title": "Building an Enterprise Data Access Layer: Automated User Auditing and Series Wrap-up", "content_html": "

Welcome to the seventh and final article in our series on building an enterprise-grade Data Access Layer (DAL) in C#. Over the last six posts, we have methodically built a robust, automated, and secure DAL using a database-first philosophy with C# and LinqToDB.

We've tackled soft-deletes, timestamp auditing, multi-tenancy, and even complex projected row-level security. In our very first article, we set a high bar for the capabilities our DAL must provide. Today, we implement the final missing piece: automated user auditing (CreatedByUser / ModifiedByUser).

Once that's in place, we'll hold a full retrospective to see how our final architecture stacks up against our original goals, and finally, we'll look at what challenges lie beyond this series.

Implementing User Auditing

So far, our audit fields (CreatedAt, ModifiedAt) tell us when a record was changed. User auditing completes the picture by telling us who made that change. This is a common requirement for compliance, traceability, and debugging.

Our goal is to automatically populate CreatedByUserId on insert and ModifiedByUserId on insert and update, using the UserId from our request-scoped context. We will follow the exact same pattern of "behavioral interfaces" that we established in Part 3 (Auditing) and Part 4 (Soft-Delete).

The Schema and Interface Contracts

Following our database-first approach, we first update the database schema. We add created_by_user_id ulid not null and modified_by_user_id ulid not null columns to the post and comment tables, both with foreign keys referencing the user table.

Next, we define the C# contracts in DAL.Base/EntityBehavior/:

namespace DAL.Base.EntityBehavior;\n\npublic interface IUserCreatable\n{\n\tUlid CreatedByUserId { get; set; }\n}
namespace DAL.Base.EntityBehavior;\n\npublic interface IUserModifiable\n{\n\tUlid ModifiedByUserId { get; set; }\n}

These simple interfaces define the properties our DAL will automatically manage.

Automated Scaffolding

With the database and interfaces in place, we update our custom scaffolding interceptor (DAL.ScaffoldInterceptor/Interceptor.cs).

Just as it was configured to detect created_at and attach ICreatable, we've now taught it to look for created_by_user_id and modified_by_user_id columns. When it finds them, it automatically adds IUserCreatable and IUserModifiable to the generated partial entity classes in DbCtx.generated.cs.

This is the power of our architecture: the database schema remains the single source of truth. A schema change, followed by a re-scaffold, is all that's needed to make the C# code aware of this new behavior.

Updating the CRUD Extensions

The final step is to update our CrudExtensions.cs file to enforce the logic. We need to fetch the current user's ID from our IDbCtx (which we added in Part 6) and assign it.

We update CreateAsync, ModifyAsync (for single entity), and ModifyAsync (for fluent queries) to handle these new interfaces.

public static async Task<long> CreateAsync<T>(\n\tthis IDbCtx ctx,\n\tIEnumerable<T> entities,\n\tCancellationToken cancellationToken = default\n)\n\twhere T : class, IEntity\n\t=> (\n\t\tawait ctx.BulkCopyAsync(\n\t\t\tentities.Select(e =>\n\t\t\t\t{\n\t\t\t\t\t// Timestamp Auditing (from Part 3)\n\t\t\t\t\tif (\n\t\t\t\t\t\te is ICreatable creatable\n\t\t\t\t\t\t&& creatable.CreatedAt == default\n\t\t\t\t\t)\n\t\t\t\t\t{\n\t\t\t\t\t\tcreatable.CreatedAt = DateTime.UtcNow;\n\t\t\t\t\t}\n\n\t\t\t\t\t// NEW: User Create Auditing\n\t\t\t\t\tif (\n\t\t\t\t\t\te is IUserCreatable userCreatable\n\t\t\t\t\t\t&& userCreatable.CreatedByUserId == default\n\t\t\t\t\t)\n\t\t\t\t\t{\n\t\t\t\t\t\tuserCreatable.CreatedByUserId = ctx.Attributes.UserId ?? Ulid.Empty;\n\t\t\t\t\t}\n\n\t\t\t\t\t// Timestamp Auditing (from Part 3)\n\t\t\t\t\tif (\n\t\t\t\t\t\te is IModifiable updateable\n\t\t\t\t\t\t&& updateable.ModifiedAt == default\n\t\t\t\t\t)\n\t\t\t\t\t{\n\t\t\t\t\t\tupdateable.ModifiedAt = DateTime.UtcNow;\n\t\t\t\t\t}\n\n\t\t\t\t\t// NEW: User Modify Auditing\n\t\t\t\t\tif (\n\t\t\t\t\t\te is IUserModifiable userUpdateable\n\t\t\t\t\t\t&& userUpdateable.ModifiedByUserId == default\n\t\t\t\t\t)\n\t\t\t\t\t{\n\t\t\t\t\t\tuserUpdateable.ModifiedByUserId = ctx.Attributes.UserId ?? Ulid.Empty;\n\t\t\t\t\t}\n\n\t\t\t\t\treturn e;\n\t\t\t\t}\n\t\t\t),\n\t\t\tcancellationToken\n\t\t)\n\t).RowsCopied;\n\npublic static Task<int> ModifyAsync<T>(\n\tthis IDbCtx ctx,\n\tT entity,\n\tCancellationToken cancellationToken = default\n)\n\twhere T : class, IEntity\n{\n\t// Timestamp Auditing (from Part 3)\n\tif (entity is IModifiable updateable)\n\t{\n\t\tupdateable.ModifiedAt = DateTime.UtcNow;\n\t}\n\n\t// NEW: User Modify Auditing\n\tif (entity is IUserModifiable userUpdateable)\n\t{\n\t\tuserUpdateable.ModifiedByUserId = ctx.Attributes.UserId ?? Ulid.Empty;\n\t}\n\n\treturn ctx.GetTable<T>()\n\t\t.UpdateOptimisticAsync(entity, cancellationToken);\n}

The IUpdatable Nuance

The fluent ModifyAsync extension (which operates on IUpdatable<T>) presents a unique challenge. This method is what allows us to write code like ctx.GetTable<Post>().Where(p => p.Id == id).ModifyAsync().

To inject the ModifiedByUserId value, we need the IDbCtx instance to get the ctx.Attributes.UserId. However, the IUpdatable<T> (which is essentially an IQueryable) doesn't expose its parent context directly.

To solve this, we use a helper provided by LinqToDB:

public static Task<int> ModifyAsync<T>(\n\tthis IUpdatable<T> source,\n\tCancellationToken cancellationToken = default\n)\n\twhere T : class\n{\n\t// Timestamp Auditing (from Part 3)\n\tif (typeof(IModifiable).IsAssignableFrom(typeof(T)))\n\t{\n\t\tsource = source.Set(\n\t\t\tx => Sql.Property<DateTime>(x, nameof(IModifiable.ModifiedAt)),\n\t\t\tDateTime.UtcNow\n\t\t);\n\t}\n\n\t// NEW: User Modify Auditing\n\tif (typeof(IUserModifiable).IsAssignableFrom(typeof(T)))\n\t{\n\t\t// Get the DbCtx from the query\n\t\tvar dbCtx = Internals.GetDataContext(source) as IDbCtx;\n\t\tsource = source.Set(\n\t\t\tx => Sql.Property<Ulid>(x, nameof(IUserModifiable.ModifiedByUserId)),\n\t\t\tdbCtx?.Attributes.UserId ?? Ulid.Empty\n\t\t);\n\t}\n\n\treturn source.UpdateAsync(cancellationToken);\n}

The key part is (LinqToDB.Internal.Linq.)Internals.GetDataContext(source). While using an Internal namespace is generally something to be wary of, it is a pragmatic and necessary solution in this advanced scenario. It allows us to bridge our fluent query API with our request-scoped context, ensuring user auditing is applied even during batch UPDATE operations.

And with that, our final automated behavior is implemented.

The Result: A Fully Automated Query

To see all our work come together, let's look at the simple ModifyAsync call from our Program.cs that updates a Comment:

// Try to update a protected comment entity\nawait ctx.GetTable<Comment>()\n\t.Set(x => x.Content, \"asd123\")\n\t.ModifyAsync();

When this C# code executes, our DAL intercepts it and, using all the logic from this series, generates the following comprehensive SQL query:

UPDATE\n\t[comment]\nSET\n\t[content] = 'asd123',\n\t[modified_at] = '2025-11-12 15:31:11.972',\n\t[modified_by_user_id] = X'019A78B1378CC2EC0F1736EE8D4F1E8F'\nFROM\n\t[post] [a_Post],\n\t[user] [a_User],\n\t[permission] [b]\nWHERE\n\t[a_User].[removed_at] IS NULL AND\n\t[a_User].[tenant_id] = X'019A78B13787E6060BF9BA9A51DAA2C6' AND\n\t[a_Post].[removed_at] IS NULL AND\n\t[comment].[removed_at] IS NULL AND\n\t[comment].[post_id] = [a_Post].[id] AND\n\t[a_Post].[user_id] = [a_User].[id] AND\n\t[b].[object_id] = [a_Post].[id] AND\n\t[b].[subject_id] = X'019A78B1378CC2EC0F1736EE8D4F1E8F'

This single UPDATE statement perfectly demonstrates our entire architecture working in harmony. The SET clause not only updates the content but also automatically injects modified_at (from Part 3) and our new modified_by_user_id field.

Furthermore, the WHERE clause is a composition of all our global filters:

This is the automation we set out to build, all abstracted from the developer who just called .ModifyAsync().

Achieving Our Goals

In our first article, we defined a clear set of requirements for an enterprise DAL. Our mission was to automate cross-cutting concerns to prevent errors and reduce technical debt. Let's review our original checklist.

Verdict: We successfully implemented 100% of our initial goals. The final composed query, which LinqToDB generates by combining all these filters, is a testament to the power of this composable, interface-driven architecture.

Future Ideas

While this series is complete, the work of a systems architect is never truly finished. Our DAL is robust, but it exists within a larger ecosystem. Here are a few topics worth exploring next:

Conclusion

Thank you for following this series. We began with an ambitious set of goals for a "perfect" DAL, and through seven articles, we've built it piece by piece. We have a Data Access Layer that is truly "enterprise-ready" - it is secure by default, resilient to common errors, and abstracts immense complexity away from the business logic.

The final code for this article, and the entire series, is available on GitHub.

Full Source Code: https://github.com/ByteAether/EnterpriseDal/tree/part7

I hope this series has provided a valuable blueprint for your own advanced data access architectures.

", "date_published": "2025-12-01T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-composable-row-level-security/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-composable-row-level-security/", "title": "Building an Enterprise Data Access Layer: Composable Row-Level Security", "content_html": "

Welcome to the sixth post in our series on building a feature-rich, automated enterprise Data Access Layer (DAL) using C# and LinqToDB. In our previous articles, we established a powerful architectural pattern: a composable global query filter system. This system allows us to define cross-cutting concerns as simple interfaces, automatically combining and applying them to every relevant query.

So far, we have successfully implemented:

In this post, we will tackle the final and most complex pillar of our automated DAL: entity-based row-level security (RLS). Our goal is to automatically filter any query to return only the records that the current user is authorized to access. We will achieve this by extending the exact same composable filter architecture we've already built, demonstrating its power and flexibility.

1. Defining the Security Contracts

To build an RLS system, we need to define three core components:

  1. A contract for the current user.
  2. A contract for a permission record.
  3. A contract for an entity that requires protection.

Updating the Database Context

First, we must make our DAL aware of the "current user." We previously extended our IDbCtx with DbCtxAttributes to hold the current TenantId. We will now update that same record to also hold a UserId.

// In DAL.Base/IDbCtx.cs\npublic interface IDbCtx : IDataContext\n{\n\tDbCtxAttributes Attributes { get; set; }\n\n\tITable<IPermissionEntity> GetPermissions(); // New!\n\n\tpublic record DbCtxAttributes(\n\t\tUlid TenantId = default,\n\t\tUlid? UserId = null // New!\n\t);\n}

We've added two things:

The Permission Contract (IPermissionEntity)

Next, we need a way to represent a permission. In our database, we've added a new permission table with a simple schema: subject_id (who has the permission) and object_id (what entity the permission is for).

To map this in code, we create a new interface:

// In DAL.Base/EntityBehavior/IPermissionEntity.cs\npublic interface IPermissionEntity\n{\n\tUlid SubjectId { get; } // The \"Who\" (e.g., a UserId)\n\tUlid ObjectId { get; }  // The \"What\" (e.g., a PostId)\n}

After running our scaffolder to generate the Permission entity from the new table, we simply create a partial class to apply this interface:

// In DAL.Context/Entity/Permission.cs\npublic partial class Permission : IPermissionEntity\n{ }

Finally, we implement the new GetPermissions() method in our DbCtx.cs file. It's a simple one-line pass-through:

// In DAL.Context/DbCtx.cs\npublic ITable<IPermissionEntity> GetPermissions() => GetTable<Permission>();

2. The IProtected Interface: The RLS Filter

With our contracts in place, we can now create the "behavior" interface that will automatically apply our RLS filter. We'll call it IProtected. This interface is the heart of our new system and leverages the same [EntityFilter] attribute we used for soft-deletes and tenancy.

// In DAL.Base/EntityBehavior/IProtected.cs\n[EntityFilter<IProtected>(nameof(Filter))]\npublic interface IProtected\n{\n\tUlid GetPermissionObjectId();\n\n\tprivate static IQueryable<T> Filter<T>(IQueryable<T> q, IDbCtx dbCtx)\n\t\twhere T : IProtected\n\t\t=> dbCtx.Attributes.UserId == null\n\t\t\t// 1. If no user is in context, return nothing.\n\t\t\t? q.Where(_ => false)\n\t\t\t// 2. Otherwise, join with permissions and filter.\n\t\t\t: q.InnerJoin(\n\t\t\t\tdbCtx.GetPermissions(),\n\t\t\t\t(entity, perm) =>\n\t\t\t\t\tperm.ObjectId == entity.GetPermissionObjectId() &&\n\t\t\t\t\tperm.SubjectId == dbCtx.Attributes.UserId,\n\t\t\t\t(entity, perm) => entity\n\t\t\t);\n}

Let's break this down, as it's the most important piece:

  1. [EntityFilter<...>]: This attribute, as before, tells our DAL's bootstrapper to find the Filter method and apply it to any entity that implements IProtected.
  2. Ulid GetPermissionObjectId(): This is the contract. Any entity implementing IProtected must tell the system which ObjectId to use for its permission check. This is a method, not a property, which allows for incredible flexibility, as we'll see next.
  3. private static IQueryable<T> Filter<...>: This is the filter logic.

3. Implementation: Projected Permissions

Now, let's apply our new IProtected interface to an entity. A common scenario is that permissions aren't granted to granular entities like Comment, but rather to their parent, like Post. A user who can access a Post can access all of its Comments.

Our IProtected interface's GetPermissionObjectId() method is designed for precisely this. We can implement it on our Comment entity to return its parent Post.Id. This is the exact same "projected" logic we used for multi-tenancy.

// In DAL.Context/Entity/Comment.cs\npublic partial class Comment : IProtected\n{\n\t// This is the implementation of IProtected.GetPermissionObjectId()\n\t[ExpressionMethod(nameof(GetPermissionObjectIdExpression))]\n\tpublic Ulid GetPermissionObjectId() => Post.Id;\n\n\tprivate static Expression<Func<Comment, Ulid>> GetPermissionObjectIdExpression()\n\t\t=> x => x.Post.Id;\n}

By using LinqToDB's [ExpressionMethod], we are telling the query engine that when it sees entity.GetPermissionObjectId() inside our Filter's InnerJoin, it should substitute the expression x => x.Post.Id.

The RLS filter logic will now transparently translate to: perm.ObjectId == comment.Post.Id. The DAL will automatically join the Comment table to the Post table to get the Id for the check, all without the developer ever thinking about it.

4. The Payoff: The Final Composed Query

We have now layered four distinct, automated behaviors onto our entities:

  1. ICreatable & IModifiable (Auditing)
  2. IRemovable (Soft-Delete)
  3. ITenanted (Multi-Tenancy)
  4. IProtected (Row-Level Security)

Let's see what happens when we execute a simple business operation, like modifying a comment. We add this code to our Program.cs:

// Set DB Context parameters\nctx.Attributes = new(\n\ttenant.Id,\n\tu.Id\n);\n\n// ...\n\n// Try to update a protected comment entity\nawait ctx.GetTable<Comment>()\n\t.Set(x => x.Content, \"asd123\")\n\t.ModifyAsync();

Remember, our custom ModifyAsync extension also automatically handles setting the ModifiedAt timestamp. The developer wrote a simple, two-line update. Our DAL, however, composes all the rules and generates the following SQL:

UPDATE\n\t[comment]\nSET\n\t[content] = 'asd123',\n\t-- 1. \"ModifyAsync\" Auditing\n\t[modified_at] = '2025-11-04 12:34:11.899'\nFROM\n\t[post] [a_Post],\n\t[user] [a_User],\n\t-- 4. RLS Join\n\t[permission] [b]\nWHERE\n\t-- 2. Soft-Delete Filters (on all joined tables)\n\t[a_User].[removed_at] IS NULL AND\n\t[a_Post].[removed_at] IS NULL AND\n\t[comment].[removed_at] IS NULL AND\n\t-- 3. Projected Multi-Tenancy Filter\n\t[a_User].[tenant_id] = X'019A4EDC4AD87A53C22679FA88F15885' AND\n\t-- Joins for projections\n\t[comment].[post_id] = [a_Post].[id] AND\n\t[a_Post].[user_id] = [a_User].[id] AND\n\t-- 4. Projected Row-Level Security Filter\n\t[b].[object_id] = [a_Post].[id] AND\n\t[b].[subject_id] = X'019A4EDC4ADE42CA89BA742103A7A723'

This query is a perfect demonstration of our architecture's power:

  1. Auditing: ModifyAsync correctly injected the modified_at SET clause.
  2. Soft-Delete: The [removed_at] IS NULL filter was applied to comment, post, AND user.
  3. Multi-Tenancy: The projected tenancy filter ([a_User].[tenant_id] = ...) is present, ensuring we don't even see comments from other tenants.
  4. Row-Level Security: The new IProtected filter joined the permission table ([b]) and correctly filtered on the Post ID ([b].[object_id] = [a_Post].[id]) and the current User ID ([b].[subject_id] = ...).

The developer is completely abstracted from this complexity. They just write business logic, and the DAL guarantees that auditing, soft-deletes, tenancy, and row-level security are all correctly and automatically enforced.

Conclusion

With the addition of composable row-level security, our enterprise DAL framework is now feature-complete on its core data integrity and security goals. We have proven that a thoughtful architecture built on interfaces, expression trees, and composable filters can eliminate entire classes of common bugs and security vulnerabilities.

By separating the definition of a behavior (the interface) from its implementation (the filter logic), we've created a system that is robust, testable, and incredibly easy to extend.

As always, the complete, working implementation for this post is available in our GitHub repository: https://github.com/ByteAether/EnterpriseDal/tree/part6

In our next post, we will address the final piece of our auditing system: automatically populating the CreatedBy and ModifiedBy fields, leveraging the new UserId we just added to our DbCtxAttributes.

", "date_published": "2025-11-24T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/announcing-byteaetherulid-132-net-10-support-and-optimized-design/", "url": "https://byteaether.github.io/2025/announcing-byteaetherulid-132-net-10-support-and-optimized-design/", "title": "Announcing ByteAether.Ulid 1.3.2: .NET 10 Support and Optimized Design", "content_html": "

We are excited to announce the release of ByteAether.Ulid version 1.3.2, which is now available on NuGet. This release introduces official support and dedicated binaries for the new .NET 10 platform, continuing our mission to provide the most performant, secure, and specification-compliant ULID (Universally Unique Lexicographically Sortable Identifier) implementation for the .NET ecosystem.

For those new to the project, ByteAether.Ulid provides a robust solution engineered to solve critical, often-overlooked issues in unique identifier generation, such as reliable overflow handling and secure monotonic generation.

Embracing .NET 10 and Finalizing C# 14 Features

The primary feature of version 1.3.2 is the inclusion of build artifacts specifically compiled against the .NET 10 SDK.

This release also solidifies our use of modern C# features. We were an early adopter of C# 14's "preview" features, specifically the field keyword. With the official release of .NET 10, we can now move from LangVersion "preview" to "latest," locking in the benefits this feature provides.

We use this feature in our GenerationOptions record to perform "fail-fast" validation. By validating a property once during its init assignment, we avoid all validation overhead on subsequent get operations or, more importantly, inside the Ulid.New() hot path.

For example, here is how we ensure that the Monotonicity property is always set to a valid enum value:

public MonotonicityOptions Monotonicity\n{\n\tget;\n\tinit => field = Enum.IsDefined(typeof(MonotonicityOptions), value)\n\t\t? value\n\t\t: throw new ArgumentOutOfRangeException(\n\t\t\tnameof(Monotonicity),\n\t\t\tvalue,\n\t\t\t\"Invalid monotonicity option.\"\n\t\t);\n} = MonotonicityOptions.MonotonicIncrement;

This pattern provides a critical optimization. The validation logic runs a single time when the GenerationOptions object is constructed, ensuring that any call to Ulid.New() using these options operates with pre-validated, zero-overhead configuration.

A Deeper Look: Our Multi-Targeting Strategy

Our NuGet package ships distinct, optimized artifacts for a wide range of targets:

This multi-targeting strategy is fundamental to our performance promise. We avoid the "lowest common denominator" approach of only shipping a .NET Standard 2.0 assembly.

When you install ByteAether.Ulid in a .NET 10 project, NuGet automatically selects the .NET 10 binary. This binary is compiled with all the latest JIT intrinsics and BCL improvements. Our codebase is rich with conditional compilation, allowing us to use the most powerful APIs available on each platform.

For example, our parsing (Ulid.Parse()) and stringification (ulid.ToString()) routines use optimized ReadOnlySpan<T> and MemoryMarshal APIs on modern targets, often leveraging SIMD for Base32 operations. This is a primary reason for our benchmark-leading performance, and it's an advantage you would lose if you only consumed a .NET Standard 2.0-compiled library.

How This Philosophy Shapes Our Core Features

This focus on performance and robust design is directly applied to the library's core features, particularly in how we handle security and reliability.

Reliable Overflow Handling

A critical, and often ignored, problem in ULID generation is the OverflowException that can occur during rapid-fire generation (many IDs in the same millisecond). While the 80-bit random component is massive, its starting value is random. This means a newly generated ULID could, by chance, have a random component already near its maximum value.

In this scenario, even a few increments (especially multi-byte random increments) could cause it to overflow. Our library programmatically prevents this by intelligently incrementing the 48-bit timestamp component by 1ms when an overflow is imminent, ensuring continuous, error-free generation.

We've written a detailed analysis of this problem and our solution here: Prioritizing Reliability: When Milliseconds Aren't Enough

Secure and Configurable Generation

The GenerationOptions class, which benefits from the field keyword mentioned earlier, is the key to our advanced security and performance tuning.

It allows you to configure MonotonicityOptions (e.g., MonotonicRandom1Byte) to defeat enumeration attacks, a vulnerability in simple +1 incrementors. It also lets you control the IRandomProvider used for both the initial random bytes and the monotonic increments. This design provides a nuanced balance, allowing for a fast, non-blocking pseudo-random incrementor while using a cryptographically secure generator for the initial seed.

You can read a deep dive into these options and their security implications here: ByteAether.Ulid v1.3.0: Enhanced ULID Generation Control and Security

Conclusion

The 1.3.2 release of ByteAether.Ulid, with its .NET 10 support, continues our commitment to providing a ULID implementation that is fast, secure, and meticulously spec-compliant.

Our aggressive multi-targeting strategy ensures your application uses the most optimized code for its platform, and our forward-looking adoption of C# features allows us to build a cleaner, faster, and more reliable API.

We encourage all architects and senior engineers who value performance, security, and specification-compliance to adopt ByteAether.Ulid for their identifier generation needs.

", "date_published": "2025-11-14T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-composable-multi-tenancy-filtering/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-composable-multi-tenancy-filtering/", "title": "Building an Enterprise Data Access Layer: Composable Multi-Tenancy Filtering", "content_html": "

In our previous articles, we established a robust foundation for our enterprise Data Access Layer (DAL). We began with a database-first approach using C# and LinqToDB, then implemented automated auditing for CreatedAt and ModifiedAt timestamps. Most recently, we engineered a powerful, composable global query filter system to handle soft-deletes transparently. This architecture was designed for extensibility, and in this post, we will leverage that investment to tackle one of the most critical cross-cutting concerns in enterprise software: multi-tenancy.

Multi-tenancy ensures that one tenant's data is strictly isolated from another's. A failure in this area is not a minor bug, but a critical security breach. Our goal is to enforce this isolation at the lowest possible level—the DAL—to guarantee that it is impossible for business logic to accidentally query data from the wrong tenant.

We will extend our entity behavior model by introducing an ITenanted interface, update our database context to be tenant-aware, and enhance our scaffolding interceptor for automation. Crucially, we will also solve the complex problem of "projected tenancy," where an entity's tenant affiliation is derived from a related entity.

The complete source code for this part is available on GitHub: https://github.com/ByteAether/EnterpriseDal/tree/part5.

1. Establishing Tenant Context in the DAL

Before our DAL can filter by tenant, it needs to know the identity of the current tenant for any given operation. This context is typically derived from an HTTP request, a message queue header, or a similar scope. We need a clean mechanism to pass this information down to our database context.

Previously, our IDbCtx was a simple marker interface. We will now extend it to carry request-scoped attributes, starting with the TenantId.

File: DAL.Base/IDbCtx.cs

namespace DAL.Base;\n\npublic interface IDbCtx : IDataContext\n{\n\tDbCtxAttributes Attributes { get; set; }\n\n\tpublic record DbCtxAttributes(Ulid TenantId = default);\n}

By adding the Attributes property, we provide a dedicated, extensible home for contextual data. We use a C# record for DbCtxAttributes to create a simple, immutable data carrier. The IDbCtx implementation (our DbCtx class) will be responsible for initializing this property. In a real application, this would be done via dependency injection at the beginning of a request scope, making the current TenantId available for the lifetime of that request.

For our demonstration, we will manually set this value in Program.cs before executing any queries. This simulates how a tenant's context would be established at the start of an operation.

File: App/Program.cs (Addition)

var tenantId = Ulid.New();\nctx.Attributes = new(TenantId: tenantId);

While we use a new Ulid here, the key is that this specific value will later appear in the generated SQL WHERE clause, confirming that our filter is using the context we've provided.

2. Defining the Tenancy Contract: The ITenanted Interface

Following the pattern established with ICreatable and IRemovable, we will define tenancy as an entity behavior through a new interface. This interface serves two purposes: it enforces a contract that the entity must expose a TenantId, and it hooks into our global filter system.

File: DAL.Base/EntityBehavior/ITenanted.cs

namespace DAL.Base.EntityBehavior;\n\n[EntityFilter<ITenanted>(nameof(Filter))]\npublic interface ITenanted : IEntity\n{\n\tUlid TenantId { get; }\n\n\tprivate static IQueryable<T> Filter<T>(IQueryable<T> q, IDbCtx ctx) where T : ITenanted\n\t\t=> q.Where(x => x.TenantId == ctx.Attributes.TenantId);\n}

This implementation is concise but powerful, directly leveraging our previous work:

  1. Ulid TenantId { get; }: This property contract mandates that any class implementing ITenanted must provide a TenantId getter. This is the key piece of information needed for filtering.
  2. [EntityFilter<ITenanted>(nameof(Filter))]: This is the connection to our composable filter architecture. We are associating the ITenanted interface with a specific filter-providing method named Filter.
  3. private static IQueryable<T> Filter(...): This method contains the filtering logic itself. It receives the current queryable (q) and the database context (ctx). It then appends a standard LINQ Where clause, comparing the entity's TenantId with the TenantId we stored in the context's Attributes.

When the DAL is initialized, our MappingSchema.ApplyEntityFilters<DbCtx>() helper will discover this attribute and automatically apply this Where clause to every query against any entity that implements ITenanted.

3. Automating Interface Scaffolding

To maintain our database-first philosophy, we must ensure that our generated entity classes automatically implement the ITenanted interface if their corresponding database tables contain a tenant_id column. This task is perfectly suited for our custom scaffolding interceptor.

We will add a simple check to the interceptor's logic.

File: DAL.ScaffoldInterceptor/Interceptor.cs (Addition)

// ITenanted\nvar tenantIdField = entityModel.Columns.FirstOrDefault(x =>\n\tx.Property.Name == nameof(ITenanted.TenantId)\n);\nif (tenantIdField is not null)\n{\n\taddedInterfaces.Add(typeof(ITenanted));\n}

This code snippet inspects the columns of the entity being scaffolded. If it finds a column that maps to a property named TenantId, it adds ITenanted to the list of interfaces for the generated partial class. With this change, entities like user which have a direct tenant_id column will be automatically protected by our tenancy filter without any manual intervention.

4. The Advanced Case: Projected Tenancy

The true test of our architecture is not in handling direct foreign keys, but in its ability to manage more complex, indirect relationships. Consider the post table. Its schema does not contain a tenant_id column. However, a post is unequivocally owned by a tenant because it belongs to a user, and that user belongs to a tenant. The Post entity's tenancy is therefore projected from its parent User entity.

Our DAL must enforce this projected relationship as a filtering rule. We need to make the Post entity implement ITenanted, but how do we implement the TenantId property when there is no backing column?

The solution lies in creating an expression-based property that LinqToDB can translate directly into SQL. We achieve this by manually creating a partial class for Post and using the [ExpressionMethod] attribute.

File: DAL.Context/Entity/Post.cs

namespace DAL.Context.Entity;\n\npublic partial class Post : ITenanted\n{\n\t[ExpressionMethod(nameof(GetTenantIdExpression))]\n\tpublic Ulid TenantId => GetTenantIdExpression().Compile()(this);\n\n\tprivate static Expression<Func<Post, Ulid>> GetTenantIdExpression()\n\t\t=> x => x.User.TenantId;\n}

Let's dissect this implementation, as it is central to the power of our DAL:

  1. public partial class Post : ITenanted: We explicitly declare that Post conforms to the ITenanted contract. We do this manually because our scaffolder did not find a TenantId column.
  2. private static Expression<Func<Post, Ulid>> GetTenantIdExpression(): Instead of a simple value, we define the logic for retrieving the TenantId as an expression tree. The expression x => x.User.TenantId is not just executable C# code, but it is a data structure that represents the navigation from a Post (x), through its User property, to that user's TenantId. LinqToDB is designed to parse these expression trees and convert them into the corresponding SQL, which in this case involves an INNER JOIN to the user table.
  3. [ExpressionMethod(nameof(GetTenantIdExpression))]: This LinqToDB attribute is the magic that connects the property to the expression tree. When LinqToDB encounters post.TenantId inside a LINQ query (like the one in our ITenanted.Filter method), this attribute instructs it to substitute the SQL translation of the GetTenantIdExpression method's return value.
  4. public Ulid TenantId => GetTenantIdExpression().Compile()(this);: This line handles the case where the TenantId property is accessed on an already-materialized Post object in C# memory, outside of a database query. In this scenario, LinqToDB is not involved, and the [ExpressionMethod] attribute has no effect. We must provide a valid C# implementation. We take our expression tree, .Compile() it into a runnable delegate, and then invoke it for the current Post instance (this). For performance-critical code, this compiled delegate could be cached, but we have kept it simple for clarity.

This elegant solution allows the TenantId property to work seamlessly both within database queries (translating to SQL) and on in-memory objects (executing as C# code).

5. Observing Composable Filters in Action

We have now implemented all the necessary components. The ITenanted interface defines the filter, the scaffolder applies it to entities with a direct TenantId, and we have manually implemented it for the Post entity using a projected property.

Let's verify that it all works together. We add a simple query to our application's entry point and inspect the generated SQL.

File: App/Program.cs (Addition)

// Set DB Context parameters\nctx.Attributes = new(\n\ttenant.Id\n);\n\n// ...\n\n// Get posts that are tenanted through a user\nawait ctx.GetTable<Post>()\n    .ToListAsync();\n\nConsole.WriteLine(ctx.LastQuery);

Executing this code produces the following SQL:

SELECT\n    [x].[id],\n    [x].[user_id],\n    [x].[title],\n    [x].[content],\n    [x].[created_at],\n    [x].[modified_at],\n    [x].[removed_at]\nFROM\n    [post] [x]\n\tINNER JOIN [user] [a_User] ON [x].[user_id] = [a_User].[id]\nWHERE\n    [a_User].[removed_at] IS NULL\n    AND [a_User].[tenant_id] = X'0199DD7C56CCCB4521596E010C8BED67'\n    AND [x].[removed_at] IS NULL

This output is a perfect confirmation of our architecture's capabilities. A simple call to ToListAsync() on the Post table has triggered a cascade of automated, composed filters:

This demonstrates the power of composable filters. Each rule (soft-delete, tenancy) is defined independently, yet the system correctly combines them into a single, comprehensive WHERE clause, ensuring data integrity and security automatically.

Conclusion

In this article, we successfully implemented automated, non-bypassable multi-tenancy filtering by building upon our existing architectural foundation. By defining tenancy as a behavioral interface and leveraging LinqToDB's ExpressionMethod, we were able to handle both direct and complex projected tenancy relationships with equal elegance. The final generated SQL proves that our composable filter system is robust, correctly layering multi-tenancy rules on top of soft-delete rules for all entities involved in a query.

This approach significantly reduces the risk of data leakage and removes the burden of security filtering from the business logic developer, fulfilling a core requirement of a true enterprise DAL.

You can review the complete implementation for this stage of the project on GitHub: https://github.com/ByteAether/EnterpriseDal/tree/part5.

Moving forward, we will tackle an even more granular security challenge: a generic, row-based permission system. The goal is to create a mechanism where any entity can be protected, requiring users to have explicit permission for that specific row in the database. If a user lacks the required permission, the entity will be automatically and transparently filtered from all query results, abstracting this complex conditional logic away from the business layer.

", "date_published": "2025-11-11T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-soft-delete/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-soft-delete/", "title": "Building an Enterprise Data Access Layer: Automated Soft-Delete", "content_html": "

Welcome to the fourth installment of our series on building a robust, enterprise-grade Data Access Layer (DAL) in C#. In our previous post, we established a powerful automated auditing system to track entity creation and modification times (CreatedAt/ModifiedAt). Now, we will tackle another critical enterprise requirement: soft-delete.

In many systems, physically deleting data (DELETE FROM ...) is unacceptable. Regulatory compliance, audit trails, and the simple need for data recovery demand that records are preserved, even when a user "deletes" them. The standard solution is soft-deletion, where a record is marked as inactive instead of being removed.

This post will detail how we implement a fully automated, transparent, and secure soft-delete mechanism. We will not only intercept delete operations but also ensure that soft-deleted records are automatically filtered out of all read operations, including through entity associations.

The complete code for this part of the series is available on our GitHub repository: https://github.com/ByteAether/EnterpriseDal/tree/part4.

The Soft-Delete Contract: The IRemovable Interface

Consistent with our architecture, we define entity behaviors through interfaces. For soft-delete, we introduce the IRemovable interface in the DAL.Base/EntityBehavior namespace.

namespace DAL.Base.EntityBehavior;\n\n[EntityFilter<IRemovable>(nameof(Filter))]\npublic interface IRemovable : IEntity\n{\n\tDateTime? RemovedAt { get; set; }\n\n\tprivate static IQueryable<T> Filter<T>(IQueryable<T> q, IDbCtx _)\n\t\twhere T : IRemovable\n\t\t=> q.Where(x => x.RemovedAt == null);\n}

It introduces a nullable RemovedAt property. When this property is NULL, the record is active. When it contains a timestamp, the record is considered deleted. Any entity that should support soft-deletion will implement this interface.

Our database schema has been updated accordingly, with a nullable removed_at DATETIME column added to all tables.

The Core Challenge: Global and Composable Query Filters

Implementing soft-delete presents two significant challenges that a naive approach cannot solve:

  1. Ubiquitous Filtering: Soft-deleted records must be hidden from every query by default. Manually adding a Where(x => x.RemovedAt == null) clause to every data retrieval call is brittle, error-prone, and violates the core principle of our DAL, which is to handle cross-cutting concerns automatically.
  2. Filtering Through Associations: This is a more subtle but critical problem. Imagine fetching a Post entity and then accessing its associated Comments collection. The DAL must ensure that any soft-deleted comments are automatically filtered from that collection. A simple WHERE clause on the initial Post query will not propagate to lazily or eagerly loaded associations.

The solution is to use Global Query Filters, a feature supported by ORMs like LinqToDB and Entity Framework Core. A global filter is a WHERE condition that the ORM automatically appends to every query for a specific entity type.

However, these ORMs typically allow only a single filter expression per entity. Our architectural vision requires multiple, independent behaviors (soft-delete, multi-tenancy , row-level security), each potentially contributing its own filter. An entity could implement IRemovable and ITenanted, requiring two separate filters to be applied simultaneously.

A Composable Architecture for Global Filters

To overcome this limitation, we have engineered a system to aggregate multiple filter expressions into a single, cohesive filter for each entity. This logic is encapsulated within the new DAL.Base/EntityFilter folder.

The system works in three main steps:

  1. Declaration: We use a custom attribute, [EntityFilter<T>(nameof(MethodName))], to associate a filtering method with an interface or class. As seen in the IRemovable interface code above, we declare that its filter logic is located in a static method named Filter. This makes each behavioral interface self-contained.

  2. Discovery & Aggregation: We created a helper that, upon database context initialization, scans all our DAL entities (classes implementing IEntity). For each entity, it traverses its entire inheritance tree including all implemented interfaces. It collects all lambda expressions from the filter methods declared via the EntityFilter attribute. These individual expressions are then combined into a single, larger lambda expression using Expression.AndAlso.

  3. Application: The final, aggregated expression is then applied as a single global filter to the entity using LinqToDB's mapping schema. We trigger this entire process with a single call in our database context DbCtx.InitDataContext():

    ...\nMappingSchema.ApplyEntityFilters<DbCtx>();\n...

This architecture is incredibly flexible. It allows us to define filtering logic alongside the relevant behavior (like IRemovable or, in the future, ITenanted), and the DAL automatically composes these rules at runtime.

Integrating Soft-Delete into the DAL

With the filter aggregation system in place, we can now integrate the soft-delete logic.

1. Automated Scaffolding (Interceptor.cs)

Our custom scaffolding interceptor is extended to automatically add the IRemovable interface to any generated entity partial class that has a corresponding RemovedAt column in the database. This ensures our C# entity model stays perfectly in sync with our database-first philosophy.

2. Overriding Delete Operations (CrudExtensions.cs)

Next, we introduce new RemoveAsync extension methods that replace direct calls to LinqToDB's DeleteAsync. These methods contain the core soft-delete logic.

public static Task<int> RemoveAsync<T>(\n\tthis IQueryable<T> source,\n\t CancellationToken cancellationToken = default\n) where T : class\n{\n\t// Check if the entity type T implements IRemovable\n\tif (typeof(IRemovable).IsAssignableFrom(typeof(T)))\n\t{\n\t\t// If so, build an UPDATE query instead of a DELETE\n\t\treturn source\n\t\t\t.Set(x => ((IRemovable)x).RemovedAt, DateTime.UtcNow)\n\t\t\t.ModifyAsync(cancellationToken); // Reuse our existing ModifyAsync\n\t}\n\n\t// Otherwise, perform a hard delete\n\treturn source.DeleteAsync(cancellationToken);\n}

This implementation is transparent to the developer. When they call RemoveAsync on a query for an IRemovable entity, the DAL automatically translates it into an UPDATE statement that sets the RemovedAt timestamp. By cleverly chaining this into our existing ModifyAsync method, we also get ModifiedAt auditing for free.

Verification: Seeing It in Action

Let's validate our implementation. We can write a simple test in Program.cs to remove a user and inspect the generated SQL.

The output confirms our success. The DELETE operation was transparently converted into an UPDATE that sets both removed_at and modified_at. Crucially, notice the WHERE [user].[removed_at] IS NULL clause. Our global filter is automatically applied even to the UPDATE operation itself, ensuring we don't try to re-delete an already deleted record. Any subsequent SELECT query for this user will now automatically fail to find it, as it will not satisfy the global filter.

Conclusion and Next Steps

In this post, we have successfully implemented a comprehensive and automated soft-delete system. By building a composable global filter architecture, we have not only solved the immediate challenge but also laid a robust foundation for future DAL capabilities. Our DAL now automatically handles soft-deletes and ensures data integrity across all queries and associations, preventing common security and consistency issues.

The powerful filter system we built is now ready for its next major task: multi-tenancy. In the next article, we will leverage this exact architecture to automatically isolate data between different tenants, preventing one of the most critical types of data leakage in SaaS applications.

For a deeper dive into the implementation, please explore the source code on our GitHub repository: https://github.com/ByteAether/EnterpriseDal/tree/part4.

", "date_published": "2025-11-04T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-auditing/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-automated-auditing/", "title": "Building an Enterprise Data Access Layer: Automated Auditing", "content_html": "

In our previous posts, we laid the foundation for our enterprise Data Access Layer (DAL). We established our core principles: a database-first philosophy, the use of C# and LinqToDB, and a commitment to automating cross-cutting concerns to enhance security and data integrity. We also structured our project, set up a scaffolding interceptor to enrich our auto-generated entities, and chose ULIDs as our primary keys for their performance benefits.

This post will build directly on that foundation. We will implement one of the most common and critical DAL features: automated audit fields. Specifically, we will ensure that CreatedAt and ModifiedAt timestamps are automatically and correctly populated for every entity, without requiring any manual intervention from the developer using the DAL. This automation is key to preventing data inconsistencies and reducing cognitive load on engineers.

A Philosophical Shift: Technical vs. Business-Logical CRUD

Before diving into the implementation, it is important to introduce a conceptual distinction that will guide our design. In software development, we often use the acronym CRUD (Create, Read, Update, Delete) to describe data operations. However, these terms often conflate low-level database actions with high-level business logic.

To build a truly robust DAL, we must separate these concepts:

Our goal is to expose the business-logical methods (CREATE, MODIFY, REMOVE) to the application's business logic layer, while discouraging the direct use of the technical methods (INSERT, UPDATE, DELETE). By doing so, we guarantee that all our automated rules (such as setting audit fields, enforcing soft-delete, or applying multi-tenancy filters) are always executed.

For now, this separation is a matter of developer discipline. In a future post, we may explore building a custom Roslyn (static code) analyzer that warns an engineer if they attempt to call a technical CRUD method directly from a higher-level service, thus enforcing this architectural pattern at compile time.

Step 1: Defining Behavior with ICreatable and IModifiable

To apply auditing logic selectively, we first need a way to identify which entities require it. We achieve this by defining "entity behavior" interfaces. For this feature, we will add two new interfaces to our DAL.Base/EntityBehavior directory:

These interfaces are simple contracts. An entity implementing ICreatable guarantees it has a CreatedAt property. This allows us to write generic extension methods that can operate on any entity fulfilling this contract.

Step 2: Evolving the Database and Scaffolding

Following our database-first approach, the next step is to update our database schema. We add created_at DATETIME not null and modified_at DATETIME not null columns to all four of our tables: tenant, user, post, and comment.

With the database schema updated, we can enhance our scaffolding interceptor. In addition to adding the IEntity and IIdentifiable<Ulid> interfaces as it did before, the interceptor will now inspect the properties of each table during code generation.

After running the scaffolding tool, our generated User entity, for example, will now have a signature like this in DbCtx.generated.cs:

public partial class User : IEntity, IIdentifiable<Ulid>, ICreatable, IModifiable\n{\n\t\t// ... generated properties\n}

This powerful combination of a database-first approach and an intelligent interceptor ensures our C# entity definitions always stay synchronized with the database schema and its intended behaviors, without manual intervention.

Step 3: Implementing the Business Logic in CrudExtensions.cs

Now that our entities automatically implement the correct interfaces, we can create the extension methods that constitute our business-logical CREATE and MODIFY operations. We'll place these in a new file, DAL.Base/CrudExtensions.cs.

The CreateAsync Method

The CreateAsync method is an extension on DbCtx. It takes either an entity object or an enumerable of them, automatically sets the audit fields, and then calls the underlying technical INSERT operation.

public static async Task<long> CreateAsync<T>(\n\tthis DbCtx ctx,\n\tIEnumerable<T> entities,\n\tCancellationToken cancellationToken = default\n) where T : class, IEntity\n\t=> (\n\t\tawait ctx.BulkCopyAsync(\n\t\t\tentities.Select(e =>\n\t\t\t\t{\n\t\t\t\t\tif (e is ICreatable creatable && creatable.CreatedAt == default)\n\t\t\t\t\t{\n\t\t\t\t\t\tcreatable.CreatedAt = DateTime.UtcNow;\n\t\t\t\t\t}\n\n\t\t\t\t\tif (e is IModifiable updateable && updateable.ModifiedAt == default)\n\t\t\t\t\t{\n\t\t\t\t\t\tupdateable.ModifiedAt = DateTime.UtcNow;\n\t\t\t\t\t}\n\n\t\t\t\t\treturn e;\n\t\t\t\t}\n\t\t\t),\n\t\t\tcancellationToken\n\t\t)\n\t).RowsCopied;\n\npublic static Task<long> CreateAsync<T>(\n\tthis DbCtx ctx,\n\tT entity,\n\tCancellationToken cancellationToken = default\n) where T : class, IEntity\n\t=> ctx.CreateAsync([entity], cancellationToken);

By checking the interfaces, these methods correctly handle any entity. If an entity only implements ICreatable, only CreatedAt is set. If it implements both, both timestamps are set.

The ModifyAsync Method

The ModifyAsync methods are slightly different. They are extensions on either DbCtx or IQueryable<T>, designed to work with LinqToDB's fluent update syntax.

public static Task<int> ModifyAsync<T>(\n\tthis DbCtx ctx,\n\tT entity,\n\tCancellationToken cancellationToken = default\n) where T : class, IEntity\n{\n\tif (entity is IModifiable updateable)\n\t{\n\t\tupdateable.ModifiedAt = DateTime.UtcNow;\n\t}\n\n\treturn ctx.GetTable<T>().UpdateOptimisticAsync(entity, cancellationToken);\n}\n\npublic static Task<int> ModifyAsync<T>(\n\tthis IUpdatable<T> source,\n\tCancellationToken cancellationToken = default\n) where T : class\n{\n\tif (typeof(IModifiable).IsAssignableFrom(typeof(T)))\n\t{\n\t\tsource = source.Set(\n\t\t\tx => Sql.Property<DateTime>(x, nameof(IModifiable.ModifiedAt)),\n\t\t\tDateTime.UtcNow\n\t\t);\n\t}\n\n\treturn source.UpdateAsync(cancellationToken);\n}\n\npublic static Task<int> ModifyAsync<T>(\n\tthis IQueryable<T> source,\n\tCancellationToken cancellationToken = default\n) where T : class\n\t=> source.AsUpdatable().ModifyAsync(cancellationToken);

These methods cleverly inspect the generic type T. If T implements IModifiable, it injects a .Set() call to update the ModifiedAt field with the current timestamp before executing the final UpdateAsync() call or assigns the current timestamp to an entity's property. This ensures that any update operation expressed through this fluent API is properly audited.

A Note on IQueryExpressionInterceptor

You might be wondering if there is a more integrated way to achieve this within LinqToDB, rather than creating wrapper extension methods. LinqToDB provides a IQueryExpressionInterceptor interface, which is designed to intercept every query expression before its translation into SQL. In theory, this would be the perfect place to inject our auditing logic.

Unfortunately, as of this writing, the implementation of this interceptor is not consistently triggered across all operation types. For example, it does not fire for INSERT operations or for calls to IDataContext.UpdateAsync<T>(), although it does trigger for the fluent IUpdatable<T>.UpdateAsync() method. This inconsistency makes it unsuitable for building a reliable, all-encompassing auditing system.

Should this behavior be fixed in a future version of LinqToDB to reliably intercept all query expressions, it would offer a more elegant solution. We could then centralize our auditing logic in the interceptor and use the standard InsertAsync and UpdateAsync methods directly. This would eliminate the need for our custom CreateAsync and ModifyAsync wrappers and the potential need for a static code analyzer to enforce their use. If such an update occurs, we will certainly explore it in a follow-up article.

Seeing it in Action

With all the pieces in place, we can verify that our implementation works as expected. We can write a small test in Program.cs to create and then modify a User entity. Notice that our C# code never explicitly provides a value for CreatedAt or ModifiedAt.

await ctx.BeginTransactionAsync();\n\n// Create entity\nvar u = new User\n{\n\t\tId = Ulid.New(),\n\t\tUsername = \"asd123\"\n};\nawait ctx.CreateAsync(u);\n\nConsole.WriteLine(ctx.LastQuery);\n\n// Modify entity\nawait ctx.GetTable<User>()\n\t\t.Where(x => x.Id == u.Id)\n\t\t.Set(x => x.Username, \"asd1234\")\n\t\t.ModifyAsync();\n\nConsole.WriteLine(ctx.LastQuery);\n\nawait ctx.RollbackTransactionAsync();

Executing this code produces the following SQL, captured from LinqToDB's LastQuery property:

INSERT INTO [user]\n(\n\t\t\t\t[id],\n\t\t\t\t[tenant_id],\n\t\t\t\t[username],\n\t\t\t\t[created_at],\n\t\t\t\t[modified_at]\n)\nVALUES\n(X'0199C31CC424CFC7293147E8A6AE578C',X'00000000000000000000000000000000','asd123','2025-10-08 09:17:46.472','2025-10-08 09:17:46.473')\n\nUPDATE\n\t\t\t\t[user]\nSET\n\t\t\t\t[username] = 'asd1234',\n\t\t\t\t[modified_at] = '2025-10-08 09:17:46.506'\nWHERE\n\t\t\t\t[user].[id] = X'0199C31CC424CFC7293147E8A6AE578C'

The output confirms our success. The INSERT statement correctly includes values for both created_at and modified_at. The UPDATE statement automatically includes the SET clause for modified_at. Our DAL now handles these audit fields transparently, reliably, and without any developer effort.

The complete implementation for this part of the series can be found on GitHub: https://github.com/ByteAether/EnterpriseDal/tree/part3.

Conclusion and Next Steps

In this post, we successfully implemented automated auditing for creation and modification timestamps. We introduced the critical distinction between technical and business-logical CRUD operations, defined entity behaviors using interfaces, and leveraged our scaffolding interceptor to automate their implementation. Finally, we created generic extension methods that apply auditing rules consistently across all relevant entities.

In the next episode, we will tackle another core DAL capability: Soft-Delete. This feature is similar in that it will rely on a new interface, but it introduces an additional layer of complexity: we must also implement a global query filter to automatically exclude soft-deleted records from all read operations, a crucial step in preventing data leakage and ensuring application correctness.

", "date_published": "2025-10-24T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-database-and-code-structure/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-database-and-code-structure/", "title": "Building an Enterprise Data Access Layer: Database and Code Structure", "content_html": "

Welcome back to our series on building a robust, enterprise-grade Data Access Layer (DAL) with C# and LinqToDB. In our previous post, we established the core principles and non-negotiable capabilities our DAL must possess, including automatic multi-tenancy filtering, soft-delete, and auditing. Now, we'll begin laying the foundation for this powerful system by defining the database schema and the initial code structure.

This post will explore the deliberate technical choices we have made for this project's foundation. While these decisions may seem straightforward at first glance, each one serves a specific purpose in building a scalable, secure, and maintainable data layer that abstracts complexity away from the business logic.

The Database-First Philosophy

Before we define the structure, let's address the approach. This project is firmly rooted in a database-first philosophy. In my view, the database is a core engineering artifact and should be treated as such, designed and managed using purpose-built database engineering tools. The schema is the source of truth, and our code should be a faithful reflection of that schema.

This approach ensures that database constraints, indexes, and data types are optimized for the database engine itself, rather than being an afterthought generated by an Object-Relational Mapper (ORM). It provides a clear separation of concerns, allowing database administrators and backend engineers to collaborate on a well-defined and performant schema.

Our Core Data Model

For this series, we will build a simple data model to represent a multi-tenant blogging platform. The model consists of four essential tables, each with a clear purpose and relationship to the others.

The relationships are as follows:

This simple structure allows us to demonstrate how the DAL can automatically enforce rules like multi-tenancy and permissions by navigating these relationships.

The Power of the ULID

A cornerstone of our database design is the choice of ULID (Universal Unique Lexicographically Sortable Identifier) as the primary key for all tables. A ULID is a 128-bit identifier that combines the uniqueness of a UUID with the sortable properties of a timestamp.

In our C# code, we will use the ByteAether.Ulid library. This library provides a native Ulid type that is specifically designed for this purpose. In the database, ULIDs are stored as 128-bit binary, which is the most efficient representation.

The key advantage of ULIDs is their sortability. Because the first 48 bits of a ULID are a timestamp, when you sort a column of ULIDs, you are also sorting by the order of creation. This is incredibly beneficial for performance on append-only tables, as new records are always written to the end of the table, minimizing fragmentation. It also makes it easy to find the most recent records without a separate timestamp column.

For databases like SQLite, this property allows us to declare our tables WITHOUT ROWID. This SQLite-specific feature, when used with a ULID primary key, removes the default internal 64-bit integer row identifier, making the ULID the sole source of record identity and order. Since ULIDs are inherently sortable, the table is naturally clustered by creation time, which optimizes read operations on the most recent data.

Here is the full SQL schema for our four tables:

create table tenant\n(\n\tid   ulid        not null\n\t\tprimary key,\n\tname VARCHAR(64) not null\n\t\tunique\n)\n\twithout rowid;\n\ncreate table user\n(\n\tid        ulid        not null\n\t\tprimary key,\n\ttenant_id ulid        not null\n\t\treferences tenant,\n\tusername  VARCHAR(64) not null\n)\n\twithout rowid;\n\ncreate table post\n(\n\tid      ulid        not null\n\t\tprimary key,\n\tuser_id ulid        not null\n\t\treferences user,\n\ttitle   varchar(64) not null,\n\tcontent text        not null\n)\n\twithout rowid;\n\ncreate table comment\n(\n\tid      ulid not null\n\t\tprimary key,\n\tpost_id ulid not null\n\t\treferences post,\n\tcontent text not null\n)\n\twithout rowid;

The Scaffolding and Code Structure

With our database schema defined, we now turn to the C# code. We will use LinqToDB's built-in scaffolding tool to generate our C# entity classes and a database context. However, we will enhance this process with a custom scaffolding interceptor.

Our core code structure will be organized into a few key areas:

DAL.Base

This namespace holds the fundamental interfaces that all of our entities may inherit from. We're keeping it simple for now, with two interfaces:

DAL.Context

This namespace contains our database context and all generated entity classes. We've made a key architectural decision here: all generated classes will reside in a single file named DbCtx.generated.cs. This simplifies the regeneration process, as you only need to update a single file when the database schema changes.

A key piece of the DbCtx.cs file is the partial void InitDataContext() method. This method, which is automatically called by the generated constructor in DbCtx.generated.cs, is where we will define the runtime type mapping for our Ulids. This ensures that when LinqToDB reads a ulid type from the database (as a byte array), it is correctly converted into a Ulid C# object, and vice versa. This runtime mapping is the counterpart to the design-time mapping handled by our ScaffoldInterceptor.

Note on Companion Libraries:

Following the release of ByteAether.Ulid v1.4.0, official integration packages (including ByteAether.Ulid.linq2db) are now available to encapsulate ORM type conversions. While production applications can leverage DataOptions.RegisterUlid() to simplify setup, this repository retains the explicit mapping code inside InitDataContext(). Keeping this logic visible eliminates "framework magic" and clarifies the precise conversion mechanisms LinqToDB relies on behind the scenes.

DAL.Context.Entity

This namespace is where we will create additional partial classes to extend the functionality of our generated entity classes. For example, we could create a Post.cs file to add a convenience method for getting the post's user from the database.

DAL.ScaffoldInterceptor

This is the "secret sauce" of our scaffolding process. We will create a class that implements ScaffoldInterceptors. This interceptor allows us to hook into the scaffolding process and customize it.

Our custom interceptor will perform two key functions:

  1. Interface Injection: It will automatically add our IEntity and IIdentifiable<Ulid> interfaces to the generated partial entity classes.
  2. ULID Type Mapping: It will intercept any database column with the ulid type and ensure it is properly mapped to the Ulid type from our C# library in the generated code.
  3. Association Naming Convention: We will use the interceptor to enforce a strict and predictable naming convention for association properties. For One-To-Many relationships, we adopt a simple 's' suffix (e.g., a Post entity will have a property named Comments, not Commentaries or Commentss). This deliberate choice avoids the ambiguity and confusion that can arise from irregular pluralization in English (e.g., avoiding a scenario where a Mouse entity might generate a Mice collection). For Many-To-One relationships, we simply use the entity's singular name (e.g., a Comment entity will have a Post property).

The use of partial classes is a core tenet of this approach. It gives us the best of both worlds: we have the convenience of an automatically generated code base that stays in sync with our database schema, while also having the freedom to extend and customize our models and database context without ever touching the generated code.

Looking Ahead

In this post, we've defined the bedrock of our enterprise DAL, from the database schema and the choice of ULID primary keys to the core code structure and our enhanced scaffolding process. We started simple, without implementing audit fields or a permission system. The project's root also includes a simple Program.cs file used for demonstration. This utility file connects to our SQLite database and runs simple LINQ queries (like selecting all users). Crucially, we use it to inspect the generated SQL from our LINQ expressions, rather than executing the queries and reviewing the result set. This allows us to verify that our DAL is correctly translating our C# code into performant SQL.

The full source code for this post, including the scaffolding interceptor and the completed project structure, is available at GitHub.

In the next post, we will dive into the details of implementing the crucial audit fields (CreatedAt, CreatedBy, ModifiedAt, and ModifiedBy) and the logic to automatically populate them during data operations. We'll show how the foundation we've built here makes this complex, cross-cutting concern remarkably simple to implement.

", "date_published": "2025-10-07T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-the-foundation/", "url": "https://byteaether.github.io/2025/building-an-enterprise-data-access-layer-the-foundation/", "title": "Building an Enterprise Data Access Layer: The Foundation", "content_html": "

In enterprise software development, the Data Access Layer (DAL) serves as the critical interface between an application's business logic and its underlying data store. Beyond basic CRUD operations, a robust DAL is fundamental for ensuring data integrity, security, and operational consistency. It proactively addresses cross-cutting concerns, preventing inconsistencies, defects, and potential security vulnerabilities that arise when these responsibilities are left to the business logic.

This series will detail the design and implementation of an enterprise-level Data Access Layer in C#, with a primary focus on LinqToDB. We'll demonstrate how to embed essential cross-cutting concerns directly within the DAL, ensuring their consistent application without reliance on manual enforcement within the business logic. LinqToDB's capabilities, particularly its strong support for advanced SQL features and expression tree manipulation, make it an excellent choice for building a highly customized and efficient DAL. We outline the fundamental capabilities such a DAL must provide, establishing the framework for subsequent practical implementations.

The Necessity of an Automated, Error-Resistant DAL

The lifecycle of an enterprise application involves continuous data creation, modification, and retrieval by various users and automated processes. Without a centralized, opinionated, and highly capable DAL, developers are tasked with manually implementing critical data-related behaviors across numerous service methods and business logic components. This decentralized approach is susceptible to human error, such as overlooking a filter, omitting an audit field, or mismanaging a soft-delete operation. Such oversights can lead to:

The core premise of this series is that these capabilities must be inherent to the DAL. The DAL should function as a guardian, automatically applying these rules and transformations, thereby offloading this responsibility from the business logic and ensuring consistent, secure, and reliable data interaction.

Core Capabilities of an Enterprise Data Access Layer

We'll now examine the critical capabilities an enterprise DAL should provide, abstracting them from the application's business logic.

1. CreatedAt/ModifiedAt Auditing

Maintaining an audit trail of when data records were created and last modified is a fundamental requirement for most enterprise applications. This temporal auditing provides insights for debugging, compliance, and understanding data evolution. Manually setting CreatedAt and ModifiedAt timestamps in the business logic for every INSERT and UPDATE operation is repetitive and prone to omission.

A sophisticated DAL should automate this process. Upon entity insertion, the DAL should automatically populate its CreatedAt property with the current timestamp. Similarly, upon any update to an existing entity, the DAL should automatically update its ModifiedAt property. This ensures that every record consistently carries its creation and last modification timestamps, without explicit action required from the business logic developer. This automation guarantees comprehensive temporal auditing across the entire dataset.

2. CreatedBy/ModifiedBy Auditing

Extending temporal auditing, it is often critical to identify the user responsible for a particular data change. This "who" information, typically represented by a User ID, enhances accountability and traceability within the audit trail. As with CreatedAt/ModifiedAt, manually populating CreatedBy and ModifiedBy fields in the business logic is a source of potential errors.

The enterprise DAL should seamlessly manage this. For new entities, it should automatically set the CreatedBy property to the ID of the current user. For updates, it should update the ModifiedBy property. The current User ID must be settable for the scope of an entire API request or a specific unit of work. The DAL will then automatically retrieve and apply this contextual User ID to the appropriate fields during persistence operations. This mechanism ensures that every data change is attributed to the responsible user, providing a complete and reliable audit history.

3. Soft-Delete

In many enterprise scenarios, physical data deletion is undesirable due to auditing requirements, data recovery needs, or the necessity of maintaining historical context. Soft-delete addresses this by marking a record as logically "deleted" rather than physically removing it.

An enterprise DAL should automate this process. When the business logic initiates a "delete" operation, the DAL should intercept this call. Instead of issuing a DELETE SQL command, it should update a designated DeletedAt property (e.g., a DateTimeOffset? nullable timestamp) on the entity. Furthermore, all standard read operations performed through the DAL must automatically filter out entities where DeletedAt is not null, unless explicitly overridden (e.g., for administrative recovery tools). This ensures that the business logic perceives soft-deleted entities as non-existent without requiring manual WHERE DeletedAt IS NULL clauses in every query. This capability is essential for data integrity and provides a safeguard against accidental data loss.

4. TenantId Filtering (Multi-Tenancy)

Multi-tenancy, where a single application instance serves multiple distinct organizations (tenants), is a common architectural pattern in SaaS solutions. Strict data isolation is critical: Tenant A must never access Tenant B's data. Enforcing this isolation is exceptionally challenging if left to the business logic, as every query, join, and relationship traversal would require manual TenantId filtering. A single missed filter can lead to catastrophic data leakage.

Our enterprise DAL must provide automatic TenantId filtering. For entities designated as "tenant-owned," the DAL should automatically inject a WHERE TenantId = [CurrentTenantId] clause into all read operations. Crucially, this filtering must propagate throughout all associations. If a query involves joining across multiple tables, the DAL must ensure the TenantId filter is applied appropriately at each relevant level to prevent data from other tenants from being inadvertently included. Similar to the CreatedBy/ModifiedBy capability, the TenantId must be settable for the scope of the current API request or unit of work, enabling the DAL to transparently apply it to all relevant data operations. This capability is a cornerstone of security and data isolation in multi-tenant architectures.

5. Entity-Based-Permissions (Row-Level Security)

Beyond multi-tenancy, many applications require more granular control over data visibility, often termed row-level security or entity-based permissions. This allows specific users to view certain entities while restricting others, even within the same tenant. For example, in a project management system, a user might only access tasks assigned to them or their team. Implementing this within the business logic is complex, often resulting in deeply nested conditional logic and a high risk of oversight.

An advanced DAL should fully enforce entity-based permissions. Entities themselves will define whether they are protected by the permission system. For protected entities, they will also define how the ID, upon which the permission system will operate, is derived. The DAL will then automatically apply these permission checks during all read operations, effectively filtering out rows that the current user is not authorized to see. This abstraction ensures that the business logic operates under the assumption that it only receives data the user is permitted to view, significantly simplifying development and bolstering security. The current User ID, set for the scope of the API request or unit of work, will provide the context for these permission evaluations.

6. Projected Data Support for Contextual Filtering

Both TenantId filtering and entity-based permissions introduce a common challenge: the relevant ID (e.g., TenantId or permission ObjectId) may not always be a direct property on the entity being queried. For instance, a Comment entity might not have its own TenantId but instead inherits it from its parent Post entity (this.Post.TenantId). Similarly, permissions for a Task might be based on the Id of its Project (this.Project.Id).

The enterprise DAL must provide a mechanism for entities to define how these contextual IDs are retrieved. This "projected data support" allows an entity to specify a path or a function (e.g., a LINQ expression) that the DAL can use to resolve the TenantId or the ObjectId for filtering purposes. This flexibility ensures that the DAL can correctly apply multi-tenancy and row-level security even when the relevant identifiers are not directly present on the entity itself but are accessible through its relationships, maintaining consistency and correctness across complex data models.

Why LinqToDB?

While Entity Framework Core (EF Core) is a widely adopted and powerful ORM in the .NET ecosystem, this series will primarily focus on implementing our enterprise DAL using LinqToDB. LinqToDB offers a highly flexible and performant approach to data access, particularly excelling in scenarios requiring fine-grained control over generated SQL.

One of LinqToDB's significant advantages for our purposes is its robust support for features like projected properties and direct translation of SQL window functions into LINQ queries. While not all of these advanced features may be strictly necessary for the current enterprise DAL implementation, they collectively illustrate LinqToDB's broader capabilities and its suitability for complex data manipulation. Such capabilities are crucial for efficiently implementing complex filtering and auditing mechanisms directly within the DAL, often requiring less boilerplate or specialized extension libraries compared to EF Core. While EF Core does support some of these advanced scenarios through community-contributed extension libraries, LinqToDB provides many of these features out-of-the-box, simplifying our implementation journey.

Our decision to focus on LinqToDB for this series is driven by its suitability for demonstrating how to build a highly optimized and feature-rich DAL that minimizes manual intervention and maximizes precision in data operations.

Scope of History Support

It is important to clarify that while this series will implement certain auditing functionalities, such as CreatedAt/ModifiedAt and CreatedBy/ModifiedBy fields, it will not cover full history support. Implementing a complete archive of all data at all points in time, including granular versioning and temporal querying capabilities, is a significant undertaking. However, the capabilities developed in this series are foundational and can be extended with full history support later without invalidating the core principles and implementations discussed. Our focus remains on the automated enforcement of the capabilities outlined above to enhance data consistency and security at the transactional level.

The Journey Ahead

Centralizing these essential capabilities within an enterprise-grade Data Access Layer is key to building applications that are inherently more consistent, secure, and maintainable. With LinqToDB as our primary ORM for this implementation journey, the business logic can focus solely on its core rules rather than on data access mechanics.

We encourage you to prepare your development environment for the next installment, where we will establish the foundational C# project and begin implementing the DAL with LinqToDB. Upon completion of this series, a comparative analysis may also be provided by reproducing the same solution using Entity Framework Core.

", "date_published": "2025-09-25T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/byteaetherulid-v130-enhanced-ulid-generation-control-and-security/", "url": "https://byteaether.github.io/2025/byteaetherulid-v130-enhanced-ulid-generation-control-and-security/", "title": "ByteAether.Ulid v1.3.0: Enhanced ULID Generation Control and Security", "content_html": "

ByteAether.Ulid v1.3.0, a significant update to our high-performance .NET ULID implementation, has been released. This version introduces enhanced flexibility and security for ULID generation, reinforcing its suitability for modern applications.

ByteAether aims to provide a compliant and dependable ULID solution for .NET developers. ULIDs combine universal uniqueness with lexicographical sortability, making them suitable for distributed systems and time-ordered data by optimizing indexing and querying.

Enhanced Generation Control with GenerationOptions

Previous versions of ByteAether.Ulid utilized a bool? isMonotonic and Ulid.DefaultIsMonotonic flags for monotonic generation control. Version 1.3.0 replaces this with a new configuration object, GenerationOptions, offering granular control over ULID generation. GenerationOptions includes three parameters: Monotonicity, InitialRandomSource, and IncrementRandomSource.

GenerationOptions Explained

Let's explore the new GenerationOptions in detail:

Monotonicity

This option controls the monotonic behavior of ULIDs, offering choices to balance predictability and uniqueness:

InitialRandomSource and IncrementRandomSource

Both InitialRandomSource and IncrementRandomSource properties are of type IRandomProvider. This design provides a flexible way to define how randomness is produced, allowing users to implement and provide their own custom random number generators.

The InitialRandomSource option specifies the source of randomness for the initial random component of the ULID. The IncrementRandomSource setting determines the source of randomness for the increment value when using MonotonicRandomXByte options for monotonicity.

Default Behavior and Backward Compatibility

Version 1.3.0's default settings ensure a seamless transition, matching previous ByteAether.Ulid monotonic generation behavior:

Ulid.DefaultGenerationOptions = new Ulid.GenerationOptions\n{\n\tMonotonicity = MonotonicityOptions.MonotonicIncrement,\n\tInitialRandomSource = new CryptographicallySecureRandomProvider(),\n\tIncrementRandomSource = new PseudoRandomProvider()\n}

Upgrading to v1.3.0 will not alter existing ULID generation logic unless GenerationOptions are explicitly configured and assigned to Ulid.DefaultGenerationOptions.

For backward compatibility, public API methods with a bool? isMonotonic parameter and the Ulid.DefaultIsMonotonic static property remain, but are marked [Obsolete]. These are deprecated to encourage migration to the GenerationOptions-based methods for enhanced control and flexibility. Obsolete methods and properties will be removed in a future version and users are encouraged to update their code for future compatibility and to leverage new capabilities.

Why ULIDs?

Identifier selection is crucial in distributed systems. GUIDs provide uniqueness but lack sortability, hindering database indexing and querying. Integer IDs are sortable but may conflict in distributed environments due to their limited scope.

ULIDs combine universal uniqueness with lexicographical sortability:

ByteAether.Ulid is a robust and compliant .NET implementation. It addresses potential issues from the official ULID specification, such as the "random part overflow" issue, by allowing timestamp increments to ensure unique ULIDs even at high generation rates.

Benchmarking against other .NET ULID implementations (e.g., NetUlid, Ulid, NUlid), Guid, and GuidV7 consistently shows ByteAether.Ulid as highly performant and fully compliant with the ULID specification, ensuring dependable generation.

Installation and Usage

Install ByteAether.Ulid v1.3.0 via NuGet:

dotnet add package ByteAether.Ulid

Example of GenerationOptions usage:

using ByteAether.Ulid;\nusing static ByteAether.Ulid.Ulid.GenerationOptions;\n\nvar optionsWithRandomIncrement = new Ulid.GenerationOptions\n{\n\t// 4-byte random increment\n\tMonotonicity = MonotonicityOptions.MonotonicRandom4Byte,\n\t// Cryptographically secure initial random\n\tInitialRandomSource = new CryptographicallySecureRandomProvider(),\n\t// Cryptographically secure increment random\n\tIncrementRandomSource = new CryptographicallySecureRandomProvider()\n};\n\nvar ulidWithRandomInc = Ulid.New(optionsWithRandomIncrement);\nConsole.WriteLine($\"ULID with random increment: {ulidWithRandomInc}\");\n\n// Set default generation options globally\nUlid.DefaultGenerationOptions = new()\n{\n\tMonotonicity = MonotonicityOptions.MonotonicIncrement,\n\t// Pseudo-randomness for speed\n\tInitialRandomSource = new PseudoRandomProvider(),\n\t// IncrementRandomSource is not used by MonotonicIncrement\n};\n\nvar defaultUlid = Ulid.New();\nConsole.WriteLine($\"ULID from default pseudo-random source: {defaultUlid}\");

Ecosystem Integration

ByteAether.Ulid integrates with popular .NET libraries and frameworks:

Contribute to ByteAether.Ulid

We are committed to the continuous improvement of ByteAether.Ulid. Contributions, including pull requests, bug reports, and feature suggestions, are welcome. Your input is valuable for shaping the project's future.

Upgrade to ByteAether.Ulid v1.3.0 for enhanced control, security, and performance in ULID generation.

", "date_published": "2025-07-31T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/prioritizing-reliability-when-milliseconds-arent-enough/", "url": "https://byteaether.github.io/2025/prioritizing-reliability-when-milliseconds-arent-enough/", "title": "Prioritizing Reliability When Milliseconds Aren't Enough", "content_html": "

Universally Unique Lexicographically Sortable Identifiers (ULIDs) represent a significant advancement over traditional UUIDs, offering guaranteed uniqueness (within practical limits), natural sortability by time, and a concise, URL-friendly format. These features make them highly suitable for diverse applications, including database keys, distributed tracing, and event identifiers. While the intricacies of ID generation might seem like a deep concern primarily for high-throughput systems, understanding the underlying design choices is valuable for all developers leveraging such libraries.

A standard ULID consists of a 48-bit timestamp (milliseconds since the Unix epoch) and an 80-bit random component. This structure ensures IDs generally sort chronologically. However, the official specification includes a mechanism to handle multiple IDs generated within the exact same millisecond: it increments the 80-bit random part sequentially. This leads to a critical question: What happens if this increment causes the 80-bit random part to exceed its maximum value?

The official specification mandates throwing an overflow exception in this situation. While this adheres strictly to the definition, it introduces a potential point of failure, especially in high-performance systems. Crucially, this isn't just a theoretical concern that requires generating 2^80 IDs. For the first ULID generated within a specific millisecond, the 80-bit portion is initialized with a cryptographically random value. If this initial random value happens to be very close to the maximum possible value (all 1's in binary), even the second ID generated within that same millisecond could trigger the increment overflow and, consequently, an exception. This makes the overflow a far more probable scenario than one might initially assume based purely on the size of the 80-bit space.

In developing ByteAether.Ulid, our C# implementation available on GitHub and NuGet, we recognized the practical implications of this potential exception. We made a deliberate design choice, a documented deviation from the strict specification, to enhance dependability: instead of throwing an exception, ByteAether.Ulid allows the increment operation to flow into the 48-bit timestamp part when the 80-bit random part overflows. This post explains the rationale behind this decision.

The Overflow Scenario: More Likely Than You Think

Consider systems that demand high throughput: real-time analytics pipelines, busy web servers handling rapid requests, IoT data ingestion points, or batch processing jobs churning through records. In these environments, generating multiple unique identifiers within a single millisecond is commonplace.

The ULID specification's approach ensures monotonicity within a millisecond by incrementing the random part. For example:

  1. Ulid.New() at T ms: [Timestamp T | Random Value R1]
  2. Ulid.New() at T ms again: [Timestamp T | Random Value R1 + 1]
  3. Ulid.New() at T ms yet again: [Timestamp T | Random Value R1 + 2] ... and so on.

The issue arises when Random Value R1 + n reaches the maximum value representable by 80 bits. As noted earlier, if the initial Random Value R1 generated by the secure random number generator is already near this maximum, it might only take one or two subsequent calls to Ulid.New() within the same millisecond to hit the ceiling, if the initial random draw was exceptionally unlucky (i.e., very close to the maximum).

An implementation strictly following the spec would then throw an exception. This creates significant challenges:

  1. Reduced Reliability: ID generation, a typically background concern, becomes a potential source of runtime failure, especially under peak load conditions where reliability is most critical.
  2. Increased Complexity & Performance Impact: Developers must anticipate and handle these exceptions. try-catch blocks, particularly when an exception is actually thrown, can incur a non-trivial performance cost, especially in very hot code paths like ID generation. This might lead to implementing retry logic or other workarounds that add complexity and can negatively impact the very performance ULIDs are often chosen to support.

Update: The Impact of Random Increments on Overflow Probability

With the release of ByteAether.Ulid v1.3.0, our decision to handle random-part overflow by incrementing the timestamp has become even more critical and relevant. The previous monotonic generation behavior, which incremented the random part by a fixed value of 1, made the overflow scenario more likely than one might assume, especially if the initial random value was close to the maximum.

The new GenerationOptions configuration object introduces several MonotonicRandomXByte options, which enhance security by allowing the increment value to be a random number. This value can range from 1 to 256 for MonotonicRandom1Byte, up to 1 to 4 294 967 296 for MonotonicRandom4Byte. While this adds an important layer of security against enumeration attacks, it also significantly increases the probability of an 80-bit random part overflow occurring within a single millisecond.

Consider a scenario where MonotonicRandom4Byte is used. A cryptographically secure random provider could generate an initial 80-bit random component close to the maximum value, and the very next ULID generated in that same millisecond could have an increment value of 2^32, causing an immediate overflow.

Our approach of allowing the increment to flow into the timestamp directly addresses this increased risk. Instead of a predictable, linear march toward a potential exception, the new random increments make the overflow threshold far more volatile. By not throwing an exception and instead seamlessly incrementing the timestamp, our implementation ensures continued, reliable operation under these new, more complex generation scenarios, reinforcing its value for robust and high-throughput systems.

Considering the Alternatives

When faced with the random-part overflow, beyond throwing an exception (as per the spec), several strategies exist:

  1. Delay Generation: This approach involves pausing the generation thread until the system clock advances to the next millisecond. While ensuring the timestamp strictly reflects the wall-clock time (or a later one), it introduces artificial latency. Halting execution, even briefly, can create significant bottlenecks in high-throughput systems, effectively throttling the application's performance based on clock resolution rather than processing capability. This often defeats the purpose of using efficient ID generation.

  2. Force Initial MSB to Zero: This technique modifies the generation of the initial random value (R1 in the example above) within a millisecond. The most significant bit (MSB) of the 80-bit random part is always set to 0. This effectively means the initial random value is always less than 2^79. Consequently, at least 2^79 increments are guaranteed to be possible within that millisecond before the 80-bit value overflows.

  3. Allow Overflow into Timestamp: This is the strategy implemented in ByteAether.Ulid. When the 80-bit random part reaches its limit and needs to increment, the increment operation carries over, increasing the 48-bit timestamp component by one millisecond. The practical effect is that the generated ULID might have a timestamp slightly "ahead" of the precise system clock time at the moment of generation.

Comparing these, the ByteAether.Ulid approach directly addresses the consequence of the overflow by providing a non-blocking, non-exception-throwing resolution, regardless of how likely the overflow is. We contend that this provides the optimal blend of performance, reliability, and practical sortability for demanding applications.

Understanding the Impact: Monotonicity in Practice

A natural question regarding the timestamp overflow approach is its effect on ULID's sortability and monotonicity guarantees. It's essential to analyze this within realistic contexts.

Single-Process Monotonicity is Strictly Preserved: This is the cornerstone of the argument. Within a single running process, ByteAether.Ulid guarantees that each call to Ulid.New() produces a ULID value strictly greater than the previous one. Whether the increment occurs within the 80-bit random part or spills over into the 48-bit timestamp part, the resulting 128-bit value increases. This ensures that IDs generated sequentially by the same generator instance remain perfectly sortable relative to each other. This is often the most critical monotonicity requirement for use cases like database record ordering within an import batch.

Cross-Process Monotonicity is Inherently Approximate: Attempting to achieve strict, guaranteed millisecond-level chronological ordering using ULIDs across different machines or even different processes on the same machine is inherently unreliable, irrespective of how overflow is handled. Factors like clock skew, network latency, and scheduling jitter mean you cannot reliably assert that ULID_A generated on Machine_1 at its perceived time T is strictly less than ULID_B generated on Machine_2 at its perceived time T + 1ms.

Given this reality, the fact that ByteAether.Ulid might, in high-contention scenarios, advance the timestamp by a millisecond due to overflow does not practically diminish the utility of ULIDs for cross-process chronological estimation. The timestamp remains a highly accurate representation of the generation time. Crucially, this "future-drift" is typically only by a single millisecond; once the timestamp increments, the subsequent ULID generated in this "new" millisecond has the full 2^80 range for its random component, making further immediate overflows into the timestamp exceptionally unlikely. This minimal adjustment is well within the margin of error already introduced by system clocks and network effects, and it prevents generation failure without meaningfully impacting the approximate sortability expected in distributed contexts.

Alternative - Centralized ID Generation: One could achieve stronger cross-instance monotonicity by funneling all ID requests through a dedicated, single-instance "ID Generator Service." This service could maintain strict ordering. However, this introduces significant drawbacks: primarily, network latency for every ID request and a single point of failure/bottleneck. This contrasts sharply with the benefit of ULIDs: fast, decentralized, in-process generation. For systems prioritizing low-latency generation, the centralized approach is often impractical.

In summary, ByteAether.Ulid's approach trades theoretical timestamp purity (which is already compromised in distributed systems) for practical reliability and performance, while fully preserving the critical single-process monotonicity guarantee.

Why Reliable Monotonicity Matters: The Data Import Example

Let's revisit the large-scale data import workflow to illustrate the value of ByteAether.Ulid's reliable single-process monotonicity. A single process reads millions of source records, transforms them, assigns a ULID, and inserts them into a database.

// Simplified Example using ByteAether.Ulid\npublic class DataImporter\n{\n\tprivate readonly IDatabaseRepository _repository;\n\t// ... constructor ...\n\n\tpublic void ImportData(IEnumerable<SourceRecord> sourceRecords)\n\t{\n\t\tvar recordsToInsert = new List<TargetRecord>();\n\t\tforeach (var sourceRecord in sourceRecords)\n\t\t{\n\t\t\t// ByteAether.Ulid handles potential same-ms overflow gracefully\n\t\t\t// Potential high-frequency generation here!\n\t\t\tUlid newId = Ulid.New();\n\n\t\t\tvar targetRecord = new TargetRecord {\n\t\t\t\tId = newId,\n\t\t\t\t/* ... other fields ... */\n\t\t\t};\n\t\t\trecordsToInsert.Add(targetRecord);\n\t\t}\n\t\t_repository.BulkInsert(recordsToInsert);\n\t}\n\t// ... other methods and record definitions ...\n}

Advantages in this Scenario:

  1. Dependable High-Speed Operation: The import can run at full speed. Even if bursts occur where the random part of the ULID would overflow (potentially after just one previous generation in the same millisecond if the initial random value was high), ByteAether.Ulid seamlessly increments the timestamp and continues generation without throwing exceptions. This prevents job failures and avoids the need for complex error handling around Ulid.New().
  2. Preserved Processing Order: Because each newId is guaranteed to be greater than the last within this import process, the generated IDs maintain the sequence in which records were handled. If used as a clustered primary key, this aids database insertion performance and allows efficient time-based range queries on the imported batch. The sequence is inherent in the IDs themselves.
  3. No Artificial Bottlenecks: Unlike a delay-based approach, ByteAether.Ulid doesn't artificially throttle the import speed based on millisecond clock ticks, allowing the system to utilize its full processing power.

Using an implementation that throws exceptions could lead to sporadic, hard-to-reproduce failures during large imports. Using one that delays would unnecessarily extend the import duration. Using the MSB=0 approach would make failures extremely unlikely, but still theoretically possible, requiring a fallback strategy (like throwing or delaying anyway) that doesn't offer the same definitive resolution. ByteAether.Ulid provides a direct, reliable handling mechanism.

A Pragmatic Choice for Robust Systems

The decision in ByteAether.Ulid to handle random-part overflow by incrementing the timestamp is a pragmatic one, driven by the need for reliability and performance in real-world applications. It recognizes that the overflow scenario is more likely than a simple 2^80 calculation suggests due to the random starting point within each millisecond.

This approach:

While representing a documented deviation from the specification's exception-throwing behavior, this design choice prioritizes building robust, performant systems that can handle demanding workloads without faltering. We believe this makes ByteAether.Ulid a strong choice for developers needing fast, reliable, and sortable unique identifiers in C#.

", "date_published": "2025-06-19T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/announcing-byteaetherweakevent-v100/", "url": "https://byteaether.github.io/2025/announcing-byteaetherweakevent-v100/", "title": "Announcing ByteAether.WeakEvent v1.0.0", "content_html": "

We are thrilled to announce the release of ByteAether.WeakEvent v1.0.0, a NuGet library designed to revolutionize event management in .NET. This release marks the culmination of months of development, testing, and community feedback. With a focus on memory safety and ease-of-use, ByteAether.WeakEvent empowers software engineers and architects to build decoupled, maintainable, and high-performance applications without the pitfalls of memory leaks caused by traditional event subscriptions.

Tackling a Common Challenge in .NET

In modern .NET applications, event-driven programming is ubiquitous. Whether you are developing rich desktop experiences with WPF, dynamic web applications with Blazor, or any event-based system, managing event subscriptions efficiently is critical. A common challenge is that the standard event subscription model holds strong references to subscribers, often leading to unintended memory retention and, ultimately, memory leaks.

The weak event pattern offers a solution by holding references to event subscribers weakly. In this model, the garbage collector can reclaim subscriber memory even when they remain subscribed to an event, provided no other strong references exist. This release of ByteAether.WeakEvent brings this pattern to your fingertips with a clean API, automatic cleanup of defunct subscriptions, and a robust publish–subscribe mechanism tailored for .NET applications.

For more details on the concepts behind weak events and publish–subscribe patterns, refer to our in-depth article on the subject. You can also check out the GitHub repository for source code and additional documentation.

Understanding the Weak Event Pattern

What Is a Weak Event?

In conventional .NET event handling, the publisher holds a strong reference to each subscriber through its delegate list. If subscribers are not explicitly removed, these references persist, keeping objects in memory even after they are no longer needed. This is the primary cause of memory leaks in event-driven systems.

A weak event solves this by holding subscribers via weak references. A weak reference permits the garbage collector to reclaim an object's memory if no other strong references exist. This behavior is especially useful in scenarios where the publisher has a longer lifetime than its subscribers.

The Theory Behind Weak References

As explained in foundational materials on weak event management:

Consider this simplified code example that demonstrates the essence of weak references in event handling:

public class Publisher\n{\n\t// List of weak references to event handlers\n\tprivate List<WeakReference<EventHandler>> _handlers = new();\n\n\tpublic void Subscribe(EventHandler handler)\n\t{\n\t\t_handlers.Add(new WeakReference<EventHandler>(handler));\n\t}\n\n\tpublic void RaiseEvent()\n\t{\n\t\tforeach (var weakHandler in _handlers)\n\t\t{\n\t\t\tif (weakHandler.TryGetTarget(out EventHandler handler))\n\t\t\t{\n\t\t\t\thandler.Invoke(this, EventArgs.Empty);\n\t\t\t}\n\t\t}\n\t}\n}

In this model, event subscribers are not held strongly, which allows the garbage collector to reclaim memory for any subscriber that is no longer in use.

Key Benefits of ByteAether.WeakEvent

Memory Efficiency and Automatic Cleanup

One of the primary advantages of using ByteAether.WeakEvent is its ability to manage memory efficiently. By storing subscribers as weak references, the library ensures that objects are not unnecessarily held in memory. This is particularly beneficial for applications with long lifetimes, where small memory leaks can accumulate over time and degrade performance.

Simplified Event Management

Traditionally, developers must be vigilant in manually unsubscribing events—an often error-prone process. ByteAether.WeakEvent automates the cleanup process by removing dead references whenever an event is raised. This simplifies event management, reduces boilerplate code, and minimizes the risk of memory leaks.

Versatility Across Application Types

The library is designed to be flexible:

Exploring the API and Usage Examples

Simple Event Subscription

For events that do not require additional data, the API is as simple as subscribing a handler and publishing the event:

using ByteAether.WeakEvent;\n\n// Create an instance of the weak event without event data\nvar myEvent = new WeakEvent();\n\n// Create a subscriber and subscribe\nvar subscriber = () => Console.WriteLine(\"Event received!\");\nmyEvent.Subscribe(subscriber);\n\n// Raise the event\nawait myEvent.PublishAsync();

In this example, the handler is invoked if its target is still alive. Dead subscribers are automatically pruned from the subscription list when the event is raised.

Event Subscription with Data

When you need to transmit data along with the event, the WeakEvent<TEvent> class comes into play:

using ByteAether.WeakEvent;\n\n// Create an instance of the weak event with event data\nvar myEvent = new WeakEvent<MyEventData>();\n\n// Create a subscriber and subscribe\nvar subscriber = (MyEventData data) => Console.WriteLine(\"Received: \" + data.Message);\nmyEvent.Subscribe(subscriber);\n\n// Raise the event with data\nawait myEvent.PublishAsync(new MyEventData(\"Hello, World!\"));\n\n// Define your event data\npublic record MyEventData(string Message);

This approach allows your events to be rich in context, with subscribers receiving any necessary data without compromising the integrity of the memory management system.

How ByteAether.WeakEvent Works Under the Hood

The Publish–Subscribe Paradigm

At its core, ByteAether.WeakEvent embraces the publish–subscribe model, a design pattern that decouples the event publisher from its subscribers. In this architecture:

This decoupling not only simplifies code but also enhances scalability and maintainability—attributes that are critical in both desktop and web applications.

Detailed Implementation Insights

The library leverages the built-in .NET WeakReference<T> to hold event subscribers. When an event is published:

This strategy ensures that your application's memory footprint remains minimal, as there is no lingering of obsolete event handlers. Detailed examples and explanations can be found in our documentation and our in-depth article on the weak event pattern.

Real-World Applications and Scenarios

UI Development with Blazor and WPF

In UI frameworks such as Blazor and WPF, the lifecycle management of components is crucial. Consider a Blazor component that subscribes to events during its initialization:

@code {\n\t[Inject]\n\tprotected readonly Publisher _publisher { get; set; } = default!;\n\n\tprotected override void OnInitialized()\n\t{\n\t\t_publisher.OnPublish.Subscribe(OnEvent);\n\t}\n\n\tpublic void OnEvent(MyEventData eventData)\n\t{\n\t\tConsole.WriteLine(\"Event received in Blazor component.\");\n\t}\n\n\tpublic void Dispose()\n\t{\n\t\t// No need to manually unsubscribe, the weak reference handles cleanup.\n\t}\n}

In this scenario, even if a component is disposed, its subscription does not prevent garbage collection. This is a critical advantage in long-running applications where manual unsubscription could be easily overlooked, leading to potential memory leaks.

Server-Side and Distributed Systems

For backend services and distributed systems, resource management is paramount. ByteAether.WeakEvent's ability to automatically manage event subscriptions means that your server-side applications can scale more reliably. When objects are no longer in use, they are automatically cleaned up, reducing the overhead associated with manual memory management and ensuring smoother performance.

Cross-Platform Benefits

While our focus is on .NET, the principles behind weak event management apply broadly across programming environments. Whether you are working with JavaScript's reactive libraries, Java's weak references, or even observer patterns in C++, the core concept remains the same: decoupling event publishers from subscribers results in more resilient and maintainable systems.

Getting Started with ByteAether.WeakEvent

Installation Made Easy

The ByteAether.WeakEvent library is available as a NuGet package. To install, simply run:

dotnet add package ByteAether.WeakEvent

For those who wish to experiment with the latest features, you can also specify a preview version using the --version option.

Documentation and Community

For comprehensive documentation, API details, and additional usage examples, please visit the GitHub repository. We encourage contributions and feedback from the community. If you encounter any issues or have suggestions, feel free to open an issue or submit a pull request.

A New Era of Event Management in .NET

ByteAether.WeakEvent v1.0.0 sets a new standard for event handling in .NET. By embracing the weak event pattern, the library not only addresses common pitfalls like memory leaks but also simplifies your codebase, allowing you to focus on building great features. Whether you are developing desktop applications, dynamic web apps, or large-scale distributed systems, this library is designed to make your event-driven programming more efficient, robust, and maintainable.

We invite software engineers and architects to integrate ByteAether.WeakEvent into their projects and experience the benefits of a lean, memory-safe event management system. Explore the GitHub repository and get started today!

Happy coding!

", "date_published": "2025-03-18T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/hyperaction-api-a-reimagined-approach-to-http-api-design/", "url": "https://byteaether.github.io/2025/hyperaction-api-a-reimagined-approach-to-http-api-design/", "title": "HyperAction API: A Reimagined Approach to HTTP API Design", "content_html": "

Traditional REST designs emphasize CRUD operations over distinct business actions, forcing developers to implement significant business logic on the client side while backends simply translate raw database records into HTTP responses. As a result, the intended business rules become obscured and must be redundantly re-implemented on the server side—complicating maintenance and creating security vulnerabilities.

One major flaw arises from shared data models that are designed around generic resources rather than tailored to specific business features. For example, a single "User" endpoint might return all associated details—including sensitive fields like email addresses—even when the client requires only a list of online usernames. This forces clients to make multiple API calls to compile the necessary data and increases the risk of unintended data exposure.

In contrast, adopting a vertical slice architecture—where each feature or business action is developed as a self-contained unit (encapsulating all layers from user interface to data access)—directly addresses these issues. Vertical slices isolate business logic, ensuring that changes in one domain do not affect unrelated parts of the system. This approach simplifies development and maintenance while aligning API behavior with specific business capabilities.

Drawing on extensive experience with both large-scale and smaller applications, the HyperAction API convention reimagines HTTP API design by centering endpoints around clear, action-oriented business operations. This method enhances security, minimizes redundancy, and fosters a more maintainable system architecture.

This document outlines the technical guidelines for the new HyperAction API convention. Departing from traditional REST paradigms, HyperAction API enforces batch processing at every endpoint, aligns endpoints with specific business actions, and incorporates explicit mechanisms for consistency and asynchronous operations. The objective is to create a clear and efficient contract between clients and servers that minimizes the common pitfalls of REST while meeting modern application needs.

Each design decision is explained by comparing conventional REST approaches with their challenges, then detailing our design decisions with updated examples that adhere to the batching principle.

Key Concepts

1. Use Singular Naming Conventions

Traditional REST Approach

Many REST APIs use plural names (e.g., /users, /orders) to indicate collections of resources. This approach can be ambiguous when handling irregular plurals.

Challenges

Plural names may lead to confusion with irregular words—such as "mouse" versus "mice" or "octopus" versus "octopi"—and can complicate the mental model of the API. In a batch-based system where every endpoint already handles multiple items, the plurality is implied by the data structure, not the endpoint name.

HyperAction API Decision

In HyperAction API, all endpoint and model names are singular. Even though requests and responses always use arrays, the endpoint name itself represents the individual item type. This convention avoids ambiguity, simplifies naming, and reinforces that the batch behavior is a global characteristic of the API.

Example

An endpoint to retrieve users is named singularly:

HTTP Request

GET /user?limit=10&offset=0 HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 10\nX-Limit: 10\nX-Offset: 0\n\n[\n\t{ \"id\": 1, \"name\": \"Alice\" },\n\t{ \"id\": 2, \"name\": \"Bob\" },\n\t...\n]\n

Similarly, a business action such as account verification remains singular:

HTTP Request

POST /account/verify HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{ \"userId\": 789, \"token\": \"ABC123XYZ\" }\n]\n

HTTP Response

HTTP/1.1 207 Multi-Status\nContent-Type: application/json\n\n[\n\t{ \"userId\": 789, \"status\": 200 }\n]\n

This naming convention clarifies that although each payload contains multiple items, the fundamental unit of operation remains singular.

2. Headers Instead of Envelope Models

Traditional REST Approach

Many APIs wrap data in envelope objects that include metadata like status and pagination details, leading to additional nesting and complexity.

Challenges

The extra layer of an envelope model forces clients to parse both the data and metadata, which complicates the data extraction and processing logic.

HyperAction API Decision

Responses for list endpoints return a raw array of items. Metadata such as total count, offset, and limit is communicated exclusively through HTTP headers like X-Count, X-Total-Count, X-Limit, and X-Offset.

Example

A product listing request:

HTTP Request

GET /product?limit=20&offset=0 HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 20\nX-Total-Count: 250\nX-Limit: 20\nX-Offset: 0\n\n[\n\t{ \"productId\": 101, \"name\": \"Widget A\", \"price\": 9.99 },\n\t{ \"productId\": 102, \"name\": \"Widget B\", \"price\": 12.99 },\n\t...\n]\n

Based on RFC 6648, the "X-" prefix on custom headers is not a requirement. This document does not enforce the use of the "X-" prefix; however, the examples include it because this naming convention is widely recognized and helps make custom headers easier to read and understand.

3. Batch Endpoints for All Operations

Traditional REST Approach

RESTful designs typically offer endpoints such as GET /items/{id} or PUT /items/{id} that focus on individual resources. When multiple items must be handled, the client is forced to make several calls or use loosely defined batch endpoints.

Challenges

Focusing on single resources increases round-trips and may lead to inconsistency when operations naturally involve multiple entities. It also forces clients to stitch together logic for batch processing, which can become cumbersome.

HyperAction API Decision

Every endpoint in HyperAction API is a batch endpoint. There is no dedicated endpoint for a single-item operation; instead, all operations—whether for retrieval, creation, or update—are designed to work with arrays. This not only ensures atomic, batch-based processing but also enforces a mindset where the client always works with collections of data.

Example

To retrieve items, a client issues:

HTTP Request

GET /item?filter=id:123,456 HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 2\nX-Total-Count: 2\nX-Limit: 50\nX-Offset: 0\n\n[\n\t{ \"id\": 123, \"name\": \"Item One\", \"price\": 14.99 },\n\t{ \"id\": 456, \"name\": \"Item Two\", \"price\": 7.99 }\n]\n

4. Pagination Using Limit and Offset

Traditional REST Approach

Many REST APIs use page and pageSize parameters, which hide the actual dataset position and force clients to convert page numbers into offsets.

Challenges

The abstract concept of a page makes it harder to implement progressive loading or infinite scrolling. Clients must perform additional calculations to determine the proper slice of data.

HyperAction API Decision

Endpoints return list results using explicit limit and offset parameters. This clear slicing method allows clients to directly control the amount of data and its position, simplifying custom pagination logic.

Example

A request for a subset of orders:

HTTP Request

GET /order?limit=50&offset=100 HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 50\nX-Total-Count: 1000\nX-Limit: 50\nX-Offset: 100\n\n[\n\t{ \"orderId\": 201, \"amount\": 99.99, \"status\": \"completed\" },\n\t{ \"orderId\": 202, \"amount\": 149.99, \"status\": \"pending\" },\n\t...\n]\n

With efficient pagination established, we now turn to designing endpoints that reflect discrete business actions—ensuring that each endpoint aligns with a specific operational capability.

5. Business Logic–Specific Endpoints

Traditional REST Approach

Generic endpoints such as /users or /orders attempt to cover all CRUD operations and often reuse generic models, which may lead to oversharing of data or the mixing of unrelated business logic.

Challenges

Using a single resource endpoint for various actions increases the risk of exposing unnecessary details and complicates segregation of business processes.

HyperAction API Decision

Each endpoint is crafted for a specific business action. Every business-logic action endpoint now receives and responds with arrays of items, ensuring that the interface naturally supports batch processing.

Example

For account verification, instead of a single-object payload, the endpoint handles an array of verification requests:

HTTP Request

POST /account/verify HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{ \"userId\": 789, \"token\": \"ABC123XYZ\" }\n]\n

HTTP Response

HTTP/1.1 207 Multi-Status\nContent-Type: application/json\n\n[\n\t{ \"userId\": 789, \"status\": 200 }\n]\n

This design keeps the interface consistent with batch processing while encapsulating business logic within dedicated endpoints. By designing endpoints specifically for discrete business actions, we sidestep the pitfalls of generic data handling and establish a clear, action-oriented approach to API design.

6. Action-Oriented Endpoints over CRUD

Traditional REST Approach

Traditional CRUD endpoints focus solely on direct data manipulation and tend to oversimplify underlying business processes. This design forces clients to manage critical business logic—often duplicating effort and increasing security risks—while the backend merely acts as a conduit for raw data.

Challenges

Generic CRUD endpoints can force clients to manage business rules or risk unwanted side effects by treating operations as mere data transformations.

HyperAction API Decision

Endpoints are explicitly designed for business actions. Even when these endpoints appear similar, they are modeled with distinct request and response payloads—and both are arrays. This makes the intended action explicit and separates it from simple data manipulation.

Example

For an address change, the API now expects an array of address change requests:

HTTP Request

POST /account/change-address HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{\n\t\t\"userId\": 789,\n\t\t\"address\": \"123 New Street, Cityville, Country\"\n\t}\n]\n

HTTP Response

HTTP/1.1 207 Multi-Status\nContent-Type: application/json\n\n[\n\t{ \"userId\": 789, \"status\": 200 }\n]\n

This explicit batching reinforces the notion that operations are designed for multiple items at once.

The HyperAction API employs action-oriented endpoints that integrate business logic directly into each operation. By embedding the intended business actions within the API itself, these endpoints offer clearer semantics and enhanced security. This shift not only streamlines client interactions by reducing the need for multiple calls but also ensures that each endpoint directly reflects a distinct business capability.

7. CRUD-Like Operations via Form Loading and Saving

Traditional REST Approach

Typically, REST APIs implement simple CRUD endpoints for basic data manipulation without substantial business logic. This often results in "proxy" endpoints that merely transfer data from the client to the database without additional contextual processing.

The approach described here is most applicable to systems where the frontend and backend are developed together as a single, cohesive unit. In tightly integrated environments, close coordination between UI requirements and backend implementations can be achieved, making form loading and saving an effective pattern for streamlining data handling and reducing the need for redundant API calls. Conversely, for general public APIs—where the consumer or client code is developed by external parties—a more standardized and decoupled approach using traditional CRUD endpoints may be preferable to accommodate the varied requirements and integration patterns of third-party developers.

Challenges

HyperAction API Decision

In HyperAction API, we reconceptualize these operations as "form loading" and "form saving" to address these challenges:

Both endpoints adhere to the batching principle, allowing multiple forms to be loaded or saved in a single request.

Example

HTTP Request (Loading form)

GET /form/user-profile?ids=123, HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response (Loading form)

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 2\nX-Total-Count: 2\nX-Form: { \"roleId\": { 5: \"Administrator\", 6: \"User\" } }\n\n[\n\t{\n\t\t\"userId\": 123,\n\t\t\"etag\": \"a1b2c3d4\",\n\t\t\"name\": \"Alice\",\n\t\t\"email\": \"alice@example.com\",\n\t\t\"roleId\": 5\n\t},\n\t{\n\t\t\"name\": \"\",\n\t\t\"email\": \"\",\n\t\t\"roleId\": 6\n\t}\n]\n

HTTP Request (Saving form)

POST /form/user-profile HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\nX-Idempotency-Key: 123e4567-e89b-12d3-a456-426614174000\n\n[\n\t{\n\t\t\"userId\": 123,\n\t\t\"etag\": \"a1b2c3d4\",\n\t\t\"name\": \"Alice Updated\",\n\t\t\"email\": \"alice.new@example.com\",\n\t\t\"roleId\": 5\n\t},\n\t{\n\t\t\"userId\": 456,\n\t\t\"name\": \"Bob Created\",\n\t\t\"email\": \"bob.created@example.com\",\n\t\t\"roleId\": 6\n\t}\n]\n

This dual-endpoint approach for form operations ensures that even simple data manipulations are handled with the same rigor and consistency as more complex business actions, aligning seamlessly with the overarching principles of the HyperAction API convention.

8. Using HTTP Status Code 207 (Multi-Status) for Batch Operations

Traditional REST Approach

Batch operations often return a generic 200 OK even when different items in the batch yield mixed results, masking individual errors.

Challenges

A generic status code does not allow clients to easily identify which items in a batch succeeded or failed, complicating error handling and recovery.

HyperAction API Decision

For operations that process batches with mixed outcomes, the API uses HTTP status code 207 Multi-Status. The response body provides an array of per-item status information, giving clients granular feedback.

Example

A batch update of order statuses:

HTTP Request

POST /order/update-status HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{ \"orderId\": 201, \"newStatus\": \"shipped\" },\n\t{ \"orderId\": 202, \"newStatus\": \"cancelled\" }\n]\n

HTTP Response (207 Multi-Status)

HTTP/1.1 207 Multi-Status\nContent-Type: application/json\n\n[\n\t{ \"orderId\": 201, \"status\": 200, \"message\": \"Status updated successfully\" },\n\t{ \"orderId\": 202, \"status\": 400, \"message\": \"Invalid transition to 'cancelled'\" }\n]\n

9. Asynchronous Processing for Long-Running Operations

Traditional REST Approach

Long-running operations are typically processed synchronously, which may result in timeouts and a poor user experience.

Challenges

Holding a connection open during lengthy processing risks timeouts and duplicated requests, degrading both client and server performance.

HyperAction API Decision

For operations that take an extended time, asynchronous processing is strongly encouraged. Two strategies are provided:

  1. Callback URL: The client submits an array of requests with callback URLs. Once processed, the server calls the provided URL for each item with a minimal payload.
  2. Polling with Job ID: Alternatively, the server immediately returns a batch of job identifiers or URLs. Clients can then poll or use long-polling to track each job's progress.

Example (Callback URL)

HTTP Request

POST /report/generate HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{\n\t\t\"reportType\": \"sales\",\n\t\t\"parameters\": { \"date\": \"2025-02-27\" },\n\t\t\"callbackUrl\": \"https://client.example.com/report-callback\"\n\t}\n]\n

HTTP Response

HTTP/1.1 207 Multi-Status\nContent-Type: application/json\n\n[\n\t{ \"jobId\": \"abc123\", \"status\": 202 }\n]\n

This decoupled approach improves reliability and scalability for long-running processes.

10. Enforcing Idempotency

Traditional REST Approach

While certain HTTP methods are idempotent, operations that modify state (typically via POST) are not, leading to potential duplicate processing if retried.

Challenges

Without an explicit idempotency mechanism, clients risk accidentally processing the same request multiple times during network retries or failures.

HyperAction API Decision

Every state-modifying action must include an idempotency key, passed as a header (e.g., X-Idempotency-Key). In line with the batching principle, the endpoint accepts an array of requests, each of which will be deduplicated based on the provided key.

Example

HTTP Request

POST /account/update-email HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\nX-Idempotency-Key: 4f2a9c3e-d8f1-4b5e-9a2d-123456789abc\n\n[\n\t{\n\t\t\"userId\": 789,\n\t\t\"email\": \"new.email@example.com\"\n\t}\n]\n

The server uses the idempotency key to ensure that duplicate requests do not cause repeated processing.

11. Caching and Consistency with ETAGs

Traditional REST Approach

ETAGs are used for caching and concurrency but are often applied at a coarse level (e.g., per entire response), which is insufficient when handling batches.

Challenges

Without item-level ETAGs, clients may inadvertently overwrite data when updating parts of a batch, risking consistency issues.

HyperAction API Decision

Assign ETAGs to each individual item in a response. When updating items, the client includes the corresponding ETAG for each item in the batch, ensuring updates are based on the latest state.

Example

Retrieving items with individual ETAGs:

HTTP Request

GET /item?limit=10&offset=0 HTTP/1.1\nHost: api.example.com\nAccept: application/json

HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/json\nX-Count: 10\nX-Limit: 10\nX-Offset: 0\n\n[\n\t{ \"id\": 123, \"name\": \"Item One\", \"etag\": \"a1b2c3d4\" },\n\t{ \"id\": 124, \"name\": \"Item Two\", \"etag\": \"e5f6g7h8\" },\n\t...\n]\n

When updating, the request carries an array of items with ETAGs:

HTTP Request

PUT /item HTTP/1.1\nHost: api.example.com\nContent-Type: application/json\n\n[\n\t{ \"id\": 123, \"name\": \"Item One Updated\", \"etag\": \"a1b2c3d4\" }\n]\n

This mechanism minimizes race conditions and guarantees consistency for batch operations.

12. No API Versioning in URLs

Traditional REST Approach

Version numbers are commonly embedded in URLs (e.g., /v1/users, v1.example.com/users), which can lead to fragmentation and maintenance overhead.

Challenges

Including version numbers in the URL clutters the design and complicates the evolution of the API, especially when changes are either mandatory or better expressed through distinct endpoints.

Version numbers embedded in URLs tend to clutter the API design and force clients to hardcode endpoint paths that may rapidly become outdated. By removing version numbers from the URL and instead leveraging HTTP headers—such as the Accept and Content-Type headers—for payload negotiation, the API remains clean and adaptable. This strategy allows changes to be introduced without breaking existing client implementations, as the evolution of data formats or business rules can be communicated through header negotiation rather than through separate endpoint versions.

Additionally, relying on versioned endpoints can inadvertently signal that a fundamental business process has changed while still supporting outdated behavior. When a new version of an endpoint introduces breaking changes, it implies that the underlying business process has evolved. In such cases, maintaining the old version not only creates confusion but also contradicts the notion of a genuine process change. Either the business process has indeed changed—rendering the old endpoint obsolete and in need of removal—or a completely new business process is introduced, which should be reflected through distinct endpoint naming rather than a simple version increment.

HyperAction API Decision

Version numbers are removed from URLs. Instead, use HTTP headers such as Accept and Content-Type to indicate the payload format or model variant, if necessary. This keeps the endpoint clean and directs versioning concerns to header negotiations.

Example

Consider a client request to the /account/summary endpoint that negotiates the desired payload format via the Content-Type and Accept headers rather than embedding a version number in the URL:

HTTP Request

GET /account/summary HTTP/1.1\nHost: api.example.com\nAccept: application/vnd.example.account-summary-with-balance+json

In a successful scenario, the server recognizes the specified version in the headers and responds with the corresponding data model:

Successful HTTP Response

HTTP/1.1 200 OK\nContent-Type: application/vnd.example.account-summary-with-balance+json\nX-API-Status: Active\n\n[\n\t{\n\t\t\"accountId\": 123,\n\t\t\"name\": \"Alice\",\n\t\t\"balance\": 250.75\n\t}\n]\n

Alternatively, if the client submits a request using a deprecated accept type (for example, application/vnd.example.account-summary+json), the server can indicate that this request model is no longer supported. This response clearly communicates that the API no longer supports the outdated version, urging the client to update its request headers:

Deprecated Request HTTP Response

HTTP/1.1 410 Gone\nContent-Type: application/json\nX-API-Status: Deprecated\n\n{\n\t\"error\": \"Not Acceptable\",\n\t\"message\": \"The requested content type 'application/vnd.example.account-summary+json' is deprecated and no longer supported. Please update your Accept header to a supported version.\"\n}\n

It is also acceptable for the server to respond with just the appropriate headers and a 410 status code, without any accompanying content.

Note that a draft standardizing the use of a Deprecation header in HTTP responses is currently underway, as described in The Deprecation HTTP Header Field draft, and is still in active development at the time of writing this article.

Conclusion

The HyperAction API convention represents a deliberate reimagining of HTTP API design by enforcing batch processing at every endpoint and a singular naming convention throughout. By eliminating single-item endpoints, abstract pagination models, and generic resource endpoints—and by always using singular names—the API design closely aligns with the underlying business logic and operational realities. With every endpoint accepting and returning arrays of items, employing dedicated HTTP headers for metadata, explicit idempotency keys, and supporting asynchronous operations, the HyperAction API provides a robust, efficient, and clear contract between clients and servers.

Software engineers adopting these guidelines will benefit from clearer API semantics, reduced redundancy in client–server communication, and an interface that naturally mirrors the business operations it supports. This document serves as a comprehensive technical reference for designing modern HTTP APIs under the HyperAction API convention.

Update (July 2026): The HTTP QUERY Method (RFC 10008)

A year after the original publication of this article, RFC 10008 officially standardized the QUERY HTTP method. This standard aligns exceptionally well with the HyperAction API architecture.

QUERY provides a safe, idempotent request method that natively supports a request body—effectively bridging the gap where GET fell short. Under this updated model, HyperAction APIs can strictly categorize endpoints into two primary behaviors:

Adoption Considerations & Friction: While QUERY is the ideal protocol-level choice for read-heavy batch operations, full adoption introduces temporary operational friction. Many legacy client/server frameworks, API gateways, and WAFs do not yet support the QUERY verb natively or may strip payload bodies from non-POST requests. As framework ecosystems and infrastructure catch up over time, GET with URL parameters remains a fallback, but QUERY represents the long-term target for modern HyperAction API implementations.

Future Work & Feedback

Please note that this whole concept is still a work in progress. I would be very happy to receive any feedback or suggestions to help refine and improve this approach.

Appendix: Embracing Vertical Slice Architecture

The protocol conventions presented in this document are partly inspired by the principles of vertical slice architecture. While this article focuses on redefining HTTP request-response protocols—such as action-oriented endpoints, batch processing, and explicit metadata handling—the underlying inspiration comes from an architectural mindset that emphasizes self-contained business features.

Vertical slice architecture is a holistic approach to software organization. It advocates for encapsulating all layers of functionality—ranging from user interface and application logic to data access—within discrete slices that correspond to individual business capabilities. This separation minimizes interdependencies and simplifies testing, maintenance, and independent evolution of features.

It is important to note that this document addresses only the HTTP protocol conventions, not the deeper aspects of code organization. The protocol decisions described here are fully compatible with, and indeed inspired by, a vertical slice approach. However, the specifics of organizing your codebase into vertical slices—covering topics such as module boundaries, dependency management, and internal service communication—fall outside the scope of this article.

We plan to publish a separate article in the future that will explore these code organization topics in detail, providing practical guidance on how to implement vertical slice architecture to complement the protocol conventions outlined here.

", "date_published": "2025-03-11T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/announcing-querylink-v100-simplify-your-ui-and-data-integration/", "url": "https://byteaether.github.io/2025/announcing-querylink-v100-simplify-your-ui-and-data-integration/", "title": "Announcing QueryLink v1.0.0: Simplify Your UI and Data Integration", "content_html": "

We are thrilled to announce the official release of QueryLink v1.0.0! This milestone release marks the culmination of our efforts to provide a seamless bridge between UI components and backend data sources powered by IQueryable. Whether you're building dynamic data grids or data tables, QueryLink is here to simplify your life by handling filters, sorting, and query string conversions with minimal code.

What is QueryLink?

QueryLink is a NuGet package designed to integrate UI components (like datagrids and datatables) with backend IQueryable data sources. It helps you to:

By abstracting these common tasks, QueryLink lets you focus on building great applications without getting bogged down in repetitive query logic.

Key Features

With support for .NET 6.0, .NET 8.0, and .NET Standard 2.1, QueryLink fits neatly into your modern .NET projects.

Installation

Getting started is as simple as running a single command. Install the latest stable version of QueryLink via NuGet:

dotnet add package ByteAether.QueryLink

If you need a preview version, you can specify it with the --version option. This ease-of-installation helps you to integrate QueryLink into your project quickly and reliably.

How to Use QueryLink

QueryLink provides a few core concepts that make filtering and sorting a breeze:

1. Defining Filters and Orders

The Definitions class allows you to specify filter and order criteria. For example:

var definitions = new Definitions\n{\n\tFilters = new List<FilterDefinition>\n\t{\n\t\tnew(\"Name\", FilterOperator.Eq, \"John\"),\n\t\tnew(\"Age\", FilterOperator.Gt, 30)\n\t},\n\tOrders = new List<OrderDefinition>\n\t{\n\t\tnew(\"Name\", IsReversed: false),\n\t\tnew(\"Age\", IsReversed: true)\n\t}\n};

This snippet creates filters to select records where the name equals "John" and age is greater than 30, and then orders the results accordingly.

2. Using Overrides for Custom Behavior

Sometimes the default filtering logic isn't enough. QueryLink's Overrides class lets you tailor operations using expression-based overrides:

var overrides = new Overrides\n{\n\tFilter = new List<FilterOverride>\n\t{\n\t\tnew(p => p.Name, p => p.FullName)\n\t},\n\tOrder = new List<OrderOverride>\n\t{\n\t\tnew(p => p.Name, p => p.FullName)\n\t}\n};

This approach lets you substitute a property (like Name) with another (like FullName) when applying filters or orders, offering flexible customization.

3. Converting to and from Query Strings

Easily integrate with web APIs by converting your definitions to a query string and back:

string queryString = definitions.ToQueryString();\nDefinitions parsedDefinitions = Definitions.FromQueryString(queryString);

This feature ensures that your filtering and sorting logic is easily transferable over HTTP.

4. Applying Definitions to IQueryable

Finally, apply your definitions directly to an IQueryable source:

IQueryable query = dbContext.People.AsQueryable();\nquery = query.Apply(definitions, overrides);

By extending IQueryable, QueryLink seamlessly integrates with your existing data-access code, providing a clean and efficient query transformation pipeline.

Real-World Examples

QueryLink shines in practical scenarios—like integrating with a MudBlazor DataGrid alongside EF Core. The package simplifies server-side data loading by reading grid state, generating query strings, and applying filter and order definitions to your data source. This comprehensive approach means fewer lines of code and less boilerplate for you to manage.

Join the Community

We believe that community feedback is invaluable. As we celebrate this stable release, we invite you to contribute to the project:

Your contributions and insights will help shape the future of QueryLink.

License

QueryLink is released under the MIT License, ensuring that it remains free and open for your use and customization. For more details, please refer to the LICENSE file in the repository.

QueryLink v1.0.0 is here to make your UI and data integration simpler, more flexible, and efficient. We look forward to seeing the innovative solutions you build with it. Happy coding!

", "date_published": "2025-03-04T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/harnessing-weak-referenced-event-managers-and-publish-subscribe-patterns-in-net/", "url": "https://byteaether.github.io/2025/harnessing-weak-referenced-event-managers-and-publish-subscribe-patterns-in-net/", "title": "Harnessing Weak-Referenced Event Managers and Publish–Subscribe Patterns in .NET", "content_html": "

In modern application development, event-driven programming is ubiquitous. Whether you're building rich desktop applications with WPF or web applications with Blazor, events provide a powerful way to decouple components. However, one common pitfall in event-based systems is memory leaks caused by lingering event subscriptions. The weak event pattern, along with publish–subscribe architectures, offers a robust solution by using weak references to hold subscribers. This ensures that objects can be garbage collected when no longer needed, even if they remain subscribed to events.

In this article, we will explore the theory behind weak-referenced event managers, demonstrate their practical implementation in .NET using C#, and illustrate how these concepts can be applied to real-world scenarios—including a look at the ByteAether.WeakEvent library.

Understanding the Weak Event Pattern

What Is a Weak Event?

In the typical event subscription model in .NET, the publisher holds a strong reference to each subscriber via its delegate invocation list. If a subscriber does not explicitly unsubscribe, it will remain in memory—even when it's no longer needed—causing memory leaks. The weak event pattern addresses this issue by holding references to subscribers weakly. In simple terms, a weak reference allows the garbage collector to reclaim the subscriber's memory if no other strong references exist.

According to Microsoft's documentation on WPF weak event patterns, the weak event pattern is particularly valuable in scenarios where the publisher's lifetime is longer than that of its subscribers. This pattern minimizes the risk of memory leaks by decoupling the event's lifecycle from its subscribers.

Key Benefits

Deep Dive into .NET Garbage Collection and Weak References

The Role of the Garbage Collector

The .NET garbage collector (GC) is responsible for managing memory automatically. It frees objects that are no longer referenced by the application. However, in traditional event subscriptions, the publisher's reference to the subscriber via the event delegate prevents the GC from collecting the subscriber—even if the subscriber is no longer actively used.

How Weak References Work

A weak reference allows an object to be referenced without preventing its collection by the GC. Microsoft explains in its .NET garbage collection documentation on weak references that a weak reference object does not count as a "root" for garbage collection. This means that if the only references to an object are weak, the GC can reclaim that object's memory.

Example: Using WeakReference in C#

public class MySubscriber\n{\n\tpublic void HandleEvent(object sender, EventArgs e)\n\t{\n\t\tConsole.WriteLine(\"Event handled by MySubscriber.\");\n\t}\n}\n\npublic class Publisher\n{\n\t// List of weak references to event handlers\n\tprivate List<WeakReference<EventHandler>> _handlers = new List<WeakReference<EventHandler>>();\n\n\tpublic void Subscribe(EventHandler handler)\n\t{\n\t\t_handlers.Add(new WeakReference<EventHandler>(handler));\n\t}\n\n\tpublic void RaiseEvent()\n\t{\n\t\tforeach (var weakHandler in _handlers)\n\t\t{\n\t\t\tif (weakHandler.TryGetTarget(out EventHandler handler))\n\t\t\t{\n\t\t\t\thandler.Invoke(this, EventArgs.Empty);\n\t\t\t}\n\t\t}\n\t}\n}

In this simple example, the Publisher class holds a list of weak references to event handlers. When raising the event, it attempts to invoke only those handlers that are still alive. This strategy prevents dead objects from keeping themselves in memory alive indefinitely.

The Publish–Subscribe Pattern and Its Synergy with Weak References

The publish–subscribe (pub–sub) pattern is an asynchronous messaging paradigm where publishers emit events and subscribers listen for those events. This decoupling means that the publisher does not need to know the specifics of its subscribers. Traditionally, this pattern improves scalability and maintainability, particularly in distributed systems and event-driven architectures.

Combining Weak References with Pub–Sub

By integrating weak references into a publish–subscribe system, developers can manage event subscriptions more safely. The use of weak references ensures that even if a subscriber forgets to unsubscribe, the system remains robust against memory leaks. This is especially critical in long-running applications or dynamic environments like modern web frameworks.

Example: A Weak-Referenced Publish–Subscribe System in C#

Consider a more structured implementation where a mediator handles event subscriptions:

public interface IEventSubscriber<TEventArgs>\n{\n\tvoid OnEvent(object sender, TEventArgs e);\n}\n\npublic class WeakEventManager<TEventArgs> where TEventArgs : EventArgs\n{\n\tprivate List<WeakReference<IEventSubscriber<TEventArgs>>> _subscribers\n\t\t= new List<WeakReference<IEventSubscriber<TEventArgs>>>();\n\n\tpublic void Subscribe(IEventSubscriber<TEventArgs> subscriber)\n\t{\n\t\t_subscribers.Add(new WeakReference<IEventSubscriber<TEventArgs>>(subscriber));\n\t}\n\n\tpublic void Unsubscribe(IEventSubscriber<TEventArgs> subscriber)\n\t{\n\t\t_subscribers.RemoveAll(wr =>\n\t\t{\n\t\t\tif (wr.TryGetTarget(out IEventSubscriber<TEventArgs> target))\n\t\t\t{\n\t\t\t\treturn target.Equals(subscriber);\n\t\t\t}\n\t\t\treturn true; // Clean up dead references\n\t\t});\n\t}\n\n\tpublic void RaiseEvent(object sender, TEventArgs e)\n\t{\n\t\t_subscribers.RemoveAll(wr => !wr.TryGetTarget(out _)); // Cleanup\n\t\tforeach (var weakSubscriber in _subscribers)\n\t\t{\n\t\t\tif (weakSubscriber.TryGetTarget(out IEventSubscriber<TEventArgs> subscriber))\n\t\t\t{\n\t\t\t\tsubscriber.OnEvent(sender, e);\n\t\t\t}\n\t\t}\n\t}\n}

In this code, the WeakEventManager<TEventArgs> class encapsulates the publish–subscribe pattern. It leverages weak references to avoid preventing the garbage collection of its subscribers. The cleanup logic ensures that any references to objects that have been collected are removed.

Real-World Applications: ByteAether.WeakEvent NuGet Library

The ByteAether.WeakEvent NuGet library provides a ready-to-use implementation of the generic weak event pattern using the publish–subscribe mechanism. It is built on the principles discussed above and is tailored for .NET developers seeking an out-of-the-box solution for managing event lifecycles.

According to its description:

The weak event pattern is a proven approach to managing event subscriptions in .NET, as detailed in Microsoft's documentation on WPF weak event patterns. This design avoids memory leaks by holding event subscribers with weak references, which means that even if an object remains subscribed to an event, it can still be garbage collected when no longer in use.

Leveraging the power of .NET's weak reference mechanism, as explained in the .NET garbage collection documentation, the WeakEvent library ensures that event subscribers do not prevent the garbage collector from reclaiming memory. This decoupling of publishers and subscribers provides a robust solution for managing event lifecycles without the need for manual unsubscription.

Benefits in Blazor and Other Modern Frameworks

When developing Blazor components, for example, unsubscribing from events during component disposal can be tricky. The ByteAether.WeakEvent library addresses this by ensuring that subscriptions are weak; once a component is no longer rendered, the weak reference does not prevent its memory from being reclaimed. This automatic cleanup is invaluable in preventing the accumulation of "dead" components that can degrade performance over time.

Example Usage in a Blazor Component

@code {\n\t[Inject]\n\tprotected readonly Publisher _publisher { get; set; } = default!;\n\n\tprotected override void OnInitialized()\n\t{\n\t\t// Assume Publisher has a public property WeakEvent<MyEventData> OnPublish\n\t\t_publisher.OnPublish.Subscribe(OnEvent);\n\t}\n\n\tpublic void OnEvent(MyEventData eventData)\n\t{\n\t\t// Handle the event (e.g., update UI state)\n\t\tConsole.WriteLine(\"Event received in Blazor component.\");\n\t}\n\n\tpublic void Dispose()\n\t{\n\t\t// No need to manually unsubscribe, the weak reference handles cleanup.\n\t}\n}

In this Blazor example, the component subscribes to event in a Publisher instance, that makes use of ByteAether.WeakEvent library. Even if the component is disposed, the weak reference does not hinder the garbage collection process, eliminating one of the common pitfalls in event-driven UI development.

Broader Implications and Cross-Platform Benefits

Although our examples and discussion have been centered on C# and .NET, the challenges of managing event lifecycles and preventing memory leaks are common across all programming environments. Languages and frameworks that use event-driven paradigms—such as JavaScript, Java, and even C++—face similar issues when subscribers outlive their useful life due to lingering references.

Applying Weak-Referenced Event Patterns in Other Environments

Many modern frameworks provide mechanisms similar to .NET's weak references. For instance:

Thus, while the syntax and APIs differ, the fundamental benefit remains the same: decoupling the event subscription mechanism from the object lifecycle leads to more robust, maintainable, and memory-efficient applications.

Conclusion

The weak event pattern combined with the publish–subscribe model represents a powerful approach to managing events in software engineering. By utilizing weak references, developers can mitigate common pitfalls such as memory leaks and dangling subscriptions, ensuring that components are cleaned up promptly by the garbage collector. In the .NET world, where managed memory and automatic garbage collection are key features, these patterns are especially valuable.

Libraries like ByteAether.WeakEvent encapsulate these best practices, providing developers with a plug-and-play solution to implement weak event patterns easily. Whether you're developing desktop applications with WPF, building modern web applications with Blazor, or designing systems in other environments, understanding and leveraging weak-referenced event managers can significantly enhance application reliability and performance.

As the landscape of software development evolves—with more dynamic and component-based systems emerging—the need for robust event management becomes increasingly critical. Embracing patterns that promote decoupling and automatic resource management not only leads to cleaner code but also results in applications that are more scalable and resilient in the face of growing complexity.

By integrating these approaches into your development practices, you are better equipped to build systems that manage resources intelligently, ultimately leading to a smoother user experience and more maintainable codebases.

", "date_published": "2025-02-25T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/seamlessly-connecting-frontend-and-backend-data-in-net-applications-a-new-paradigm-for-effortless-integration/", "url": "https://byteaether.github.io/2025/seamlessly-connecting-frontend-and-backend-data-in-net-applications-a-new-paradigm-for-effortless-integration/", "title": "Seamlessly Connecting Frontend and Backend Data in .NET Applications: A New Paradigm for Effortless Integration", "content_html": "

In today's fast‐paced software development world, engineers are often forced to write repetitive integration logic to connect user interfaces - especially data grids and tables - to their backend data sources. Every project, every new feature, and every iteration seems to reintroduce the same challenges: how do we reliably filter, sort, and display data without reinventing the wheel each time? The answer lies in automating these common tasks, thereby freeing developers to focus on business logic rather than boilerplate code.

The Problem: Repetitive Integration Logic

Consider a typical .NET application where you need to fetch, filter, and sort data from a SQL database using Entity Framework Core. In many projects, developers are repeatedly writing similar code snippets to achieve this functionality. For example:

// Example: Manually applying filters and sorting on a data source\npublic IQueryable<Person> GetFilteredAndSortedPeople(\n\tApplicationDbContext dbContext,\n\tstring name,\n\tint? minAge,\n\tstring sortField\n)\n{\n\tIQueryable<Person> query = dbContext.People.AsQueryable();\n\n\t// Repetitive filtering logic:\n\tif (!string.IsNullOrWhiteSpace(name))\n\t{\n\t\tquery = query.Where(p => p.Name == name);\n\t}\n\tif (minAge.HasValue)\n\t{\n\t\tquery = query.Where(p => p.Age >= minAge.Value);\n\t}\n\n\t// Repetitive sorting logic:\n\tif (sortField == \"Name\")\n\t{\n\t\tquery = query.OrderBy(p => p.Name);\n\t}\n\telse if (sortField == \"Age\")\n\t{\n\t\tquery = query.OrderByDescending(p => p.Age);\n\t}\n\n\treturn query;\n}

Now, imagine this pattern repeated across multiple controllers, services, and even projects. Each time you need to tweak a filter or adjust a sort order, you are forced to manually update these code blocks - often duplicating logic and introducing room for inconsistencies or bugs.

Another example might involve converting query parameters from the UI into LINQ expressions:

// Example: Converting UI query parameters into LINQ filter expressions\npublic IQueryable<Product> FilterProducts(\n\tIQueryable<Product> query,\n\tDictionary<string, string> filters\n)\n{\n\tforeach (var filter in filters)\n\t{\n\t\tswitch (filter.Key.ToLower())\n\t\t{\n\t\t\tcase \"category\":\n\t\t\t\tquery = query.Where(p => p.Category == filter.Value);\n\t\t\t\tbreak;\n\t\t\tcase \"price\":\n\t\t\t\tif (decimal.TryParse(filter.Value, out var price))\n\t\t\t\t{\n\t\t\t\t\tquery = query.Where(p => p.Price >= price);\n\t\t\t\t}\n\t\t\t\tbreak;\n\t\t\tdefault:\n\t\t\t\tbreak;\n\t\t}\n\t}\n\n\treturn query;\n}

Problems with this approach:

Clearly, there is a need for a generic, reusable solution that abstracts away this repetitive integration logic.

Identifying the Requirements for a Generic Solution

To create a generic solution for handling data integration, certain core requirements must be met:

1. Serializability

For a solution to work seamlessly between the client and server, its filter and order definitions must be serializable. This allows easy transmission over HTTP without losing state.

2. Integration with IQueryable and EF Core

The solution must work directly with IQueryable, ensuring that filtering and sorting logic is applied at the database level, improving performance.

3. Flexible Filter Definitions

A robust solution should support a variety of filtering operators such as:

... and more.

4. Support for Overrides

Developers should be able to specify overrides for sorting and filtering logic to handle cases where default behavior isn't sufficient. For example, sorting should support custom overrides, like sorting Firstname as [Firstname] [Lastname] to ensure consistent ordering.

A Modern Approach to Automating Data Query Integration

An innovative solution was born: QueryLink. This library encapsulates all the above requirements into a single, easy-to-use package.

Key Features:

  1. Filter and Order Definitions: Define filters and sorting orders without writing complex LINQ expressions.
  2. Expression-based Overrides: Customize filtering or sorting logic using lambda expressions with type safety.
  3. Query String Conversion: Convert definitions to query strings for seamless transmission over GET parameters.
  4. Direct IQueryable Integration: Ensures efficient query execution at the database level.

Getting Started: Hands-On Code Examples

Defining Filters and Sorting Orders

var definitions = new Definitions\n{\n\tFilters =\n\t[\n\t\tnew(\"Name\", FilterOperator.Eq, \"John\"),\n\t\tnew(\"Age\", FilterOperator.Gt, 30)\n\t],\n\tOrders =\n\t[\n\t\tnew(\"Name\"),\n\t\tnew(\"Age\", IsReversed: true)\n\t]\n};

Customizing with Overrides (Optional)

var overrides = new Overrides<Person>\n{\n\tFilter =\n\t[\n\t\tnew(p => p.Name, p => p.FullName)\n\t],\n\tOrder =\n\t[\n\t\tnew(p => p.Name, p => p.FullName)\n\t]\n};

Applying Definitions to an IQueryable Source

IQueryable<Person> query = dbContext.People.AsQueryable();\nquery = query.Apply(definitions, overrides);

Embracing the New Way of Integration Logic

With QueryLink:

QueryLink transforms UI-to-database integration by:

Given these advantages, QueryLink is an essential library for every full-stack .NET application. By integrating this tool into your workflow, you empower your team to build more efficient and maintainable software.

Embrace the future of data integration with QueryLink.

Check out the QueryLink repository on GitHub.

", "date_published": "2025-02-18T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/ulids-as-the-default-choice-for-modern-systems-lessons-from-shopifys-payment-infrastructure/", "url": "https://byteaether.github.io/2025/ulids-as-the-default-choice-for-modern-systems-lessons-from-shopifys-payment-infrastructure/", "title": "ULIDs as the Default Choice for Modern Systems: Lessons from Shopify's Payment Infrastructure", "content_html": "

In distributed systems, the choice of identifier is rarely a neutral decision. It impacts everything from database performance to operational debuggability. For years, UUIDs (Universally Unique Identifiers) were the default solution for generating unique keys, but their limitations become glaring in high-scale environments. Shopify's engineering team encountered these challenges firsthand while rebuilding their payment systems - a project where milliseconds and reliability mattered. Their solution? ULIDs (Universally Unique Lexicographically Sortable Identifiers). This article explores why ULIDs are not just an alternative to UUIDs but a superior default choice for most applications, drawing on Shopify's experience to illustrate their transformative potential.

What Are ULIDs, and Why Do They Matter?

A ULID is a 128-bit identifier designed to combine uniqueness with inherent sortability. Its structure is simple but powerful:

For example, the ULID 01JJP9VSVRWSMP4QEJTYYFSJE8 starts with a timestamp (01JJP9VSVR) that corresponds to a precise moment in time, followed by a random suffix (WSMP4QEJTYYFSJE8). This design allows ULIDs to be sorted lexicographically, meaning they retain the chronological order of their creation. Unlike UUIDs, which are fundamentally random, ULIDs embed meaningful metadata (the timestamp) while preserving uniqueness.

The Problem with UUIDs in Real-World Systems

Shopify's payment systems initially relied on UUIDv4 for idempotency keys—unique identifiers that prevent duplicate processing of payment requests. While UUIDs ensured uniqueness, their randomness created two critical issues:

  1. Database Performance Degradation: Databases like MySQL use B-tree indexes, which optimize for sequential writes. Random UUIDs force the database to insert entries at arbitrary positions in the index, leading to frequent page splits. These splits fragment the index, increasing write latency and disk I/O. At Shopify's scale, where millions of transactions occur daily, this fragmentation caused measurable slowdowns.
  2. Operational Complexity: UUIDs lack embedded metadata. To trace the timing of a transaction, engineers had to join tables or query additional timestamp columns. This added complexity to debugging and auditing, especially during incidents requiring rapid root-cause analysis.

ULIDs addressed both problems. By making identifiers sortable, they eliminated the database fragmentation caused by UUIDs. The embedded timestamp also provided immediate context for debugging, reducing reliance on secondary metadata.

Shopify's Journey: How ULIDs Transformed Their Payment Systems

The Role of Idempotency Keys

In payment processing, idempotency is critical. If a buyer's network connection drops during checkout, the client must retry the payment without risking a double charge. Idempotency keys solve this: the client sends the same key with each retry, and the server returns the original result instead of reprocessing the request.

Initially, Shopify used UUIDv4 for these keys. However, as transaction volume grew, the limitations of random UUIDs became untenable.

The Switch to ULIDs

Shopify replaced UUIDv4 with ULIDs for idempotency keys, achieving three key improvements:

  1. 50% Faster Database Inserts: By ensuring identifiers were chronologically ordered, ULIDs allowed MySQL to append new records sequentially. This aligned with the natural behavior of B-tree indexes, reducing page splits and write latency. In high-throughput systems, this optimization halved the time required for INSERT operations.
  2. Simplified Debugging: The timestamp within ULIDs eliminated the need for separate timestamp columns. Engineers could instantly determine when a transaction occurred by parsing the ULID, accelerating incident response. For example, filtering logs by ULID prefixes narrowed results to specific time windows without complex queries.
  3. Reduced Storage Overhead: With ULIDs, Shopify no longer needed to maintain separate timestamp columns for sorting. This streamlined their schema and reduced storage costs.

Technical Implementation Insights

Shopify's implementation highlights practical considerations for adopting ULIDs:

Why ULIDs Are Almost Always the Better Default

Performance and Scalability

In distributed systems, write performance is often a bottleneck. ULIDs eliminate the overhead of random writes, making them ideal for high-throughput applications like payment processing, event logging, or real-time analytics. Sequential writes reduce disk I/O and index maintenance, which is critical at scale.

Built-In Observability

ULIDs embed a creation timestamp, effectively acting as free metadata. This is invaluable for debugging:

Future-Proof Design

ULIDs are encoding-agnostic. While often represented in Base32 (which is URL-safe and avoids UUID's hexadecimal ambiguity), they can be stored as binary or other formats. This flexibility makes ULIDs adaptable to diverse use cases, from API request IDs to distributed trace identifiers.

Collision Resistance: Sufficient for Most Cases

While UUIDv4 offers 122 bits of randomness (compared to ULID's 80 bits), ULID's collision probability remains negligible for nearly all applications. At 80 bits, the chance of collision is roughly 1 in 1.2×10^24 — a risk so low that even systems processing billions of daily transactions would take millennia to encounter a duplicate.

When UUIDs Might Still Be Appropriate

UUIDs retain niche utility in scenarios where hiding timestamps is a requirement, such as:

However, these cases are exceptions. For the vast majority of applications — payment systems, IoT event streams, user session tracking — ULIDs offer a strict superset of UUID benefits.

Adopting ULIDs: Best Practices

Shopify's experience provides a blueprint for successful ULID adoption:

  1. Prioritize High-Impact Systems: Begin with critical components where performance gains matter most (e.g., payment idempotency keys).
  2. Leverage Database Optimizations: Use clustered indexes (e.g., InnoDB's PRIMARY KEY) to fully benefit from sequential writes.
  3. Educate Teams: Ensure engineers understand how to extract timestamps from ULIDs and leverage their sortability in queries.

Conclusion: ULIDs as the New Standard

Shopify's success with ULIDs underscores a broader truth: in modern system design, uniqueness alone is insufficient. Identifiers must also enable performance, observability, and scalability. ULIDs deliver on all fronts, making them the default choice for engineers building resilient systems.

UUIDs served a vital role in the past, but their randomness is a relic of an era before hyperscale systems. ULIDs, with their blend of sortability and uniqueness, address the demands of today's applications. As Shopify demonstrated, the benefits are tangible—reduced latency, streamlined debugging, and simpler schemas.

Unless a system explicitly requires timestamp obfuscation or must maintain legacy compatibility, ULIDs should be the default. They represent an evolution in identifier design, one that aligns with the realities of distributed, high-scale computing. The question isn't whether to adopt ULIDs—it's how quickly you can migrate.

For a deeper dive into Shopify's approach, read their original article: 10 Tips for Building Resilient Payment Systems.

", "date_published": "2025-02-11T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/uuid-vs-ulid-vs-integer-ids-a-technical-guide-for-modern-systems/", "url": "https://byteaether.github.io/2025/uuid-vs-ulid-vs-integer-ids-a-technical-guide-for-modern-systems/", "title": "UUID vs ULID vs Integer IDs: A Technical Guide for Modern Systems", "content_html": "

Unique identifiers are critical components in software systems, serving as the foundation for data management, distributed architectures, and secure API design. While UUIDs (specifically UUIDv4) and integer IDs have been widely adopted, ULIDs (Universally Unique Lexicographically Sortable Identifiers) are increasingly recognized as a superior choice for modern applications. This article explores the technical distinctions between these identifiers, focusing on performance in .NET ecosystems and the database-level implications of their structural differences.

Understanding the Identifier Types

Integer IDs

Integer IDs are sequential numeric values typically managed by databases through auto-increment mechanisms. Their simplicity and minimal storage requirements (4 bytes for INT, 8 bytes for BIGINT) make them straightforward to implement and efficient for indexing. However, their predictability exposes systems to security risks such as enumeration attacks (e.g., /users/123). Additionally, their reliance on centralized generation makes them unsuitable for distributed systems where coordination between nodes is impractical.

UUIDs

UUIDs are 128-bit identifiers designed to guarantee global uniqueness through randomness. The UUIDv4 variant, which uses 122 bits of random data, is the most common implementation. While UUIDs excel at collision resistance, their lack of inherent sortability leads to significant database performance issues, particularly in write-heavy systems. The random distribution of UUIDv4 values causes index fragmentation, increasing I/O overhead and reducing cache efficiency.

ULIDs

ULIDs are 128-bit identifiers composed of a 48-bit timestamp (millisecond precision) and 80 bits of cryptographically secure randomness. Encoded as a 26-character base32 string, ULIDs are lexicographically sortable, compact, and URL-safe. The timestamp prefix ensures that new records are inserted in chronological order, aligning with database indexing patterns to minimize fragmentation.

Performance in .NET Applications

ULID Generation and Monotonicity

In .NET, the ByteAether.Ulid library provides a high-performance implementation of ULID generation. Benchmarks published in its GitHub repository demonstrate that it generates ULIDs with zero heap allocations, leveraging stack-based operations and optimized bitwise manipulation. A key feature of ULIDs is monotonicity, which ensures that identifiers generated within the same millisecond increment sequentially. This guarantees that data inserted in a logical order (e.g., batch processing workflows) remains ordered in the database. For example, in a distributed logging system, monotonicity ensures log entries from the same source retain their sequence even if generated microseconds apart. ByteAether.Ulid further enforces this by incrementing the timestamp when the random component overflows, avoiding errors during generation while preserving order.

Why UUIDs Fall Short

While Guid.NewGuid() in .NET is fast, UUIDs lack structural ordering. This forces databases to insert records at random positions within indexes, leading to frequent page splits and fragmentation. ULIDs, by contrast, align insertion patterns with the natural ordering of database indexes, reducing overhead and improving throughput.

Database Performance: Index Fragmentation Explained

How Databases Organize Data

Databases store records in fixed-size blocks called pages (e.g., 8kB in SQL Server, 16kB in MySQL). These pages are managed within B+ tree structures, where leaf nodes store actual data rows, and internal nodes route queries using key values. Sequential insertion ensures new rows are added to the end of the leaf node chain, minimizing disk seeks and memory usage.

The Impact of Random Identifiers

When identifiers are random (e.g., UUIDv4), new rows are inserted at arbitrary positions within the B+ tree. This forces databases to split existing pages to accommodate entries, a process known as page splitting. For example, inserting a record with UUID f47ac10b-... into a page already containing a5670e02-... may split the page into two, with half the rows moved to a new location.

Page splits have three major consequences:

  1. Increased I/O Overhead: Fragmented pages require more disk reads to retrieve logically contiguous data.
  2. Memory Pressure: More pages are loaded into the buffer pool, reducing the cache's effectiveness.
  3. Write Amplification: Splits generate additional transaction log entries, slowing bulk inserts and increasing storage costs.

ULID’s Structural Advantage

ULIDs embed a timestamp in their high-order bits, ensuring that new records are inserted sequentially. This aligns with the B+ tree’s design, reducing page splits compared to UUIDv4.

Implementing ULIDs in .NET with ByteAether.Ulid

Integration with Database Systems

The ByteAether.Ulid library simplifies storing ULIDs as binary data, minimizing storage overhead and accelerating queries. Below are examples of integrating ULIDs with popular .NET data access frameworks:

Entity Framework Core Configuration

public class Order\n{\n\tpublic Ulid Id { get; set; } // Stored as BINARY(16)\n\tpublic string Product { get; set; }\n}\n\npublic class UlidToBytesConverter : ValueConverter<Ulid, byte[]>\n{\n\tprivate static readonly ConverterMappingHints DefaultHints = new(size: 16);\n\n\tpublic UlidToBytesConverter() : this(defaultHints) { }\n\n\tpublic UlidToBytesConverter(ConverterMappingHints? mappingHints = null)\n\t\t: base(\n\t\t\tconvertToProviderExpression: x => x.ToByteArray(),\n\t\t\tconvertFromProviderExpression: x => Ulid.New(x),\n\t\t\tmappingHints: defaultHints.With(mappingHints)\n\t\t)\n\t{ }\n}\n\n// In DbContext configuration\nprotected override void OnModelCreating(ModelBuilder modelBuilder)\n{\n\t// ...\n\tconfigurationBuilder\n\t\t.Properties<Ulid>()\n\t\t.HaveConversion<UlidToBytesConverter>();\n\t// ...\n}

Dapper Type Handler

public class UlidTypeHandler : SqlMapper.TypeHandler<Ulid>\n{\n\tpublic override void SetValue(IDbDataParameter parameter, Ulid value)\n\t{\n\t\tparameter.Value = value.ToByteArray();\n\t}\n\n\tpublic override Ulid Parse(object value)\n\t{\n\t\treturn Ulid.New((byte[])value);\n\t}\n}\n\nDapper.SqlMapper.AddTypeHandler(new UlidTypeHandler());

Simplified Integration with Companion Libraries (v1.4.0+)

While writing custom type converters and handlers provides a clear view of the ORM plumbing, manual setup is no longer required for standard projects. With the release of ByteAether.Ulid v1.4.0, official companion packages are available on NuGet to handle registration and persistence strategies automatically:

These integration packages also ensure consistent byte layout strategies (such as SqlServerGuid handling) across complex query types and sorting operations.

Performance Metrics

The ByteAether.Ulid library generates ULIDs efficiently with zero heap allocations, making it ideal for high-throughput scenarios like event sourcing or real-time analytics. Detailed benchmark results are available in its GitHub repository.

Recommendations and Conclusion

When to Use ULIDs

ULIDs are particularly advantageous in:

Why ByteAether.Ulid?

The Future of Identifiers

UUIDs and integer IDs remain viable for legacy use cases, but ULIDs address their core limitations: sortability, fragmentation, and scalability. For .NET developers, ByteAether.Ulid offers a robust, future-proof solution that aligns application logic with database storage mechanics.

# Install ByteAether.Ulid\ndotnet add package ByteAether.Ulid

By adopting ULIDs, teams can achieve faster queries, reduced infrastructure costs, and architectures that scale effortlessly—proving that the right identifier can transform system design.

", "date_published": "2025-02-04T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/an-introduction-to-ulids-a-modern-identifier-for-software-systems/", "url": "https://byteaether.github.io/2025/an-introduction-to-ulids-a-modern-identifier-for-software-systems/", "title": "An Introduction to ULIDs: A Modern Identifier for Software Systems", "content_html": "

In the world of software development, identifiers are a cornerstone of data storage and retrieval. Whether you are designing a database, an API, or a distributed system, choosing the right identifier scheme is critical to ensuring performance, scalability, and reliability. Two widely used options are auto-incrementing integers and UUIDs (Universally Unique Identifiers). While each has its strengths, they also come with significant trade-offs. ULIDs (Universally Unique Lexicographically Sortable Identifiers) have emerged as a modern alternative that combines the best of both worlds. This article introduces ULIDs by first exploring the strengths and limitations of traditional identifier schemes and then delving into the advantages of ULIDs.

The Strengths and Limitations of Auto-Incrementing Integers

Auto-incrementing integers have been a staple in relational databases for decades. They are simple to implement, easy to understand, and offer excellent performance in single-node systems. With each new row, the database generates a unique, sequential integer that serves as the primary key. This approach ensures:

However, this simplicity comes at a cost when dealing with distributed systems or high-scale applications:

UUIDs: Strengths and Weaknesses

UUIDs were designed to address the distributed nature of modern systems. By using a combination of time, machine identifiers, and random numbers, UUIDs provide globally unique identifiers without requiring central coordination. Their advantages include:

However, the randomness inherent in UUIDs introduces its own set of challenges:

Introducing ULIDs: The Best of Both Worlds

ULIDs were designed to overcome the limitations of both auto-incrementing integers and UUIDs. By combining a 48-bit timestamp with an 80-bit random component, ULIDs achieve:

The official ULID specification, available on GitHub, provides detailed guidelines for implementing ULIDs in various programming languages.

The Role of Crockford's Base32 Encoding

A key feature of ULIDs is their use of Crockford's Base32 encoding, which is designed to enhance human readability. This encoding scheme avoids visually similar characters, such as "I" and "1" or "O" and "0," reducing the chance of errors when reading or transcribing ULIDs. Additionally, Base32-encoded ULIDs are case-insensitive, making them easier to handle in systems that do not preserve case.

By encoding ULIDs in Base32, developers can work with identifiers that are compact, consistent, and user-friendly, bridging the gap between machine efficiency and human usability.

Monotonicity: Preserving Order in High-Throughput Systems

ULIDs also address a unique challenge in high-throughput systems: ensuring that identifiers remain ordered even when multiple are generated within the same millisecond. The monotonicity feature of ULIDs guarantees that if multiple ULIDs share the same timestamp, they are incremented to maintain their lexicographical order. This ensures:

Why ULIDs Matter for Modern Applications

ULIDs are particularly well-suited for applications that require scalability, temporal ordering, and distributed generation. Use cases include:

  1. Event Logging: Ensures efficient storage and retrieval of time-stamped log entries.
  2. Distributed Databases: Eliminates the need for central coordination while preserving order.
  3. Analytics and Reporting: Facilitates time-based filtering and aggregation of data.
  4. High-Throughput Systems: Maintains performance and order even under heavy write loads.

Implementing ULIDs in Practice

For developers looking to adopt ULIDs, a variety of libraries are available across programming languages. In particular, the ByteAether's Ulid GitHub repository offers a high-performance C# implementation.

By following the official specification and leveraging well-maintained libraries, developers can integrate ULIDs into their systems with minimal effort and maximum reliability.

Conclusion

ULIDs offer a modern, versatile solution to the challenges of identifier generation in software systems. By combining the benefits of auto-incrementing integers and UUIDs while avoiding their pitfalls, ULIDs provide a unique identifier scheme that is globally unique, lexicographically sortable, and human-readable. With features like Crockford's Base32 encoding and monotonicity, ULIDs are tailored for the demands of modern applications, from distributed systems to time-based analytics.

Whether you are building a new system or optimizing an existing one, ULIDs represent a compelling option for identifier management. With their growing ecosystem of libraries and strong community support, they are poised to become a standard choice for developers seeking scalability, performance, and simplicity.

Start integrating ULIDs into your .NET applications today with ByteAether.Ulid. Let us know your thoughts and feedback—we’re excited to see what you build!

", "date_published": "2025-01-28T00:00:00Z" } , { "id": "https://byteaether.github.io/2025/announcing-byteaetherulid-v100-a-high-performance-net-ulid-library-for-modern-applications/", "url": "https://byteaether.github.io/2025/announcing-byteaetherulid-v100-a-high-performance-net-ulid-library-for-modern-applications/", "title": "Announcing ByteAether.Ulid v1.0.0: A High-Performance .NET ULID Library for Modern Applications", "content_html": "

Frustrated with GUIDs that aren’t sortable or human-readable? Meet ByteAether.Ulid - a ULID library designed to simplify your .NET development.

We’re excited to introduce ByteAether.Ulid v1.0.0, a high-performance .NET library for generating and working with ULIDs (Universally Unique Lexicographically Sortable Identifiers).

For developers managing distributed systems or working with time-sensitive data, ULIDs offer the perfect blend of uniqueness, readability, and sortability. ByteAether.Ulid takes it further by delivering exceptional performance, seamless integration with modern .NET projects, and comprehensive reliability.

Why ULIDs, and Why ByteAether.Ulid?

ULIDs provide unique identifiers that are:

ByteAether.Ulid goes beyond standard implementations:

FeatureGUIDsInteger IDsULIDsByteAether.Ulid Enhancements
Global Uniqueness✅ Yes❌ No✅ Yes✅ Yes
Time-Based Sortability❌ No✅ Yes✅ Yes✅ Yes
Human-Readable Format❌ No✅ Yes✅ Yes✅ Yes
Overflow Handling❌ N/A❌ N/A❌ Sometimes✅ Graceful & Reliable

ByteAether.Ulid fully adheres to the ULID specification while addressing common pitfalls, such as random-part overflow, ensuring a truly error-free experience.

Key Features of ByteAether.Ulid

1. Error-Free ULID Generation

Handles edge cases gracefully, like random-part overflow, by incrementing timestamps to maintain monotonicity without errors.

2. High-Performance Design

ByteAether.Ulid is optimized for speed and low memory usage, outperforming other libraries like NetUlid, NUlid, and Cysharp’s Ulid.

3. Seamless .NET Integration

4. Versatile Conversion Methods

Effortlessly convert ULIDs to and from:

Getting Started

Installation

Getting started is as simple as installing the NuGet package:

dotnet add package ByteAether.Ulid

Run dotnet list package to verify the installation. Visit the NuGet page for more details.

Usage

Here’s how simple it is to generate and use ULIDs with ByteAether.Ulid:

using System;\n\nclass Program\n{\n\tstatic void Main()\n\t{\n\t\t// Create a new ULID\n\t\tvar ulid = Ulid.New();\n\n\t\t// Convert to string and back\n\t\tstring ulidString = ulid.ToString();\n\t\tvar ulidFromString = Ulid.Parse(ulidString);\n\n\t\tConsole.WriteLine($\"Generated ULID: {ulid}\");\n\t}\n}

For detailed usage instructions and a complete API reference, check out the GitHub repository.

Performance Highlights

ByteAether.Ulid outpaces competitors while maintaining strict adherence to the ULID specification.

LibraryGeneration SpeedComplianceError Handling
ByteAether.Ulid✅ Fastest✅ Full✅ Graceful
NetUlid⚠️ Slower✅ Full⚠️ Limited
NUlid⚠️ Slower✅ Full⚠️ Limited
Cysharp.Ulid⚠️ Fast, Non-secure, Non-monotonic❌ Incomplete⚠️ Limited

For a full breakdown of benchmarks and testing methodology, visit our GitHub repository.

Open Source and Contributions

ByteAether.Ulid is fully open-source and licensed under the MIT License. Contributions are welcome! Whether you want to report bugs, suggest features, or submit a pull request, we’d love your input.

👉 Submit an Issue
👉 Contribute Code

What’s Next?

Stay tuned for deep dives into:

Start simplifying your .NET projects today. Install ByteAether.Ulid and experience the difference!

", "date_published": "2025-01-21T00:00:00Z" } ] }