At IdeaNova, we have a long history of developing License Servers for our DRM customers. Over the years, we have introduced numerous valuable features and important security enhancements.
However, despite reaching six major releases, our fundamental request-processing architecture has remained unchanged. We have relied on a conservative
thread-per-request model, which has served us well, allowing us to meet scalability requirements effortlessly.
The thread-per-request model has been widely used in foundational Internet architectures, such as the Apache HTTP Server and the original Java Servlets. It is conceptually simple and provides strong isolation between requests. This synchronous model processes I/O operations—such as reading from or writing to a file—in a blocking manner, meaning a thread waits until the operation completes. While a blocked thread allows the operating system scheduler to execute other threads in the meantime, this architecture has scalability limitations.
Scalability Challenges of the Thread-Per-Request Model
Despite advancements in OS-level thread handling, the thread-per-request model introduces scalability bottlenecks:
- High Memory Overhead: Each thread requires a fixed amount of memory, primarily for its stack. The stack size varies depending on the OS configuration but is typically in the range of a few megabytes (e.g., 8MB). Supporting 1,000 threads could consume 8GB of memory for stacks alone.
- Expensive Context Switching: When the OS switches from one running thread to another, it incurs a performance cost. The system must save the entire thread state, including CPU registers, and load the new thread’s state, which adds latency and reduces efficiency.
Embracing an Asynchronous, Event-Based Architecture
To overcome these limitations, most modern network servers use an asynchronous, event-driven architecture. Instead of blocking threads, the server registers interest in I/O events and is notified when data is available. This approach enables high concurrency while significantly reducing memory and CPU overhead.
A major upgrade to a core component of our system (a DRM vendor SDK) prompted us to rethink our software architecture. Given the increasing scalability demands on our DRM servers, we decided to adopt a more modern approach.
Introducing Rust and Tokio in Inplay License Server v7
For the next-generation
Inplay License Server v7, we have replaced our old C++-based synchronous architecture with a modern
Rust and Tokio-based asynchronous architecture. Our development team has had a highly positive experience with this stack in our
Inplay Edge product, making it a natural choice for this modernization effort.
While we do not yet have exact benchmark numbers to share, we have observed promising improvements in performance and scalability. Our initial results show potential for 5x-10x scalability improvements on 10+ core count system. On a typical IFE system this improvement maybe less pronounced, but we still expect to see noticeably better performance and scalability numbers nonetheless.
We are excited to complete development and testing and will share performance insights in a future post.