Paper Explained #1: Building Fair Financial Exchanges in the Cloud: How Jasper Solved the Microsecond Problem

Research Paper: “Jasper: Scalable and Fair Multicast for Financial Exchanges in the Cloud” — — — Full Paper

The Race to the Cloud

In 2021, Nasdaq announced a multi-billion dollar partnership with AWS to move their infrastructure to the cloud. CME Group followed suit, with Google investing $1 billion in their cloud transformation. The message was clear: Wall Street’s future is in the cloud.

But there’s a problem that these partnerships glossed over traditional stock exchanges rely on something the cloud fundamentally can’t provide: perfect timing.

On Wall Street, they literally measure cable lengths. Every trader gets exactly the same length of copper wire to the exchange server. Same distance = same speed = fair game. It sounds absurd, but when microseconds determine millions in profit, physics becomes your trading strategy. Some financial firms employ engineers whose job is to measure wire lengths with precision instruments, ensuring every market participant gets data at exactly the same moment.

Now imagine trying to recreate that fairness in Amazon’s data centers, where you don’t control the hardware, can’t measure the virtual “cables,” and share infrastructure with millions of other applications.

This is the challenge that researchers from NYU and Stanford took on with Jasper and their solution reveals just how creative systems engineering can get when physics meets cloud computing.

The Fairness Problem: Why Microseconds Matter

What “Fair” Actually Means

In financial markets, fairness isn’t a philosophical concept — it’s a technical requirement with regulatory teeth. Here’s what it means in practice:

Fair delivery: When Apple’s stock price changes from $150.00 to $150.01, every trader should learn about this change at exactly the same moment. Not “roughly” the same moment. Not “within a few milliseconds.” Exactly simultaneous.

The stakes are real. High Frequency Trading (HFT) firms make money by being microseconds faster than competitors. A delay measured in millionths of a second can mean the difference between profit and bankruptcy. The number of market participants typically ranges from hundreds to thousands, and they’re all competing on speed.

The Traditional Solution: Engineering Physics

On-premises exchanges solve this through brute force precision:

• Identical cable lengths: Engineers literally measure copper wires with specialized tools
• Custom switching hardware: Specialized network switches that can multicast data simultaneously to all participants
• Controlled environments: Complete control over every component from server to network interface

The result? When the exchange says “Apple = $150.01,” everyone hears it within nanoseconds of each other.

Why the Cloud Breaks Everything

Cloud computing introduces three deal-breaker problems for financial exchanges:

1. No Hardware Multicast Support Traditional exchanges use specialized switches that can duplicate a single packet to multiple destinations simultaneously, true hardware multicast. Cloud providers don’t give tenants this level of hardware control. Instead, you get virtual machines that must send individual copies to each recipient.

2. Latency Variance In controlled data centers, network delays are predictable. In the cloud, VM-to-VM communication varies wildly, sometimes 50 microseconds, sometimes 500, occasionally spiking to 5000 microseconds or more. This variance makes fair timing impossible with traditional approaches.

3. Scale Requirements Modern markets need to handle 1000+ participants simultaneously. Sending individual messages to each participant creates a cascading delay problem: by the time participant #1000 receives the message, participant #1 has already acted on it.

The math is brutal. If it takes 1 microsecond to send each message, and you have 1000 participants, the last person gets their data 1000 microseconds (1 millisecond) after the first person. In HFT terms, that’s an eternity.

Jasper’s Three-Part Solution

The Jasper system, developed by researchers from NYU and Stanford, tackles this seemingly impossible problem through three interconnected techniques. Let me break down each one.

Part 1: Tree Architecture: Solving the Distribution Problem

The Problem:

Traditional approach (Direct Unicast):
[Exchange] → individually sends to 1000 traders
Result: Trader #1000 gets data 50ms after Trader #1

Jasper’s Solution:

Tree Structure:
[Exchange] → 10 Proxy Servers → Each serves 100 traders
Result: Everyone gets data within ~5ms window

The insight is borrowed from high-performance computing: instead of the exchange directly contacting every trader, create a hierarchy of proxy servers that handle the distribution.

The Math Behind Tree Design

Jasper’s tree structure is defined by two key parameters:

• D = tree depth (number of levels)
• F = fanout (number of branches per level)

Through extensive experimentation, the researchers found an optimal formula:

• F = 10 (sweet spot between serialization delay and tree depth)
• D = ⌊log₁₀(N)⌋ (optimal depth for N participants)

Real Example for 1000 traders:

• D = ⌊log₁₀(1000)⌋ = 3 levels
• F = 10 branches per level

Level 0: [Exchange Server]
Level 1: [10 Proxy VMs]  
Level 2: [100 Proxy VMs] (10 children each from Level 1)
Level 3: [1000 Traders] (10 traders each from Level 2)

Performance Impact:

• Direct unicast median latency: 254μs
• Tree structure median latency: 150μs
• Improvement: 40% faster with better scalability

Why F=10? The researchers tested various fanout values:

• F=5: Too many levels, accumulating delays
• F=20: Serialization bottleneck (sending 20 messages back-to-back takes time)
• F=10: Mathematical sweet spot that minimizes total latency

Part 2: VM Hedging: Solving the Reliability Problem

Even with optimal tree structure, cloud networks introduce unpredictable delays. A single slow proxy can ruin fairness for all its downstream participants.

Traditional Approach:

[Proxy A] → [Trader] 
If Proxy A has network issues, Trader is stuck with high latency

Jasper’s Hedging:

[Proxy A] → [Trader]  ← First copy arrives at 10:00:01.001
[Proxy B] → [Trader]  ← Second copy arrives at 10:00:01.003
[Proxy C] → [Trader]  ← Third copy arrives at 10:00:01.005

Trader processes the first copy, ignores duplicates

Implementation: Sibling Hedging

Instead of adding dedicated backup nodes, Jasper uses “sibling hedging”- -each proxy helps its neighbors:

• Proxy A serves Traders 1–100 AND helps Proxy B by serving Traders 101–200
• Proxy B serves Traders 101–200 AND helps Proxy C by serving Traders 201–300
• Every trader receives the same message from multiple sources

Cost Formula:

Goodput = 1/(H+1) × Total throughput
where H = hedging factor

With H=2 (two backup paths):

• Goodput = 1/3 × 100% = 33% efficiency per proxy
• Need 3x more proxy VMs
• Get near-zero packet loss and dramatic variance reduction

Performance Analysis

The researchers ran Monte Carlo simulations with 100,000 iterations:

• H=0 (no hedging): High variance, occasional 10ms+ spikes
• H=1: 50% variance reduction, rare spikes above 1ms
• H=2: 75% variance reduction, virtually no spikes

Real Results:

• 99th percentile latency without hedging: ~300μs
• 99th percentile latency with H=1: ~150μs
• 50% improvement in worst-case performance

Part 3: Hold-and-Release: Solving the Synchronization Problem

Even with tree distribution and hedging, small timing differences remain:

• Trader A gets message at 10:00:01.001
• Trader B gets message at 10:00:01.003
• 2ms difference = potential unfair advantage

Jasper’s solution: a “traffic light” system for financial data.

How Hold-and-Release Works

Step 1: Measure Network Delays Each trader periodically measures and reports their typical one-way delay (OWD). Jasper collects the 95th percentile OWD for each participant, the delay that 95% of messages meet or beat.

Step 2: Calculate Global Deadline

Deadline = Current_time + max(all_trader_OWD95) + safety_buffer

Example:

• Trader A’s OWD95: 80μs
• Trader B’s OWD95: 120μs
• Trader C’s OWD95: 95μs
• Global deadline: Current_time + max(80, 120, 95) + 20μs buffer = Current_time + 140μs

Step 3: Attach Deadline to Messages

Message = "AAPL stock price = $150.01" + "Don't act until timestamp 10:00:01.000140"

Step 4: Synchronized Action All traders receive the message at different times but wait until the deadline timestamp. At exactly 10:00:01.000140, everyone can act simultaneously.

The Scalability Challenge

With 1000+ traders, collecting OWD measurements creates a new problem: network congestion from measurement traffic.

Jasper’s Solution: Reverse-Tree Aggregation

Instead of 1000 traders sending measurements directly to the exchange:

Level 3: [Traders] → report individual delays to their proxy parents
Level 2: [Proxies] → report max(child_delays) to their parents  
Level 1: [Proxies] → report max(child_delays) to exchange
Level 0: [Exchange] → receives single aggregated measurement

Network Traffic Reduction: 1000 individual reports → ~100 aggregated reports

This approach scales the hold-and-release mechanism from CloudEx’s ~10 participants to Jasper’s 1000+ participants.

Technical Implementation Deep Dive

DPDK: Bypassing the Operating System

Traditional network applications go through the operating system, creating overhead:

Normal path: [App] → [System Calls] → [Kernel] → [Network Stack] → [Hardware]
Latency per message: ~100μs

Jasper uses DPDK (Data Plane Development Kit) for direct hardware access:

DPDK path: [App] → [Direct Hardware Access]
Latency per message: ~10μs

The Difference: It’s like the difference between asking a secretary to pass along every message versus talking directly to the person. DPDK eliminates the middleman, letting applications control network cards directly. This removes context switching overhead and system call delays.

Threading Architecture:

Proxy VM (8 cores):
├── 1 core: RX thread (receive messages)
├── 6 cores: TX threads (send messages)  
└── 1 core: Management (statistics, monitoring)

The RX thread receives packets and dispatches them to TX threads via lockless queues: no waiting, no blocking.

Zero-Copy Optimization

The Problem: Copying a 1KB message to 1000 traders = 1MB of memory operations per message.

Traditional Approach: Make a complete copy of the message for each destination.

Zero-Copy Approach: Keep the message in one memory location and just create different headers pointing to it. It’s like having one newsletter that everyone reads versus photocopying 1000 individual newsletters.

The message payload stays in one memory location. Only packet headers (with different destination addresses) are created fresh. This reduces memory bandwidth usage by 99% and eliminates copy overhead entirely.

eBPF: Smart Packet Filtering

Hedging creates a new problem: each trader receives multiple copies of the same message. Traditional approaches would burden the application with deduplication.

Traditional Approach: All duplicate packets go through the full network stack to reach the application, which then has to figure out which ones to keep.

eBPF Approach: Smart filtering happens at the network card level. Think of it as having a smart doorman who knows which visitors you’ve already seen and turns away duplicates before they even enter the building.

How it works: The network card maintains a small table of recently seen message IDs. When a packet arrives, it checks: “Have I seen this message before?” If yes, drop it immediately. If no, let it through and remember the ID.

This deduplication happens at the network card level, before packets consume CPU cycles or memory bandwidth.

Deployment Models: Balancing Trust and Performance

Real-world deployment of Jasper faces a critical challenge: the hold-and-release mechanism requires traders to cooperate. What prevents a rogue trader from ignoring deadlines and gaining unfair advantage?

Jasper proposes three deployment models with different trust/performance trade-offs.

Model 1: Full Exchange Control

Architecture:

[Exchange-Controlled VM]
├── Hold-and-release program (Exchange runs this)
└── Trader's algorithm (Exchange runs this too)

Process:

1. Trader provides their proprietary trading algorithm to the exchange
2. Exchange runs both fair delivery system AND trader’s algorithm
3. Perfect fairness guaranteed since exchange controls everything

Performance:

• Median latency: 129μs
• Delivery window size: <1μs
• Perfect fairness probability: >95%

Trade-offs:

• Pros: Best performance and fairness, simple architecture
• Cons: Trader must reveal secret algorithms, high trust requirement, potential regulatory issues

Use Case: Internal trading within the same financial institution.

Model 2: Gateway Approach

Architecture:

[Exchange VM] → [Gateway VM] → [Trader's VM]
                     ↑
           Fairness enforced here

Process:

1. Exchange controls gateway VMs that enforce hold-and-release
2. Traders run their algorithms on separate VMs they control
3. Fair delivery guaranteed at gateway level

Performance:

• Median latency: ~170μs (extra hop adds ~40μs)
• Delivery window size: ~5μs
• Good fairness with some variance from gateway→trader path

Trade-offs:

• Pros: Trader algorithms stay private, fairness guaranteed by trusted gateway, moderate performance overhead
• Cons: Extra network hop introduces some variance, still requires trust in gateway operator

Use Case: Traditional financial exchanges serving external clients.

Model 3: Confidential Computing

Architecture:

[Exchange-Controlled VM]
├── Hold-and-release program (Exchange controls)
└── [ENCRYPTED ENCLAVE: Trader's algorithm] (Exchange cannot see)

This is the most innovative approach, using AWS Nitro Enclaves or similar confidential computing technology.

Technical Implementation:

• Exchange controls the VM and timing mechanisms
• Trader’s algorithm runs inside a cryptographically isolated enclave
• Even with root access to the VM, the exchange cannot see inside the enclave
• Hardware-level memory encryption protects the trader’s code and data

Performance Challenge: VM → Enclave communication adds variable latency (~20–80μs). This could break fairness if different VMs have different enclave delays.

Solution: Low-Jitter Optimizations The system applies several techniques to make enclave communication predictable: dedicating specific CPU cores to enclave work, disabling interrupts that cause timing variations, and pre-allocating memory to avoid unpredictable delays.

Latency Compensation: Different VMs will have different enclave communication delays:

• VM A → Enclave A: 40μs
• VM B → Enclave B: 60μs
• VM C → Enclave C: 45μs

Jasper measures these delays and adds compensation:

• VM A waits extra 20μs (60–40) before releasing to enclave
• VM B waits extra 0μs (60–60)
• VM C waits extra 15μs (60–45)

Result: All enclaves receive data simultaneously.

Performance:

• Median latency: ~190μs
• Delivery window size: ~10μs
• Cryptographic privacy guarantees

Trade-offs:

• Pros: Strong privacy protection (cryptographically enforced), regulatory compliance friendly, no trust required between exchange and traders
• Cons: Highest latency of the three models, complex setup and management, depends on specific cloud provider support

Use Case: Public exchanges serving competing firms, regulated environments.

Performance Results: How Well Does It Work?

Latency Comparison

The researchers tested Jasper against existing solutions:

Direct Unicast: 254μs median latency
AWS Transit Gateway: 228μs median latency  
Jasper: 129μs median latency

Improvement: 48% faster than direct unicast, 43% faster than AWS's solution

Fairness Achievement

Delivery Window Size (Unfairness Measurement):

• Without Jasper: ~350μs window (early traders get 350μs head start)
• With Jasper: <1μs window (nearly perfect simultaneous delivery)

Real Trading Simulation: The researchers built a prototype trading system with 4 active participants among 100 total receivers:

Perfect scenario: Each trader should earn 25% of total profits

Results:
├── Jasper: 25%, 24.97%, 24.66%, 25.36% (nearly perfect fairness!)
├── Direct Unicast: 49.79%, 50.21%, 0%, 0% (completely unfair)
└── Tree only: 56.95%, 42.39%, 0.40%, 0.25% (better but still unfair)

With Jasper’s full system, profit distribution was nearly ideal. Without fair delivery, the first two traders captured all profits.

Scalability Results

Previous Systems:
├── CloudEx: <100 participants maximum
├── AWS Transit Gateway: ~100 participants
└── Octopus (SmartNIC-based): <10 participants

Jasper:
├── Real testing: 1000 participants  
├── Simulation: 5000+ participants
└── Performance scales gracefully with participant count

Throughput Performance

• Single tree capacity: 35,000 messages/second
• Multiple trees (sharding): Linear scaling with number of trees
• For 100K messages/second: Deploy 3 parallel trees serving different stock symbols

Cost Analysis

For 1000 participants with hedging factor H=2:

VM Requirements:
├── Base proxy VMs: 110 VMs (tree structure)
├── With hedging: 110 × (2+1) = 330 VMs total
└── Cost: 330 × $0.34/hour = $112.2/hour

Monthly cost: ~$80,640
Compare to: Traditional on-premises setup costs millions in hardware

The cloud approach offers massive cost savings even with the overhead of proxy VMs.

Critical Analysis: What’s Missing?

While Jasper represents a significant breakthrough, several limitations and opportunities for improvement exist:

Technical Limitations

1. Single Cloud Region Constraint Jasper works within one AWS or GCP region but doesn’t address cross-region fairness. Global financial markets need solutions that work across continents, dealing with internet routing complexities that vary by geography and time of day.

2. Byzantine Fault Tolerance The system assumes all participants follow protocols correctly. What happens if some proxy nodes are compromised or some traders deliberately violate hold-and-release rules? The paper doesn’t address malicious behavior.

3. Dynamic Scaling The tree structure is calculated at deployment time based on expected participant count. Real markets have fluctuating participation — how does the system handle sudden spikes in trader count or proxy failures requiring tree reconfiguration?

Evaluation Gaps

1. Limited Test Duration
Experiments ran for 150 seconds. What about day-long stability? Week-long operation? Do performance characteristics change over extended periods?

2. Synthetic Workload Testing used uniform message patterns. Real trading has bursty workloads, market opens/closes, and varying message sizes. How would performance change with realistic traffic patterns?

3. Comparison Baselines The paper compares against CloudEx and AWS Transit Gateway, but what about other academic solutions or proprietary systems used by major exchanges?

Practical Deployment Concerns

1. Regulatory Compliance Different countries have different fairness requirements. How does Jasper adapt to varying regulatory frameworks? Can it provide audit trails proving fairness?

2. Integration Complexity Real exchanges have complex existing infrastructure. How difficult is migration from current systems to Jasper? What’s the migration path for active trading systems that can’t tolerate downtime?

3. Operational Challenges Who monitors system health across hundreds of proxy VMs? How do you debug fairness violations in a complex distributed system? What happens during cloud provider outages?

Broader Impact: Beyond Financial Trading

Jasper’s innovations extend far beyond stock exchanges. The core techniques — scalable fair multicast with guaranteed synchronization — apply to numerous domains:

Real-World Applications

Online Gaming Multiplayer games need fair, low-latency updates to all players. Jasper’s techniques could eliminate the advantage of “low ping” players in competitive gaming.

Live Streaming Synchronized viewing experiences for major events (sports, concerts, breaking news) where timing matters for social interaction.

Industrial IoT Coordinated actions across manufacturing systems, where robots or sensors must act simultaneously for safety or precision.

Medical Systems Synchronized data delivery in operating rooms or emergency response systems where timing coordination can be life-critical.

Distributed Databases Consistent replication across database nodes with guaranteed ordering, improving both performance and correctness.

System Design Principles

Jasper contributes three general principles for distributed systems:

1. Hierarchical Distribution: Tree structures can dramatically improve scalability while maintaining performance
2. Redundancy for Reliability: Smart hedging reduces variance without proportional cost increases
3. Coordinated Release: Sometimes fairness requires making fast systems wait for slow ones

These principles apply whenever you need to distribute information fairly and efficiently to large numbers of recipients.

Conclusion: Rethinking Fairness in the Cloud Era

Jasper solves a problem that seemed impossible: recreating the precision timing of custom financial hardware using shared cloud infrastructure. The solution required rethinking fundamental assumptions about network architecture, reliability, and synchronization.

The Technical Achievement:

• 48% latency improvement over naive approaches
• <1μs fairness window (nearly perfect synchronization)
• 10x scalability improvement over previous systems
• $80K/month vs. millions in capital costs

The Conceptual Breakthrough: More important than the numbers is the approach. Instead of trying to eliminate variance (impossible in the cloud), Jasper embraces it and builds systems that work despite uncertainty. Instead of requiring custom hardware, it shows how software can create the same guarantees.

The Broader Lesson: When faced with seemingly impossible constraints, breakthrough solutions often come from creative combination of simple techniques rather than revolutionary new technology. Three concepts — trees, redundancy, and synchronization — solved a problem that stumped the financial industry.

Looking Forward: As more critical infrastructure moves to the cloud, Jasper’s approach provides a template: understand your fundamental requirements, embrace the platform’s strengths, and engineer around its weaknesses.

The next time someone says “you can’t guarantee fairness in the cloud,” point them to Jasper. Sometimes the impossible just needs better engineering.

The Jasper system was developed by researchers at New York University and Stanford University, with the complete technical details available in their research paper “Jasper: Scalable and Fair Multicast for Financial Exchanges in the Cloud.”