18/08/2022, 21:10 System Design Stock Trading Platforms - Scaler

Cache Replacement

Today’s Session - System Design for Stock Trading Platforms VS. For reference only

X
Stateful Stateless

CAP

Loadbalancer
Lambda architecture Sagas architecture

1
Scalability

2 Rate Limiter
Delta architecture

X
INTRO
Caching Kappa architecture
BASICS

Proxies

With microservices, choose different priorities for different services

Cache Stampede

Solutions Strict / strong Consistency - relational DBs - payment, transactions

Weak Consistency - other DBs and consider scalabilty, availability

Eventual Consistency - other DBs and consider scalabilty, availability

Scaling caches

Consitent Hashing

BASICS

Questions around the topic

System design for a buy and sell order service
System design for a stock recommendation service
System design for analytics system for stock trading
System design for basic crypto exchange

Things We might want to use - domain specific

1 2 3
A B C
3
A

B 4 5 6
ETLs

4 5 C Analytics Engines

6 7 8
Time series databases

9
D
E 7 8 The Issue
9
F

10 11 12
F
Possible bottlenecks D E
13 14 15

Graph DBs

10
ML engines
16 17 18 19 20 21 Real time Recommendation engines
ESTIMATIONS
VISIBLE TRADING FLOWS
A General system design when you’re starting out.
List of possible components / modules / services
Each module can choose a different priority

INVISIBLE FLOWS
H I
A “naive” system design G Serverless functions

Separation of concern Logging Systems

Scalable business logic
Monitoring Systems
Reduced team inter-dependency
Separate deployment methodologies and frequency
Can use completely different technologies for each service
Separate teams work on separate part of the software
Reduced blast radius - one failure doesn’t bring the entire system down

J K

Content Delivery Networks Media buckets

The Solution

Important tech that makes things easier for us

Multi-tenancy - A lot of the trading is through corporate accounts.

16
Event Driven
Real Time Systems

11

Real time concepts

JOBS
19

17
Synchronous Microsevices API Gateways, service registries and API composers

Architectures

Microservice considerations

21
Asynchronous/event driven

20
Microservices

12

Distributed Scrapers Datalakes

Get all crawled scraped data into a data lake, apply rules engines and train ML models
We’re not using data lake as a long term persistence but only as a short term storage for massive
amounts of data

A B
A better system design - but not scalable.

18
D
C

23
E F
22 13

Rules engines
The big problem
G H
Warehousing and Auditing

For long term stored data that’s been stored in archiveDB, you’d want to warehouse the data to
generate business related data reports.

Priority Planning exercise

Mo
Must Have

Specific tech for Big data, real time systems, microservices architecture

S
Should Have

Considerations

15 Stretch goals General security considerations

14 Miscellaneous Considerations

Co

24 25
Could Have

Batch Reference
Feature Lookup
Data Preparation

Cloud Dataflow Cloud Datastore

W
Entity Entity Id
attributes lookup

Enterprise Data
ML Gateway Entity Id
Warehouse
Client
Prediction
BigQuery App Engine

Request Response
Won’t Have
(entity attributes)

Deployed Model

Cloud Machine
Learning Engine

Considerations - What will you prioritize - gotta pick a few only Security considerations

Price prediction algorithms and services

Algorithmic trading + bots

26
Distributed Messaging

Micro front ends

Distributed Tracing

27
Distributed Data Operations
Map Reduce on Warehoused Data

E
C

Distributed Systems

Domain Specific

28 Distributed Systems
Distributed Systems

29 FINAL SYSTEM DESIGN

Authentication microservice

Auditing service

Authorization microservice

Archive DB

ANALYTICS ENGINE

Dashboards
charts Timeseries DB
reports

Transforms data at a basic level

but data is still RAW

Predictions on better trades

ML Engine Queue for trades on the platform - maintains the trades that are happening on the platform - the list of trades.

https://whimsical.com/system-design-stock-trading-platforms-scaler-J4jsLowyxtaQnZYqxkrLWN 1/1