Download as PDF, PPTX
![Arcturus - Batching marshaller
vBatching: use multi thread marshaller
● Pros
- Fast (400k-600k ops/s*)
- Avoid multi key locking
- Write only most updated value
● Cons
- Inconsistent transaction offset
→ use when you just want to make your code as fast
as possible.
* Macbook pro 2016 15’’ 2.7GHz Core i7, 16G ram. Without I/O, single update per
transaction
*** Test without I/O: [Single ops vbatch] Total time running for 1000000 transactions: 1085 ms; at rate: 921,658.99 ops/s](https://image.slidesharecdn.com/highconcurrencyarchitecturetiki-190908005606/75/High-Concurrency-Architecture-at-TIKI-19-2048.jpg)
![Arcturus - Batching marshaller
hBatching: use single marshaller thread
● Pros
- Ensure transaction offset
- Ordered db updating
● Cons
- A little bit slower than vBatching.
→ use when you need to re-create application state
after fail.
*** Test without I/O: [Single ops hbatch] Total time running for 1000000 transactions: 1056 ms; at rate: 946,969.70 ops/s](https://image.slidesharecdn.com/highconcurrencyarchitecturetiki-190908005606/75/High-Concurrency-Architecture-at-TIKI-20-2048.jpg)
Uploaded byNghia Minh
High Concurrency Architecture at TIKI
This document discusses high concurrency architectures at TIKI. It describes Pegasus, the highest throughput API, which uses caching, compression, and a non-blocking architecture to handle over 200k requests per minute with sub-2ms latency. It also describes Arcturus, the high concurrency inventory API, which uses an in-memory ring buffer, Kafka for ordering, and asynchronous database flushing to handle millions of inventory transactions per second with eventual consistency. Key techniques discussed include non-blocking designs, caching, compression, ordering queues, and asynchronous data replication.
In this document
Powered by AI
Introduction to high concurrency architecture principles and presentation agenda.
Discusses principles like local memory use, non-blocking architecture, and the trade-offs of consistency.
Details on Pegasus, TIKI's highest throughput API handling 10k request/s, compressing data for efficiency.
Discussion on Arcturus, designed for high concurrency during order placements with challenges of inventory handling.
Compares old multi-thread applications with new non-blocking architecture in Arcturus, highlighting the efficiency of batch processing.
Technologies utilized in the APIs including Java, Kafka, and the use of LMax architecture.
Summary of key insights regarding consistency trade-offs, in-memory improvements, and non-blocking architecture for better performance.
More Related Content
Similar to High Concurrency Architecture at TIKI
High Concurrency Architecture at TIKI
- 1.
- 2. Contributors Bùi Anh Dũng PrincipalEngineer Nguyễn Hoàng Bách Senior Principal Engineer Phan Công Huân Senior Software Engineer Lê Minh Nghĩa Senior Architect Trần Nguyên Bản Principal Engineer
- 3. Agenda - Principles - Pegasus- Highest throughput API - Arcturus - High concurrency inventory API - Conclusion
- 4. Principles - Use localmemory to handle high concurrency transaction - Non blocking architecture - Trade-offs consistency vs eventual consistency - Reliable replication - Authority: each service owns its problems.
- 5. Pegasus - HighestThroughput API at TIKI Problems: - Handle the most of traffic to fetch product information - Has to handle at least 10k request/s, tp95 < 5ms
- 6.
- 7. Pegasus - Architecture Practises: -Cache product data in local memory - Subscribe product change event to invalid cache - Non-Blocking http web server - Compress to reduce payload size Technology: - Java - Guava - In Memory Cache - Gridgo - Async IO & Event driven framework
- 8. Pegasus - Compression Twocases: - Get single product by id - Get multiple product by a list id Solutions: - Using gzip to compress product data to store in cache. Reduce from 200kb text to 3Kb - Handle single product request: return compressed gzip bytes to client directly - Handle multiple product request: store plain product data and merge them to build a list of products, then use Snappy format to compress data and return to client - Gzip format compress better than Snappy and is supported natively by most http client, but Snappy compress faster and use less CPU - Output cache can be enabled in hot-deal situation
- 9. Pegasus - Technology -Java - Gridgo - Guava - Kafka
- 10. Pegasus - Benchmark Benchmarkuse WRK tools, 4VM, L4 LB, GCP: - 90%<6.6ms - 96k request/s Production - 200k request/minutes (all platforms) - <2ms Cache hit: 85%. Increased to 95% with Consistent Hashing With Bounded Load. Average payload: Single (3KB), Multi (60KB)
- 11.
- 12. Arcturus - Highconcurrency Inventory API Problems: - Handle inventory transaction when customers place orders - Handle high concurrency transaction for extreme hot deals with very cheap and low quantity. Exp: Only ten 1đ iPhone XS, only in ten minutes from 9AM to 9h10 AM. - Guarantee eventual consistency of inventory data between many systems
- 13. Arcturus - ContextDiagram
- 14.
- 15. Arcturus - Problemsand Solutions Problems: - Many customers may place order at the same time, especially for hot skus. - System has to be deterministic and can be recovered after crashing Solutions: - All write requests are published to Kafka with only one partition to guarantee the ordering of message and recover if system crashes. - Non Blocking In Memory Cache for both read and write - All changes are flushed to database asynchronously - Recovery based on the offset of write command
- 16. Arcturus - InMemory Cache Data Structure Problem: - There will be a race condition when many customer place the same skus at the same time - Building an in memory cache structure that can buffer data changes and flush to database asynchronously Approach: - Each record have a key (string, bytes…) and a value (8 bytes long number). - A transaction is a set of record updates. - Transactions must be ordered (e.g. key_1 must +3 before can be -1). - It can be millions of keys and process upto 10k TPS. - Using ring buffer data structure to guarantee the ordering of changes * The hardest thing is to keep everything ordered when make it fast enough.
- 17. Arcturus - Oldsolution - Multi threading application - DB transaction, row locking Pros - Strong consistency - Simple implementation Cons - Slow - Deadlock/timeout implicit risk
- 18. Arcturus - LMaxArchitecture - Non Blocking architecture using single thread business logic processor - Use Ring Buffer data structure to communicate between processors - Handle million transaction per seconds
- 19. Arcturus - Batchingmarshaller vBatching: use multi thread marshaller ● Pros - Fast (400k-600k ops/s*) - Avoid multi key locking - Write only most updated value ● Cons - Inconsistent transaction offset → use when you just want to make your code as fast as possible. * Macbook pro 2016 15’’ 2.7GHz Core i7, 16G ram. Without I/O, single update per transaction *** Test without I/O: [Single ops vbatch] Total time running for 1000000 transactions: 1085 ms; at rate: 921,658.99 ops/s
- 20. Arcturus - Batchingmarshaller hBatching: use single marshaller thread ● Pros - Ensure transaction offset - Ordered db updating ● Cons - A little bit slower than vBatching. → use when you need to re-create application state after fail. *** Test without I/O: [Single ops hbatch] Total time running for 1000000 transactions: 1056 ms; at rate: 946,969.70 ops/s
- 21.
- 22. Arcturus - Technology -Java - Kafka - ZeroMQ - Gridgo - MySQL - LMax/Ring Buffer
- 23. Conclusion - Has totrade-off between consistency and eventual consistency - In Memory is a great way to improve the performance - Reliable replication is the key to split and scale system - Non Blocking architecture is a great way to utilize hardware resources efficiently.