Thursday, August 31, 2017

Distributed Counter



http://rockthecode.io/blog/highly-available-counters-using-cassandra/
This is the simplest counter to implement, especially for people coming from the RDBMS world. Acquire a lock on the counter, increment the counter and release the lock. In distributed systems, lockscan be implemented using an external coordinator, like Zookeeper.
 1 lock.acquire();
 2 try {
 3    counter.increment(5);
 4 } finally {
 5    lock.release();
 6 } 

  1. Locking involves coordination and coordination is expensive
  2. If the counter is a hotspot, there will be high lock contention
  3. In distributed systems, messages may be redelivered. The solution will not handle duplicate messages and is not idempotent

LINEARIZABLE COUNTERS

This involves implementing a CAS (compare-and-swap) operation on a counter. This can be achieved using Zookeeper or Paxos based consensus in Cassandra.
 1 int retries = 5;
 2 while(retries > 0) {
 3     int currentValue = counter.getValue();
 4     if (!counter.compareAndSwap(currentValue, currentValue + 5)) {
 5         try {
 6             Thread.sleep(500); // You want to ideally use an exponential backoff algorithm here
 7         } catch (InterruptedException e) {}
 8         retries--;
 9     } else {
10         break;
11     }
12 }  
  1. High contention
  2. Not very scalable
  3. Non-idempotent
  4. May exhaust retries if the counter is a hotspot

CASSANDRA COUNTERS USING CRDTS

In this example, we will see how we can implement a CRDT based counter using Cassandra. We will show how we can model a counter using a CmRDT (operation-based) and a CvRDT (state-based). We’ll then model these using CQL.


Labels

Review (572) System Design (334) System Design - Review (198) Java (189) Coding (75) Interview-System Design (65) Interview (63) Book Notes (59) Coding - Review (59) to-do (45) Linux (43) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (31) Big Data (29) Product Architecture (28) MultiThread (27) Soft Skills (27) Concurrency (26) Cracking Code Interview (26) Miscs (25) Distributed (24) OOD Design (24) Google (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) System Design - Practice (20) Tips (19) Algorithm (17) Company - Facebook (17) Security (17) How to Ace Interview (16) Brain Teaser (14) Linux - Shell (14) Redis (14) Testing (14) Tools (14) Code Quality (13) Search (13) Spark (13) Spring (13) Company - LinkedIn (12) How to (12) Interview-Database (12) Interview-Operating System (12) Solr (12) Architecture Principles (11) Resource (10) Amazon (9) Cache (9) Git (9) Interview - MultiThread (9) Scalability (9) Trouble Shooting (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cassandra (8) Company - Uber (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Design (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Mac (7) Machine Learning (7) NoSQL (7) C++ (6) Chrome (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Python (5)

Popular Posts