Sunday, June 28, 2015

Flickr Architecture - High Scalability


http://blog.gainlo.co/index.php/2016/03/01/system-design-interview-question-create-a-photo-sharing-app/
Flickr Architecture - High Scalability -
Platform
  • PHP && MySQL
  • Shards
  • Memcached for a caching layer.
  • Squid in reverse-proxy for html and images.
  • Smarty for templating
  • PEAR for XML and Email parsing
  • ImageMagick, for image processing
  • SystemImager for deployment
  • Ganglia for distributed system monitoring
  • Subcon stores essential system configuration files in a subversion repository for easy deployment to machines in a cluster.
  • Cvsup for distributing and updating collections of files across a network.






  • Use dedicated servers for static content.
  • Talks about how to support Unicode.
  • Use a share nothing architecture.
  • Everything (except photos) are stored in the database.
  • Statelessness means they can bounce people around servers and it's easier to make their APIs.
  • Scaled at first by replication, but that only helps with reads.
  • Create a search farm by replicating the portion of the database they want to search.
  • Use horizontal scaling so they just need to add more machines.





  • Lessons Learned

  • Think of your application as more than just a web application
  • Go stateless. Statelessness makes for a simpler more robust system that can handle upgrades without flinching.
  • Re-architecting your database sucks.
  • Capacity plan
  • Start slow
  • Measure reality. Capacity planning math should be based on real things, not abstract ones.
  • Build in logging and metrics. Usage stats are just as important as server stats. Build in custom metrics to measure real-world usage to server-based stats.
  • Cache. Caching and RAM is the answer to everything.
  • Abstract. Create clear levels of abstraction between database work, business logic, page logic, page mark-up and the presentation layer. This supports quick turn around iterative development.
  • Layer. Layering allows developers to create page level logic which designers can use to build the user experience. Designers can ask for page logic as needed. It's a negotiation between the two parties.
  • Release frequently
  • Forget about small efficiencies
  • Test in production. Build into the architecture mechanisms (config flags, load balancing, etc.) with which you can deploy new hardware easily into (and out of) production.
  • Forget benchmarks. Benchmarks are fine for getting a general idea of capabilities, but not for planning. Artificial tests give artificial results, and the time is better used with testing for real.
  • Find ceilings.
    - What is the maximum something that every server can do ?
    - How close are you to that maximum, and how is it trending ?
    - MySQL (disk IO ?)
    - SQUID (disk IO ? or CPU ?)
    - memcached (CPU ? or network ?)
  • Be sensitive to the usage patterns for your type of application.
  • Be sensitive to the demands of exponential growth
  • Plan for peaks

  • http://s.niallkennedy.com/blog/uploads/flickr_php.pdf
    JOIN’s are slow
    • Normalised data is for sissies
    • Keep multiple copies of data around
    • Makes searching faster
    • Have to ensure consistency in the application logic

    http://www.scribd.com/doc/2592098/DVPmysqlucFederation-at-Flickr-Doing-Billions-of-Queries-Per-Day
    What Problems was Flickr Having?
    Master Slave Topology•Slave Lag•Multiple SPOF•Unable to keep up with demand•Unable to Serve Search Traffic
    •Multiple Second page load times

    Design to attain Goal
    •Since write intensive, need more then 1master, need many write points.•To get rid of SPOFs - be redundant.•To allow maintenance real-time, trafficneeds to stick to servers, and ‘a’ server needs to be able to handle the all traffic.•To serve pages fast with many queriesneed small data that fits in memory.

    Federation Key Components
    •Shards•Global Ring•PHP logic to connect to the shards andkeep the data consistent

    Shards
    •Shards are a slice of a main database•Shards are set up in Active Master-Master Ring Replication
     –Done by sticking a user to a server in a shard –Shard assignments are from a randomnumber for new accounts –Migration is done from time to time –Can run on any hardware grade

    Global Ring
    • a.k.a Lookup Ring
     –For stuff that can’t be federated –Like where stuff isOwner_id

    Allow for maintenance
    •Each server in a Shard is 50% loaded
     –i.e. 1 server in a shard can take the full load if a server of that shard is down or inmaintenance mode

    http://laughingmeme.org/2009/09/29/try-coding-dear-boy/


    “We generally try do the dumbest thing that will work first. And that’s usually as far as we get. Almost everything we do is pretty straightforward, and as such is well documented around the Web, sometimes by us, generally by others. And when we do get fiendishly clever, as we do now and again, it’s usually a highly tuned (read idiosyncratic) solution for the problems we’re trying to solve.”
    http://www.slideshare.net/techdude/scalable-web-architectures-common-patterns-and-approaches/138
    https://www.scribd.com/doc/2592098/DVPmysqlucFederation-at-Flickr-Doing-Billions-of-Queries-Per-Day

    - 可延伸网络架构和分布式系统 | Scalable Web Architecture and Distributed systems(Flickr)
    English: http://www.aosabook.org/en/distsys.html
    Chinese: http://goo.gl/u3MoqU

    - Flickr 网站架构研究
    (一)Flickr 网站架构综述
    (二)数据库最初的扩展-Replication
    (三)Shard - 大型网站数据库扩展的终极武器?
    (四)海量文件系统设计的考虑因素
    (五)"Flickr File System"探秘(上)
    (六)"Flickr File System"探秘(下)
    (七)系统监控和故障管理

      Building Scalable Web Sites (by Cal Henderson, from Flickr).

    -    Database War Stories #3: Flickr (by Tim O'Reilly)




    Read full article from Flickr Architecture - High Scalability -

    No comments:

    Post a Comment

    Labels

    Review (554) System Design (293) System Design - Review (189) Java (178) Coding (75) Interview-System Design (65) Interview (60) Book Notes (59) Coding - Review (59) to-do (45) Knowledge (39) Linux (39) Interview-Java (35) Knowledge - Review (32) Database (30) Design Patterns (29) Product Architecture (28) Big Data (27) Soft Skills (27) Miscs (25) MultiThread (25) Concurrency (24) Cracking Code Interview (24) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Distributed (20) Interview Q&A (20) OOD Design (20) System Design - Practice (19) Security (17) Algorithm (15) How to Ace Interview (15) Brain Teaser (14) Google (13) Linux - Shell (13) Spark (13) Spring (13) Code Quality (12) How to (12) Interview-Database (12) Interview-Operating System (12) Redis (12) Tools (12) Architecture Principles (11) Company - LinkedIn (11) Testing (11) Resource (10) Solr (10) Amazon (9) Cache (9) Search (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Company - Uber (8) Interview - MultiThread (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Scalability (8) Cassandra (7) Git (7) Interview Corner (7) JVM (7) Java Basics (7) Machine Learning (7) NoSQL (7) C++ (6) Design (6) File System (6) Highscalability (6) How to Better (6) Kafka (6) Network (6) Restful (6) Trouble Shooting (6) CareerCup (5) Code Review (5) Company - Facebook (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Be Architect (4) Big Fata (4) C (4) Company Product Architecture (4) Data structures (4) Design Principles (4) Facebook (4) GeeksforGeeks (4) Generics (4) Google Interview (4) Hardware (4) JDK8 (4) Optimization (4) Product + Framework (4) Shopping System (4) Source Code (4) Web Service (4) node.js (4) Back-of-Envelope (3) Company - Pinterest (3) Company - Twiiter (3) Company - Twitter (3) Consistent Hash (3) GOF (3) Game Design (3) GeoHash (3) Growth (3) Guava (3) Interview-Big Data (3) Interview-Linux (3) Interview-Network (3) Java EE Patterns (3) Javarevisited (3) Map Reduce (3) Math - Probabilities (3) Performance (3) Puzzles (3) Python (3) Resource-System Desgin (3) Scala (3) UML (3) geeksquiz (3) AI (2) API Design (2) AngularJS (2) Behavior Question (2) Bugs (2) Coding Interview (2) Company - Netflix (2) Crawler (2) Cross Data Center (2) Data Structure Design (2) Database-Shard (2) Debugging (2) Docker (2) Elasticsearch (2) Garbage Collection (2) Go (2) Hadoop (2) Html (2) Interview - Soft Skills (2) Interview-Miscs (2) Interview-Web (2) JDK (2) Logging (2) POI (2) Papers (2) Programming (2) Project Practice (2) Random (2) Software Desgin (2) System Design - Feed (2) Thread Synchronization (2) Video (2) ZooKeeper (2) reddit (2) Ads (1) Advanced data structures (1) Algorithm - Review (1) Android (1) Approximate Algorithms (1) Base X (1) Bash (1) Books (1) C# (1) CSS (1) Chrome (1) Client-Side (1) Cloud (1) CodingHorror (1) Company - Yelp (1) Counter (1) DSL (1) Dead Lock (1) Difficult Puzzles (1) Distributed ALgorithm (1) Eclipse (1) Facebook Interview (1) Function Design (1) Functional (1) GoLang (1) How to Solve Problems (1) ID Generation (1) IO (1) Important (1) Internals (1) Interview - Dropbox (1) Interview - Project Experience (1) Interview Tips (1) Interview-Brain Teaser (1) Interview-How (1) Interview-Mics (1) Interview-Process (1) Jeff Dean (1) Joda (1) LeetCode - Review (1) Library (1) LinkedIn (1) LintCode (1) Mac (1) Micro-Services (1) Mini System (1) MySQL (1) Nigix (1) NonBlock (1) Process (1) Productivity (1) Program Output (1) Programcreek (1) Quora (1) RPC (1) Raft (1) RateLimiter (1) Reactive (1) Reading (1) Reading Code (1) Refactoring (1) Resource-Java (1) Resource-System Design (1) Resume (1) SQL (1) Sampling (1) Shuffle (1) Slide Window (1) Spotify (1) Stability (1) Storm (1) Summary (1) System Design - TODO (1) Tic Tac Toe (1) Time Management (1) Web Tools (1) algolist (1) corejavainterviewquestions (1) martin fowler (1) mitbbs (1)

    Popular Posts