Scalability Rules: 50 Principles for Scaling Web Sites

Scalability.Rules.notes

设计原则

• Rule 1—Don’t Over-engineer the Solution — 适合的是最好的

1. What: Guard against complex solutions during design.
2. How to use: Resist the urge to over-engineer solutions by testing ease of understanding with fellow engineers.
3. Why: Complex solutions are costly to implement and have excessive long-term costs.
4. Key takeaways: Systems that are overly complex limit your ability to scale. Simple systems are more easily and cost effectively maintained and scaled.

• Rule 2—Design Scale into the Solution (D-I-D Process) — 在设计阶段就考虑scale成本最低并最有效率

designing something is significantly less expensive than actually implementing that design in code.

How to use:

+Design for 20x capacity.
+Implement for 3x capacity.
+Deploy for ~1.5x capacity.

架构设计

• Rule 43—Communicate Asynchronously As Much As Possible — 使用可信赖的消息队列

1. What: Use asynchronous instead of synchronous communication as often as possible.
2. When to use: Consider for all calls between services and tiers.
3. Why: Synchronous calls stop the entire program’s execution waiting for a response, which ties all the services and tiers together resulting in cascading failures.
4. Key takeaways: Use asynchronous communication techniques to ensure that each service and tier is as independent as possible. This allows the system to scale much farther than if all components are closely coupled together.

• Rule 29—Failing to Design for Rollback Is Designing for Failure — 减少失败就是提高可用性

1. What: Always have the ability to roll back code.
2. Key takeaways: Don’t accept that the application is too complex or that you release code too often as excuses that you can’t roll back. No sane pilot would take off in an airplane without the ability to land, and no sane engineer would roll code that they could not pull back off in an emergency.

This pattern of push a fix at night, without traffic think it’s fixed, only to find out the next day that the site still had issues carried on for more than a week.

Database changes must only be additive— Columns or tables should only be added, not deleted, until the next version of code is released that deprecates the dependency on those columns. Once these standards are implemented every release should have a portion dedicated to cleaning up the last release’s data that is no longer needed.

DDL and DML scripted and tested

Restricted SQL queries in the application— The development team needs to disambiguate all SQL by removing all SELECT * queries and adding column names to all UPDATE statements.

Semantic changes of data— The development team must not change the definition of data within a release. An example would be a column in a ticket table that is currently being used as a status semaphore indicating three values such as assigned, fixed, or closed. The new version of the application cannot add a fourth status until code is first released to handle the new status and then code can be released to utilize the new status.

Wire On/Wire Off— The application should have a framework added that allows code paths and features to be accessed by some users and not by others, based on an external configuration. This setting can be in a configuration file or a database table and should allow for both role-based access as well as random percentage based. This framework allows for beta testing of features with a limited set of users and allows for quick removal of a code path in the event of a major bug in the feature, without rolling back the entire code base.

• Rule 37—Never Trust Single Points of Failure  — 增加可用性，还可以提高用户网络体验

1. What: Never implement and always eliminate single points of failure.
2. When to use: During architecture reviews and new designs.
3. How to use: Identify single instances on architectural diagrams. Strive for active/active configurations.
4. Why: Maximize availability through multiple instances.
5. Key takeaways: Strive for active/active rather than active/ passive solutions. Use load balancers to balance traffic across instances of a service. Use control services with active/passive instances for patterns that require singletons.

everything fails
The solution to most SPOFs is simply requisitioning another piece of hardware and running two or more of every service by cloning that service as described in our X axis of scale.
Stick session in load balance.

Web开发设计

• Rule 21—Use Expires Headers  — cache in client

1. What: Use Expires headers to reduce requests and improve the scalability and performance of your system.

• Rule 22—Cache Ajax Calls       — cache is king； cache in multiple ties
• Rule 23—Leverage Page Caches

What: Deploy page caches in front of your Web services.

Why: Decrease load on Web servers by caching and delivering previously generated dynamic requests and quickly answering calls for static objects.

Key takeaways: Page caches are a great way to offload dynamic requests, decrease customer response time, and scale cost-effectively.

reverse proxy cache, reverse proxy server

We use the term page cache deliberately, because whereas a proxy might also be responsible for load balancing or SSL (Secure Sockets Layer) acceleration, we are solely focused on the impact that these caching servers have on our scalability

The ETag, or entity tag, was developed to facilitate the method of If-None-Match conditional GET requests by clients of a server. ETags are unique identifiers issued by the server for an object at the time of first request by a browser. If the resource on the server side is changed, a new ETag is assigned to it. Assuming appropriate support by the browser (client), the object and its ETag are cached by the browser, and subsequent If-None-Match requests by the browser to the Web server will include the tag. If the tag matches, the server may respond with an HTTP 304 Not Modified response. If the tag is inconsistent with that on the server, the server will issue the updated object and its associated ETag.

• Rule 24—Utilize Application Caches
• Rule 25—Make Use of Object Caches
• Rule 26—Put Object Caches on Their Own “Tier”  — don’t abuse cache
• Rule 4—Reduce DNS Lookups  — volecity大会中介绍过很多类似的方法，提高web页面相应速度

1. What: Reduce the number of DNS lookups from a user perspective.
2. When to use: On all Web pages where performance matters.
3. How to use: Minimize the number of DNS lookups required to download pages, but balance this with the browser’s limitation for simultaneous connections.
4. Why: DNS lookups take a great deal of time, and large numbers of them can amount to a large portion of your user experience.

Design for Fault Tolerance and Graceful Failure
Rule 38—Avoid Putting Systems in Series
What: Reduce the number of components that are connected in series.
When to use: Anytime you are considering adding components.
How to use: Remove unnecessary components or add multiple versions of them to minimize the impact.
Why: Components in series have a multiplicative effect of failure.
Key takeaways: Avoid adding components to your system that are connected in series. When necessary to do so add multiple versions of that component so that if one fails others are available to take its place.

• Rule 39—Ensure You Can Wire On and Off Functions — 通过文件或者数据库配置，可以针对用户打开或者关闭某个服务，类似oracle 11g中的invsisiable index

1. What: Create a framework to disable and enable features of your product.
2. When to use: Risky, very high use, or shared services that might otherwise cause site failures when slow to respond or unavailable.
3. How to use: Develop shared libraries to allow automatic or on-demand enabling and disabling of services.
4. Why: Graceful failure (or handling failures) of transactions can keep you in business while you recover from the incident and problem that caused it.
5. Key takeaways: Implement Wire On/Wire Off frameworks whenever the cost of implementation is less than the risk and associated cost of failure.Work to develop shared libraries that can be reused to lower the cost of future implementation.

Automatic markdown based on timeouts
Synchronous markdown command
User intervention sends a command to services to stop using the failed or slow service.
Config file markdown
File markdown
Database markdown
Runtime Variable

• Rule 40—Strive for Statelessness  — 无状态web应用；不大懂web开发；可能是将session信息存储在应用服务器端的成本太高

1. What: Design and implement stateless systems.
2. When to use: During design of new systems and redesign of existing systems.
3. How to use: Choose stateless implementations whenever possible. If stateful implementations are warranted for business reasons, refer to Rules 41 and 42.
4. Why: The implementation of state limits scalability and increases cost.

Ask: Why do you need it at all?

• Rule 41—Maintain Sessions in the Browser When Possible

1. What: Try to avoid session data completely, but when needed, consider putting the data in users’ browsers.
2. When to use: Anytime that you need session data for the best user experience.
3. How to use: Use cookies to store session data on the users’ browsers.
4. Why: Keeping session data on the users’ browsers allows the user request to be served by any Web server in the pool and takes the storage requirement away from your system.
5. Key takeaways: Using cookies to store session data is a common approach and has advantages in terms of ease of scale but also has some drawbacks. One of the most critical cons is that unsecured cookies

session data can be easily captured on an open WiFi network and used to nefariously log in to someone else’s account.
According to RFC2965 browsers should support cookies at least up to 4KB in size and up to 20 cookies from the same domain.
the larger the cookie the slower your pages will load since this data has to be transmitted back and forth with each request.

you can transmit your pages and cookies all in HTTPS.

• Rule 42—Make Use of a Distributed Cache for States –可以将session信息持久化到数据库中，同时在应用服务器和数据库之前提供一层高可用的cache；

1. What: Use a distributed cache when storing session data in your system.
2. When to use: Anytime you need to store session data and cannot do so in users’ browsers.
3. How to use: Watch for some common mistakes such as a session management system that requires affinity of a user to a Web server.
4. Why: Careful consideration of how to store session data can help ensure your system will continue to scale.
5. Key takeaways: Many Web servers or languages offer simple server-based session management, but these are often fraught with problems such as user affiliation with specific servers. Implementing a distributed cache will allow you to store session data in your system and continue to scale.

Don’t implement systems that require affinity to a server to function properly.

Don’t use state or session replication to create duplicates of data on different systems.

Don’t locate the cache on the system doing the work (this doesn’t mean you shouldn’t have a local application cache—just that session information is best handled in its own tier of servers).

网络实施

• Rule 20—Leverage CDNs  — 收益和成本的balance

1. What: Use CDNs to offload traffic from your site.
2. When to use: Ensure it is cost justified and then choose which content is most suitable.
3. How to use: Most CDNs leverage DNS to serve content on your site’s behalf.
4. Why: CDNs help offload traffic spikes and are often economical ways to scale parts of a site’s traffic.
5. Key takeaways: CDNs are a fast and simple way to offset spikiness of traffic as well as traffic growth in general. Ensure you perform a cost-benefit analysis and monitor the CDN usage.

• Rule 6—Use Homogenous Networks

1. What: Don’t mix the vendor networking gear.
2. When to use: When designing or expanding your network.
3. How to use:
   + Do not mix different vendors’ networking gear (switches and routers).
   + Buy best of breed for other networking gear (firewalls, load balancers, and so on).
4. Why: Intermittent interoperability and availability issues simply aren’t worth the potential cost savings.
5. Key takeaways: Heterogeneous networking gear tends to cause availability and scalability problems. Choose a single provider.

数据库相关

• Rule 31—Be Aware of Costly Relationships  — FK约束会对数据库的分库分表等行为有一定限制

1. What: Be aware of relationships in the data model.
2. When to use: When designing the data model, adding tables/columns, or writing queries consider how the relationships between entities will affect performance and scalability in the long run.
3. How to use: Think about database splits and possible future data needs as you design the data model.
4. Why: The cost of fixing a broken data model after it has been implemented is likely 100x as much as fixing it during the design phase.
5. Key takeaways: Think ahead and plan the data model carefully. Consider normalized forms, how you will likely split the database in the future, and possible data needs of the application.

• Rule 14—Use Databases Appropriately   — 选择传统关系数据库？

1. What: Use relational databases when you need ACID properties to maintain relationships between your data. For other data storage needs consider more appropriate tools.
2. How to use: Consider the data volume, amount of storage, response time requirements, relationships, and other factors to choose the most appropriate storage tool.
   Why: RDBMSs provide great transactional integrity but are more difficult to scale, cost more, and have lower availability than many other storage options.
3. Key takeaways: Use the right storage tool for your data. Don’t get lured into sticking everything in a relational database just because you are comfortable accessing data in a database.

Distribute Your Work 

• Rule 7—Design to Clone Things (X Axis)   –（复制:一写N读）

1. What: Typically called horizontal scale, this is the duplication of services or databases to spread transaction load.
2. How to use:
   + Simply clone services and implement a load balancer.
   + For databases, ensure the accessing code understands the difference between a read and a write.

• Rule 8—Design to Split Different Things (Y Axis) — 按照function或者service分库
• Rule 9—Design to Split Similar Things (Z Axis) — 分表

1. What: This is often a split by some unique aspect of the customer such as customer ID, name, geography, and so on.
2. When to use:Very large, similar data sets such as large and rapidly growing customer bases.
3. How to use: Identify something you know about the customer, such as customer ID, last name, geography, or device and split or partition both data and services based on that attribute.
4. Why: Rapid customer growth exceeds other forms of data growth or you have the need to perform “fault isolation” between certain customer groups as you scale.

• Rule 35—Don’t Select Everything

1. What: Don’t use Select * in queries.
2. When to use: Never select everything (unless of course you are going to use everything).
3. How to use: Always declare what columns of data you are selecting or inserting in a query.
4. Why: Selecting everything in a query is prone to break things when the table structure changes and it transfers unneeded data.
5. Key takeaways: Don’t use wildcards when selecting or inserting data.

The problem is when the next developer needs to add a column to the table they might issue a command such as this:
ALTER TABLE bestpostpage ADD remote_host varchar(25) AFTER id;

The result is that your mapping from column 1 is no longer the remote_ip but instead is now remote_host.

The second big problem with Select * is that usually you don’t need all the data in all the columns.

Insert: A much better way of inserting the data is to use the actual column names, like this:

INSERT INTO bestpostpage (id, remote_ip, post_data,
insert_date) VALUES (1, '10.97.23.45', 'test data',
'2010-11-19 11:15:00');

As a best practice, do not get in the habit of using Select or Insert without specifying the columns. Besides wasting resources and being likely to break or potentially even corrupt data, it also prevents you from rolling back.

Rule 32—Use the Right Type of Database Lock

Rule 33—Pass on Using Multiphase Commits

Rule 34—Try Not to Use “Select For Update”

In many databases, when the cursor with a FOR UPDATE clause is opened, the rows identified within the statement are locked until either a commit or rollback is issued within the session.
Additionally, after issuing the commit or rollback, you lose your position within the cursor and will not be able to execute any more fetches against it.

Datacenter实施

Chapter 3. Design to Scale Out Horizontally
Rule 10: Design Your Solution to Scale Out—Not Just Up
Rule 11—Use Commodity Systems (Goldfish Not Thoroughbreds)
Rule 13—Design to Leverage the Cloud

• Rule 12—Scale Out Your Data Centers

1. What: Design your systems to have three or more live data centers to reduce overall cost, increase availability, and implement disaster recovery.
2. When to use: Any rapidly growing business that is considering adding a disaster recovery (cold site) data center.
3. How to use: Split up your data to spread across data centers and spread transaction load across those data centers in a “multiple live” configuration. Use spare capacity for peak periods of the year.

Let’s start with three data centers. Each data center is the “home” for roughly 33% of our data. We will call these data sets A, B, and C. Each data set in each data center has its data replicated in halves, 50% going to each peer data center.

Route customers to closest data center if possible to reduce dynamic call times

运维

• Rule 30—Discuss and Learn from Failures

1. What: Leverage every failure to learn and teach important lessons.
2. How to use: Employ a postmortem process and hypothesize failures in low failure environments.
3. Why:We learn best from our mistakes—not our successes.
   Key takeaways: Never let a good failure go to waste. Learn from every one and identify the technology, people, and process issues that need to be corrected.

• Rule 27—Learn Aggressively  — 27&30 2个rules都是为了避免incident成为mistake

1. When to use: Be constantly learning from your mistakes as well as successes.
2. How to use: Watch your customers or use A/B testing to determine what works. Use postmortems to learn from incidents and problems in production.
3. Why: Doing something without measuring the results or having an incident without learning from it are wasted opportunities that your competitors are taking advantage of.

• Rule 16—Actively Use Log Files  — for DW or troubleshooting

1. What: Use your application’s log files to diagnose and prevent problems.
   When to use: Put a process in place that monitors log files and forces people to take action on issues identified.
2. How to use: Use any number of monitoring tools from custom scripts to Splunk to watch your application logs for errors. Export these and assign resources for identifying and solving the issue.
3. Why: The log files are excellent sources of information about how your application is performing for your users; don’t throw this resource away without using it.
4. Key takeaways: Make good use of your log files and you will have fewer production issues with your system.

• Rule 29—Failing to Design for Rollback Is Designing for Failure — 减少失败就是提高可用性
• Rule 49—Design Your Application to Be Monitored

1. What: Think about how you will need to monitor your application as you are designing it.
2. When to use: Anytime you are adding or changing modules of your code base.
3. How to use: Build hooks into your system to record transaction times.
4. Why: Having insight into how your application is performing will help answer many questions when there is a problem.
5. Key takeaways: Adopt as an architectural principle that your application must be monitored. Additionally, look at your overall monitoring strategy to make sure you are first answering the question of “Is there a problem?” and then the “Where” and “What.”

Cacti, Ntop, or Nagios
automatic alerting system
monitor your system from the perspective of a business metric.

Is there a problem?
“Where is the problem?” and “What is the problem?”
Why is there a problem?
“Will there be a problem?

• Rule 47—Purge, Archive, and Cost-Justify Storage  — 真对OLTP数据库，归档数据可以保证cache命中率

1. What: Match storage cost to data value, including removing data of value lower than the costs to store it.
2. Why: Not all data is created equal (that is, of the same value) and in fact it often changes in value over time.Why then should we have a single storage solution with equivalent cost for that data?
3. Key takeaways: It is important to understand and calculate the value of your data and to match storage costs to that value. Don’t pay for data that doesn’t have a stakeholder return.

Rule 46—Be Wary of Scaling Through Third Parties

What: Scale your own system; don’t rely on vendor solutions to achieve scalability.
How to use: Rely on the rules of this book for understanding how to scale and use vendor provided products and services in the most simplistic manner possible.

Why: Three reasons for following this rule: Own your destiny, keep your architecture simple, and reduce your total cost of ownership.

Key takeaways: Do not rely on vendor products, services, or features to scale your system. Keep your architecture simple, keep your destiny in your own hands, and keep your costs in control. All three of these can be violated by using a vendor’s proprietary scaling solution.

Do not overengineer the solution and simplify the solution three times over.

Rule 48—Remove Business Intelligence from Transaction Processing
What: Separate business systems from product systems and product intelligence from database systems.

How to use: Remove stored procedures from the database and put them in your application logic. Do not make synchronous calls between corporate and product systems.

Rule 50—Be Competent
What: Be competent, or buy competency in/for each component of your architecture.
When to use: For any Internet service or commerce solution.
How to use: For each component of your infrastructure, identify the team responsible and level of competency with that component.
Why: To a customer, every problem is your problem. You can’t blame suppliers or providers. You provide a service—not software.

Your customers expect you to deliver a service to them.
For anything we employ, we need to know that we are using it correctly, maintaining it properly, and restoring it to service promptly when it fails. We can do this by developing those skills within our own team or by entering into relationships to help us.

Scale Out Your Hosting Solution
Design your systems to have three or more live data centers to reduce overall cost, increase availability, and implement disaster recovery.
Multiple live site benefits include
• Higher availability as compared to a hot and cold site configuration
• Lower costs compared to a hot and cold site configuration
• Faster customer response times if customers are routed to the closest data center for dynamic calls
• Greater flexibility in rolling out products in a SaaS environment
• Eliminate the need for state and affinity wherever possible
• Route customers to closest data center if possible to reduce dynamic call times
• Investigate replication technologies for databases and state if necessary
Reference
Scalability Rules: 50 Principles for Scaling Web Sites
Chapter 1. Reduce the Equation
Rule 1—Don’t Overengineer the Solution
Rule 2—Design Scale into the Solution (D-I-D Process)
Rule 3—Simplify the Solution 3 Times Over
Simplify scope using the Pareto Principle.
-- what 80% of your revenue will be achieved by 20% of your features.

Simplify design by thinking about cost effectiveness and scalability.
--  Complexity elimination is about doing less work, and design simplification is about doing that work faster and easier.
--  the step of design simplification asks us how to get the job done in an easy to understand, cost-effective, and scalable way.

Simplify implementation by leveraging the experience of others.
Rule 4—Reduce DNS Lookups
Rule 5—Reduce Objects Where Possible
Rule 6—Use Homogenous Networks

2. Distribute Your Work
Rule 7—Design to Clone Things (X Axis)
Rule 8—Design to Split Different Things (Y Axis)
Rule 9—Design to Split Similar Things (Z Axis)

3. Design to Scale Out Horizontally
Rule 10: Design Your Solution to Scale Out—Not Just Up
Rule 11—Use Commodity Systems (Goldfish Not Thoroughbreds)
Rule 12—Scale Out Your Data Centers
Rule 13—Design to Leverage the Cloud

4. Use the Right Tools
Rule 14—Use Databases Appropriately
Rule 15—Firewalls, Firewalls Everywhere!
Rule 16—Actively Use Log Files

5. Don’t Duplicate Your Work
Rule 17—Don’t Check Your Work
Avoid checking re-checking the work you just performed or immediately reading objects you just wrote within your products.
Never read what you just wrote for the purpose of validation. Store data in a local or distributed cache if it is required for operations in the near future.

The cost of validating your work is high relative to the unlikely cost of failure. Such activities run counter to cost-effective scaling.
Never justify reading something you just wrote for the purposes of validating the data. Trust your persistence tier to notify you of failures, and read and act upon errors associated with the write activity. Avoid other types of reads of recently written data by storing that data locally.

If you just wrote something and you know you are likely to need it again, just keep it around locally.
if the information you are writing is going to be needed in the near future, it makes sense to keep it around, to cache it.

Rule 18—Stop Redirecting Traffic
Rule 19—Relax Temporal Constraints
There are time lapses between a user viewing an item, putting it in his shopping cart, and purchasing it. One could argue that for the absolute best user experience, the state of the object, specifically whether or not the object is available, would ideally remain consistent throughout this process. To do so would require that the application mark the item as “taken” in the database until the user browses off the page, abandons the cart, or makes the purchase.

A BASE(Basically Available, Soft State, and Eventually Consistent) architecture allows for the databases to become consistent, eventually.
6. Use Caching Aggressively
Rule 20—Leverage Content Delivery Networks
Rule 21—Use Expires Headers
Rule 22—Cache Ajax Calls
Rule 23—Leverage Page Caches
Rule 24—Utilize Application Caches
Rule 25—Make Use of Object Caches
Rule 26—Put Object Caches on Their Own “Tier”

7. Learn from Your Mistakes
Rule 27—Learn Aggressively
Rule 28—Don’t Rely on QA to Find Mistakes
Rule 29—Failing to Design for Rollback Is Designing for Failure
Rule 30—Discuss and Learn from Failures

8. Database Rules
Rule 31—Be Aware of Costly Relationships
Rule 32—Use the Right Type of Database Lock
Rule 33—Pass on Using Multiphase Commits
Rule 34—Try Not to Use “Select For Update”
Rule 35—Don’t Select Everything

9. Design for Fault Tolerance and Graceful Failure
Rule 36—Design Using Fault Isolative “Swimlanes”
Rule 37—Never Trust Single Points of Failure
Rule 38—Avoid Putting Systems in Series
Rule 39—Ensure You Can Wire On and Off Functions

10. Avoid or Distribute State
Rule 40—Strive for Statelessness
Rule 41—Maintain Sessions in the Browser When Possible
Rule 42—Make Use of a Distributed Cache for States

11. Asynchronous Communication and Message Buses
Rule 43—Communicate Asynchronously As Much As Possible

Prefer asynchronous over synchronous communication whenever possible.
Rule 44—Ensure Your Message Bus Can Scale
Rule 45—Avoid Overcrowding Your Message Bus

12. Miscellaneous Rules
Rule 46—Be Wary of Scaling Through Third Parties
Rule 47—Purge, Archive, and Cost-Justify Storage
Rule 48—Remove Business Intelligence from Transaction Processing
Separate business systems from product systems and product intelligence from database systems.

Remove stored procedures from the database and put them in your application logic. Do not make synchronous calls between corporate and product systems.
Putting application logic in databases is costly and represents scale challenges. Tying corporate systems and product systems together is also costly and represents similar scale challenges as well as availability concerns.

Keep like transactions together (or alternatively separate unlike transactions) for the highest possible availability and scalability and best possible cost.

Rule 49—Design Your Application to Be Monitored
Rule 50—Be Competent