The WhatsApp Architecture Facebook Bought For $19 Billion
it’s getting all those users to love and use your product that is the hard part. You can’t buy love
WhatsApp is a no gimmicks (no ads, no gimmicks, no games) product with loyal users from across the world.
It is aggressively cross platform so everyone you know can use it and it will just work.
WhatsApp is used to create group conversations for project teams and venture capitalists carry out deal flow conversations over WhatsApp.
Messaging has become the center of engagement on mobile, not search, changing how things are found and the very nature of what applications will win the future.
On mobile it’s easier to go find a special app than it is to find a menu buried deep within a complicated portal style application.
Conway's law may be operating here. The idea that “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.”
Backend
Erlang
FreeBSD
Yaws, lighttpd
PHP
Custom patches to BEAM (BEAM is like Java’s JVM, but for Erlang)
Custom XMPP
Hosting may be in Softlayer
WhatsApp server is almost completely implemented in Erlang.
Great achievement is that the number of active users is managed with a really small server footprint. Team consensus is that it is largely because of Erlang.
A primary gauge of system health is message queue length. The message queue length of all the processes on a node is constantly monitored and an alert is sent out if they accumulate backlog beyond a preset threshold.
Erlang helps being aggressive in getting fixes and features into production. Hot-loading means updates can be pushed without restarts or traffic shifting. Mistakes can usually be undone very quickly, again by hot-loading. Systems tend to be much more loosely-coupled which makes it very easy to roll changes out incrementally.
How does the registration process work internally in Whatsapp? WhatsApp used to create a username/password based on the phone IMEI number. This was changed recently. WhatsApp now uses a general request from the app to send a unique 5 digit PIN. WhatsApp will then send a SMS to the indicated phone number (this means the WhatsApp client no longer needs to run on the same phone). Based on the pin number the app then request a unique key from WhatsApp. This key is used as "password" for all future calls. (this "permanent" key is stored on the device). This also means that registering a new device will invalidate the key on the old device.
Google’s push service is used on Android.
Need to plan for bumps in traffic.
Servers were brittle in the face of burst loads. Network glitches and other problems would occur. Needed to decouple components so things weren’t so brittle at high capacity.
Wrote system activity reporter tool (wsar):
Limits in the cause of simplicity are OK.
Start simply and then customize.
Keep server count low. Constantly work to keep server counts as low as possible while leaving enough headroom for events that create short-term spikes in usage. Analyze and optimize until the point of diminishing returns is hit on those efforts and then deploy more hardware.
Purposely overprovision hardware.
Growth stalls when you charge money.
https://www.quora.com/How-does-messaging-work-in-WhatsApp
WhatsApp or most of the other messaging apps rarely work on a peer to peer basis. So it wouldn't open a connection (from your device) to each of your friends' devices. Instead your device connects to their server. It could then use a custom TCP protocol or maybe HTTP to communicate your messages to the server. The server in return would dispatch them to your friends' devices. If your friend had their app open or at least the app process running there might be a live connection to the server. WhatsApp will use that connection to send them your messages. If their app is "offline" then they might choose to send them a push notification instead.
WhatsApp has chosen Erlang a language built for writing scalable applications that are designed to withstand errors. Erlang uses an abstraction called the Actor model for it's concurrency - http://en.wikipedia.org/wiki/Act.... Instead of the more traditional shared memory approach, actors communicate by sending each other messages. Actors unlike threads are designed to be lightweight. Actors could be on the same machine or on different machines and the message passing abstractions works for both. A simple implementation of WhatsApp could be:
Each user/device is represented as an actor. This actor is responsible for handling the inbox of the user, how it gets serialized to disk, the messages that the user sends and the messages that the user receives.
The advantage of using XMPP is it comes with built in support for rosters and presence so you don't have to manually setup infrastructure and code to manage subscriptions.
WhatsApp uses XMPP (eXtensible Messaging and Presence Protocol) to handle the message delivery system.
XMPP is mostly like HTTP where the client opens the socket with the XMPP server and keeps it open as long as the client is logged in. It's not like the regular REST API where the client opens the socket send/receive the data and close the socket. The socket is opened as long as you are signed in. In case of WhatsApp that's eternity (not really, WhatsApp reconnects automatically if the connection terminates)
XMPP protocol has been used in various chat applications such as Google Talk, Facebook messenger etc.
http://siddharth-ravichandran.com/2011/01/21/multiuser-chat-using-xmpp-and-orbited-using-ruby-on-rails/
Ejabberd
Ejabberd is a Jabber/XMPP server built using Erlang and is open-source and we would be using Ejabberd in this example. Another popular alternative for Ejabberd is Openfire – Ejabberd
Long-Polling
Comet is a broad term used for technologies like Long-Polling and streaming. While traditional polling requires periodic requests to be sent to the server and then return with the response, in long-polling a connection is established with the server which persists until a response is provided (or the request times out). Once the response is provided the connection is closed and a new one is step up waiting for the next response from the server. Similarly a new connection is set up on timeout. In Streaming the connection persists between the client and the server while the information is transferred.
The key to the smooth data processing lies with the message queue length. The length of the message queue of all the processes associated with a node is monitored.
Instead of being dependent on the IMEI number, WhatsApp now asks for the app to send a 5 digit PIN. A message will then be sent to the phone number from WhatsApp. The app on the mobile device requests for a unique key from WhatsApp. This ditches out the dependency of WhatsApp being used on the device.
http://blog.contus.com/how-whatsapp-works-technically-and-how-to-build-an-app-similar-to-it/
While Mnesia is used to maintain user login sessions, MySQL contributes in maintaining the user details, right from account credentials to other personal data.
As and when a user triggers a message, the app’s SDK sends a request to the Ejabbered server. In return the server checks for the availability of the recipient and if available, the delivery request is sent to the recipients SDK and then delivered. If the recipient is offline, the datas are stored in the Ejabbered as message will be queued in the server.
it’s getting all those users to love and use your product that is the hard part. You can’t buy love
WhatsApp is a no gimmicks (no ads, no gimmicks, no games) product with loyal users from across the world.
It is aggressively cross platform so everyone you know can use it and it will just work.
WhatsApp is used to create group conversations for project teams and venture capitalists carry out deal flow conversations over WhatsApp.
Messaging has become the center of engagement on mobile, not search, changing how things are found and the very nature of what applications will win the future.
On mobile it’s easier to go find a special app than it is to find a menu buried deep within a complicated portal style application.
Conway's law may be operating here. The idea that “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.”
Backend
Erlang
FreeBSD
Yaws, lighttpd
PHP
Custom patches to BEAM (BEAM is like Java’s JVM, but for Erlang)
Custom XMPP
Hosting may be in Softlayer
WhatsApp server is almost completely implemented in Erlang.
Great achievement is that the number of active users is managed with a really small server footprint. Team consensus is that it is largely because of Erlang.
A primary gauge of system health is message queue length. The message queue length of all the processes on a node is constantly monitored and an alert is sent out if they accumulate backlog beyond a preset threshold.
Erlang helps being aggressive in getting fixes and features into production. Hot-loading means updates can be pushed without restarts or traffic shifting. Mistakes can usually be undone very quickly, again by hot-loading. Systems tend to be much more loosely-coupled which makes it very easy to roll changes out incrementally.
How does the registration process work internally in Whatsapp? WhatsApp used to create a username/password based on the phone IMEI number. This was changed recently. WhatsApp now uses a general request from the app to send a unique 5 digit PIN. WhatsApp will then send a SMS to the indicated phone number (this means the WhatsApp client no longer needs to run on the same phone). Based on the pin number the app then request a unique key from WhatsApp. This key is used as "password" for all future calls. (this "permanent" key is stored on the device). This also means that registering a new device will invalidate the key on the old device.
Google’s push service is used on Android.
Need to plan for bumps in traffic.
Servers were brittle in the face of burst loads. Network glitches and other problems would occur. Needed to decouple components so things weren’t so brittle at high capacity.
Wrote system activity reporter tool (wsar):
Limits in the cause of simplicity are OK.
Start simply and then customize.
Keep server count low. Constantly work to keep server counts as low as possible while leaving enough headroom for events that create short-term spikes in usage. Analyze and optimize until the point of diminishing returns is hit on those efforts and then deploy more hardware.
Purposely overprovision hardware.
Growth stalls when you charge money.
https://www.quora.com/How-does-messaging-work-in-WhatsApp
WhatsApp or most of the other messaging apps rarely work on a peer to peer basis. So it wouldn't open a connection (from your device) to each of your friends' devices. Instead your device connects to their server. It could then use a custom TCP protocol or maybe HTTP to communicate your messages to the server. The server in return would dispatch them to your friends' devices. If your friend had their app open or at least the app process running there might be a live connection to the server. WhatsApp will use that connection to send them your messages. If their app is "offline" then they might choose to send them a push notification instead.
WhatsApp has chosen Erlang a language built for writing scalable applications that are designed to withstand errors. Erlang uses an abstraction called the Actor model for it's concurrency - http://en.wikipedia.org/wiki/Act.... Instead of the more traditional shared memory approach, actors communicate by sending each other messages. Actors unlike threads are designed to be lightweight. Actors could be on the same machine or on different machines and the message passing abstractions works for both. A simple implementation of WhatsApp could be:
Each user/device is represented as an actor. This actor is responsible for handling the inbox of the user, how it gets serialized to disk, the messages that the user sends and the messages that the user receives.
The advantage of using XMPP is it comes with built in support for rosters and presence so you don't have to manually setup infrastructure and code to manage subscriptions.
WhatsApp uses XMPP (eXtensible Messaging and Presence Protocol) to handle the message delivery system.
XMPP is mostly like HTTP where the client opens the socket with the XMPP server and keeps it open as long as the client is logged in. It's not like the regular REST API where the client opens the socket send/receive the data and close the socket. The socket is opened as long as you are signed in. In case of WhatsApp that's eternity (not really, WhatsApp reconnects automatically if the connection terminates)
XMPP protocol has been used in various chat applications such as Google Talk, Facebook messenger etc.
http://siddharth-ravichandran.com/2011/01/21/multiuser-chat-using-xmpp-and-orbited-using-ruby-on-rails/
Ejabberd
Ejabberd is a Jabber/XMPP server built using Erlang and is open-source and we would be using Ejabberd in this example. Another popular alternative for Ejabberd is Openfire – Ejabberd
Long-Polling
Comet is a broad term used for technologies like Long-Polling and streaming. While traditional polling requires periodic requests to be sent to the server and then return with the response, in long-polling a connection is established with the server which persists until a response is provided (or the request times out). Once the response is provided the connection is closed and a new one is step up waiting for the next response from the server. Similarly a new connection is set up on timeout. In Streaming the connection persists between the client and the server while the information is transferred.
The key to the smooth data processing lies with the message queue length. The length of the message queue of all the processes associated with a node is monitored.
Instead of being dependent on the IMEI number, WhatsApp now asks for the app to send a 5 digit PIN. A message will then be sent to the phone number from WhatsApp. The app on the mobile device requests for a unique key from WhatsApp. This ditches out the dependency of WhatsApp being used on the device.
http://blog.contus.com/how-whatsapp-works-technically-and-how-to-build-an-app-similar-to-it/
While Mnesia is used to maintain user login sessions, MySQL contributes in maintaining the user details, right from account credentials to other personal data.
As and when a user triggers a message, the app’s SDK sends a request to the Ejabbered server. In return the server checks for the availability of the recipient and if available, the delivery request is sent to the recipients SDK and then delivered. If the recipient is offline, the datas are stored in the Ejabbered as message will be queued in the server.
Architectures behind Whatsapp or similar applications.
In that scenarios, where mobiles are the key, you have a very strong limitation about energy consumption (Battery). So, you cannot implement a chat pulling the server for new messages every second, to have real time chat.
I think that pub/sub architectures using technologies such as AMPQ implementations, or PubNub could be the base to support these real time messaging functionalities.
Actually, if I am not wrong, Google is able to do it with XMPP in his GCM.
You could consider MQTT ( also available in RabbitMQ), for example Facebook messenger uses MQTT.
http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger
Scaling to Millions of Simultaneous Connections: Rick ReedIn that scenarios, where mobiles are the key, you have a very strong limitation about energy consumption (Battery). So, you cannot implement a chat pulling the server for new messages every second, to have real time chat.
I think that pub/sub architectures using technologies such as AMPQ implementations, or PubNub could be the base to support these real time messaging functionalities.
Actually, if I am not wrong, Google is able to do it with XMPP in his GCM.
You could consider MQTT ( also available in RabbitMQ), for example Facebook messenger uses MQTT.
http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger
