What are the advantages of Erlang over other programming languages?
Lightweight user-space threads.
Built-in distribution and failure detection.
Mostly functional programming with decent matching constructs, but with fully dynamic typing.
Reliability-oriented standard library.
Hot code swapping.
There's a lot of runtime introspection above and beyond hot code swapping. You can attach a console to a running node and inspect and modify state interactively. I'm a big fan of debugging with printfs, but if you've got a stateful server in production and you want to debug it, it's a lot easier to connect using a real REPL shell than with say gdb.
Erlang has an excellent platform that rivals and in some cases, exceeds, the JVM. The BEAM VM is excellent. Since processes have a per-process heap, garbage collection pauses are greatly reduced compared to HotSpot. The library contains support for operational functionality like exposing SNMP traps and gauges.
A well tested garbage collector (acceptable "soft real time" performance) and the operational features make Erlang an excellent platform for long running services (as opposed to short running services, such as web application request handlers running as pre-spawned processes with mod_php/mod_perl/mod_python under Apache).
the language makes it easy to build a system out of single purpose actors that communicate only via immutable message passing. The language also makes it easy to build systems that can have isolated subsystems that can crash without bringing the whole system down and be restarted without intervention. Single assignment makes it very easy to reason locally. First class functions make it easy to write programs as a series of small functions that operate on both data and functions as data. It makes it easier to write distributed systems because you are already just sending messages to actors so they might just end up living on a different machine.
The language makes it hard to use shared mutable data structures that require locks, it makes it hard to crash the whole vm because of a single bug, it makes it hard to write confusing large functions where a single variable takes on many meanings, it makes it hard to write imperative loops without clear invariants (because there are no loops and recursive functions with argument pattern matching makes invariants obvious). Erlang is a functional language, implementing the actor model for concurrency. »The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.«
While Discord is rich with features, most of it boils down to pub/sub. Users connect to a WebSocket and spin up a session process (a GenServer), which then communicates with remote Erlang nodes that contain guild (internal for a “Discord Server”) processes (also GenServers). When anything is published in a guild, it is fanned out to every session connected to it.
When a user comes online, they connect to a guild, and the guild publishes a presence to all other connected sessions.
Eventually we ended up with many Discord servers like /r/Overwatch with up to 30,000 concurrent users. During peak hours, we began to see these processes fail to keep up with their message queues. At a certain point, we had to manually intervene and turn off features that generated messages to help cope with the load
