How to load balance when Netflix fails

Even during the Corona 19 period, the number of Netflix viewers is increasing. Accordingly, the server load is also increasing. Of course, there may be a one-hour suspension depending on the region, but in 2020, there were no large-scale failures involving service suspensions around the world. This is not a coincidence, but a result of Netflix’s efforts. On November 2 (local time), Netflix unveiled a prioritized load balancing structure that manages traffic without interrupting service in the event of a failure.

Netflix is not originally centralized, but takes the form of microservices where small services are gathered to form the whole. For this reason, even if a hardware failure occurs in some servers, the possibility of the entire service being interrupted is lowering. It is a famous story that Chaos Engineering is employed, which always checks the operation to ensure that the failure continues to automatically recover and prevents major failures.

However, there may also be secondary failures, such as some part of the system is stopped and other parts of the traffic increase. Therefore, Netflix introduced a method to prevent service disruption by dividing by traffic importance and limiting traffic that does not directly affect the user viewing experience. It is the API gateway service line (Zuul) that determines the importance, and the source code is released on GitHub.

In fact, even in 2020, even though an error such as the global service disruption occurred in 2019, this structure was able to recover without service interruption. Netflix will continue to do so in the future, such as adjusting the load limit threshold.

Meanwhile, Netflix also enforced a bit rate limit in Europe in March 2020, but this is not for load balancing, but to reduce the amount of data through the network throughout Europe. Related information can be found here .