LINEの大規模リアルタイム配信基盤

LINE must deliver messages in realtime every day to hundreds of millions of users. It is not unusual to encounter situations where a single post fans out explosively — group chats with thousands of participants, or simultaneous broadcasts from official accounts. This article gives an overview, within what can be inferred from public materials, of the delivery infrastructure that supports this realtime nature.

LINE clients are designed to maintain a connection to the server throughout startup and immediately receive new messages and notifications. Supporting this is a cluster of dedicated gateways terminating WebSocket (or HTTP/2 or proprietary protocols). Each server is required to handle hundreds of thousands of simultaneous connections, and they are thought to be implemented using non-blocking I/O leveraging Netty/epoll, etc.

The fundamental point of chat delivery is "when to perform fan-out." Fan-out on Write writes to each recipient's inbox queue at post time — reads are fast, but write cost explodes for huge groups. Fan-out on Read writes only to the shared group message log, with clients fanning out at read time — strong for huge groups but complex on the read side. LINE is seen to use both depending on the chat's size and type.

Posted messages are thought to be processed asynchronously through Apache Kafka as a relay rather than delivered directly from the gateway to all recipients. Because Kafka offers high throughput, persistence, and per-partition order guarantees, it is suited as a "common backbone" to various subsystems — fan-out, push notifications, read-receipt management, history storage, etc. LINE has published its track record of operating Kafka at scale over many years and is reportedly running it with a dedicated internal team.

Information about "which user is currently connected to which gateway" changes frequently yet must be referenced every time a message is delivered. Such session information and latest-message caches are commonly thought to be held in in-memory stores like Redis. LINE is publishing about its proprietary Redis operations extensions and long-term use of distributed KVS (HBase), and is seen to combine storage by role.

LINE is used across Asia — Japan, Korea, Taiwan, Thailand, Indonesia — and is thought to operate multiple datacenters to provide low-latency experiences to users in each region. Cross-border data transfer must also balance with each country's data protection regulations. A typical strategy is thought to be placing primary data in the user's home datacenter and edge caches or read-only replicas in other regions.

While clients are disconnected or in the background, push notifications via Apple APNs (iOS) and Google FCM (Android) are required. Because these external providers have proprietary rate limits, formats, and retry policies, LINE itself also needs multi-layer delivery strategies — retry queues, fallbacks for failures (SMS or follow-up notifications), and so on.

Realtime delivery infrastructure must be designed to keep service running even when failures occur. Gateway clusters are made multi-redundantly so clients can automatically reconnect when one node goes down; Kafka replicates across multiple brokers; and configurations are thought to be in place to fail over to other datacenters even if an entire datacenter fails. Meanwhile, acceptable trade-offs at failover — temporary order reversals, duplicate deliveries — must be defined in advance.

With hundreds of millions of connections, even a single heartbeat (keep-alive) packet has cost. The accumulation of fine-grained optimizations — protocol-layer compression (HTTP/2 with header compression or proprietary binary encodings), heartbeat-interval tuning, bundled sending of small messages like typing indicators and read-receipt sync — directly impacts overall service cost.

LINE has announced end-to-end encryption for one-to-one and group chats under the name "Letter Sealing," and the delivery infrastructure is designed to fundamentally just relay encrypted payloads without decrypting their contents. Key management is complex and must respond to various scenarios — device additions, history re-sync, group membership changes, etc.

LINE's realtime delivery infrastructure is a giant stack combining multiple layers — gateways holding long-running connections, an asynchronous backbone centered on Kafka, state management via Redis/HBase, regional distribution, and end-to-end encryption. Vast engineering can be said to be accumulated behind the seemingly simple requirement of "just delivering a message."