For more than two hours on Thursday, June 6, a large chunk of European mobile traffic was rerouted through the infrastructure of China Telecom, China’s third-largest telco and internet service provider (ISP).
The incident occurred because of a BGP route leak at Swiss data center colocation company Safe Host, which accidentally leaked over 70,000 routes from its internal routing table to the Chinese ISP.
The Border Gateway Protocol (BGP), which is used to reroute traffic at the ISP level, has been known to be problematic to work with, and BGP leaks happen all the time.
However, there are safeguards and safety procedures that providers usually set up to prevent BGP route leaks from influencing each other’s networks.
But instead of ignoring the BGP leak, China Telecom re-announced Safe Host’s routes as its own, and by doing so, interposed itself as one of the shortest ways to reach Safe Host’s network and other nearby European telcos and ISPs.
Mobile operators in France, Holland, Switzerland affected
For the subsequent hours, until China Telecom operators realized what they have done, traffic meant for many European mobile networks was rerouted through China Telecom’s network.
“Some of the most impacted European networks included Swisscom (AS3303) of Switzerland, KPN (AS1130) of Holland, and Bouygues Telecom (AS5410) and Numericable-SFR (AS21502) of France,” said Doug Madory, Director of Oracle’s Internet Analysis division (formerly Dyn).
“Often routing incidents like this only last for a few minutes, but in this case many of the leaked routes in this incident were in circulation for over two hours,” Madory added.
For the users on the affected mobile network, this manifested as slow connections or the inability to connect to some servers.
“Today’s incident shows that the internet has not yet eradicated the problem of BGP route leaks,” Madory said.
“It also reveals that China Telecom, a major international carrier, has still implemented neither the basic routing safeguards necessary both to prevent propagation of routing leaks nor the processes and procedures necessary to detect and remediate them in a timely manner when they inevitably occur.
“Two hours is a long time for a routing leak of this magnitude to stay in circulation, degrading global communications.”
But if any other ISP would have caused this incident, it would have likely been ignored. Alas, it was China Telecom, and there’s a backstory.
An academic paper published by experts from the US Naval War College and Tel Aviv University in October last year blamed China Telecom for “hijacking the vital internet backbone of western countries.”
The report argued that the Chinese government was using local ISPs for intelligence gathering by systematically hijacking BGP routes to reroute western traffic through its country, where it can log it for later analysis.
While some experts have criticized the paper, Madory is one of the people who stood by its technical accuracy — albeit not by its politically-charged accusations– confirming that China Telecom has rerouted western traffic through its network for years many times before.
However, Madory couldn’t say if this was intentional, or a technical or human error.
Back last year, Madory recommended that internet service providers support up-and-coming BGP security standards such as RPKI, as a way to prevent such internet traffic “misdirections” from taking place in the first place.