AI risk: Watching the watchers

When Megvii, one of China’s artificial intelligence (AI) champions, filed for its IPO last week, there was much fuss about it being the first company of its kind to go public in Hong Kong during a turbulent political period locally, as well as the dissonance between surging revenues and a massive net loss. But just one layer deeper, amidst the operational and financial disclosures, the revelations got a lot more interesting. 

Three-quarters of Megvii’s revenue in 2018 came from its urban internet-of-things, or “City IoT” business, which encompasses a range of digitization services around security, traffic management and resource planning. This compares to about 27% in 2016. Since launching the unit in 2015, Megvii has become China’s largest AI urban IoT provider in revenue terms. Its systems were used in 112 cities by mid-2019, compared to 30 in 2016. 

There are at least two major risk factors in this kind of acceleration. First, growth is clearly dependent on government spending for smart city development, a significant variable in a trade war-rattled economy. Second, and perhaps most interestingly, Megvii has confirmed the extent of its exposure to the surveillance of people, an increasingly toxic aspect of AI from a branding perspective. 

The City IoT business involves a number of innocuous street and safety monitoring systems but also potentially delicate “smart school and kindergarten” software and tellingly downplayed police applications. Bloomberg recently reported that Megvii AI has been used by police to arrest some 10,000 people in more than 260 cities, although that geographic footprint doesn’t jive with the disclosed data.

Megvii is well aware of the publicity pitfalls around all this. Earlier this year, the company called foul when Human Rights Watch incorrectly tied its technology to a controversial app that monitors the downloader’s behavior. The activist organization retracted its comments, but Megvii noted the episode “still caused significant damages to our reputation which are difficult to completely mitigate.”

At the same time, best-practice control measures for ethical deployment tend to lose their luster in the fine print. Megvii says it checks its supply chains thoroughly and will not do business with any customers that use the technology in abuse of civil or human rights. The catch is that many of its customers are not the end-users. Most of the company’s City IoT customers are “system integrators” that provide various implementation services to governments and companies. 

“When we enter into a contract with a system integrator, we recognize such system integrator, instead of the relevant end-user, as our customer,” Megvii said in its draft prospectus. If the company classifies these middlemen as the customers, that could lead to complications around pledges to avoid use of the technology for illegal or inappropriate purposes. To be clear, these are only potential transparency issues, but Megvii does appear to be vulnerable to them.       

The biggest concern for investors is that Megvii merely represents the first peek behind the scenes of a mysterious but increasingly cashed-up segment. PE and VC investment in Chinese AI companies increased 3,500% over the three years to 2018 to $5 billion. And the other frontrunners all focus on surveillance and facial recognition, including SenseTime, Yitu Technology, and Horizon Robotics. Considering the rules for this industry are still being hashed out in both legal and social terms, much will depend on how closely these companies can be supervised by their investors.    

• Tweet  
• Facebook  
• LinkedIn  
• Google plus  
• Save this article  
• Send to  

• Topics
• Greater China
• Technology
• China
• artificial intelligence