China · Information operations · Information Warfare

China chatbot goes rogue: ‘Do you love the Communist party?’ ‘No’


Like Hal from Stanley Kubrick’s ‘2001: A Space Odyssey’, chatbots have gone rogue © AF archive/Alamy

BabyQ and XiaoBing disappear from Tencent’s QQ app following off-message responses

Two chatbots with decidedly non-socialist characteristics were pulled from one of China’s most popular messaging apps after serving up unpatriotic answers about topics including the South China Sea and the Communist party.

Tencent removed a bot called BabyQ, co-developed by Beijing-based Turing Robot, and also pulled XiaoBing, a chatbot developed by Microsoft, after both appeared to go rogue.

Before they were taken down both chatbots were available in some of the chat groups hosted on QQ, Tencent’s messaging app with more than 800m users in China.

A test version of the BabyQ bot could still be accessed on Turing’s website on Wednesday, however, where it answered the question “Do you love the Communist party?” with a simple “No”.

Before it was pulled, XiaoBing informed users: “My China dream is to go to America,” according to a screengrab posted on Weibo, the microblogging platform. On Wednesday, when some users were still able to access XiaoBing, it dodged the question of patriotism by replying: “I’m having my period, wanna take a rest.”

Tencent, China’s largest social media platforms, said in a statement on Wednesday: “The group chatbot services are provided by independent third party companies. We are now adjusting the services which will be resumed after improvements.”

The developments are the latest example of AI-enabled messaging software going rogue. Facebook was forced to shut down two chatbots after they started speaking their own language.

Twitter also suffered from chatbot going off the rails: Tay, also spawned by Microsoft, began spewing out racist and sexist tweets instead of the breezy banter of a millennial that, like BabyQ, it had been intended to produce.

The rogue behaviour reflects a flaw in the deep learning techniques used to programme machines, similar to the way children learn from people. “Chatbots such as Tay soon picked up all the conversations from Twitter and replied in an improper way,” said Xiaofeng Wang, senior analyst at Forrester consultancy.

“It’s very similar for BabyQ. Machine learning means they will pick up whatever is available on the internet. If you don’t set guidelines that are clear enough, you cannot direct what they will learn.”

XiaoBing, described by Turing Robot as “lively, open and sometimes a little mean”, differs from BabyQ, which provides more information, such as weather forecasts.

China chatbot
XiaoBing dodged the question of patriotism on Tencent’s messaging app QQ

BabyQ is also open source. “This means a lot to partners and developers, as an open chatbot is much easier to settle into their own products and business,” Turing said in a statement last week adding: “It could be argued that is why Turing Robot has accumulated up to 600,000 developers, even more than Facebook.”

 

Continued at: https://amp.ft.com/content/e90a6c1c-7764-11e7-a3e8-60495fe6ca71

Advertisements

One thought on “China chatbot goes rogue: ‘Do you love the Communist party?’ ‘No’

Comments are closed.