First Prize: Huawei AI Hackathon 2018

December 21^st, 2018 - Huawei has organized the Huawei AI Hackathon in Finland. We won the First Prize! The hackathon’s purpose was to create AI based applications. One requirement was to deploy at least one model on the Neural Processing Unit (NPU) of the Huawei P20 Pro.

I took part of this hackathon along with my research colleagues from the Signal Analysis and Machine Intelligence (SAMI) Group at Tampere University: Kateryna Chumachenko and Firas Laakom.

Our idea

We came up with Assist.ai, the Uber of customer service. A platform where companies can hire remote agents for their customer service and expand their call centers virtually. The aim is to help small and medium companies cut the cost overhead of setting up a call center. Assist.ai will help them create a scalable virtual customer call center at anytime.

Assist.ai simplified Business Idea

The Prototype

Emotional Analysis

For the sake of prototyping, we trained a simple 4 layer convolution neural network (CNN) model on the FEI Face Database for 2 classes: Happy and Sad. The model was built in Tensoflow (TF) then deployed on the NPU for real time inference. We used Huawei SDK for face detection, the cropped face is then fed to the model.

Model Architecture behind the Emotion Detection feature

Speech and Language Analysis

We used the Android on device Speech-to-Text (STT) API to make the transcription. Since the STT is made on the client’s side, the transcription will enhance the agent’s experience in case of bad internet connection.

The transcription is being used to do topic classification and urgency analysis. We have utilized the Monkey Learn API for classification and urgency analysis. The API enabled us to tag the incoming call transcription into 8 categories: Complaint, Discount, Fraud, Missing Item, Order problem, Product Availability, Return or Replace and Shipping Problem.

Features based on Speech Analysis

Entity and Product Recognition

One of the biggest bottlenecks agents face when establishing a call with a client is finding the product/order on the database. It was natural for us to have this feature on our prototype. The entity and product recognition features helps to automatize the product search on the database by identifying the concerned order from the speech.

Product Recognition Feature

You can find the presentation slide deck here.

The First Prize

It was an enjoyable experience. We have learnt a lot building a working prototype in such a short period of time. We were able to win the First Prize with our solution.

Winning Picture! (You can see the wide smiles on our faces)

As part of the first prize, we flew to China to visit Huawei’s HQ in Shenzhen and Huawei’s Research Centre in Shanghai. Here are some pictures from the trip.

Big Thanks to Miikka Viitala and Jiajia Wu from the Huawei Finland Team for facilitating our trip, and to Professor Heikki Huttunen for helping organizing the competition at Tampere University.