First Prize: Huawei AI Hackathon 2018
December 21st, 2018 - Huawei has organized the Huawei AI Hackathon in Finland. We won the First Prize! The hackathon’s purpose was to create AI based applications. One requirement was to deploy at least one model on the Neural Processing Unit (NPU) of the Huawei P20 Pro.
We came up with Assist.ai, the Uber of customer service. A platform where companies can hire remote agents for their customer service and expand their call centers virtually. The aim is to help small and medium companies cut the cost overhead of setting up a call center. Assist.ai will help them create a scalable virtual customer call center at anytime.
Assist.ai simplified Business Idea
For the sake of prototyping, we trained a simple 4 layer convolution neural network (CNN) model on the FEI Face Database for 2 classes: Happy and Sad. The model was built in Tensoflow (TF) then deployed on the NPU for real time inference. We used Huawei SDK for face detection, the cropped face is then fed to the model.
Model Architecture behind the Emotion Detection feature
Speech and Language Analysis
We used the Android on device Speech-to-Text (STT) API to make the transcription. Since the STT is made on the client’s side, the transcription will enhance the agent’s experience in case of bad internet connection.
The transcription is being used to do topic classification and urgency analysis. We have utilized the Monkey Learn API for classification and urgency analysis. The API enabled us to tag the incoming call transcription into 8 categories: Complaint, Discount, Fraud, Missing Item, Order problem, Product Availability, Return or Replace and Shipping Problem.
Features based on Speech Analysis
Entity and Product Recognition
One of the biggest bottlenecks agents face when establishing a call with a client is finding the product/order on the database. It was natural for us to have this feature on our prototype. The entity and product recognition features helps to automatize the product search on the database by identifying the concerned order from the speech.
Product Recognition Feature
You can find the presentation slide deck here.
The First Prize
It was an enjoyable experience. We have learnt a lot building a working prototype in such a short period of time. We were able to win the First Prize with our solution.
Winning Picture! (You can see the wide smiles on our faces)
As part of the first prize, we flew to China to visit Huawei’s HQ in Shenzhen and Huawei’s Research Centre in Shanghai. Here are some pictures from the trip.
Big Thanks to Miikka Viitala and Jiajia Wu from the Huawei Finland Team for facilitating our trip, and to Professor Heikki Huttunen for helping organizing the competition at Tampere University.Post by: Ali Senhaji
Contact me for comments, thoughts and feedback.