Three challenges you’re going to face when building a chatbot
Adam Radziszewski on Dec. 01, 20166 min read
Developing your own chatbot? You're likely to meet these challenges. Continue reading to find out what we've learned from developing a medical bot.
Recently we have released Symptomate Bot – one of the world’s first health check-up chatbots. The road towards a working service was a new experience for us. We decided to share some lessons we’ve learned during the process, hoping other bot developers will benefit from them. We'd love to hear your opinion and also learn from your experiences.
At Infermedica we've developed a diagnostic engine that collects symptoms, asks diagnostic questions and presents likely health issues underlying this evidence. The engine uses a complex probabilistic model built on top of a knowledge base curated by medical professionals and enriched by machine learning. It is available via API and has provided helpful information to over 1M patients through a number of symptom checker apps, intelligent patient intake forms and other applications. Symptomate Bot is our attempt at building a conversational interface to the engine that will work as a symptom checker chatbot.
Here are three of the challenges we had to face during the process.
1. Scalable architecture for handling messages
Chatbots are a new trend, which makes it extremely difficult to predict the amount of traffic the bot will be exposed to. If you take the undertaking seriously, you should anticipate the possibility of a sudden traffic increase even if it doesn’t seem likely to come soon. Keep in mind that chat platforms allow users to report a defunct or misbehaving bot, which could result in a sudden account shutdown, not to mention a loss of credibility.
On most platforms (Facebook Messenger included) the bot developer is responsible for exposing a webhook. Your webhook will be called each time a user messages the bot. Handling each message involves understanding the message given the current state of the conversation with the user, altering the conversation state and possibly sending back responses. It may require some expensive operations (in our case calling our API which hosts both diagnostic reasoning and NLP services).
“You should anticipate the possibility of a sudden traffic increase even if it doesn’t seem likely to come soon.”
The setting described above fits well into an architectural pattern called message broker. In this model, user messages are stored in a message queue from which a number of concurrent workers will pick them up and handle them. Here are some especially important requirements for chatbots:
- Instant handling – The server sitting behind the webhook should handle incoming messages instantly by moving them straight to the message queue. Some chat platforms may be pretty touchy about delays, so better safe than sorry.
- Assigned worker – The message queue should guarantee that each user will always be handled by the same worker (consumer in message queue lingo). This is necessary to ensure that messages sent by a user are always handled in the order in which they are sent.
- Load-balancing – The message queue should also perform load-balancing to keep the workers reasonably occupied with tasks.
- Propagation – If a worker dies, the queue should be able to transfer its responsibilities to another worker.
- Event processing loop – A worker’s event processing loop includes these steps: pop a message from the queue, fetch the conversation state associated with the user, handle the message, possibly replying to the user, and push the updated state back.
- Storage – The storage place for the per-user conversation state should allow flexible structure and fast read/write access. The best choice seems to be a NoSQL database. We opted for MongoDB because its simple and extensible document format seems perfect for keeping track of the conversation state.
We did some research on message queue software. Our first finding was that requirements 2 and 3 are satisfied by a feature called Message Groups.
What came as a surprise is that the most popular implementations, such as ActiveMQ and RabbitMQ, do not support this feature (the tumultuous history of message queue projects is perhaps worth a read on its own). We found a working and user-friendly implementation of message groups in Apache Apollo and we can recommend it as a working solution even though it's not the most recent project in the message queue family. We’d love to hear your opinions and experiences in this matter.
2. A conversational agent must converse
There’s no way around it. If users just wanted to tap and swipe, they wouldn't be using a messenger; they'd go to your fancy responsive website. Go to botlist.co, pick a category, start interacting with random bots and you’ll likely see a lot of oversized tables filled with buttons and awkwardly cropped images, all squeezed into a tiny chat window.
Having said that, we think there's nothing wrong with using response templates for simple questions with a small number of possible answers. Using them wisely may direct the user towards the answers they need without giving the impression of talking to an IVR system (everyone hates to hear “press 1 for help”).
Natural Language Processing
If you want to join the chatbot hype and position your product as an intelligent one, you need to implement at least very basic Natural Language Processing (NLP) techniques. Take the time to come up with use cases and likely conversation scenarios. Try writing down intents the user may express when chatting. With an anticipated spectrum of intents in front of you, it will be easier to prepare to recognize them.
“If users just wanted to tap and swipe, they wouldn't be using a messenger; they'd go to your fancy responsive website.”
In the case of a symptom checker, we anticipated users trying to communicate their symptoms, attempting to correct misunderstood observations or telling the bot that they have nothing more to say.
Also, think about objects in your domain that the user may want to address. Do they have names? Do they have synonyms? Are they typically expressed as noun phrases or do you also need to deal with clauses (e.g., “my back hurts”) in some descriptions?
We realised that understanding of symptom mentions in text is an important task. There are excellent tools out there to help you recognize medical concepts — such as Lexigram API. It’s worth mentioning that Infermedica offers an entity recognition tool tailored to work together with our customized medical knowledge base. We've made the tool available as a text analysis endpoint in our API.
Chat language
Last but not least, chat language is far from standard English. You can refuse to deal with spelling errors, or face the chat reality and implement spelling correction. While there is an abundance of open-source projects related to spell-checking and spelling correction, you'd be surprised how slow most implementations are.
Before adopting a solution it makes sense to perform some tests on your own. You can't afford to spend 200 milliseconds on each word. For industrial-strength spelling correction you should consider the Symmetric Delete algorithm.
3. Users will get frustrated
During initial testing it became clear to us that much of the time users are going to be misunderstood. To put it bluntly, a huge proportion of messages will not be understood at all and there is little we can do about it. What we can do is anticipate user frustration and alleviate the situation by offering a little help and managing expectations. In some situations we can turn an utter lack of understanding into partial understanding, which is a huge win.
The scale of the comprehension problems will depend on the size of the domain. If the bot is to print out the current exchange rates, there is little room for misunderstanding. Our domain is common health problems. The bot is a conversational interface to our diagnostic engine, built on top of a knowledge base curated by medical professionals. While the knowledge base contains over 1400 symptoms and thousands of synonyms, some users will inevitably want to report observations that are not covered. Also, no NLP technology is perfect and ours is no different; language allows for enormous freedom of expression and some descriptions of known symptoms are not going to be understood.
Analytics
In Symptomate Bot we keep track of some basic statistics gathered during each “visit” (a conversation which starts with the user being asked to report his/her symptoms and finishes with the presentation of possible diagnoses). These statistics include a measure of anticipated frustration, which is used to decide when to digress in order to offer a bit of contextual help, or at least acknowledge that we are aware of current shortcomings.
“To put it bluntly, a huge proportion of messages will not be understood at all and there is little we can do about it.”
Also, we've found it worthwhile to fall back on a partial understanding mechanism when our main NLP module fails to detect any symptom in the user’s story. This mechanism assumes that the story was a description of exactly one symptom (the main NLP module can capture multiple mentions). We use a vector space model to find the textually most similar symptoms and ask if the user meant any of those.
Conclusion
We've just shared a few hints and thoughts stemming from our experience building a conversational interface to our medical inference engine. We hope you’ll find these observations useful when building your own chatbot, whatever the domain. Stay tuned for more articles — we’ll be covering topics in healthcare, AI and NLP.
Read more in: Architecture, Chatbots, NLP, Symptom Checker, Technical
Adam Radziszewski
Share
Subscribe to our newsletter
We’ll send you the latest articles from this blog and updates on what we’re working on.
SUBSCRIBE
(转)Three challenges you’re going to face when building a chatbot的更多相关文章
-
JavaScript Modules
One of the first challenges developers new to JavaScript who are building large applications will ha ...
-
What are the 10 algorithms one must know in order to solve most algorithm challenges/puzzles?
QUESTION : What are the 10 algorithms one must know in order to solve most algorithm challenges/puzz ...
-
SQLI LABS Challenges Part(54-65) WriteUp
终于到了最后一部分,这些关跟之前不同的是这里是限制次数的. less-54: 这题比较好玩,10次之内爆出数据.先试试是什么类型: ?id=1' and '1 ==>>正常 ?id=1' ...
-
译文 - Recommender Systems: Issues, Challenges, and Research Opportunities
REF: 原文 Recommender Systems: Issues, Challenges, and Research Opportunities Shah Khusro, Zafar Ali a ...
-
Solving the Top ERP and CRM Metadata Challenges with erwin &; Silwood
Registrationhttps://register.gotowebinar.com/register/3486582555108619010 Solving the Top ERP and CR ...
-
Database Sharding Challenges DATABASE SHARDING
w分布式查询.数据聚合.跨碎片join是可且应避免的.自增主键管理.基于-会话/事务/语句-选择碎片.通过-主键/模块/碎片索引-碎片化数据 http://www.agildata.com/datab ...
-
Digital Current-Mode Control Challenges Analog Counterparts
http://electronicdesign.com/digital-ics/digital-current-mode-control-challenges-analog-counterparts ...
-
The promises and challenges of std::async task-based parallelism in C++11 C++11 std::async/future/promise
转载 http://eli.thegreenplace.net/2016/the-promises-and-challenges-of-stdasync-task-based-parallelism- ...
-
XSS Challenges(1-12关)
XSS Challenges解析: 1. 什么过滤也没有,而且是直接输出.<scrip>alert(document.domain);</script>即可通过 2. 直接输入 ...
随机推荐
-
poj1942 Paths on a Grid
处理阶乘有三种办法:(1)传统意义上的直接递归,n的规模最多到20+,太小了,在本题不适用,而且非常慢(2)稍快一点的算法,就是利用log()化乘为加,n的规模虽然扩展到1000+,但是由于要用三重循 ...
-
Android权限安全(5)组件的android:exported属性
Android四大组件都有 android:exported 属性 android:exported="true" 时 表示该组件是公开的,其它组件可以访问这个组件 android ...
-
S2 结业考试前改错汇总
1. PS:正确答案是A:枚举是值类型,一个类的对象是引用类型. 2. 每张表最多包含1个聚集索引.并且聚集索引会决定记录储存的物理位置.聚集索引不一定要建立在主键字段上.一张表可以没有任何索引. 3 ...
-
Android单元測试之JUnit
随着近期几年測试方面的工作慢慢火热起来.常常看见有招聘測试project师的招聘信息.在Java中有单元測试这么一个JUnit 方式,Android眼下主要编写的语言是Java,所以在Android开 ...
-
IE 中开发,兼容与性能测试工具汇总
前言 对于开发者来说, IE的兼容性是最让人头疼的. 因为是微软的产品, 且绑定在操作系统上, 所以IE的占用率还是相当大, 对于开发者来说, 这部分的兼容的考虑就不可避免了. 对于IE 的各版本来说 ...
-
mina教程
关于mina介绍这里不做阐述..... 我们先做一个关于mina的helloworld 首先先下载mina包:http://mina.apache.org/ (如果你已经下载,此步骤忽略) 下载下来以 ...
-
201521123122 《java程序设计》第十三周学习总结
## 201521123122 <java程序设计>第十三周实验总结 ## 1. 本周学习总结 以你喜欢的方式(思维导图.OneNote或其他)归纳总结多网络相关内容. 2. 书面作业 1 ...
-
分布式监控系统Zabbix-3.0.3--短信报警设置
前面已分别介绍了zabbix的邮件.微信报警设置,这些都是手机在有网络时才能收到报警信息,那如果手机没有网的情况下怎么办,这就需要考虑使用短信接口报警了.当服务出现故障达到预警级别是通过发送短信的形式 ...
-
JavaSE学习总结(四)——Java面向对象十分钟入门
面向对象编程(Object Oriented Programming,OOP)是一种计算机模拟人类的自然思维方式的编程架构技术,解决了传统结构化开发方法中客观世界描述工具与软件结构的不一致性问题.Ja ...
-
linux中nmcli命令详解
https://www.iyunv.com/thread-269695-1-1.html http://www.178linux.com/44668