Authors: Rajiv Marwah & Ritesh Kumar:
In a rather animated conversation with a peer of mine, I was told that “Chatbots or Virtual Assistants have no role to play in improving Clinical Trial patient recruitment or patient retention”. For instance, we have all experienced virtual assistants as consumers with airlines, telecom, and banks. Each of us could admit that we have a varied response to chatbots, as they can provide both positive and negative experiences. After over six months of experimentation and multiple implementations, I am convinced that we should not ignore the power of smart Chatbots. It is one more step in our journey to digitalize clinical trials, an undeniable trend and one that is gaining momentum. There are clear benefits of implementing chatbots for both patient recruitment and patient retention. This tool allows for greater patient centricity which is what most stakeholders desire. One positive of Covid has been the increasing digitization of clinical trials and as this trend continues additional productivity tools will be required.
Before proceeding, it might be of value to list the elements of Conversational AI to allow for increased clarity. The terms Conversational AI, Chatbot, and Virtual/digital assistant are all used interchangeably and refer to the same functionality.
The objective is to simulate human-like conversations with a friendly user interface that can be used across the IoT. When implemented properly, it can result in an improved patient and provider experience, while being available 24X7. To reduce conversational friction, the software must be contextual, include current information, and provide personalized responses. This will be possible if the software can do both intent & entity recognition.
Intent recognition is the ability to interpret what the user is asking so that an appropriate response can be provided. Entity recognition, in our use case, is the ability to recognize and effectively interpret medical/clinical entities. These include eligibility criteria, procedures, conditions, drugs, and other clinical trial variables. The accuracy and number of medical entities recognized will directly impact the quality of responses. The AI and Natural Language Processing (NLP) engine under the hood of a smart virtual assistant enables successful interactions. The associated Machine Learning (ML) engine will learn continuously to improve future interactions. To deal with exceptions or when the software is unable to understand patient intent, it’s important to be able to seamlessly transfer the conversation to a human agent. At this point, a human agent could be introduced through video or email. The human agent will be shared an email with a copy of the chat conversation details for a future appointment.
Given the above, let’s review how these concepts can be implemented in clinical trials as they relate to both patient recruitment and retention. The conversion and retention rates with the deployment of Conversational AI technology should see an improvement of at least 25%
- Recruitment: The Chatbot can be visualized in distinct sections:
- Enrollment: The NCTID for the trial is ingested, parsed for inclusion and exclusion criteria and standard information (e.g. demographics and reimbursement)
- Potential patients can interact with the chatbot 24×7 to determine eligibility, and obtain general and clinical answers about the trial, based on available public information through clinicaltrials.gov or other generally available information. This information is shared in a user-friendly manner and is easier for the patient to understand. This allows for several patient-centric interactions and digital nudges to reduce the enrolment time.
- The digital interface will encourage the participation of a younger patient population.
- Retention: of enrolled patients
- Chatbot can provide sentiment analysis and data-driven digital interventions to reduce drop-off rates.
- The chatbot can recognize potential adverse reactions and raise an alert and facilitate the scheduling of meetings with clinical research staff. In these cases, the human remains in the loop and the goal is to act as an alert mechanism and at no stage is clinical advice provided by the chatbot
- Data collection for improvement based on patient feedback across a growing number of users.
- Regulatory framework:
- Chatbot can be deployed in a secure manner such that there are no privacy or other compliance-related violations
- The chatbot conversations are saved and available for audit, reporting, and monitoring. It is easier to monitor textual information in real-time as compared to voice calls and static web forms.
- Deployment:
- The chatbot can be hooked to any page on a website that the customer chooses
- The chatbot can be skinned to align with the customer’s brand
- The chatbot can be deployed for a single or multiple trials
- There are multiple sources of rich data sets which enable higher patient satisfaction it is now possible to access these using the web, API(s), and other technologies
- Chatbots can be scaled up or down based on trial requirements.
- Technology:
Clinical studies are often listed on clinicaltrials.gov. They have the information presented in a specific format, starting with Study Description to ending with More Information. We can easily observe three unique ways in which the text is layered:
- General FAQ-type questions with typical answers (study start etc.) from the non-clinical text
- Domain-specific text as in the eligibility criteria, rich with clinical entities
- Text in the tables like the Condition-Intervention-Phase table
The differing layers provide a unique challenge to the CeeBee Bot to correctly understand the context in which the question is asked, identify the relevant candidate passage or table where the answer might be, and then generate the correct answer.
The first layer task
Our CeeBee bot uses a multi-task transformer architecture that can handle both intent classification and entity recognition together. It thus starts by capturing the intent of a user’s query and the entity within the query. While intents allow it to understand the motivation behind a user’s input, entities allow it to gather specific pieces of information that the user may have mentioned. Next, we define the user’s utterances for each intent which are the automated responses from the bot. But wait! The questions from the user could come in any variation. The Question Generator helps CeeBee Bot understand the multiple different ways in which a question may be asked by the user.
The second layer task
The eligibility criteria section of the clinical study has text rich in clinical entities. Extremely domain-centric, for the bot to understand the text, clinical texts must be parsed correctly, and clinical entities are extracted and overlaid on the text embeddings. For example, consider an inclusion criterion under the eligibility section:
“At least one lesion ≥ 1.5 cm that is seen on standard imaging (e.g. CT, MRI, mammogram, ultrasound, FDG-PET/CT).”
For the CeeBee bot to understand it, we need to mine these entity pairs:
{“condition”: ”lesion”} with attributes {“value1”: “at least one”}, {“value2” : “>=1.5 cm”}
{“procedure”: “CT”} etc
Candidate lines are fed along with the user’s query into the deep learning models for predicting the most probable answer span. We use BiLSTM, BERT, and RoBerta to generate the answer span. Our model uses the candidate lines to construct a feature vector which is then encoded. Similarly, question encoding is also created. Finally, the model uses candidate line encoding and question encoding to predict the correct offset for the answer span.
The third layer task
CeeBee bot has an in-built retriever that identifies if the section in the clinical study is text or table type. Tables need to be dealt little differently than the text. They can be thought of as multiple entity-entity relations stacked together as columns of the table. The answer to a user’s query could be in table or text or both. The CeeBee bot takes the answers from different layers and joins them together to create a coherent response.
CeeBee’s performance
Enriched with more than 30 clinical entities and their relationships, the CeeBee bot is an innovative solution in its domain with a near-real response time of less than a fraction of a second. Interspersed with thoughtful utterances, it engages very well with the patients and answers their queries effectively. As stated above, it can fast-track the recruitment by 2-3X.
How do you engage with patients during pre-recruitment and later? Please share your thoughts in the comments and do stay tuned for the second part of this article which would discuss the tasks our CeeBee bot accomplished while the study is ON!
