MLRun Customer Support Gen AI Copilot

A generative AI copilot is an interactive gen AI assistant that is designed to amplify human capabilities while working together interactively. The term “gen AI co-pilot” is inspired by the aviation concept of a copilot, who assists the main pilot to ensure smooth and successful flying. You can develop your own copilot with open-source MLRun, which will orchestrate the AI pipelines at scale with pre-built components.

In this blog post, we’ll dive into the concept of a gen AI copilot and show a demo of building one with MLRun.

A copilot in generative AI is an AI-powered assistant designed to work interactively and collaboratively with humans in real-time to enhance our capabilities. This could include conducting tasks like automating repetitive assignments, generating drafts, retrieving information, transcription of conversations, analyzing data, providing insights, writing and testing code, or generating content. With a copilot, we can work faster, more effectively and at a larger scale.

Some of the most popular copilots in use today are:

• Microsoft Copilot: Assists with document creation, data analysis and communication.
• GitHub Copilot: A coding assistant that helps developers write, debug and optimize code.
• Design Copilots: Tools like Canva’s AI features that assist in creating visually appealing designs based on user input.
• Customer Support Copilots: AI systems that help agents by suggesting responses, retrieving data, or automating routine queries.

A gen AI copilot leverages LLMs to understand user input, process it, and generate relevant outputs for tasks such as answering questions, creating content, or writing code. It combines specialized tools or APIs to tailor responses. With RAG, it can also fetch and incorporate real-time data, ensuring accuracy and relevance.

The system adapts through user feedback, integrates with external tools for automation, and maintains privacy and compliance standards to deliver secure, efficient, and personalized assistance across various domains.

Workflows are the sequences of tasks or actions that the copilot automates or assists with, based on user input and specific goals. They typically involve multi-step operations, integrations with external tools, and contextual understanding to ensure tasks are completed effectively.

A customer support copilot, for example, might include the following workflows:

1. Client Profile Retrieval – Automatically fetch detailed client information, such as name, address, account details, family status, preferences and previous engagements with the organization. This involves retrieving data from CRM systems, previous interactions (e.g., emails, chats, or calls), and other internal databases. The goal is to provide the support representative with a holistic view of the client to personalize the conversation.
2. Transcripting the conversation – Creating a transcript of the conversation so it can be used for further analysis and any required follow ups.
3. Retrieving information from online and internal sources – Identifying requirements in the call, like documents or benchmarks, and bringing them to the human representative to use on the call and enhance the customer experience.
4. Follow-up Email Management – Automating personalized email communications with action items based on the conversation. The copilot will also ensure these emails are clear, concise and aligned with the tone and professionalism of the organization.

5. Data Compliance and Logging – Ensuring all client interactions adhere to regulatory standards. For example, automatically logging the client interaction into the organization’s system while ensuring compliance with data protection and regulatory standards (e.g., GDPR, HIPAA), flagging any sensitive or non-compliant elements for review and maintaining a secure audit trail for accountability.

MLRun is an open-source AI orchestration framework that simplifies and accelerates the development and deployment of AI models. Building a copilot with MLRun allows for:

1. End-to-End AI Workflow Management – MLRun provides an integrated environment to manage the entire machine learning lifecycle: data preparation, model training and validation, deployment and monitoring.
2. Scalability – MLRun leverages K8s for scalable and distributed processing, enables scalable, event-driven workflows without infrastructure overhead and works with public cloud vendors for elasticity.
3. Collaboration and Reproducibility – MLRun facilitates collaboration among data scientists, ML engineers and developers by organizing code configurations and experiments in shared environments, versioning and automations. 
4. Customizability – Every copilot has unique requirements. MLRun enables the creation of tailored pipelines and algorithms specific to the co-pilot’s domain (e.g., customer support, code generation).
5. Pre-Built Components – MLRun provides ready-to-use functions and templates for common machine learning tasks preprocessing, model training, evaluation, real-time or batch inference pipelines, monitoring and logging, and more.
6. Real-Time Capabilities – MLRun integrates with real-time data streams and deploys optimized serving functions for fast and reliable inference.
7. Monitoring and Observability – MLRun offers comprehensive monitoring for co-pilots in production, hallucination, bias, toxicity, performance and more. It also provides tools to retrain and redeploy models as needed.

Customer service copilots can serve multiple use cases, from a 24/7 support call center to escalation management to global multilingual support. In the example below, you can see a demo of an MLRun copilot. It shows what such a copilot could look like in a private banking client relationship management scenario.

Meet Miss Chen, who recently invested in green energy bonds and is looking for advice on reinvesting additional funds. Together with the copilot, the banker identifies and recommends a relevant investment opportunity based on the client’s history. In addition, the co-pilot helps the agent anticipate future opportunities, like biotech investments, based on client interests, which expands the bank’s role in the client’s portfolio.

 The banker also proactively shares research materials from reputable sources, retrieved by the copilot, to support informed decision-making. This fosters a sense of trust and expertise while generating more business for the bank.

The copilot emphasizes personalized service, strategic investment advice and proactive support for the client’s needs. It helps the human agent provide personal touches, such as acknowledging the client’s daughter’s achievements and offering tailored solutions, to build trust and loyalty. This long-term retention through proactive service ensures steady revenue from high-net-worth clients. 

In the end, the co-pilot can create a hyper-personalized follow-up email based on the conversation for accountability and to close the deal.

You can watch the demo of this copilot here.