Building a custom CRM Agent
A technical transformation of an existing business model into an Agentic system
I've always been fascinated by the gap between how AI systems work in demos versus real life. You see these amazing capabilities showcased in controlled environments, but then you try to implement something similar and suddenly it's all edge cases and unexpected behaviors. Two months ago, I decided to bridge that gap for myself by building a genuine, useful AI agent that could handle real-world complexity. After some research, LangGraph emerged as the toolkit that might make this possible. What started as a weekend experiment quickly evolved into a full-fledged obsession with building an agent that could actually think (or at least convincingly fake it) across multiple conversational turns. Here's what I learned along the way.
I needed an existing business model to imitate and because of some circumstances a good old CRM was the best choice I could make at that time and so it started.
After weeks of watching my chatbot lose context, forget crucial information, and restart conversations mid-flow, I knew there had to be a better way. Enter LangGraph—a framework that transformed how I build AI agents by giving them something crucial: memory and state management.
What is LangGraph? LangGraph is a tool for building stateful, controllable AI agents, often used in industries like customer support and software development. It's part of the LangChain ecosystem, which is popular for AI development.
For instance, Klarna used LangGraph to build an AI Assistant for customer support, serving 85 million users and reducing resolution times by 80%. Other companies, like LinkedIn and Uber, have also adopted it for tasks like recruitment and code automation, though some details are less explicit. These cases suggest LangGraph helps make AI systems more efficient and scalable.
Klarna's Implementation: Klarna's AI Assistant, powered by LangGraph, handles 2.5 million daily transactions and has conducted 2.5 million conversations to date, equivalent to 700 full-time staff. The use of LangGraph's controllable agent architecture decreased latency, improved reliability, and cut operational costs, automating approximately 70% of repetitive support tasks and reducing customer resolution times by 80%. This is a clear example of LangGraph's impact on scalability and efficiency in customer support.
LinkedIn's Adoption: While the LinkedIn blog post on their GenAI tech stack explicitly mentions LangChain, not LangGraph, the context suggests overlap, as LangGraph is part of the ecosystem. LinkedIn's AI recruiter, which includes conversational search and candidate matching, likely benefits from LangGraph's hierarchical agent system.
Other Companies: Uber's use of LangGraph for code migrations and unit test automation, Replit's coding agents for reliability, AppFolio's Realm-X with doubled response accuracy, and Elastic's threat detection agents all underscore LangGraph's versatility across industries. These projects demonstrate its ability to handle complex, multi-agent workflows, ensuring reliability and scalability.
CRM Agent
When I set out to build my CRM agent, my primary goal was to create a system that could streamline customer relationship management tasks through intelligent automation and natural language interaction. Why CRM? Because it’s one of those use cases where AI has great potential to eat up old and outdated clunky UI dashboard kind of projects. After building so many frontends in my career and realizing how much time and resources these projects consume I simply vibe with this idea that AI is killing interfaces. I wanted the agent to be both flexible and extensible, so I made several key implementation choices to support these objectives.
1. Modular Architecture
I opted for a modular architecture, breaking down the agent into distinct components: data ingestion, intent recognition, action execution, and response generation. This separation of concerns makes it easier to maintain and extend the system. For example, if I want to add support for a new CRM platform or integrate a new data source, I can do so without overhauling the entire codebase.
2. Natural Language Processing (NLP) Backbone
To enable users to interact with the agent using everyday language, I integrated a robust NLP pipeline. I chose to use OpenAI’s GPT-3.5-turbo model for intent recognition and entity extraction, as it provides state-of-the-art performance and can handle a wide variety of request phrasings.
3. API-Driven Data Access
Rather than relying on direct database queries, I built the agent to interact with CRM data via APIs. This approach improves security, allows for better error handling, and makes it easier to support multiple CRM systems in the future.
4. Contextual Memory
To make conversations more natural, I implemented a contextual memory system. The agent keeps track of recent interactions, so it can handle follow-up questions and reference previous requests without requiring the user to repeat themselves.
5. User-Centric Design
Throughout the development process, I prioritized user experience. I designed the agent to clarify ambiguous requests, provide helpful suggestions, and gracefully handle errors. The goal is for users to feel like they’re interacting with a knowledgeable assistant, not just a chatbot.
6. Intent detector
The intent detector is a crucial component of my CRM agent, designed to understand and classify user requests into specific actions. It leverages both advanced natural language processing and a fallback keyword-based approach to ensure robust intent detection.
I integrated OpenAI’s GPT-3.5-turbo model to classify user requests into predefined intents. This model is trained to recognize a variety of phrasings and nuances in user queries, making it highly effective for intent detection.
Predefined Intents
The intent detector is configured to recognize several key intents relevant to CRM operations:
CONTACT_IMPORT: For requests about importing contacts.
CONTACT_SEARCH: For requests about finding contacts.
CONTACT_RESEARCH: For requests about researching contact information.
GENERATE_FOLLOW_UP: For requests about generating follow-up content.
UNKNOWN: For any request that doesn’t fit into the above categories.
Confidence Scoring
To ensure accuracy, the intent detector calculates a confidence score for each detected intent. This score is based on the number of keyword matches found in the user’s request. If the confidence is low, the system defaults to the "UNKNOWN" intent, ensuring that only high-confidence intents are acted upon.
Fallback Mechanism
In case the OpenAI API fails or returns an unexpected result, the intent detector falls back to a keyword-based detection method. This ensures that the system remains functional even under adverse conditions.
Integration with the Orchestrator
The intent detector works seamlessly with the orchestrator, which uses the detected intent to determine the appropriate tool to execute. This integration ensures that each user request is processed efficiently and correctly.
My intent detector prompt looks like this:
def __init__(self):
"""Initialize the intent detector."""
logger.info("Initializing IntentDetector")
self.client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
self.system_prompt = """You are an intent classifier for a CRM system.
Your task is to classify user requests into one of the following intents:
- CONTACT_IMPORT: For requests about importing contacts
- CONTACT_SEARCH: For requests about finding contacts
- CONTACT_RESEARCH: For requests about researching contact information
- GENERATE_FOLLOW_UP: For requests about generating follow-up content
- UNKNOWN: For any request that doesn't fit into the above categories
IMPORTANT: If the request doesn't clearly fit into one of the CRM-related intents (CONTACT_IMPORT, CONTACT_SEARCH, CONTACT_RESEARCH, GENERATE_FOLLOW_UP), you MUST respond with "UNKNOWN".
Respond with ONLY the intent name, nothing else."""Making an API request:
(venv) ➜ crm-ai git:(main) curl -X POST "http://127.0.0.1:8000/process" \
-H "Content-Type: application/json" \
-d '{"text": "Find my contact John Doe"}'Server logs:
Invoking: `contact_search` with `{'query': 'John Doe'}`
responded: To find your contact John Doe, I will use the ContactSearchTool. This tool allows me to search for contacts in the CRM database using a search query string. In this case, the query will be "John Doe". This tool is chosen over others because it directly supports searching for contacts by name, which is the information you've provided.
Let me perform the search for you now.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
{'status': 'success', 'results': [{'_id': '68071da4d414ab92f6761cec', 'first_name': 'John', 'last_name': 'Doe', 'email': '', 'company': 'Novo Nordisk', 'position': 'Director', 'connected_on': '2013-07-10T00:00:00'}]}I found your contact John Doe in the CRM database. Here are the details:
- **First Name:** John
- **Last Name:** Doe
- **Company:** Novo Nordisk
- **Position:** Director
- **Connected On:** 2013-07-10
Please let me know if you need any more information or further assistance with this contact.
> Finished chain.
INFO: 127.0.0.1:49453 - "POST /process HTTP/1.1" 200 OK
I built an easy to extend intent orchestration where I list my business tools. My agent can handle various database searches, perform research on its customers, importing tool and follow up message generation tool.
Contact Research Tool
The contact research tool is a powerful component of my CRM agent, designed to perform targeted external research on contacts. It leverages the Tavily API to gather comprehensive information about a contact, enhancing the CRM’s ability to provide valuable insights.
1. Input Schema
The tool accepts a structured input schema, requiring the first and last name of the contact, along with an optional research target. This allows for flexible and specific research requests, such as finding recent news or company changes related to the contact.
2. Integration with Tavily API
I integrated the Tavily API to perform advanced searches, ensuring that the research results are both relevant and comprehensive. The API is configured to return the most pertinent information, which is then saved in the CRM’s database for future reference.
Server logs:
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
{'status': 'success', 'message': 'Research completed successfully', 'research_id': '68111ee4620e9f0715b70e2c'} The research on John Doe's work experience has been completed successfully. The detailed information about his work experience has been gathered and is now available. If you need more specific details or further assistance, please let me know!Generating Follow-Up Messages with LLMs in My CRM Agent
One of the most exciting features I’m working on for my CRM agent is the ability to generate personalized follow-up messages using a large language model (LLM). The idea is to move beyond simple reminders and actually help users craft thoughtful, context-aware communications that can drive their relationships forward.
1. Combining User Input and Research Insights
The follow-up message generator is designed to take into account both the user’s custom instructions (such as the purpose or tone of the follow-up) and the latest research findings about the contact. For example, if the research tool has recently found news about a contact’s company expansion, that information can be woven into the follow-up message, making it more relevant and engaging.
2. Prompt Engineering for the LLM
To achieve this, I construct a detailed prompt for the LLM. The prompt includes:
The user’s stated goal or context for the follow-up (e.g., “Check in about the contract,” or “Congratulate on the recent promotion”).
Key facts or recent findings from the research tool (e.g., “John’s company, Acme Corp, just announced a new product line”).
Any relevant CRM data, such as previous interactions or notes.
By providing the LLM with this rich context, I ensure that the generated message is not only grammatically correct but also highly personalized and situationally aware.
3. Workflow Integration
When a user requests a follow-up message, the orchestrator gathers the necessary data: it pulls the latest research results for the contact, collects any relevant CRM notes, and combines these with the user’s instructions. This composite context is then sent to the LLM, which returns a draft message ready for review or direct sending.
4. Example Workflow
Suppose I want to follow up with Jane Smith after a recent meeting, and my research tool has found that her company just secured a major partnership. I might ask:
User Input:
“Draft a follow-up email to Jane Smith thanking her for the meeting and mentioning her company’s new partnership.”
LLM Prompt (constructed by the agent):
“Write a professional follow-up email to Jane Smith. Thank her for the recent meeting. Mention that I saw Acme Corp just secured a new partnership, and express congratulations. Keep the tone warm and concise.”
Generated Message:
“Hi Jane,
Thank you again for meeting with me earlier this week. I also wanted to congratulate you and the team at Acme Corp on your new partnership—what an exciting development! Looking forward to staying in touch.
Best,
[Your Name]”
5. Benefits
This approach saves users time, ensures their outreach is always relevant, and helps build stronger professional relationships. By leveraging both structured CRM data and dynamic research findings, the follow-up messages feel genuinely personalized—something that’s hard to achieve with templates alone.
The Future of Building with LangGraph
What started as a weekend experiment to bridge the gap between AI demos and real-world applications evolved into something far more significant. Building this LangGraph-powered CRM agent has fundamentally changed how I think about AI development and the future of business applications.
Key Takeaways
The journey of building this agent revealed several important insights that might help others venturing into similar territory:
State management is everything. Before LangGraph, I was stuck in an endless cycle of context loss and repetitive interactions. The ability to maintain state across conversation turns completely transformed what my agent could accomplish. Those "weeks of watching my chatbot lose context" are thankfully behind me.
Modularity pays dividends. Breaking the system into distinct components—intent detection, research capabilities, tool orchestration, and response generation—made the entire project more maintainable and extensible. Each piece could be refined independently, allowing for incremental improvements without disrupting the whole system.
Intent recognition is the cornerstone. The quality of the entire interaction hinges on correctly understanding what users want to accomplish. My dual approach of using both LLM classification and keyword fallbacks proved to be resilient even when dealing with ambiguous requests.
Research capabilities create exponential value. By integrating external research through the Tavily API, the agent transcended from being a mere interface to CRM data to becoming a genuine assistant that could provide contextual insights. This transformed follow-up messages from generic templates to personalized, relevant communications.
The real power is in orchestration. LangGraph's ability to seamlessly route between different tools based on intent created a fluid experience that felt like working with a knowledgeable colleague rather than separate tools stitched together.
Where This Technology Is Headed
As more businesses adopt frameworks like LangGraph, I believe we're witnessing the early days of a significant shift in how we interact with business software. The clunky dashboards and complex UIs that have dominated enterprise software may soon give way to conversational interfaces that handle the complexity behind the scenes.
Companies like Klarna have already demonstrated the transformative impact this approach can have—reducing resolution times by 80% and automating approximately 70% of repetitive support tasks. These aren't incremental improvements; they're revolutionary changes to operational efficiency.
What I'd Do Differently
If I were starting this project again, I would:
Invest more time upfront in defining clear boundaries between the agent's components.
Build more robust error handling and recovery mechanisms from day one.
Create better telemetry to track which intents and tools are most frequently used.
Implement more thorough validation of LLM outputs before they reach users.
Final Thoughts
The path from frustrating, context-less AI interactions to a fluid, stateful agent wasn't straightforward, but LangGraph provided the architecture needed to make it possible. As these tools mature and more developers adopt them, we'll likely see an explosion of intelligent agents that can handle increasingly complex workflows across various domains.
For anyone feeling limited by the capabilities of traditional LLM implementations, I encourage you to explore LangGraph. The learning curve is worth it, and the resulting systems are capable of levels of sophistication that simply aren't possible with stateless approaches.
What started as a personal project to solve my own frustrations has opened my eyes to a new paradigm in software development—one where AI doesn't just respond to queries but actively participates in completing complex workflows. And that, I believe, is just the beginning of where this technology can take us.
Github repo: https://github.com/Bacis/agentic-crm