Building Rearc.io's AI Chat Agent

Hello everyone, my name is Sanjiv Raote and I'm currently pursuing a B.A. in Data Science and AI at the University of Miami, FL. As an intern at Rearc I worked on an exciting project that involved developing and deploying an AI ChatAgent for the company's website. The main goal of the project was to create an effective way to collect the information of potential candidates interested in working at Rearc.

The Vision: A Smart, Always Available Assistant

Imagine a website where visitors can get instant, relevant answers to their questions about the company or career opportunities. That was the vision I had while creating Rearc's AI Chat Agent and is also why I chose GPT-4o-mini to power the chatbot. I found GPT-4o-mini to be the perfect combination between cost effectiveness and response quality.

The Serverless Architecture: Powered with AWS

The chatbot is built using multiple AWS services with a Lambda function for the core chatbot logic.

• Frontend Widget: A Next.js application hosts the chat widget made using TypeScript which runs on the rearc.io website.
• AWS API Gateway: Serves as a secure public HTTP endpoint, acting as the "front door" for all chat requests. It handles the chatbot routing to Lambda and initial user authentication.
• AWS Lambda: The "brain" of the chatbot. This serverless function executes a Python script which is used to run the core logic of the chatbot.
• OpenAI API: Provides the Large Language Model (GPT-4o Mini) that drives the chatbot's intelligence, understanding, and response generation.
• AWS Secrets Manager: Securely stores sensitive credentials like the OpenAI API key, which Lambda retrieves at runtime.
• Amazon DynamoDB: A fast NoSQL database used for:
  • Conversation History: Stores all of the chatbot message history to maintain context with different user interactions (WebsiteChatAgentHistory table).

The Journey: Overcoming AWS Complexities

Building this system from the ground up, especially as an intern, involved navigating and overcoming a series of significant technical hurdles in AWS:

1. The Persistent Python Dependency Problem

One of the most stubborn issues was resolving Runtime.ImportModuleError: No module named 'pydantic_core._pydantic_core'. This error, tied to core Python libraries (pydantic, openai, langchain), stemmed from a incompatibility between dependencies built on a Windows development environment and the Linux runtime used by AWS Lambda.

• Solution: Using a Dockerfile allowed me to build all Python dependencies within a controlled Linux Docker environment, which ensured that the manylinux compatible dependencies were generated instead of the windows versions.

2. VPC Network Timeout

Getting the Lambda function to securely communicate with external internet resources (like OpenAI) and private databases (like Aurora for RAG, in a later phase) while inside a VPC was a major test. Initially this resulted in a 504 Endpoint request timed out error when in the VPC which indicated that Lambda could not communicate with the internet.

• Solution: This was resolved by configuring the VPC networking correctly. I made sure the Lambda was placed in private subnets, then I set up a NAT Gateway in a public subnet for outbound internet access, and correctly configured route tables and security group rules to allow specific traffic flows (e.g., HTTPS to internet, PostgreSQL to Aurora).

3. Fixing IAM Permission errors

Ensuring precise access controls and service configurations was a continuous challenge:

• AccessDeniedExceptions: Frequent secretsmanager:GetSecretValue denials required debugging of IAM policies, ensuring exact ARN matches for all of my AWS resources and proper policy attachment to the Lambda's execution role.
• DynamoDB Validation: Resolving ValidationException errors for DynamoDB involved attention to detail when defining the table names (e.g., exact case-sensitive primary key names like SessionId and HistoryId) and ensuring the Lambda code's boto3 calls perfectly matched those names.
• API Gateway Setup: Correctly setting up the API Gateway resources, methods, and CORS headers was necessary for frontend-to-backend communication.

Future Enhancements

In its current state the AI ChatAgent is operational, It intelligently responds to user prompts, maintains conversation context, and captures lead information for potential candidates.

Potential features I would add:

• Sending DynamoDB logs to a recruiters email directly so the candidate information wouldn't need to be viewed manually.
• Implementing Retrieval Augmented Generation (RAG) using Aurora PostgreSQL with pgvector to allow the chatbot to parse through and answer questions based on Rearc's blog content.

This project has been a great learning experience so I would like to give a huge thanks to:

• Milin Patel, Michael Kaufman and Cristina Orlando for the opportunity and all of the help along the way.