Freelance / Remote
Hourly (via Hubstaff)
We are seeking a skilled AI Developer with experience using Anthropic’s Claude models and AWS infrastructure to build a web scraper AI agent. The goal of this project is to create an intelligent agent that can autonomously search the web for home builders and their subdivisions, extract relevant data, and insert it into our existing MongoDB database.
- Build and deploy an AI agent using Claude on AWS (SageMaker, Bedrock, or other relevant tools).
- Design prompts and logic to guide the agent to locate home builders and their active/past subdivisions from public websites.
- Scrape relevant structured and unstructured data (e.g., builder name, subdivision names, states, URLs, etc.).
- Clean, structure, and inject the scraped data directly into our MongoDB database.
- Set up and test recurring scraping logic with minimal human supervision.
- Build in error handling, logging, and rate-limiting protection.
- Work closely with our team to adapt the agent based on results and needed improvements.
- Proven experience with AI models, especially Claude by Anthropic, and prompt engineering.
- Strong AWS knowledge, ideally with experience in SageMaker, Lambda, EventBridge, or Bedrock.
- Solid experience in Python, web scraping (e.g., BeautifulSoup, Scrapy, Playwright), and MongoDB integration.
- Understanding of how to make scrapers mimic human behavior to avoid IP bans.
- Ability to work autonomously and deliver results fast.
- Clear written communication and daily check-ins via Hubstaff.
- Experience with data deduplication and validation.
- Familiarity with housing and construction industry websites (e.g., DR Horton, Lennar, etc.).
- Ability to recommend improvements in how the data is structured or stored.
Please include examples of previous scraping agents or AI automation tools you've built. Preference will be given to those with Claude + AWS experience. Shortlisted candidates may be asked to complete a brief test task. Please note pay is up to $15/hour, based on experience.