Synapse Sourcing Agent

An autonomous sourcing pipeline that discovers LinkedIn profiles for a job description, ranks them with heuristics plus LLM reasoning, deep-scrapes the most promising profiles, and produces personalised outreach – all through a single FastAPI endpoint or CLI.

---

🌟 Key capabilities

1. Multi-engine discovery – Google / DuckDuckGo / SerpAPI queries (site:linkedin.com/in).
2. Semantic filter – MiniLM embeddings keep only semantically relevant profiles.
3. Heuristic rubric – Education, trajectory, company pedigree, skills, location & tenure.
4. LLM refinement – Gemini re-scores the top N candidates and crafts outreach copy.
5. Deep enrichment (optional) – Headless Chrome + linkedin-scraper to pull full Experience / Education sections when a valid LinkedIn session cookie (LI_AT) is provided.

---

🚀 Quick start (local)

# Python ≥3.10
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Mandatory – Google Gemini
export GOOGLE_API_KEY="***"

# (Optional) richer search & enrichment
export SERPAPI_KEY="***"          # SerpAPI quota >0
export LI_AT="Aqu3..."            # Your LinkedIn session cookie
export MAX_DEEP_SCRAPES=10         # Limit Selenium calls per run

# 🔍  CLI
python main.py --top 10 job_description.txt

# 🌐  REST API
uvicorn synapse.api:app --host 0.0.0.0 --port 8000
curl -X POST http://localhost:8000/source -H 'Content-Type: application/json' -d '{"job_description": "Software Engineer, ML Research …"}'

---

🐳 Docker

The included Dockerfile builds a slim container with Chromium + ChromeDriver so deep scraping works out-of-the-box.

# Build
docker build -t synapse-agent .

# Run (without enrichment)
docker run -p 7860:7860 -e GOOGLE_API_KEY=$GOOGLE_API_KEY synapse-agent

# Run (with enrichment)
docker run -p 7860:7860 \
  -e GOOGLE_API_KEY=$GOOGLE_API_KEY \
  -e SERPAPI_KEY=$SERPAPI_KEY \
  -e LI_AT=$LI_AT \
  synapse-agent

The REST endpoint will be available at POST /source on port 7860.

---

🤗 Hugging Face Spaces

1. Create a new Docker Space.
2. Add the same GOOGLE_API_KEY (and optionally LI_AT, SERPAPI_KEY) under Settings → Secrets.
3. Push this repo – Spaces automatically builds the Dockerfile and exposes port 7860.
4. Call the public URL:

   curl https://<username>-synapse.hf.space/source \
        -H 'Content-Type: application/json' \
        -d '{"job_description": "…"}'

⚠️ If no LI_AT secret is provided, the agent will skip deep enrichment and still return high-level data (headline, snippet, location) using SerpAPI & public scraping.

---

🔑 Environment variables

Variable	Required	Purpose
GOOGLE_API_KEY	✅	Gemini model access.
SERPAPI_KEY	⬜	Extra search / profile fields.
LI_AT	⬜	LinkedIn session cookie for deep scraping.
CHROME_PROFILE_DIR / CHROME_PROFILE_NAME	⬜	Alternative to LI_AT – use your own Chrome profile.
MAX_DEEP_SCRAPES	⬜	Hard cap on Selenium calls (default 10).
VERBOSE_SEARCH	⬜	Set to 1 for debug prints.

---

🗂️ Project layout

synapse/
    __init__.py
    agent.py          ← pipeline orchestration
    search.py         ← discovery + semantic filter
    profile_enrich.py ← SerpAPI + jina.ai fallback
    deep_enrich.py    ← Selenium + linkedin-scraper
    rubric.py         ← heuristic scoring
    outreach.py       ← Gemini message generation
    api.py            ← FastAPI wrapper
main.py               ← CLI entry-point
Dockerfile            ← production image
requirements.txt

---

🧪 Test run

curl -X POST http://localhost:7860/source \
     -H 'Content-Type: application/json' \
     -d '{"job_description": "Senior ML Research Engineer – focus on LLMs, Bay Area preferred"}' | jq

Expected output (truncated):

{
  "job_id": "ab34cd1e",
  "candidates_found": 30,
  "top_candidates": [
    {
      "name": "Alice Zhang",
      "linkedin_url": "https://www.linkedin.com/in/alicez",
      "headline": "Staff ML Engineer | GPT-4, LLAMA-3, applied research",
      "semantic_score": 0.82,
      "score_breakdown": { … },
      "fit_score": 9.3,
      "outreach_message": "Hi Alice, …"
    }
  ]
}

Happy sourcing! 🚀