feat: add Firecrawl backend for WebSearch and WebFetch tools
WebSearch is currently disabled for all non-Anthropic providers (OpenAI shim, DeepSeek, Ollama, etc.) because those providers have no native search backend. This adds Firecrawl as a fallback that activates when FIRECRAWL_API_KEY is set, unlocking web search for every model openclaude supports. WebFetch uses basic HTTP + Turndown for HTML-to-markdown conversion, which fails silently on JS-rendered SPAs and bot-protected pages. Firecrawl scrape replaces the fetch layer when FIRECRAWL_API_KEY is set, returning clean markdown that handles dynamic content correctly. Changes: - WebSearchTool: add runFirecrawlSearch() using @mendable/firecrawl-js, respects allowed_domains (post-filter) and blocked_domains (-site: operators), includes result snippets alongside links. shouldUseFirecrawl() ensures firstParty/Vertex/Foundry/Codex providers keep their native backends. - WebFetchTool: add scrapeWithFirecrawl(), drops into the existing applyPromptToMarkdown() pipeline so prompt processing is unchanged. - Remove "Web search is only available in the US" restriction from prompt when Firecrawl is active (it works globally).
This commit is contained in:
@@ -112,7 +112,8 @@
|
||||
"ws": "8.20.0",
|
||||
"xss": "1.0.15",
|
||||
"yaml": "2.8.3",
|
||||
"zod": "3.25.76"
|
||||
"zod": "3.25.76",
|
||||
"@mendable/firecrawl-js": "^4.18.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/bun": "1.3.11",
|
||||
|
||||
Reference in New Issue
Block a user