Search Cluster (Industrial Standard v3.1)

A multi-provider search aggregator designed for high-availability and security.

小葱技能有更好的技能skills插件。

Installation

The scrapling provider requires a dedicated virtual environment.

1. Create a venv: python3 -m venv venv/scrapling
2. Install scrapling: venv/scrapling/bin/pip install scrapling
3. Provide the path to the venv binary in SCRAPLING_PYTHON_PATH.

Security Posture

• Subprocess Isolation: Query inputs are passed as arguments to stealth_fetch.py.
• Strict TLS: Mandatory SSL verification on all providers.
• Sanitization: Integrated native internal scrubber (Path Neutral).

Requirements and Environment

Declare these variables in your environment or vault:

Providers

• google: Official Google Custom Search.
• wiki: Wikipedia OpenSearch API.
• reddit: Reddit JSON search API.
• gnews: Google News RSS aggregator.
• scrapling: Headless stealth scraping (via DuckDuckGo).

Included Scripts

• scripts/search-cluster.py: Main entry point.
• scripts/stealth_fetch.py: Scrapling fetcher (REQUIRED for scrapling provider).

Workflow

1. Execute: scripts/search-cluster.py all ""
2. Output is structured JSON with source, title, link, and sanitized snippet.

这是一个质量较高的搜索工具，优点是功能丰富、文档详细、安全防护做得不错，多个搜索源并发查询速度快。缺点是安装配置稍复杂，特别是需要单独配置虚拟环境才能使用爬虫功能，另外错误提示不够友好，出问题时用户可能摸不着头脑。总体而言适合有技术基础的用户，普通用户需要一定学习成本。