fix(parse-url): block SSRF via private/internal URLs (#845)

/api/parse-url accepted any URL the user submitted, fetched it via
@extractus/article-extractor, and returned the body as Markdown. With
ALLOW_PRIVATE_URLS unset (the default after #600) the SSRF guard
short-circuited entirely, so an unauthenticated POST could probe
container ports, read AWS IMDS / GCP metadata, and reach same-VPC
internal services.

- parse-url now always rejects private URLs regardless of
  ALLOW_PRIVATE_URLS. The flag's only legitimate use case is local
  LLM provider baseUrl overrides (validate-model, chat); article
  extraction has no business fetching internal hosts. Local LLM
  setups (Ollama, LM Studio, etc.) are unaffected.
- Strip a trailing dot from the hostname before equality checks so
  the FQDN form "localhost." (which still resolves to 127.0.0.1) is
  caught by the existing string match.

Known follow-ups (not addressed here):
- DNS rebinding: hostnames are matched as strings; a public domain
  resolving to 127.0.0.1 (e.g. localtest.me) is not caught.
- HTTP redirects: @extractus/article-extractor uses cross-fetch with
  default redirect: "follow" and exposes no hook, so a public URL
  302-ing to an internal host still leaks.
This commit is contained in:
Dayuan Jiang
2026-05-21 23:54:23 +09:00
committed by GitHub
parent 1115b2d2cd
commit 7b6eb39fa5
2 changed files with 8 additions and 4 deletions

View File

@@ -9,7 +9,9 @@
export function isPrivateUrl(urlString: string): boolean {
try {
const url = new URL(urlString)
const hostname = url.hostname.toLowerCase()
// Strip a trailing dot so FQDN forms like "localhost." (which still
// resolve to 127.0.0.1) cannot bypass the equality checks below.
const hostname = url.hostname.toLowerCase().replace(/\.$/, "")
// Block localhost
if (