fix: SSRF in /api/parse-url via DNS bypass and redirects (#878)
* fix: resolve DNS before SSRF check and block redirects in parse-url
isPrivateUrl() did string-only hostname matching and never resolved DNS,
so a public-looking name that maps to an internal IP (e.g.
127-0-0-1.sslip.io -> 127.0.0.1) passed the check while fetch/extract
later resolved it and reached internal services (GHSA-wqcv-5qvx-vx75).
- isPrivateUrl is now async: it keeps the fast string/literal-IP path,
then resolves the hostname via DNS and rejects if any address is private.
- parse-url now fetches the page itself with redirect: "error" and parses
via extractFromHtml(), since article-extractor follows redirects
internally and drops a redirect option, which allowed a public URL to
302 to an internal host.
- Update validate-model call site to await; add regression tests.
* fix: preserve charset detection and block CGNAT range in parse-url SSRF fix
Follow-up to the multi-reviewer review of the SSRF fix:
- Restore charset handling lost when switching from extract() to
response.text(): non-UTF-8 pages (Shift_JIS/GBK/EUC/Big5, common on CJK
sites this project targets) decoded as mojibake. Now read the body as
bytes, detect charset from Content-Type / <meta charset>, and decode
with TextDecoder before extractFromHtml.
- Wrap extractFromHtml in try/catch: it throws (not returns null) on
empty/non-HTML bodies, which previously surfaced as a 500 instead of the
intended 400.
- Add 100.64.0.0/10 (RFC 6598 CGNAT) to isPrivateIp; it is routable inside
some cloud internal networks and was a residual SSRF target.
- Add tests for CGNAT, its boundaries, 0.0.0.0, and DNS-resolved IPv6.
2026-06-28 12:41:29 +09:00
|
|
|
import { beforeEach, describe, expect, it, vi } from "vitest"
|
2026-06-05 23:30:20 +08:00
|
|
|
import { isPrivateUrl } from "@/lib/ssrf-protection"
|
|
|
|
|
|
fix: SSRF in /api/parse-url via DNS bypass and redirects (#878)
* fix: resolve DNS before SSRF check and block redirects in parse-url
isPrivateUrl() did string-only hostname matching and never resolved DNS,
so a public-looking name that maps to an internal IP (e.g.
127-0-0-1.sslip.io -> 127.0.0.1) passed the check while fetch/extract
later resolved it and reached internal services (GHSA-wqcv-5qvx-vx75).
- isPrivateUrl is now async: it keeps the fast string/literal-IP path,
then resolves the hostname via DNS and rejects if any address is private.
- parse-url now fetches the page itself with redirect: "error" and parses
via extractFromHtml(), since article-extractor follows redirects
internally and drops a redirect option, which allowed a public URL to
302 to an internal host.
- Update validate-model call site to await; add regression tests.
* fix: preserve charset detection and block CGNAT range in parse-url SSRF fix
Follow-up to the multi-reviewer review of the SSRF fix:
- Restore charset handling lost when switching from extract() to
response.text(): non-UTF-8 pages (Shift_JIS/GBK/EUC/Big5, common on CJK
sites this project targets) decoded as mojibake. Now read the body as
bytes, detect charset from Content-Type / <meta charset>, and decode
with TextDecoder before extractFromHtml.
- Wrap extractFromHtml in try/catch: it throws (not returns null) on
empty/non-HTML bodies, which previously surfaced as a 500 instead of the
intended 400.
- Add 100.64.0.0/10 (RFC 6598 CGNAT) to isPrivateIp; it is routable inside
some cloud internal networks and was a residual SSRF target.
- Add tests for CGNAT, its boundaries, 0.0.0.0, and DNS-resolved IPv6.
2026-06-28 12:41:29 +09:00
|
|
|
// Mock DNS so tests are deterministic and never hit the network.
|
|
|
|
|
const lookupMock = vi.hoisted(() => vi.fn())
|
|
|
|
|
vi.mock("node:dns/promises", () => ({
|
|
|
|
|
default: { lookup: lookupMock },
|
|
|
|
|
lookup: lookupMock,
|
|
|
|
|
}))
|
|
|
|
|
|
2026-06-05 23:30:20 +08:00
|
|
|
describe("isPrivateUrl", () => {
|
fix: SSRF in /api/parse-url via DNS bypass and redirects (#878)
* fix: resolve DNS before SSRF check and block redirects in parse-url
isPrivateUrl() did string-only hostname matching and never resolved DNS,
so a public-looking name that maps to an internal IP (e.g.
127-0-0-1.sslip.io -> 127.0.0.1) passed the check while fetch/extract
later resolved it and reached internal services (GHSA-wqcv-5qvx-vx75).
- isPrivateUrl is now async: it keeps the fast string/literal-IP path,
then resolves the hostname via DNS and rejects if any address is private.
- parse-url now fetches the page itself with redirect: "error" and parses
via extractFromHtml(), since article-extractor follows redirects
internally and drops a redirect option, which allowed a public URL to
302 to an internal host.
- Update validate-model call site to await; add regression tests.
* fix: preserve charset detection and block CGNAT range in parse-url SSRF fix
Follow-up to the multi-reviewer review of the SSRF fix:
- Restore charset handling lost when switching from extract() to
response.text(): non-UTF-8 pages (Shift_JIS/GBK/EUC/Big5, common on CJK
sites this project targets) decoded as mojibake. Now read the body as
bytes, detect charset from Content-Type / <meta charset>, and decode
with TextDecoder before extractFromHtml.
- Wrap extractFromHtml in try/catch: it throws (not returns null) on
empty/non-HTML bodies, which previously surfaced as a 500 instead of the
intended 400.
- Add 100.64.0.0/10 (RFC 6598 CGNAT) to isPrivateIp; it is routable inside
some cloud internal networks and was a residual SSRF target.
- Add tests for CGNAT, its boundaries, 0.0.0.0, and DNS-resolved IPv6.
2026-06-28 12:41:29 +09:00
|
|
|
beforeEach(() => {
|
|
|
|
|
lookupMock.mockReset()
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks private IPv6 URLs (string-only fast path, no DNS)", async () => {
|
|
|
|
|
expect(await isPrivateUrl("http://[::1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[0:0:0:0:0:0:0:1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[::]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[::ffff:127.0.0.1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[fc00::1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[fd12:3456:789a::1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[fe80::1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[fe9f::1]/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://[febf::1]/")).toBe(true)
|
|
|
|
|
expect(lookupMock).not.toHaveBeenCalled()
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks literal private IPv4 without DNS", async () => {
|
|
|
|
|
expect(await isPrivateUrl("http://127.0.0.1/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://10.0.0.5/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://192.168.1.1/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://169.254.169.254/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://0.0.0.0/")).toBe(true)
|
|
|
|
|
// 100.64.0.0/10 CGNAT (RFC 6598), routable in some cloud internal nets
|
|
|
|
|
expect(await isPrivateUrl("http://100.64.0.1/")).toBe(true)
|
|
|
|
|
expect(await isPrivateUrl("http://100.127.255.255/")).toBe(true)
|
|
|
|
|
expect(lookupMock).not.toHaveBeenCalled()
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("treats CGNAT boundaries correctly", async () => {
|
|
|
|
|
// 100.63.x and 100.128.x are outside 100.64.0.0/10 → public
|
|
|
|
|
lookupMock.mockResolvedValue([{ address: "100.63.255.255", family: 4 }])
|
|
|
|
|
expect(await isPrivateUrl("http://just-below.example/")).toBe(false)
|
|
|
|
|
lookupMock.mockResolvedValue([{ address: "100.128.0.1", family: 4 }])
|
|
|
|
|
expect(await isPrivateUrl("http://just-above.example/")).toBe(false)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks a hostname that resolves to a private IPv6 address", async () => {
|
|
|
|
|
lookupMock.mockResolvedValue([{ address: "fd00::1", family: 6 }])
|
|
|
|
|
expect(await isPrivateUrl("http://v6.example.com/")).toBe(true)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("allows public URLs that resolve to public IPs", async () => {
|
|
|
|
|
lookupMock.mockResolvedValue([{ address: "93.184.216.34", family: 4 }])
|
|
|
|
|
expect(await isPrivateUrl("https://example.com/article")).toBe(false)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks public-looking hostnames that resolve to a private IP (DNS-rebinding-style bypass)", async () => {
|
|
|
|
|
// e.g. 127-0-0-1.sslip.io resolves to 127.0.0.1
|
|
|
|
|
lookupMock.mockResolvedValue([{ address: "127.0.0.1", family: 4 }])
|
|
|
|
|
expect(await isPrivateUrl("http://127-0-0-1.sslip.io/")).toBe(true)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks when any resolved address is private", async () => {
|
|
|
|
|
lookupMock.mockResolvedValue([
|
|
|
|
|
{ address: "93.184.216.34", family: 4 },
|
|
|
|
|
{ address: "10.1.2.3", family: 4 },
|
|
|
|
|
])
|
|
|
|
|
expect(await isPrivateUrl("http://mixed.example.com/")).toBe(true)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it("blocks when DNS resolution fails", async () => {
|
|
|
|
|
lookupMock.mockRejectedValue(new Error("ENOTFOUND"))
|
|
|
|
|
expect(await isPrivateUrl("http://does-not-resolve.example/")).toBe(
|
|
|
|
|
true,
|
|
|
|
|
)
|
2026-06-05 23:30:20 +08:00
|
|
|
})
|
|
|
|
|
})
|