# XML security and untrusted input

pyxsd is a schema processor, not a security boundary. Its defaults favor speed
and compatibility with the stdlib parser. Protections are opt-in, finite, and
documented here so a caller can choose them deliberately.

The short version:

- **Default parsing is the stdlib.** `Schema.compile(..., xml_policy=None)` and
  `Schema.parse(..., xml_policy=None)` use `xml.etree.ElementTree` with its
  current DTD and entity behavior. `defusedxml` is installed, but it is used only
  when a policy selects it.
- **No built-in profile fetches external DTDs or entities.** The stdlib backend
  does not retrieve external references, and `XmlParserPolicy.protected()` and
  `.strict()` reject them.
- **Protections are opt-in per document.** `XmlParserPolicy.protected()` and
  `.strict()` select a defusedxml-backed parser. `XmlLimits` adds byte, depth,
  and element budgets. Neither becomes a process default.
- **Remote XSD loading is opt-in.** `RemoteResourcePolicy` is the only built-in
  way to retrieve a schema over HTTPS. Local and offline operation is the
  default, and there is no persistent remote cache.
- **XPath is code, not a sandbox.** Queries and schema assertions run in-process.
  The MCP server bounds them; the core library does not impose a timeout.

## Parser profiles

`pyxsd.XmlParserPolicy` is an immutable value. Its flags require
`use_defusedxml=True`; asking for declaration denial with the stdlib parser raises
`ValueError` rather than silently ignoring the request.

| Profile | Backend | DOCTYPE | Entity declarations | External DTD/entity references |
| --- | --- | --- | --- | --- |
| `XmlParserPolicy.default()` (or `None`) | stdlib ElementTree | allowed | allowed, expanded | never retrieved |
| `XmlParserPolicy.protected()` | defusedxml | allowed, except `SYSTEM`/`PUBLIC` identifiers | forbidden | forbidden |
| `XmlParserPolicy.strict()` | defusedxml | forbidden | forbidden | forbidden |

`protected()` is the compatibility-minded profile: a harmless internal
`<!DOCTYPE note [...]>` with an `ATTLIST` still parses, but an entity declaration
or any external identifier is refused. `strict()` is the profile for documents
that need no DTD at all.

A denial raises `pyxsd.ResourcePolicyError`, a subclass of
`pyxsd.PyXSDError`. Defusedxml's own exceptions are converted to that error with
the original exception as `__cause__`; the shared declaration guards raise
`ResourcePolicyError` directly. Malformed XML still raises
`xml.etree.ElementTree.ParseError`, which `Schema.parse` maps to `PyXSDError`.

`XmlLimits` is independent of the parser profile. Every bound is inclusive and
counted on the input as supplied:

- `max_bytes` counts a file's or binary stream's bytes. A text stream is counted
  by its UTF-8 encoding, not its character count.
- `max_depth` counts element nesting; the document element is depth 1.
- `max_elements` counts start tags.

```python
import pyxsd

profile = pyxsd.XmlParserPolicy.strict()
limits = pyxsd.XmlLimits(max_bytes=4 * 1024 * 1024, max_depth=64, max_elements=200_000)

schema = pyxsd.Schema.compile("inventory.xsd", xml_policy=profile, xml_limits=limits)
document = schema.parse("inventory.xml", xml_policy=profile, xml_limits=limits)
```

The compile-time policy applies to the main schema and every included or
imported schema it loads. It is compile-scoped: a later `Schema.parse` does not
inherit it, so pass the policy again for instance documents. The module-level
`pyxsd.parse(xml, xsd=..., xml_policy=..., xml_limits=...)` convenience applies
the same values to both the instance and the schema compile.

Use `protected()` when legacy documents may carry harmless DTD syntax but must
not declare entities or external references:

```python
import pyxsd

schema = pyxsd.Schema.compile("legacy.xsd")
document = schema.parse("legacy.xml", xml_policy=pyxsd.XmlParserPolicy.protected())
```

These parser options cannot be combined with a custom `resource_policy`.
A resource policy's `parse()` owns XML parsing and is trusted application code;
see [resource policies](#resource-policies). A pre-parsed
`xml.etree.ElementTree.Element` is also a caller-trusted boundary: no parser
options are applied to it, so validate or harden it before handing it over.

## DTDs, entities, and Expat

The default backend processes the internal DTD subset as far as Expat exposes
it: internal entity declarations are expanded, and `ATTLIST` defaults can apply.
It installs no external-resource resolver, so an external subset, external
parameter entity, or other external entity is not fetched. The document is
nevertheless untrusted input: an internal entity can expand exponentially,
and entity replacement text can consume memory.
That is what `protected()` and `strict()` are for.

DTD syntax and entity declarations are different decisions:

- A plain DOCTYPE with no entity or external identifier is accepted by
  `protected()`.
- Any entity declaration is refused by `protected()` and `strict()`.
- Any DOCTYPE at all is refused by `strict()`.

For schemas that can validate `xs:ENTITY`/`xs:ENTITIES` values, pyxsd lazily
collects unparsed (`NDATA`) entity declarations from the instance prolog and
internal subset. It checks entity names, not the resources they identify;
collection adds no external fetcher and never replaces the parser's security
handlers. Schemas unrelated to these types skip the declaration-collection
parse entirely. The profiles above keep the same behavior: `protected()`
still refuses entity declarations, and `strict()` still refuses any DOCTYPE.

An unretrieved external subset or skipped external parameter entity leaves
declaration evidence incomplete. A name absent from the internal declarations
then remains lexical-only, with one `dtd-external-skipped` advisory per
document if a check needs the missing evidence. Internal parameter entities
are expanded so their declarations are collected; no external parameter
entity is retrieved. Caller-supplied trees without pyxsd source provenance and
custom-policy parser results have no core declaration evidence and remain
lexical-only. Top-level `pyxsd.parse()` with a non-seekable instance stream
has the same limitation, even with an explicit schema; this is not a security
feature. A compiled `Schema.parse()` can instead replay that stream's prolog
when declaration checking is needed. See {doc}`validation` for input and
reread limitations. None of this performs DTD validation or hardens the
default parser.

The parser protections are Python-level declaration interception. They do not
replace a patched Expat. Expat has had memory-safety and entity-expansion
vulnerabilities, so run pyxsd on a current CPython patch release with the
operating system's Expat security updates. No parser profile makes an outdated
Expat safe. XML 1.1 documents remain unsupported.

Check the Expat your interpreter actually links against, especially in a
container image or a bundled build:

```console
$ python -c "import pyexpat; print(pyexpat.EXPAT_VERSION)"
expat_2.6.4
```

An old or unknown Expat version is a security finding for untrusted input; no
parser profile compensates for it. `xml.parsers.expat.EXPAT_VERSION` reports the
same string from inside Python.

## Resource policies

`pyxsd.ResourcePolicy` is a protocol for callers that need to control reads:
`read(source, base=..., purpose=...)` supplies original bytes, and
`parse(data, context)` parses them. `base` is the canonical name of the including
document, not its directory. `purpose` is one of:

- `"schema"` — a caller-supplied schema, a `namespace_schemas` entry, or an
  ordinary schema dependency;
- `"schema-hint"` — a schema the instance selected through `xsi:schemaLocation`
  or `xsi:noNamespaceSchemaLocation`;
- `"instance"` — the instance document itself.

A policy is trusted application code, not a sandbox. It is responsible for
confining paths, applying parser hardening, and recording namespace evidence.
Denials should raise `ResourcePolicyError`; an `OSError` is treated as an
unavailable resource and can be downgraded to an advisory warning during schema
composition.

The default loader resolves local filesystem paths. An untrusted instance can
still name a local schema through its `schemaLocation` hint, and a schema can
name local `xs:include`/`xs:import` files; those reads are normal behavior, not
an authorization decision. If paths must be confined to a directory or a virtual
workspace, supply a `ResourcePolicy` that enforces that boundary. Do not treat a
local file path as trusted merely because it came from XML.

## Remote XSD loading

`pyxsd.RemoteResourcePolicy` is the only built-in remote loader. Selecting it
authorizes the application's remote root schema; it is never enabled implicitly.

```python
import pyxsd

remote = pyxsd.RemoteResourcePolicy(
    allowed_origins={"https://schemas.example.org"},
    allow_local_remote=False,
    allow_remote_hints=False,
    limits=pyxsd.RemoteLimits(
        response_bytes=4 * 1024 * 1024,
        closure_bytes=16 * 1024 * 1024,
        resources=32,
        depth=8,
    ),
)
schema = pyxsd.Schema.compile(
    "https://schemas.example.org/orders/orders.xsd",
    resource_policy=remote,
)
```

**The policy also changes instance parsing.** The compiled schema retains the
policy, so every later `schema.parse()` reads the instance through it and parses
it with `XmlParserPolicy.strict()`, capped at `limits.response_bytes`. That
applies to local instance files as well: a local instance with any DOCTYPE, or
one larger than `response_bytes`, is refused. A policy's `parse()` owns XML
parsing, so `xml_policy`/`xml_limits` cannot override this per instance; compile
a schema without the policy if an instance needs a different posture.

### Trust rules

- Every retrieval is **HTTPS**.
- A compile's remote loads have at most one authorized **root read**: the
  explicit main schema supplied by the caller, or a main schema selected from
  the instance's location hint (which also reads with `base=None`). Selecting
  the policy authorizes a remote root; when the root came from an instance hint,
  `allow_remote_hints=True` is required as well. A remote main hint needs no
  `allowed_origins` entry.
- A dependency of a remote document may stay on the requesting document's
  origin or the root's origin. Any other origin must be listed in
  `allowed_origins` (an HTTPS origin, not a URL).
- An **additional** remote hint resolved from a local schema is not a root read:
  it is also a local-to-remote read, so it requires **both**
  `allow_remote_hints=True` and `allow_local_remote=True`.
- An ordinary remote `xs:include`/`xs:import` referenced by a local schema
  requires `allow_local_remote=True`; once that document is admitted, its own
  cross-origin dependencies need `allowed_origins` as above.
- Remote instance documents are refused. Remote loading is limited to schemas.
- A remote document cannot pivot to a local path, a `file:` URL, or another
  non-HTTPS URL. A remote schema never loads a Python overlay.
- Private or non-global resolved addresses are refused unless
  `allow_private_addresses=True`.
- Redirects are followed one request at a time. The scheme, origin, and
  addresses are revalidated before each hop, up to `limits.redirects` hops.

Once a remote policy is selected, transport and resolver failures abort the
compile, including for a document-authored `xs:include`/`xs:import`. Only the
no-policy path keeps the offline advisory behavior for a missing schema
dependency.

### Budgets and lifetime

`RemoteLimits` defaults are finite. `response_bytes` bounds one response;
`closure_bytes` bounds every downloaded body for one compilation, including
redirect and error bodies. `resources` bounds unique URLs, `depth` bounds
dependency levels, and `redirects` is a closure-wide total, not a
per-resource cap. `connect_timeout`, `read_timeout`, and `request_seconds` bound
connection, each socket read, and the whole request. `xml_depth` and
`xml_elements` bound the strict parser that consumes a downloaded resource.

Each in-flight compilation gets its own closure state, so concurrent compiles
can share one policy without sharing origins or budgets. A direct `read()` call
outside a compilation uses one fallback closure. There is **no persistent remote
cache**: every compilation retrieves what it needs again.

### DNS checking is not rebinding immunity

The policy calls `resolver(host, port)` before every request and rejects
non-global addresses by default. That answer is used only for the preconnection
authorization; the transport then resolves the hostname again to establish the
connection. A DNS answer can therefore change between the check and the
connection. The preconnection check raises the bar; it does not provide
DNS-rebinding immunity.

The public seam cannot fix this by itself: `RemoteTransport.fetch(url, *,
limits)` receives the URL, not the checked addresses, so a transport that
resolves the hostname again performs the same second lookup. To pin the address,
supply a resolver/transport pair that resolves once, validates that answer, and
connects to the exact validated address (or a transport that performs its own
single resolve-check-connect as one step); replacing only the transport or only
the resolver does not close the gap. The built-in `HttpClientTransport` resolves
at connect time and is not pinned.

`RemoteTransport` is the retrieval seam: `fetch(url, *, limits)` performs one
HTTPS request and returns a `RemoteResponse` without following redirects. The
default is `HttpClientTransport`. Both `transport` and `resolver` are policy
constructor arguments; replacing them is how a caller records, audits, or
constrains retrieval.

## XPath and denial-of-service limits

`Document.xpath` evaluates an XPath expression in-process over a projection of
the bound tree. Identity constraints and `xs:assert`/`xs:assertion` evaluate
their supported XPath subsets during validation. These are not sandboxes:

- A caller-supplied XPath expression is code. A pathological expression or a
  large tree can consume CPU and memory, and the core library imposes no
  wall-clock timeout on it.
- XSD identity-constraint XPath is restricted to the specified subset. The
  `xs:assert`/`xs:assertion` subset excludes external-resource functions such as
  `fn:doc` and `fn:collection`; unsupported expressions are reported and not
  evaluated.
- The MCP server rejects `doc()`, `document()`, `collection()`,
  `uri-collection()`, `json-doc()`, `unparsed-text()`, and
  `unparsed-text-lines()` calls in `pyxsd_query_document`, and applies its own
  compile/query deadline, page, depth, element, and output budgets. That is a
  service boundary, not a guarantee for library callers.
- To contain untrusted queries or schemas, run them behind a process boundary
  with an external timeout, or validate/limit the inputs before querying.

## MCP servers

The packaged MCP server is local, offline, and does not use
`RemoteResourcePolicy`. The semantic parse of every schema and instance
document from the input capture uses `defusedxml` with `forbid_dtd=True`,
`forbid_entities=True`, and `forbid_external=True`, under the operator's XML
byte, depth, and element limits. A `result: "subtree"` query additionally runs
`SourceIndex`, a raw Expat pass configured with `XmlParserPolicy.strict()`; it
applies the same declaration policy and limits and never installs an external
resource fetcher. Both parse paths apply the same declaration policy.
The `pyxsd[mcp]` extra no longer declares `defusedxml` separately because it is a
base dependency. See {doc}`mcp-deployment` for the worker and transport
boundaries.

## What these controls do not cover

- Python overlays, mixin extensions, and transforms are executable application
  code. They run with the caller's privileges and are outside the XML loading
  contract.
- A custom `ResourcePolicy` or `RemoteTransport` is trusted code. Its own
  authorization bugs are not fixed by pyxsd.
- Parsing and validation can still be expensive for a large document within its
  budgets. Choose limits for the host, not only for the format.
- Reporting is lax by design: recoverable errors produce a tree and a report
  rather than aborting. A report is not a security decision, and a tree built
  from invalid input is not trusted data.