Network proxy administration

Lunar Paper has one outbound-network policy for the main server, background workers, Form Publisher, Data Layer, and feature-owned browser traffic. Use it when the deployment must reach third-party services through an HTTP or SOCKS5 proxy, or when an administrator must explicitly block selected integrations.

This policy is separate from the inbound reverse-proxy and trusted-hop settings described in Reverse proxy and TLS. It does not turn Lunar Paper into a general-purpose web gateway.

Before you begin

1. Set and back up DATA_SOURCE_ENCRYPTION_KEY. Lunar Paper will not save a new proxy password in plaintext. A deployment that still relies on the supported SESSION_SECRET fallback should move to the dedicated key.
2. Set the same INTERNAL_API_TOKEN for the main application and Data Layer when Data Layer is deployed separately. A publishing instance never receives INTERNAL_API_TOKEN; it uses its own PUBLISHER_BROKER_TOKEN. See Set up a publishing instance.
3. Confirm that every application container can resolve and connect to the proxy host and port. Inside a container, localhost means that container; use the proxy’s container-network name, LAN address, or host.docker.internal as appropriate.
4. Sign in as a Lunar Paper administrator.
5. Open Administration → Network → Proxy Configuration.

Never paste a production proxy password into logs, screenshots, support tickets, test commands, or browser developer-tool snippets.

Routing model

Choose one global mode:

Mode	Inherited service behavior
Off	Every service connects directly. An explicit block still blocks it.
Selected services	A service connects through a proxy only when its service route is proxy or a migrated compatibility rule preserves its previous proxy route.
All services	Every proxy-capable inherited service uses a configured compatible proxy. Explicit direct and block routes still win.

Each service has one route:

Route	Result
inherit	Follow the global mode and the service’s migration default.
direct	Never use a proxy for this service.
proxy	Require a compatible configured profile; fail closed if none is usable.
block	Reject the outbound operation before opening a connection.

A service can prefer auto, http, or socks5. auto selects the first configured profile supported by that transport. If the requested profile is missing or unsupported, the route is shown as unavailable and the operation is blocked instead of silently connecting directly.

Resolution order

For each outbound operation, Lunar Paper applies this order:

1. direct-locked bootstrap infrastructure;
2. an explicit service block;
3. browser transports that support only direct or block;
4. an explicit direct route or global Off mode;
5. proxy or inherited Selected/All routing;
6. NO_PROXY and local-address protection, unless the service explicitly enables Ignore NO_PROXY; and
7. compatible profile selection.

The Effective column displays the resolved route and a reason such as policy_off, service_direct, not_selected, bypass_rule, proxy_selected, service_blocked, or no_supported_proxy_profile.

Configure proxy profiles

HTTP profile

The HTTP profile accepts an http or https proxy endpoint, host, port, optional username, and optional password. HTTPS means the connection from Lunar Paper to the proxy is protected with TLS. HTTPS destinations use an HTTP CONNECT tunnel.

SOCKS5 profile

The SOCKS5 profile accepts a host, port, and optional credentials. Lunar Paper uses remote hostname resolution (socks5h) by default so the application host does not perform a separate target DNS lookup.

After entering a profile, select Save policy, then Test saved profile. The profile test connects only to Lunar Paper’s registered test destination. It does not accept an administrator-supplied URL and does not follow redirects.

Profile tests ignore the global routing mode

A saved-profile test deliberately exercises that profile even while global mode is Off. Service tests, previews, synchronization, and ordinary runtime calls use the complete effective policy.

Password behavior

• Admin responses contain passwordConfigured, never the password.
• Leaving the password field blank preserves the existing encrypted secret.
• Clear password explicitly removes it.
• If the credential key is absent or an existing ciphertext cannot be decrypted, authenticated proxy use fails closed and the screen reports the corrective action.
• Dynamic service endpoint labels are reduced to non-secret origins or host/port labels before they are returned to the browser.

Authenticated forward proxies

Some forward proxies require a Proxy-Authorization header even when their local policy accepts any non-empty credential pair. Leaving the username and password blank against one of these proxies returns 407 Proxy Authentication Required and makes every service routed through it appear offline.

Configure the HTTP profile with all of the values required by the proxy:

Field	Example	Notes
Protocol	HTTP	Choose HTTPS only when the connection to the proxy uses TLS.
Host	proxy.example.internal	Do not include a URL path or credentials.
Port	8080	This is the proxy listener, not the destination port.
Username	supplied by your proxy administrator	Required whenever the proxy challenges with 407.
Password	supplied by your proxy administrator	Saved encrypted; it is not returned to the browser.

Select Save policy, reload the page, and confirm Password configured is shown before selecting Test saved profile. Do not put credentials in the host field or in a URL.

TLS-inspecting proxies

An HTTP forward proxy can inspect HTTPS after accepting the CONNECT tunnel. If it signs the destination certificate with a private certificate authority, Lunar Paper must trust that CA. Otherwise HTTPS requests fail with a generic connection error even though the proxy itself is reachable.

Export the proxy’s root CA as a PEM file, make it readable by each Lunar Paper runtime, and set NODE_EXTRA_CA_CERTS to the path seen by that runtime. For a local source checkout, an absolute path in platform/.env is supported:

NODE_EXTRA_CA_CERTS=/absolute/path/to/corporate-proxy-ca.pem

For Docker Compose, mount the file and use its container path. Apply the same setting to the optional worker and to separately deployed Form Publisher or Data Layer runtimes:

services:
  lunarpaper:
    environment:
      NODE_EXTRA_CA_CERTS: /run/lunarpaper-ca/corporate-proxy-ca.pem
    volumes:
      - ./certificates/corporate-proxy-ca.pem:/run/lunarpaper-ca/corporate-proxy-ca.pem:ro
  lunarpaper-worker:
    environment:
      NODE_EXTRA_CA_CERTS: /run/lunarpaper-ca/corporate-proxy-ca.pem
    volumes:
      - ./certificates/corporate-proxy-ca.pem:/run/lunarpaper-ca/corporate-proxy-ca.pem:ro

Restart the affected runtimes after changing CA trust. Check startup logs for a message confirming that the extra certificate was loaded. A Windows host path such as C:\certificates\proxy-ca.pem is not a valid path inside a Linux container unless that file is mounted and the variable names the mount path.

Trust only your proxy CA

Adding a CA lets its holder authenticate any HTTPS destination to Lunar Paper. Obtain the certificate through your organisation’s trusted administration channel, mount it read-only, and replace it deliberately when the proxy CA is rotated.

Microsoft 365 and Entra through TLS inspection

Microsoft integrations normally make at least two HTTPS connections: credential or authorization flows use login.microsoftonline.com, while email submission and optional directory synchronization use graph.microsoft.com. Both must succeed through the selected route. An organization email connector appears in the service inventory as an email:<connector-id> entry. Entra OIDC discovery and directory-sync traffic share the provider’s sso_provider:<alias> entry.

Discovery tests the Entra OIDC metadata endpoint. Test Sync separately tests the directory application’s client credential, Graph route, and Graph permission. One can succeed while the other fails. See Directory user sync (preview).

For an app-only Microsoft 365 connector, Verify authorization requests a token but does not send a message. If it reports a generic connection failure while the saved proxy-profile test succeeds, inspect the server-side cause:

• UNABLE_TO_VERIFY_LEAF_SIGNATURE means the initiating Lunar Paper runtime did not load the inspecting proxy’s root CA. Set NODE_EXTRA_CA_CERTS, restart that runtime, and confirm the startup CA message before retrying.
• An Entra AADSTS700016 response means Microsoft received the request but could not find the supplied application ID in that tenant. Re-enter the Application (client) ID from the app overview; do not use the object ID, secret ID, secret value, or a placeholder.
• An invalid-client-secret response means the tenant and application were found, but the saved credential does not match. Enter the client secret value, not its identifier.

Keep authorization verification and message delivery as separate tests. A successful token request proves credentials and proxy trust; only a controlled test email proves Graph submission and Exchange sender authorization.

Destination-port restrictions

HTTP proxies commonly restrict which destination ports a CONNECT tunnel may reach. The saved-profile test uses HTTPS on port 443, so it can pass while SMTP, database, or other TCP-backed integrations still fail. Ask the proxy administrator to allow only the destination hosts and ports Lunar Paper actually needs, or use a reviewed SOCKS5 profile when that is the approved route. Do not open every CONNECT port merely to make the inventory green.

Examples that often need a separate proxy rule include SMTP submission on 587 or 465 and PostgreSQL on 5432. The actual port is the one configured for the connector.

When a proxy allows only port 443

A proxy that accepts CONNECT on 443 and refuses everything else lets web APIs through while every database connector fails. Testing such a connection reports:

HTTP proxy refused the tunnel to port 5432 (answered 403, policy connect-port-denied)

The port is the one the connector uses, and the status and policy are whatever the proxy returned, so the message tells you the refusal came from the proxy rather than from the database host. Cloud Postgres services are the common case: Neon and Supabase pool connections on 5432 and 6543.

To route these connections through the proxy, ask the proxy administrator for two things:

1. Allow CONNECT to the database ports the connectors use, for the destination hosts Lunar Paper needs.
2. Tunnel those hosts raw, without TLS inspection. PostgreSQL and MySQL do not begin their connections in TLS; they negotiate it inside their own protocol. A proxy that tries to inspect the tunnel breaks the handshake, and the connector reports an SSL error or a timeout instead of connecting.

Pointing a database connector at port 443 is not a workaround. The port is usually allowed, but the destination does not speak the database protocol there, and an inspecting proxy still interferes.

If the proxy cannot allow those ports, keep the proxy for everything else and give the affected connections their own route. In Administration → Network → Proxy Configuration, set the route to direct on each data_source:<id> entry, or on Data Layer source connections to cover them all. Record why in the service notes, because a direct route means those connections leave the network without passing through the proxy.

Gmail SMTP through an HTTP proxy

A personal Gmail connector using implicit TLS needs an HTTP CONNECT tunnel to the exact destination smtp.gmail.com:465. Allowing HTTPS on port 443 is not enough. A web-only corporate policy commonly rejects the SMTP tunnel with a response such as:

HTTP/1.1 403 Forbidden
X-Proxy-Policy: connect-port-denied

Configure the proxy policy as narrowly as the proxy supports:

destination: smtp.gmail.com
port: 465
method: CONNECT
action: allow
tls_inspection: disabled

This example describes the required policy rather than a vendor-specific file format. Prefer an exact host-and-port rule over allowing destination port 465 for every host. Pass the tunnel through raw: Gmail and Lunar Paper establish TLS with each other inside CONNECT, so the proxy must not replace Gmail’s certificate or try to parse the connection as HTTP.

After changing the proxy:

1. Reload or redeploy the proxy so the new CONNECT rule is active.
2. In Administration → Network → Proxy Configuration, find the connector’s smtp:<connector-id> service and set it to the reviewed HTTP proxy, or leave it inherited when All services already selects that profile.
3. In Administration → Integrations → Email Configuration, select Verify authorization. This opens and authenticates the SMTP connection without sending a message.
4. Send a test email only after verification succeeds.

If the proxy still answers 403, its running policy has not accepted the exact destination and port. If CONNECT succeeds but TLS or authentication fails, check the connector’s security mode and Gmail app password instead. For the complete connector setup, see Connect a personal Gmail mailbox.

Port 587 is an alternative only when the Gmail connector is configured for STARTTLS required and the proxy separately permits CONNECT smtp.gmail.com:587. Do not change the connector to port 443 merely because the proxy permits web traffic there.

NO_PROXY and local-address protection

Enter comma-separated exact hosts, domain suffixes, wildcard-domain forms, and optional ports. Examples:

localhost,127.0.0.1,.internal.example,*.svc.example,api.example.test:8443,[::1]

Local and private IP literals, loopback, link-local addresses, and common cloud metadata hosts bypass the proxy when Bypass local addresses is enabled. This protection remains active in All services mode.

Enable Ignore NO_PROXY only on the individual service that must cross the proxy despite a matching bypass rule. This can expose internal traffic to the proxy, so record the reason in the service notes and test it in a non-production environment first.

Service inventory

The inventory is authoritative rather than inferred from database table names. It combines fixed platform transports with current configured records:

• data_source:<id>
• ai_provider:<id>
• sso_provider:<alias>
• publisher:<id>
• external_access:<id>
• smtp:<id> for SMTP connectors and email:<id> for Microsoft Graph or Gmail API connectors
• the environment SMTP service
• licensing, telemetry, automation, webhooks, catalogues, address providers, remote assets, browser relays, and separate runtimes

SSO group mappings are not separate transports. They inherit the route of their identity provider.

Capability matrix

Transport	HTTP proxy	SOCKS5	Notes
Server HTTP, Axios, Undici, supported SDKs	Yes	Yes	Uses the approved outbound adapter.
SMTP	Yes	Yes	Includes configured email connectors; see Outbound email connectors.
PostgreSQL-compatible, MySQL, MSSQL, ClickHouse	Yes	Yes	Uses driver stream, socket, agent, or connector injection.
Form Publisher callbacks and relays	Yes	Yes	Resolves a short-lived decision through the internal broker.
Data Layer source sockets	Yes	Yes	Broker decision is bound to the configured data-source host.
Feature-owned browser APIs and assets	Yes	Yes	Browser calls use narrow same-origin relays; there is no arbitrary URL relay.
User-authored links and interactive embeds	No	No	Administrator-controlled direct or block only.
DB2, Oracle, Snowflake, BigQuery, restricted Databricks	No	No	Displayed as unsupported and blocked under an inherited All-mode proxy requirement. Use explicit direct only after review.

Platform PostgreSQL, Redis, the policy broker, and SeaweedFS are visible but direct-locked so a global mode cannot accidentally capture bootstrap traffic.

Which video sites can be embedded at all is a separate setting. See Video embeds.

Separate runtimes and caching

External publisher callbacks on a private network

When the main application’s address is private or uses local DNS, set Form Publisher callbacks (runtime:form-publisher-callbacks) to Direct unless your forward proxy is deliberately configured to reach that address:

1. Open Administration → Network → Proxy Configuration.
2. Find Form Publisher callbacks in the service inventory.
3. Choose Direct and select Save on that row.
4. Confirm the effective route is direct, then submit a test form from the external publisher.

This override works with global All services mode. Other services keep their own routing rules. Bypass local addresses does not automatically recognise every hostname that resolves to a private IP. For example, a local name such as paper.example.test can still be sent through the proxy when it is absent from NO_PROXY.

When the main application uses an HTTP listener, including behind a TLS reverse proxy, Direct callbacks to its exact APP_URL origin use that local listener. Publisher authentication and callback validation still apply. The external publisher must still resolve and reach MAIN_APP_URL and trust its TLS certificate to contact the main application’s policy broker. An explicit Proxy route continues to use the configured proxy; it does not silently fall back to Direct.

A proxy that cannot reach the private application address may reject its CONNECT tunnel with HTTP 502. Forms then show Network error, while publisher logs record Network policy relay failed; callback metrics can fail for the same reason. Correct the callback route rather than disabling CSP or TLS verification. See Set up a publishing instance.

What the callback Test button checks

The callback service test probes the main application’s /api/health endpoint through the saved effective route. Direct uses the same transport as callbacks; Proxy exercises the configured proxy, and Block prevents the probe. A passing check confirms connectivity only. It does not verify publisher credentials, form assignment, validation, or submission storage, and it creates no submission.

In version 0.7.9, this row’s Test button incorrectly treats the catalogue label “Main application callback URL” as a URL. It returns HTTP 409 before making a connection, shown as Service test failed, even when real form submissions succeed. The corrected test requires an updated main application release containing the fix; changing or repulling only the publisher does not fix the button. On an affected release, verify the route with an actual test form.

Policy caching

The main server exposes an authenticated internal policy broker. Form Publisher and Data Layer send a service key plus normalized target; the broker binds that host to the catalogue and returns a decision valid for no more than 60 seconds.

The separate runtimes keep only the resolved in-memory decision. They do not persist proxy credentials. After expiry, a broker failure blocks an operation that requires a proxy or block decision rather than falling back to a direct connection.

Main-runtime policy and catalogue reads are cached for 30 seconds. Admin saves explicitly invalidate affected entries. Allow up to 60 seconds for an already cached separate-runtime decision to expire.

Technical API reference

All admin routes require an authenticated administrator session.

Method and path	Purpose
GET /api/admin/network-proxy	Return sanitized global mode, bypass settings, and profile metadata.
PUT /api/admin/network-proxy	Save the versioned policy and encrypted profiles.
GET /api/admin/network-proxy/services	Return the catalogue, capabilities, service overrides, and effective routes.
PUT /api/admin/network-proxy/services/:serviceKey	Save routingMode, proxyKind, ignoreNoProxy, and notes.
POST /api/admin/network-proxy/profiles/:kind/test	Test the saved http or socks5 profile against the fixed destination.
POST /api/admin/network-proxy/services/:serviceKey/test	Test only that service’s registered endpoint through its effective route.

POST /api/internal/network-proxy/resolve is for authenticated Lunar Paper runtimes, not browsers or operator scripts. /api/network-relay/* contains only named, input-constrained relays for registered features such as Nominatim, Crossref, PubChem, Google Fonts, product logos, and approved static assets.

Configuration changes and tests write sanitized audit events. Connection errors are reduced to safe categories before reaching the admin browser.

Storage and upgrade compatibility

The versioned global policy is stored through the system-settings service under NETWORK_PROXY_POLICY_V2. Per-service routes are stored in core.external_service_proxy_settings.

During upgrade, legacy settings map as follows:

Legacy state	Version 2 state
Disabled	Global off
Enabled without Force All	Global selected
Enabled with Force All	Global all
Enabled service row	Service proxy

Services that previously used the generic global helper receive compatibility rows where necessary to retain their effective route. A legacy plaintext proxy password is encrypted and removed after a successful migration. If encryption is unavailable, the plaintext remains only for later migration and the authenticated profile is unusable until the key is configured.

Legacy proxy endpoints remain one-release compatibility adapters over the same resolver. New integrations should use a stable service key and the approved outbound adapter.

Manual acceptance test

Perform this test in a non-production deployment with a controlled HTTP CONNECT or SOCKS5 proxy whose access log records destination host and port but never authorization headers.

1. Establish the baseline

1. Record the current policy and every service override you will change.
2. Set global mode to Off and save.
3. Open a known integration that can make a harmless request, such as a Crossref citation lookup or a configured test data source.
4. Confirm the request succeeds and the test proxy records no connection.
5. Confirm the inventory reason is policy_off unless the service is explicitly blocked or direct-locked.

2. Validate the profiles

1. Configure the HTTP profile with non-production credentials, save, and reload the page.
2. Confirm the password field is empty but the screen reports that a password is configured.
3. Select Test saved profile and confirm a successful status and latency.
4. Confirm the proxy log records only the fixed test destination.
5. Save again with the password field blank and repeat the test to prove the secret was preserved.
6. Repeat with SOCKS5 if the deployment supports it. Confirm that the proxy observes the destination hostname, demonstrating remote DNS resolution.

3. Validate Selected-services mode

1. Set global mode to Selected services.
2. Set catalog:crossref to proxy with the HTTP profile.
3. Leave a second testable service on inherit.
4. Run both service tests.
5. Confirm Crossref reports proxy:http and appears in the proxy log.
6. Confirm the inherited service reports direct with not_selected and does not appear in the proxy log.

4. Validate All-services mode and overrides

1. Set both services back to inherit, then select All services.
2. Confirm each proxy-capable service reports a proxy route.
3. Set one service to direct and confirm only that service bypasses the proxy.
4. Set another service to block; its service test must return a blocked result without a proxy-log entry.
5. Restore the two overrides to inherit before continuing.

5. Validate bypass rules

1. Add api.crossref.org to NO_PROXY while still in All mode.
2. Confirm Crossref changes to direct with reason bypass_rule and produces no proxy-log entry.
3. Enable Ignore NO_PROXY only for Crossref and save the service.
4. Confirm it returns to the proxy route and appears in the proxy log.
5. Disable Ignore NO_PROXY and remove the temporary entry.
6. If a test REST source points to 127.0.0.1 or another private test address, confirm Bypass local addresses also keeps it direct. Do not test an ignore-local override against production infrastructure.

6. Validate feature relays and browser blocking

1. With Crossref proxied, perform a citation lookup in the document editor.
2. In browser developer tools, confirm the browser calls the same-origin /api/network-relay/ route rather than api.crossref.org directly.
3. Confirm the proxy records the server-side Crossref request.
4. Set browser:interactive-embeds to block, open a page containing an external iframe, and confirm the iframe is not loaded.
5. Set browser:user-links to block and confirm an external link click is prevented. Restore both to direct if blocking was only for the test.

7. Validate separate runtimes

1. Confirm Data Layer uses the same INTERNAL_API_TOKEN as the main application, and each publishing instance has the PUBLISHER_BROKER_TOKEN shown in its Connection settings.
2. Publish a form with address autocomplete and query an address. With address:nominatim proxied, the proxy should record Nominatim while the browser calls only the publisher’s same-origin relay.
3. Exercise a supported Data Layer source with its data_source:<id> route set to proxy and confirm the database connection reaches the proxy.
4. Change either route to block. Allow up to 60 seconds for an existing separate-runtime cache entry to expire, then confirm the operation fails without a direct connection.
5. Restore the intended routes.

8. Validate credential clearing and audit records

1. On a disposable authenticated profile, select Clear password, save, and confirm an authenticated profile test fails safely.
2. Re-enter the test password and confirm it works again.
3. Review Administration audit records for the global change, service changes, profile tests, and service tests.
4. Confirm no password, proxy authorization header, or internal token appears in the browser response, audit metadata, application logs, or proxy logs.

9. Restore and record the result

1. Restore the baseline mode, bypass list, profiles, and service overrides.
2. Repeat one representative direct and proxied operation.
3. Record the Lunar Paper version, proxy type, test date, tested runtimes, service keys, and pass/fail result. Store no credentials or personal data in the record.

Automated verification for developers

From the platform repository, run the focused proxy suite:

npm exec vitest run -- ../tests/platform/network-proxy-resolver.test.js ../tests/platform/network-proxy-credentials.test.js ../tests/platform/network-proxy-inventory.test.js ../tests/platform/network-proxy-relay.test.js ../tests/platform/network-proxy-browser-policy.test.js ../tests/platform/network-proxy-broker.test.js ../tests/platform/network-proxy-runtime-cache.test.js ../tests/platform/network-proxy-socket-integration.test.js ../tests/platform/admin-network-proxy-ui.test.js ../tests/platform/outbound-network-coverage-guard.test.js

Also run:

npm run db:check
npm run db:validate
npm run engines:validate

The outbound-network coverage guard rejects new direct server, Form Publisher, or Data Layer call sites outside the approved adapters and documented direct-infrastructure exceptions.

Troubleshooting

Symptom	Check
no_supported_proxy_profile	A compatible profile is configured and the service capability badge includes it.
http_profile_unavailable or socks5_profile_unavailable	The selected profile has a valid host and port and was saved successfully.
bypass_rule when a proxy was expected	NO_PROXY, local-address protection, and the service’s Ignore NO_PROXY setting.
Separate runtime continues using an old route	Wait up to 60 seconds, then verify broker reachability, and a matching INTERNAL_API_TOKEN for Data Layer or PUBLISHER_BROKER_TOKEN for a publishing instance.
Profile password cannot be saved	Configure DATA_SOURCE_ENCRYPTION_KEY or restore the supported legacy key before retrying.
Stored credential cannot be decrypted	Restore the key that encrypted it or explicitly clear and re-enter the password.
The proxy returns 407 or every proxied HTTPS service is offline	Enter the proxy username and password, save, and confirm Password configured before testing again. Some proxies require non-empty credentials even when they do not validate a specific account.
The proxy connects but HTTPS reports a certificate or fetch failed error	Mount the proxy root CA, set NODE_EXTRA_CA_CERTS to the runtime-visible PEM path, restart, and check the startup log.
Profile test passes but SMTP or a database connector fails	Check the proxy’s CONNECT destination-port allowlist. The profile test proves HTTPS port 443 only.
A database service is unavailable in All mode	Check the capability badge. Unsupported native transports require reviewed explicit direct routing.
A browser integration fails	Confirm its named relay service is not blocked and inspect the sanitized relay response; do not add an arbitrary proxy endpoint.