← Project Log
DecisionDEC-0024

Public Knowledge, Gated Compute, and Machine Access

humanStatus: accepted

DEC-0024: Public Knowledge, Gated Compute, and Machine Access

Decision

Adopt a durable access model based on one governing split:

  • give away structured knowledge
  • gate costly compute and higher-trust machine access

The project's default public posture is now:

  • public knowledge surfaces stay generous, easy to discover, and stable
  • semantic retrieval requires identity
  • heavier AI interaction requires payment or explicit developer approval
  • machine access must be scoped to a person or approved account, not just left broadly open

Public by default

These surfaces are intentionally public and should remain stable unless a later decision changes them:

  • KB pages and browse surfaces
  • knowledge graph browse
  • imagery
  • timeline
  • public episodes and series pages
  • markdown exports for public knowledge surfaces
  • llms.txt
  • llms-full.txt
  • public lexical search

Account-gated

These surfaces require sign-in and should be tied to a user identity plus bounded usage limits:

  • semantic search against the corpus
  • signed-in research workspace surfaces
  • future bounded AI research tools intended for free accounts

Free accounts should receive a real semantic-search experience, but one bounded enough to prevent anonymous cost leakage.

Paid

These surfaces require paid membership unless a later decision explicitly creates another access path:

  • AI chat
  • materially higher semantic-search quotas
  • heavier interactive research tooling

Developer / approved machine access

MCP is not a public anonymous utility surface.

Phase 1 machine access should be:

  • invite-only
  • provisioned to approved developer accounts
  • user-bound rather than governed only by a shared public key
  • quota-aware and observable

Phase 2 may evolve toward a broader external developer product with scoped auth and explicit packaging, but phase 1 should optimize for control, observability, and honest demand discovery.

Route Semantics

The public /search surface should mean public lexical search, not ambiguous "sometimes semantic, sometimes gated" behavior.

Semantic retrieval should move to clearly gated account surfaces and should not be leaked back into the public tier through alternate routes, stale copy, or machine-readable metadata.

llms.txt and llms-full.txt are intentionally generous public interfaces. They are part of the project's public knowledge posture, not an accidental exposure to be quietly narrowed later.

Context

The repo had drifted into an ambiguous middle state:

  • DEC-0013 correctly established KB/KG openness and said semantic search should remain gated
  • later site work reopened /search as a public surface
  • some route implementations and AI-facing docs now disagree about which search surfaces are public, signed-in, or paid
  • MCP auth exists, but its future product direction was not yet settled

The human clarified the intended product posture directly:

  • give away a lot
  • contain compute costs
  • let free users experience real value without turning the system into an anonymous cost sink
  • treat MCP as a future external product category, but not an open anonymous one

Rationale

The project's moat is not basic hoarding.

The durable advantage is:

  • the structured knowledge corpus
  • the editorial lens
  • the graph and domain model
  • the higher-trust interactive tooling built on top of them

Cheap, easy-to-cache public knowledge surfaces can be generous without breaking the business model. Expensive retrieval, chat, and machine access need identity, budget control, and product clarity.

This decision also removes a recurring source of system drift: routes and docs must stop treating "search" as a single undifferentiated thing.

Consequences

Positive

  • The site can remain generous without being careless.
  • Search semantics become legible: public lexical search vs. gated semantic retrieval.
  • Free accounts can demonstrate real value without opening an unlimited cost sink.
  • MCP gets a product path that can scale beyond a shared-key phase.

Tradeoffs

  • Some current routes and docs now need alignment work.
  • The team must maintain explicit quotas and observability for account and developer surfaces instead of relying on vague gating language.
  • AI-facing documentation must become more precise about what is public, signed-in, paid, or developer-only.

Implementation Standard

Operational follow-on work should proceed in this order:

  1. route and copy alignment
  2. auth and quota enforcement
  3. machine-access hardening
  4. exception-driven operational visibility

All AI-facing routes should have an explicit tier designation in code, docs, or both: public, account, paid, or developer.

0:00
0:00