fix(mcp): remove base64 ingest tool
Remove ingest_file_base64 from the ingest MCP tool surface so remote agents use the upload-ticket HTTP multipart path instead of oversized JSON/base64 tool arguments. Update docs and agent rules to make base64 upload explicitly unsupported as an MCP tool while preserving ticket-based uploads outside the vault. Validation: python3 -m unittest discover -v; python3 -m py_compile obsidian_wiki/mcp_server.py tests/test_mcp_server_remote.py; git diff --check
This commit is contained in:
Codex
@@ -130,7 +130,7 @@ See `wiki-query` and `wiki-export` skills for how the filter is applied.
|
||||
- **Keep product lookup fields explicit.** Carry forward `bjd_part_numbers`, `competitor_part_numbers`, `series`, `pitch`, `pin_count`, `product_lines`, `customers`, `competitors`, `standards`, `business_categories`, `related_pages`, and `catalog_paths` whenever they are known.
|
||||
- **Merge, do not overwrite.** Automatic identifier extraction may add terms, but it must not discard curated registry metadata. Registry values are the durable source for future index rebuilds and remote MCP lookup.
|
||||
- **Prefer precise identifiers in search ranking.** Exact or prefix matches in structured part-number and series fields should rank above incidental filename substrings such as timestamps or archive hashes.
|
||||
- **Keep remote ingest gated.** MCP ingest tools are write-capable and must remain disabled unless the service config explicitly enables the `ingest` tool group via `OBSIDIAN_MCP_ENABLED_TOOLS=file-assets,ingest`, `mcp_service.sh tools enable ingest`, or `mcp_service.sh start --enable-ingest`. Uploaded files must stage outside the vault under the MCP runtime root; `wiki-ingest` is responsible for accepting, archiving, and indexing sources inside the vault. Use the HTTP multipart endpoint `/mcp-ingest/uploads` for remote file transfer; if the remote agent cannot read the main MCP bearer token, it must call `create_ingest_upload_ticket` and send the returned single-use `X-Obsidian-Wiki-Upload-Token` header. Use `ingest_file_base64` only for small files when a client cannot make multipart HTTP requests. Do not expose chunked upload tools in the MCP tool surface. They may launch local OpenCode only through the fixed `/wiki-ingest` prompt; never expose arbitrary prompt or shell execution through MCP. Remote ingest must keep the OpenCode readiness preflight in front of job launch, and `check_ingest_opencode_readiness(run_smoke_test=true)` should stay available for operators to verify that OpenCode can load the project `wiki-ingest` skill before accepting uploads.
|
||||
- **Keep remote ingest gated.** MCP ingest tools are write-capable and must remain disabled unless the service config explicitly enables the `ingest` tool group via `OBSIDIAN_MCP_ENABLED_TOOLS=file-assets,ingest`, `mcp_service.sh tools enable ingest`, or `mcp_service.sh start --enable-ingest`. Uploaded files must stage outside the vault under the MCP runtime root; `wiki-ingest` is responsible for accepting, archiving, and indexing sources inside the vault. Use the HTTP multipart endpoint `/mcp-ingest/uploads` for remote file transfer; if the remote agent cannot read the main MCP bearer token, it must call `create_ingest_upload_ticket` and send the returned single-use `X-Obsidian-Wiki-Upload-Token` header. Do not expose base64 or chunked upload tools in the MCP tool surface. They may launch local OpenCode only through the fixed `/wiki-ingest` prompt; never expose arbitrary prompt or shell execution through MCP. Remote ingest must keep the OpenCode readiness preflight in front of job launch, and `check_ingest_opencode_readiness(run_smoke_test=true)` should stay available for operators to verify that OpenCode can load the project `wiki-ingest` skill before accepting uploads.
|
||||
|
||||
## Core Principles
|
||||
|
||||
|
||||
@@ -402,7 +402,6 @@ This adds write-capable MCP tools to the same `wiki-file-assets` server and bear
|
||||
|
||||
- `get_ingest_upload_instructions` — returns the preferred upload methods, HTTP endpoint path, fields, limits, and curl example for remote agents.
|
||||
- `create_ingest_upload_ticket` — returns a short-lived, single-use upload token for the HTTP endpoint so remote agents can upload with curl without reading the main MCP bearer token.
|
||||
- `ingest_file_base64` — uploads one complete base64 file and optionally starts the local OpenCode ingest job; useful for small files or MCP clients that cannot make multipart HTTP requests.
|
||||
- `check_ingest_opencode_readiness` — verifies the local OpenCode binary, required run flags, project ingest skills, and optionally runs a no-write smoke test that confirms `wiki-ingest` can be loaded.
|
||||
- `start_ingest_job` — queues a local `opencode run --auto --format json` job with a fixed `/wiki-ingest` prompt for the completed uploads.
|
||||
- `get_ingest_job_status`, `get_ingest_job_result`, `cancel_ingest_job` — inspect or cancel the local job.
|
||||
@@ -418,7 +417,7 @@ curl -H "X-Obsidian-Wiki-Upload-Token: <upload-token>" \
|
||||
http://127.0.0.1:8765/mcp-ingest/uploads
|
||||
```
|
||||
|
||||
The endpoint accepts repeated `files` fields plus optional `project`, `doc_type_hint`, `note`, and `start_job` fields. It accepts either the same bearer token as `/mcp` or the single-use upload ticket header. For MCP clients that cannot make HTTP multipart requests, `ingest_file_base64` remains available for small files, but large base64 arguments are fragile in many agent tool-call implementations.
|
||||
The endpoint accepts repeated `files` fields plus optional `project`, `doc_type_hint`, `note`, and `start_job` fields. It accepts either the same bearer token as `/mcp` or the single-use upload ticket header. Base64 file upload is intentionally not exposed as an MCP tool because large JSON arguments are fragile in many agent tool-call implementations.
|
||||
|
||||
Run `check_ingest_opencode_readiness` before the first remote upload. Use `run_smoke_test: true` when you want OpenCode itself to prove it can load the project `wiki-ingest` skill; otherwise the static preflight is also run automatically before every `start_ingest_job`.
|
||||
|
||||
|
||||
@@ -3,8 +3,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import hmac
|
||||
import os
|
||||
@@ -248,11 +246,6 @@ def create_server(
|
||||
"-F 'files=@drawing.pdf' http://<host>:<port>/mcp-ingest/uploads"
|
||||
),
|
||||
},
|
||||
"mcp_base64_tool": {
|
||||
"name": "ingest_file_base64",
|
||||
"best_for": "small files or MCP clients that cannot make HTTP multipart requests",
|
||||
"start_job_default": True,
|
||||
},
|
||||
"limits": {
|
||||
"max_file_bytes": ingest_manager.max_file_bytes,
|
||||
"max_chunk_bytes": ingest_manager.max_chunk_bytes,
|
||||
@@ -265,29 +258,6 @@ def create_server(
|
||||
},
|
||||
}
|
||||
|
||||
@server.tool()
|
||||
def ingest_file_base64(
|
||||
file_name: str,
|
||||
data_base64: str,
|
||||
project: str | None = None,
|
||||
doc_type_hint: str | None = None,
|
||||
note: str | None = None,
|
||||
start_job: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
"""Upload one complete base64 file and optionally start the local OpenCode ingest job."""
|
||||
try:
|
||||
payload = base64.b64decode(str(data_base64).strip(), validate=True)
|
||||
except (binascii.Error, ValueError) as exc:
|
||||
raise IngestError("data_base64 is not valid base64") from exc
|
||||
upload = ingest_manager.create_completed_upload(
|
||||
file_name=file_name,
|
||||
payload=payload,
|
||||
project=project,
|
||||
doc_type_hint=doc_type_hint,
|
||||
)
|
||||
job = ingest_manager.start_ingest_job([upload["upload_id"]], note=note) if start_job else None
|
||||
return {"upload": upload, "job": job}
|
||||
|
||||
@server.tool()
|
||||
def create_ingest_upload_ticket(
|
||||
expires_seconds: int = DEFAULT_UPLOAD_TICKET_SECONDS,
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -354,7 +353,7 @@ class RemoteMcpServerTests(unittest.TestCase):
|
||||
self.assertIn("check_ingest_opencode_readiness", FakeMCP.tool_names)
|
||||
self.assertIn("get_ingest_upload_instructions", FakeMCP.tool_names)
|
||||
self.assertIn("create_ingest_upload_ticket", FakeMCP.tool_names)
|
||||
self.assertIn("ingest_file_base64", FakeMCP.tool_names)
|
||||
self.assertNotIn("ingest_file_base64", FakeMCP.tool_names)
|
||||
|
||||
def test_create_ingest_upload_ticket_tool_returns_http_only_token(self):
|
||||
class FakeMCP:
|
||||
@@ -407,57 +406,6 @@ class RemoteMcpServerTests(unittest.TestCase):
|
||||
self.assertEqual(ticket["max_files"], 1)
|
||||
self.assertFalse(ticket["start_job"])
|
||||
|
||||
def test_ingest_file_base64_tool_uploads_without_chunking(self):
|
||||
class FakeMCP:
|
||||
tools = {}
|
||||
|
||||
def __init__(self, _name, **_kwargs):
|
||||
pass
|
||||
|
||||
def tool(self):
|
||||
def decorator(func):
|
||||
self.tools[func.__name__] = func
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
def resource(self, *_args, **_kwargs):
|
||||
def decorator(func):
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
def prompt(self, *_args, **_kwargs):
|
||||
def decorator(func):
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
with (
|
||||
tempfile.TemporaryDirectory() as tmp,
|
||||
patch.dict("sys.modules", {"mcp.server.fastmcp": type("Mod", (), {"FastMCP": FakeMCP})()}),
|
||||
):
|
||||
root = Path(tmp)
|
||||
mcp_server.create_server(
|
||||
vault_path=root / "vault",
|
||||
enable_ingest=True,
|
||||
project_root=root / "project",
|
||||
ingest_runtime_root=root / "runtime",
|
||||
)
|
||||
payload = b"one shot upload"
|
||||
result = FakeMCP.tools["ingest_file_base64"](
|
||||
file_name="source.pdf",
|
||||
data_base64=base64.b64encode(payload).decode("ascii"),
|
||||
project="bjd",
|
||||
doc_type_hint="drawing",
|
||||
start_job=False,
|
||||
)
|
||||
|
||||
self.assertEqual(result["job"], None)
|
||||
self.assertEqual(result["upload"]["state"], "complete")
|
||||
self.assertEqual(result["upload"]["file_name"], "source.pdf")
|
||||
self.assertEqual(result["upload"]["sha256"], "sha256:" + hashlib.sha256(payload).hexdigest())
|
||||
|
||||
def test_create_server_can_disable_file_asset_tools(self):
|
||||
class FakeMCP:
|
||||
tool_names = []
|
||||
|
||||
Reference in New Issue
Block a user