docs: add project write implementation plan
This commit is contained in:
nac
@@ -0,0 +1,347 @@
|
||||
# 项目写入与附件上传 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (\`- [ ]\`) syntax for tracking.
|
||||
|
||||
**Goal:** Add fixed-schema project creation and update MCP tools plus an authenticated multipart upload endpoint that stores files in NocoBase and incrementally associates them with projects.
|
||||
|
||||
**Architecture:** Preserve all existing read-only behavior by default. A focused \`ProjectService\` validates and maps public project inputs, \`NbClient\` writes scalar project data through the existing CLI, and \`NocoBaseHttpClient\` performs the JWT-only multipart upload and association actions. The existing bearer middleware wraps both \`/mcp\` and a new \`/uploads\` route.
|
||||
|
||||
**Tech Stack:** Python 3.12, FastMCP, Starlette, httpx, python-multipart, NocoBase CLI/HTTP API, unittest.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Scope all writes to \`t_projects\`; never accept a caller-selected collection or field.
|
||||
- Support exactly the confirmed fields, create/update, and incremental attachment add/remove. Do not implement deletes, replacement attachment lists, or orphan cleanup.
|
||||
- Creation requires nonblank \`name\`; dates are \`YYYY-MM-DD\`; valid progress labels are \`询价阶段\`、\`已报价\`、\`送样\`、\`已定点\`、\`SOP\`、\`终止\`.
|
||||
- \`POST /uploads\` accepts exactly one \`multipart/form-data\` \`file\` part, with a 25 MiB default limit.
|
||||
- Use \`NOCOBASE_MCP_TOKEN\` for the public service and keep NocoBase upload credentials in \`NOCOBASE_MCP_UPLOAD_API_BASE_URL\` and \`NOCOBASE_MCP_UPLOAD_JWT\`.
|
||||
- Do not log files, authorization headers, request bodies, or JWTs. Always delete temporary files in \`finally\`.
|
||||
- Keep stdio and \`create_server(..., readonly=True)\` read-only; enable write tools and \`/uploads\` only for the HTTP runtime's explicit write mode.
|
||||
- Test only against fake CLI runners and mocked HTTP transports.
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| \`pyproject.toml\` | Declare direct HTTP and multipart dependencies. |
|
||||
| \`nocobase_crm_mcp/config.py\` | Parse fixed \`projects\` mapping. |
|
||||
| \`nocobase_crm_mcp/nb_client.py\` | Construct safe scalar create/update CLI calls. |
|
||||
| \`nocobase_crm_mcp/nocobase_http.py\` | Upload an attachment and invoke NocoBase association add/remove actions. |
|
||||
| \`nocobase_crm_mcp/projects.py\` | Validate/mapping/orchestration for the two public project tools. |
|
||||
| \`nocobase_crm_mcp/uploads.py\` | One-file multipart endpoint and temporary-file lifecycle. |
|
||||
| \`nocobase_crm_mcp/server.py\` | Write-mode tool and route composition. |
|
||||
| \`scripts/manage_mcp_service.py\` | Generate only the non-secret upload size setting in the private env file. |
|
||||
| \`config.example.json\`, \`README.md\` | Fixed mapping, ACL, JWT, endpoint, and client-flow documentation. |
|
||||
| \`tests/test_*.py\` | Unit and ASGI coverage without real NocoBase writes. |
|
||||
|
||||
### Task 1: Fixed project configuration and NocoBase HTTP client
|
||||
|
||||
**Files:**
|
||||
- Modify: \`pyproject.toml:6-9\`
|
||||
- Modify: \`nocobase_crm_mcp/config.py:1-173\`
|
||||
- Create: \`nocobase_crm_mcp/nocobase_http.py\`
|
||||
- Create: \`tests/test_nocobase_http.py\`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces \`ProjectConfig\` with resource/field mapping and \`progress_values: dict[str, str]\`.
|
||||
- Produces \`NocoBaseHttpClient(api_base_url, jwt, timeout=30.0, transport=None)\`.
|
||||
- Produces \`upload_attachment(path, filename, mimetype) -> dict[str, Any]\`, \`add_attachment_ids(project_id, attachment_ids) -> None\`, and \`remove_attachment_ids(project_id, attachment_ids) -> None\`.
|
||||
|
||||
- [ ] **Step 1: Write failing configuration and HTTP client tests**
|
||||
|
||||
~~~python
|
||||
def test_load_config_parses_project_defaults(tmp_path: Path) -> None:
|
||||
path = tmp_path / "config.json"
|
||||
path.write_text(json.dumps({"projects": {"resource": "t_projects"}}), encoding="utf-8")
|
||||
config = load_config(path)
|
||||
assert config.project.name_field == "f_project_name"
|
||||
assert config.project.progress_values["SOP"] == "a9t9awhtt3r"
|
||||
|
||||
def test_upload_posts_file_with_attachment_field(tmp_path: Path) -> None:
|
||||
seen = {}
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["url"] = str(request.url)
|
||||
seen["auth"] = request.headers["authorization"]
|
||||
return httpx.Response(200, json={"data": {"id": 42}})
|
||||
path = tmp_path / "quote.pdf"
|
||||
path.write_bytes(b"pdf")
|
||||
client = NocoBaseHttpClient("https://nb.example/api", "jwt", transport=httpx.MockTransport(handler))
|
||||
assert client.upload_attachment(path, "quote.pdf", "application/pdf")["id"] == 42
|
||||
assert seen["url"].endswith("/attachments:create?attachmentField=t_projects.f_projects_attachments")
|
||||
assert seen["auth"] == "Bearer jwt"
|
||||
~~~
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_nocobase_http -v\`
|
||||
|
||||
Expected: FAIL because \`ProjectConfig\` and \`NocoBaseHttpClient\` do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the configuration and client**
|
||||
|
||||
Add \`httpx>=0.27,<1\` and \`python-multipart>=0.0.9,<1\` to \`pyproject.toml\`. Add frozen \`ProjectConfig\` to \`config.py\` and add \`project: ProjectConfig = ProjectConfig()\` to \`CrmConfig\`. Parse only the \`projects\` config block; do not accept MCP input for its values.
|
||||
|
||||
Use these exact progress values:
|
||||
|
||||
~~~python
|
||||
PROJECT_PROGRESS_VALUES = {
|
||||
"询价阶段": "fk9xj8akn46", "已报价": "k4w2vcz819w", "送样": "g95vqeetu83",
|
||||
"已定点": "fe0peg505d7", "SOP": "a9t9awhtt3r", "终止": "a9t9awhtsfasg3r",
|
||||
}
|
||||
~~~
|
||||
|
||||
Implement \`NocoBaseHttpClient\` with a synchronous \`httpx.Client\`. It strips a trailing slash from the API URL, rejects blank URL/JWT, posts multipart \`files={"file": (filename, handle, mimetype or "application/octet-stream")}\` to \`/attachments:create\` with \`params={"attachmentField": "t_projects.f_projects_attachments"}\`, and posts JSON \`{"tk": attachment_ids}\` to \`/t_projects/{project_id}/f_projects_attachments:add\` or \`:remove\`. Raise \`NocoBaseHttpError(status_code, message)\` for non-2xx responses after redacting the JWT.
|
||||
|
||||
- [ ] **Step 4: Run tests and commit**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_nocobase_http -v\`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
~~~bash
|
||||
git add pyproject.toml nocobase_crm_mcp/config.py nocobase_crm_mcp/nocobase_http.py tests/test_nocobase_http.py
|
||||
git commit -m "feat: add project config and NocoBase HTTP client"
|
||||
~~~
|
||||
|
||||
### Task 2: Scalar project writes through the existing CLI client
|
||||
|
||||
**Files:**
|
||||
- Modify: \`nocobase_crm_mcp/nb_client.py:50-154\`
|
||||
- Modify: \`tests/test_nb_client.py:1-160\`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces \`NbClient.create_record(resource, values, whitelist) -> dict[str, Any]\`.
|
||||
- Produces \`NbClient.update_record(resource, record_id, values, whitelist) -> dict[str, Any]\`.
|
||||
|
||||
- [ ] **Step 1: Write failing CLI command tests**
|
||||
|
||||
~~~python
|
||||
def test_create_record_uses_values_and_whitelist(self) -> None:
|
||||
client = NbClient(env_name="nb-mcp", runner=runner)
|
||||
client.create_record("t_projects", {"f_project_name": "EV inverter"}, ["f_project_name"])
|
||||
assert commands[0][:4] == ["nb", "api", "resource", "create"]
|
||||
assert json.loads(commands[0][commands[0].index("--values") + 1]) == {"f_project_name": "EV inverter"}
|
||||
assert commands[0][commands[0].index("--whitelist") + 1] == "f_project_name"
|
||||
|
||||
def test_update_record_uses_filter_by_tk(self) -> None:
|
||||
client = NbClient(env_name="nb-mcp", runner=runner)
|
||||
client.update_record("t_projects", 7, {"client_ID": 3}, ["client_ID"])
|
||||
assert commands[0][:4] == ["nb", "api", "resource", "update"]
|
||||
assert commands[0][commands[0].index("--filter-by-tk") + 1] == "7"
|
||||
~~~
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_nb_client -v\`
|
||||
|
||||
Expected: FAIL because the two methods are absent.
|
||||
|
||||
- [ ] **Step 3: Implement safe write methods**
|
||||
|
||||
Use \`_resource_command("create", resource)\` or \`_resource_command("update", resource)\`, then \`_append_json(command, "--values", values)\` and \`_append_repeated(command, "--whitelist", whitelist)\`. For update, add \`--filter-by-tk\` via \`_append_json_or_scalar\`. Reject an empty values mapping before launching a subprocess.
|
||||
|
||||
- [ ] **Step 4: Run tests and commit**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_nb_client -v\`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
~~~bash
|
||||
git add nocobase_crm_mcp/nb_client.py tests/test_nb_client.py
|
||||
git commit -m "feat: add NocoBase project record writes"
|
||||
~~~
|
||||
|
||||
### Task 3: Fixed-schema project service
|
||||
|
||||
**Files:**
|
||||
- Create: \`nocobase_crm_mcp/projects.py\`
|
||||
- Create: \`tests/test_projects.py\`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes \`ProjectConfig\`, the two \`NbClient\` write methods, and \`NocoBaseHttpClient\` association methods.
|
||||
- Produces \`ProjectService.create_project(...) -> dict[str, Any]\` and \`ProjectService.update_project(...) -> dict[str, Any]\`.
|
||||
- Produces structured error codes \`invalid_project_input\`, \`invalid_progress\`, \`attachment_not_found\`, \`nb_upload_unauthorized\`, and \`nb_upload_failed\`.
|
||||
|
||||
- [ ] **Step 1: Write failing service tests**
|
||||
|
||||
~~~python
|
||||
def test_create_maps_fixed_fields_then_adds_attachment_ids() -> None:
|
||||
result = service.create_project(name=" EV inverter ", customer_id=3, progress="SOP", attachment_ids=[8, 9])
|
||||
assert nb.create_calls == [("t_projects", {"f_project_name": "EV inverter", "client_ID": 3, "f_projects_progress": "a9t9awhtt3r"}, PROJECT_WHITELIST)]
|
||||
assert http.add_calls == [(41, [8, 9])]
|
||||
assert result["project_id"] == 41
|
||||
|
||||
def test_update_rejects_overlapping_attachment_lists_before_writing() -> None:
|
||||
result = service.update_project(41, add_attachment_ids=[8], remove_attachment_ids=[8])
|
||||
assert result["error"]["code"] == "invalid_project_input"
|
||||
assert nb.update_calls == []
|
||||
|
||||
def test_invalid_date_and_progress_have_stable_codes() -> None:
|
||||
assert service.create_project(name="x", inquiry_date="2026/07/12")["error"]["code"] == "invalid_project_input"
|
||||
assert service.create_project(name="x", progress="草稿")["error"]["code"] == "invalid_progress"
|
||||
~~~
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_projects -v\`
|
||||
|
||||
Expected: FAIL with \`ModuleNotFoundError: nocobase_crm_mcp.projects\`.
|
||||
|
||||
- [ ] **Step 3: Implement the service**
|
||||
|
||||
Define \`PROJECT_WHITELIST\` from the eight configured scalar fields. \`_project_values()\` trims name, maps public names to configured field names, parses dates with \`datetime.date.fromisoformat\`, and maps progress labels from \`config.progress_values\`. It must not produce any attachment field value.
|
||||
|
||||
Creation rejects blank names, creates scalar values, extracts the configured ID, then adds deduplicated attachment IDs. Update rejects an empty request and overlapping add/remove sets before scalar writes; it updates scalars, then adds IDs, then removes IDs. If association work fails after a scalar write, return the project ID plus a structured attachment failure; never roll back the project.
|
||||
|
||||
- [ ] **Step 4: Run tests and commit**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_projects -v\`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
~~~bash
|
||||
git add nocobase_crm_mcp/projects.py tests/test_projects.py
|
||||
git commit -m "feat: add fixed project write service"
|
||||
~~~
|
||||
|
||||
### Task 4: One-file multipart upload endpoint
|
||||
|
||||
**Files:**
|
||||
- Create: \`nocobase_crm_mcp/uploads.py\`
|
||||
- Create: \`tests/test_uploads.py\`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces \`UploadEndpoint(http_client, max_bytes, temp_dir, allowed_mime_types=None)\`.
|
||||
- Produces \`await UploadEndpoint(request) -> JSONResponse\`.
|
||||
- Success JSON is exactly \`{"upload_id": int, "filename": str, "size": int, "mimetype": str | null}\`.
|
||||
|
||||
- [ ] **Step 1: Write failing ASGI tests**
|
||||
|
||||
~~~python
|
||||
async def test_upload_returns_nocobase_attachment_id() -> None:
|
||||
response = await client.post("/uploads", headers={"Authorization": "Bearer secret"},
|
||||
files={"file": ("quote.pdf", b"pdf", "application/pdf")})
|
||||
assert response.status_code == 200
|
||||
assert response.json()["upload_id"] == 42
|
||||
|
||||
async def test_upload_rejects_missing_file_and_over_limit() -> None:
|
||||
assert (await client.post("/uploads", headers=AUTH, files={})).status_code == 400
|
||||
response = await client.post("/uploads", headers=AUTH,
|
||||
files={"file": ("big.bin", b"x" * 6, "application/octet-stream")})
|
||||
assert response.status_code == 413
|
||||
~~~
|
||||
|
||||
Inject a fake HTTP client and a temporary directory; assert it is empty after successful upload and upstream failure.
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_uploads -v\`
|
||||
|
||||
Expected: FAIL because \`UploadEndpoint\` is absent.
|
||||
|
||||
- [ ] **Step 3: Implement the endpoint**
|
||||
|
||||
Require POST and multipart form data. Require exactly one Starlette \`UploadFile\` named \`file\`. Copy it in chunks to \`tempfile.NamedTemporaryFile(delete=False, dir=temp_dir)\`, count bytes while copying, and return 413 \`upload_too_large\` when over limit. If configured, compare MIME values using \`fnmatch\` and return 415 \`unsupported_media_type\`. Call \`http_client.upload_attachment\`; map 401/403 to 502 \`nb_upload_unauthorized\` and other upstream failures to 502 \`nb_upload_failed\`. Unlink the path in \`finally\` on every path.
|
||||
|
||||
- [ ] **Step 4: Run tests and commit**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_uploads -v\`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
~~~bash
|
||||
git add nocobase_crm_mcp/uploads.py tests/test_uploads.py
|
||||
git commit -m "feat: add authenticated attachment uploads"
|
||||
~~~
|
||||
|
||||
### Task 5: Server wiring, private service configuration, and documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: \`nocobase_crm_mcp/server.py:91-367\`
|
||||
- Modify: \`scripts/manage_mcp_service.py:1-180\`
|
||||
- Modify: \`config.example.json:1-126\`
|
||||
- Modify: \`README.md:1-151\`
|
||||
- Modify: \`tests/test_server.py:1-180\`
|
||||
- Modify: \`tests/test_mcp_service.py:1-180\`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces \`create_server(..., readonly=False, project_service=...)\` with MCP tools \`create_project\` and \`update_project\`.
|
||||
- Produces \`create_http_app(server, token, upload_endpoint=None)\` where bearer authentication wraps \`/uploads\` and \`/mcp\`.
|
||||
|
||||
- [ ] **Step 1: Write failing integration tests**
|
||||
|
||||
~~~python
|
||||
def test_write_server_exposes_fixed_project_tools() -> None:
|
||||
server = create_server(crm_service=FakeCrmService(), project_service=FakeProjectService(), readonly=False)
|
||||
names = {tool.name for tool in asyncio.run(server.list_tools())}
|
||||
assert {"create_project", "update_project"}.issubset(names)
|
||||
assert "destroy_project" not in names
|
||||
|
||||
def test_upload_route_requires_the_same_bearer_token() -> None:
|
||||
app = create_http_app(server, "secret", upload_endpoint=FakeUploadEndpoint())
|
||||
assert run_asgi_request(app, path="/uploads")[0]["status"] == 401
|
||||
|
||||
def test_generated_env_has_size_but_no_upload_jwt(tmp_path: Path) -> None:
|
||||
env_path = tmp_path / "mcp.env"
|
||||
manage_mcp_service.write_token_env_file(env_path, "token")
|
||||
content = env_path.read_text(encoding="utf-8")
|
||||
assert "NOCOBASE_MCP_UPLOAD_MAX_BYTES=26214400" in content
|
||||
assert "NOCOBASE_MCP_UPLOAD_JWT" not in content
|
||||
~~~
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_server tests.test_mcp_service -v\`
|
||||
|
||||
Expected: FAIL because write tools and upload route are absent.
|
||||
|
||||
- [ ] **Step 3: Implement composition without changing read-only defaults**
|
||||
|
||||
When \`readonly=True\`, retain the exact existing read tool set. When false, require a project service and register only \`create_project\` and \`update_project\` using existing \`_safe_tool_result\`.
|
||||
|
||||
Build a Starlette app with \`Route("/uploads", upload_endpoint, methods=["POST"])\` before \`Mount("/", app=server.streamable_http_app())\`, then pass the combined app through \`BearerTokenAuthMiddleware\`. In \`run_http_server\`, create \`NocoBaseHttpClient\` and \`UploadEndpoint\` from the required upload environment variables and call write mode. Leave stdio read-only.
|
||||
|
||||
Have \`write_token_env_file\` add only \`NOCOBASE_MCP_UPLOAD_MAX_BYTES=26214400\`; operators add API URL and JWT themselves to the already-0600 env file.
|
||||
|
||||
- [ ] **Step 4: Update configuration and README**
|
||||
|
||||
Add the approved fixed \`projects\` block to \`config.example.json\`. Document the two tools, \`POST /uploads\`, 25 MiB default, JWT environment variables, required ACLs (\`t_projects\` create/update, \`attachments\` upload/create, and attachment add/remove), retained orphan attachments, and absence of delete operations.
|
||||
|
||||
Include this operational sequence:
|
||||
|
||||
~~~bash
|
||||
NOCOBASE_MCP_UPLOAD_API_BASE_URL='https://nocobase.example/api'
|
||||
NOCOBASE_MCP_UPLOAD_JWT='<user-jwt>'
|
||||
bash scripts/mcp_service.sh start
|
||||
curl -X POST -H "Authorization: Bearer <mcp-token>" -F "file=@quote.pdf" http://<host>:10087/uploads
|
||||
~~~
|
||||
|
||||
- [ ] **Step 5: Run focused and full verification**
|
||||
|
||||
Run: \`uv run python -m unittest tests.test_nb_client tests.test_nocobase_http tests.test_projects tests.test_uploads tests.test_server tests.test_mcp_service -v\`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
Run: \`uv run python -m unittest discover -s tests -v\`
|
||||
|
||||
Expected: PASS with no real NocoBase request, project write, or attachment upload.
|
||||
|
||||
- [ ] **Step 6: Inspect and commit**
|
||||
|
||||
Run: \`git diff --check\`
|
||||
|
||||
Expected: no output.
|
||||
|
||||
~~~bash
|
||||
git add nocobase_crm_mcp/server.py scripts/manage_mcp_service.py config.example.json README.md tests/test_server.py tests/test_mcp_service.py
|
||||
git commit -m "feat: expose project writes and upload endpoint"
|
||||
~~~
|
||||
|
||||
## Plan self-review
|
||||
|
||||
- **Spec coverage:** Tasks 1–3 cover mapping, field validation, label mapping, scalar creates/updates, incremental attachments, and structured failures. Task 4 covers the one-file authenticated upload, 25 MiB default, redaction, and temporary cleanup. Task 5 covers bearer-route composition, read-only compatibility, deployment configuration, ACL documentation, and full verification.
|
||||
- **Placeholder scan:** The plan names all files, interfaces, error paths, checks, and commands; it contains no unresolved requirements.
|
||||
- **Type consistency:** Project inputs enter only through \`ProjectService\`; scalar values flow to \`NbClient\`; uploads and association IDs flow to \`NocoBaseHttpClient\`; \`UploadEndpoint\` and \`server.py\` only compose those components.
|
||||
|
||||
Reference in New Issue
Block a user