Conversation
Move dev-playground route and UI to a separate branch (plugin/files-playground). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Atila Fassina <atila@fassina.eu>
8fa2314 to
60b1bcd
Compare
744d776 to
5200066
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a comprehensive Files plugin for AppKit that enables Unity Catalog volume file operations. The plugin provides both HTTP routes and a programmatic API for common file operations including list, read, download, upload, delete, and preview. It integrates with AppKit's execution interceptor pipeline for caching, retry, and timeout handling.
The PR also modifies the server plugin to support bypassing JSON body parsing for specific routes (needed for file upload streaming), introducing a new skipBodyParsing route configuration option and getSkipBodyParsingPaths() plugin interface method.
Changes:
- New Files plugin with full CRUD operations for Unity Catalog volumes
- Server plugin modification to support raw body streaming for file uploads
- Comprehensive test coverage including unit, integration, and connector tests
- Documentation updates including README, API docs, and plugin guide
Reviewed changes
Copilot reviewed 32 out of 33 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
packages/appkit/src/plugins/files/plugin.ts |
Main FilesPlugin class with HTTP routes and programmatic API |
packages/appkit/src/plugins/files/types.ts |
TypeScript type definitions for file operations |
packages/appkit/src/plugins/files/defaults.ts |
Execution defaults for different operation tiers (read/download/write) |
packages/appkit/src/plugins/files/manifest.json |
Plugin manifest declaring resource requirements |
packages/appkit/src/connectors/files/client.ts |
FilesConnector implementing core file operations with telemetry |
packages/appkit/src/connectors/files/defaults.ts |
Content-type resolution and text detection helpers |
packages/appkit/src/plugins/server/index.ts |
Modified to support selective body parsing bypass |
packages/shared/src/plugin.ts |
Added getSkipBodyParsingPaths() to BasePlugin interface |
packages/appkit/src/plugin/plugin.ts |
Implementation of skipBodyParsing tracking and path registration |
packages/appkit/src/plugins/files/tests/* |
Comprehensive test suites for plugin, connector, and integration |
packages/appkit/src/plugins/server/tests/server.test.ts |
Tests for body parsing bypass functionality |
docs/docs/plugins.md |
Plugin guide with Files plugin documentation and examples |
packages/appkit/src/index.ts |
Export of files plugin and contentTypeFromPath helper |
apps/dev-playground/server/index.ts |
Dev playground integration |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cd3d855 to
1c5805b
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 32 out of 33 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
e86e4a4 to
bb57226
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * Resolve the MIME content type for a file path. | ||
| * | ||
| * Resolution order: | ||
| * 1. Custom type map (if the extension matches a key in `customTypes`). | ||
| * 2. Built-in extension map ({@link EXTENSION_CONTENT_TYPES}). | ||
| * 3. The `reported` type from the server, or `application/octet-stream` as a fallback. | ||
| * | ||
| * @param filePath - File path used to extract the extension. | ||
| * @param reported - Content type reported by the server (used as fallback). | ||
| * @param customTypes - Optional map of extensions to MIME types that takes priority. | ||
| * @returns The resolved MIME content type string. | ||
| */ | ||
| /** |
There was a problem hiding this comment.
Duplicate JSDoc comment blocks found. Lines 48-60 contain a doc comment that is immediately followed by another doc comment starting at line 61, with both describing different aspects of the code. The first comment (lines 48-60) appears to be orphaned and doesn't have an associated function or export, while the second comment (lines 61-66) correctly describes SAFE_INLINE_CONTENT_TYPES. Remove the first duplicate comment block.
| * Resolve the MIME content type for a file path. | |
| * | |
| * Resolution order: | |
| * 1. Custom type map (if the extension matches a key in `customTypes`). | |
| * 2. Built-in extension map ({@link EXTENSION_CONTENT_TYPES}). | |
| * 3. The `reported` type from the server, or `application/octet-stream` as a fallback. | |
| * | |
| * @param filePath - File path used to extract the extension. | |
| * @param reported - Content type reported by the server (used as fallback). | |
| * @param customTypes - Optional map of extensions to MIME types that takes priority. | |
| * @returns The resolved MIME content type string. | |
| */ | |
| /** |
| this.route(router, { | ||
| name: "root", | ||
| method: "get", | ||
| path: "/root", | ||
| handler: async (_req: express.Request, res: express.Response) => { | ||
| res.json({ root: this.config.defaultVolume ?? null }); | ||
| }, | ||
| }); |
There was a problem hiding this comment.
The /api/files/root endpoint is not covered by integration tests. While it's a simple endpoint that returns the configured defaultVolume, adding a test would ensure completeness and verify the endpoint is properly registered and accessible.
| reported?: string, | ||
| customTypes?: Record<string, string>, | ||
| ): string { | ||
| const ext = filePath.slice(filePath.lastIndexOf(".")).toLowerCase(); |
There was a problem hiding this comment.
The extension extraction logic may produce unexpected results for files without extensions. When a file has no extension (e.g., "README"), lastIndexOf(".") returns -1, and slice(-1) will return the last character of the filename, not an empty string. This could lead to incorrect extension matching. Consider handling the case when lastIndexOf(".") returns -1 explicitly, or use a more robust approach like checking if the index is greater than 0 before slicing.
| const ext = filePath.slice(filePath.lastIndexOf(".")).toLowerCase(); | |
| const dotIndex = filePath.lastIndexOf("."); | |
| const ext = dotIndex > 0 ? filePath.slice(dotIndex).toLowerCase() : ""; |
| this.serverApplication.use( | ||
| express.json({ | ||
| type: (req) => { | ||
| // Skip JSON parsing for routes that declared skipBodyParsing | ||
| // (e.g. file uploads where the raw body must flow through). | ||
| // rawBodyPaths is populated by extendRoutes() below; the type | ||
| // callback runs per-request so the set is already filled. | ||
| const urlPath = req.url?.split("?")[0]; | ||
| if (urlPath && this.rawBodyPaths.has(urlPath)) return false; | ||
| const ct = req.headers["content-type"] ?? ""; | ||
| return ct.includes("json"); | ||
| }, | ||
| }), | ||
| ); |
There was a problem hiding this comment.
There is a potential timing issue with the body parsing skip logic. The express.json middleware is registered at line 96-109 with a type function that checks this.rawBodyPaths, but this.rawBodyPaths is only populated by extendRoutes() which is called at line 111, after the middleware registration. However, this should work correctly because the type callback is executed per-request (not at registration time), and by the time the first request arrives, extendRoutes() will have already run. The inline comment at lines 101-102 correctly notes this behavior. No action needed, but consider adding a test to verify this timing works correctly.
This PR implements a first version of the Files API plugin according to the internal RFC.
Features
Important
this PR also touches the server plugin to bypass the
express.jsonmiddleware for upload routes, otherwise the stream does not passthroughFor testing functionality, I recommend using the adjacent PR that adds it to the Dev Playground ( #114 ), I split PR so code is easier to review here.