Files
ragflow/web/src/routes.tsx
Zhichang Yu fd11aca8e5 feat: Implement pluggable multi-provider sandbox architecture (#12820)
## Summary

Implement a flexible sandbox provider system supporting both
self-managed (Docker) and SaaS (Aliyun Code Interpreter) backends for
secure code execution in agent workflows.

**Key Changes:**
-  Aliyun Code Interpreter provider using official
`agentrun-sdk>=0.0.16`
-  Self-managed provider with gVisor (runsc) security
-  Arguments parameter support for dynamic code execution
-  Database-only configuration (removed fallback logic)
-  Configuration scripts for quick setup

Issue #12479

## Features

### 🔌 Provider Abstraction Layer

**1. Self-Managed Provider** (`agent/sandbox/providers/self_managed.py`)
- Wraps existing executor_manager HTTP API
- gVisor (runsc) for secure container isolation
- Configurable pool size, timeout, retry logic
- Languages: Python, Node.js, JavaScript
- ⚠️ **Requires**: gVisor installation, Docker, base images

**2. Aliyun Code Interpreter**
(`agent/sandbox/providers/aliyun_codeinterpreter.py`)
- SaaS integration using official agentrun-sdk
- Serverless microVM execution with auto-authentication
- Hard timeout: 30 seconds max
- Credentials: `AGENTRUN_ACCESS_KEY_ID`, `AGENTRUN_ACCESS_KEY_SECRET`,
`AGENTRUN_ACCOUNT_ID`, `AGENTRUN_REGION`
- Automatically wraps code to call `main()` function

**3. E2B Provider** (`agent/sandbox/providers/e2b.py`)
- Placeholder for future integration

### ⚙️ Configuration System

- `conf/system_settings.json`: Default provider =
`aliyun_codeinterpreter`
- `agent/sandbox/client.py`: Enforces database-only configuration
- Admin UI: `/admin/sandbox-settings`
- Configuration validation via `validate_config()` method
- Health checks for all providers

### 🎯 Key Capabilities

**Arguments Parameter Support:**
All providers support passing arguments to `main()` function:
```python
# User code
def main(name: str, count: int) -> dict:
    return {"message": f"Hello {name}!" * count}

# Executed with: arguments={"name": "World", "count": 3}
# Result: {"message": "Hello World!Hello World!Hello World!"}
```

**Self-Describing Providers:**
Each provider implements `get_config_schema()` returning form
configuration for Admin UI

**Error Handling:**
Structured `ExecutionResult` with stdout, stderr, exit_code,
execution_time

## Configuration Scripts

Two scripts for quick Aliyun sandbox setup:

**Shell Script (requires jq):**
```bash
source scripts/configure_aliyun_sandbox.sh
```

**Python Script (interactive):**
```bash
python3 scripts/configure_aliyun_sandbox.py
```

## Testing

```bash
# Unit tests
uv run pytest agent/sandbox/tests/test_providers.py -v

# Aliyun provider tests
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v

# Integration tests (requires credentials)
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v

# Quick SDK validation
python3 agent/sandbox/tests/verify_sdk.py
```

**Test Coverage:**
- 30 unit tests for provider abstraction
- Provider-specific tests for Aliyun
- Integration tests with real API
- Security tests for executor_manager

## Documentation

- `docs/develop/sandbox_spec.md` - Complete architecture specification
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration from legacy
sandbox
- `agent/sandbox/tests/QUICKSTART.md` - Quick start guide
- `agent/sandbox/tests/README.md` - Testing documentation

## Breaking Changes

⚠️ **Migration Required:**

1. **Directory Move**: `sandbox/` → `agent/sandbox/`
   - Update imports: `from sandbox.` → `from agent.sandbox.`

2. **Mandatory Configuration**: 
   - SystemSettings must have `sandbox.provider_type` configured
   - Removed fallback default values
- Configuration must exist in database (from
`conf/system_settings.json`)

3. **Aliyun Credentials**:
   - Requires `AGENTRUN_*` environment variables (not `ALIYUN_*`)
   - `AGENTRUN_ACCOUNT_ID` is now required (Aliyun primary account ID)

4. **Self-Managed Provider**:
   - gVisor (runsc) must be installed for security
   - Install: `go install gvisor.dev/gvisor/runsc@latest`

## Database Schema Changes

```python
# SystemSettings.value: CharField → TextField
api/db/db_models.py: Changed for unlimited config length

# SystemSettingsService.get_by_name(): Fixed query precision
api/db/services/system_settings_service.py: startswith → exact match
```

## Files Changed

### Backend (Python)
- `agent/sandbox/providers/base.py` - SandboxProvider ABC interface
- `agent/sandbox/providers/manager.py` - ProviderManager
- `agent/sandbox/providers/self_managed.py` - Self-managed provider
- `agent/sandbox/providers/aliyun_codeinterpreter.py` - Aliyun provider
- `agent/sandbox/providers/e2b.py` - E2B provider (placeholder)
- `agent/sandbox/client.py` - Unified client (enforces DB-only config)
- `agent/tools/code_exec.py` - Updated to use provider system
- `admin/server/services.py` - SandboxMgr with registry & validation
- `admin/server/routes.py` - 5 sandbox API endpoints
- `conf/system_settings.json` - Default: aliyun_codeinterpreter
- `api/db/db_models.py` - TextField for SystemSettings.value
- `api/db/services/system_settings_service.py` - Exact match query

### Frontend (TypeScript/React)
- `web/src/pages/admin/sandbox-settings.tsx` - Settings UI
- `web/src/services/admin-service.ts` - Sandbox service functions
- `web/src/services/admin.service.d.ts` - Type definitions
- `web/src/utils/api.ts` - Sandbox API endpoints

### Documentation
- `docs/develop/sandbox_spec.md` - Architecture spec
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration guide
- `agent/sandbox/tests/QUICKSTART.md` - Quick start
- `agent/sandbox/tests/README.md` - Testing guide

### Configuration Scripts
- `scripts/configure_aliyun_sandbox.sh` - Shell script (jq)
- `scripts/configure_aliyun_sandbox.py` - Python script

### Tests
- `agent/sandbox/tests/test_providers.py` - 30 unit tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter.py` - Provider tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py` -
Integration tests
- `agent/sandbox/tests/verify_sdk.py` - SDK validation

## Architecture

```
Admin UI → Admin API → SandboxMgr → ProviderManager → [SelfManaged|Aliyun|E2B]
                                      ↓
                                  SystemSettings
```

## Usage

### 1. Configure Provider

**Via Admin UI:**
1. Navigate to `/admin/sandbox-settings`
2. Select provider (Aliyun Code Interpreter / Self-Managed)
3. Fill in configuration
4. Click "Test Connection" to verify
5. Click "Save" to apply

**Via Configuration Scripts:**
```bash
# Aliyun provider
export AGENTRUN_ACCESS_KEY_ID="xxx"
export AGENTRUN_ACCESS_KEY_SECRET="yyy"
export AGENTRUN_ACCOUNT_ID="zzz"
export AGENTRUN_REGION="cn-shanghai"
source scripts/configure_aliyun_sandbox.sh
```

### 2. Restart Service

```bash
cd docker
docker compose restart ragflow-server
```

### 3. Execute Code in Agent

```python
from agent.sandbox.client import execute_code

result = execute_code(
    code='def main(name: str) -> dict: return {"message": f"Hello {name}!"}',
    language="python",
    timeout=30,
    arguments={"name": "World"}
)

print(result.stdout)  # {"message": "Hello World!"}
```

## Troubleshooting

### "Container pool is busy" (Self-Managed)
- **Cause**: Pool exhausted (default: 1 container in `.env`)
- **Fix**: Increase `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` to 5+

### "Sandbox provider type not configured"
- **Cause**: Database missing configuration
- **Fix**: Run config script or set via Admin UI

### "gVisor not found"
- **Cause**: runsc not installed
- **Fix**: `go install gvisor.dev/gvisor/runsc@latest && sudo cp
~/go/bin/runsc /usr/local/bin/`

### Aliyun authentication errors
- **Cause**: Wrong environment variable names
- **Fix**: Use `AGENTRUN_*` prefix (not `ALIYUN_*`)

## Checklist

- [x] All tests passing (30 unit tests + integration tests)
- [x] Documentation updated (spec, migration guide, quickstart)
- [x] Type definitions added (TypeScript)
- [x] Admin UI implemented
- [x] Configuration validation
- [x] Health checks implemented
- [x] Error handling with structured results
- [x] Breaking changes documented
- [x] Configuration scripts created
- [x] gVisor requirements documented

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-28 13:28:21 +08:00

456 lines
12 KiB
TypeScript

import { lazy } from 'react';
import { createBrowserRouter, Navigate, type RouteObject } from 'react-router';
import FallbackComponent from './components/fallback-component';
import { IS_ENTERPRISE } from './pages/admin/utils';
export enum Routes {
Root = '/',
Login = '/login-next',
Logout = '/logout',
Home = '/home',
Datasets = '/datasets',
DatasetBase = '/dataset',
Dataset = `${Routes.DatasetBase}${Routes.DatasetBase}`,
Agent = '/agent',
AgentTemplates = '/agent-templates',
Agents = '/agents',
Memories = '/memories',
Memory = '/memory',
MemoryMessage = '/memory-message',
MemorySetting = '/memory-setting',
AgentList = '/agent-list',
Searches = '/next-searches',
Search = '/next-search',
SearchShare = '/next-search/share',
Chats = '/next-chats',
Chat = '/next-chat',
Files = '/files',
ProfileSetting = '/profile-setting',
Profile = '/profile',
Api = '/api',
Mcp = '/mcp',
Team = '/team',
Plan = '/plan',
Model = '/model',
Prompt = '/prompt',
DataSource = '/data-source',
DataSourceDetailPage = '/data-source-detail-page',
ProfileMcp = `${ProfileSetting}${Mcp}`,
ProfileTeam = `${ProfileSetting}${Team}`,
ProfilePlan = `${ProfileSetting}${Plan}`,
ProfileModel = `${ProfileSetting}${Model}`,
ProfilePrompt = `${ProfileSetting}${Prompt}`,
ProfileProfile = `${ProfileSetting}${Profile}`,
DatasetTesting = '/testing',
Chunk = '/chunk',
ChunkResult = `${Chunk}${Chunk}`,
Parsed = '/parsed',
ParsedResult = `${Chunk}${Parsed}`,
Result = '/result',
ResultView = `${Chunk}${Result}`,
KnowledgeGraph = '/knowledge-graph',
AgentLogPage = '/agent-log-page',
AgentShare = '/agent/share',
ChatShare = `${Chats}/share`,
ChatWidget = `${Chats}/widget`,
UserSetting = '/user-setting',
DataSetOverview = '/dataset-overview',
DataSetSetting = '/dataset-setting',
DataflowResult = '/dataflow-result',
Admin = '/admin',
AdminServices = `${Admin}/services`,
AdminUserManagement = `${Admin}/users`,
AdminSandboxSettings = `${Admin}/sandbox-settings`,
AdminWhitelist = `${Admin}/whitelist`,
AdminRoles = `${Admin}/roles`,
AdminMonitoring = `${Admin}/monitoring`,
}
const routeConfig = [
{
path: '/login',
Component: lazy(() => import('@/pages/login-next')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: '/login-next',
Component: lazy(() => import('@/pages/login-next')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.ChatShare,
Component: lazy(() => import('@/pages/next-chats/share')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.AgentShare,
Component: lazy(() => import('@/pages/agent/share')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.ChatWidget,
Component: lazy(() => import('@/pages/next-chats/widget')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.AgentList,
Component: lazy(() => import('@/pages/agents')),
errorElement: <FallbackComponent />,
},
{
path: '/document/:id',
Component: lazy(() => import('@/pages/document-viewer')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: '/*',
Component: lazy(() => import('@/pages/404')),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.Root,
layout: false,
Component: lazy(() => import('@/layouts/next')),
wrappers: ['@/wrappers/auth'],
children: [
{
path: Routes.Root,
Component: lazy(() => import('@/pages/home')),
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.Datasets,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Datasets,
Component: lazy(() => import('@/pages/datasets')),
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.Chats,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Chats,
Component: lazy(() => import('@/pages/next-chats')),
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.Chat + '/:id',
layout: false,
Component: lazy(() => import('@/pages/next-chats/chat')),
errorElement: <FallbackComponent />,
},
{
path: Routes.Searches,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Searches,
Component: lazy(() => import('@/pages/next-searches')),
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.Memories,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Memories,
Component: lazy(() => import('@/pages/memories')),
},
],
errorElement: <FallbackComponent />,
},
{
path: `${Routes.Memory}`,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: `${Routes.Memory}`,
layout: false,
Component: lazy(() => import('@/pages/memory')),
children: [
{
path: `${Routes.Memory}/${Routes.MemoryMessage}/:id`,
Component: lazy(() => import('@/pages/memory/memory-message')),
},
{
path: `${Routes.Memory}/${Routes.MemorySetting}/:id`,
Component: lazy(() => import('@/pages/memory/memory-setting')),
},
],
},
],
errorElement: <FallbackComponent />,
},
{
path: `${Routes.Search}/:id`,
layout: false,
Component: lazy(() => import('@/pages/next-search')),
errorElement: <FallbackComponent />,
},
{
path: `${Routes.SearchShare}`,
layout: false,
Component: lazy(() => import('@/pages/next-search/share')),
errorElement: <FallbackComponent />,
},
{
path: Routes.Agents,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Agents,
Component: lazy(() => import('@/pages/agents')),
},
],
errorElement: <FallbackComponent />,
},
{
path: `${Routes.AgentLogPage}/:id`,
layout: false,
Component: lazy(() => import('@/pages/agents/agent-log-page')),
errorElement: <FallbackComponent />,
},
{
path: `${Routes.Agent}/:id`,
layout: false,
Component: lazy(() => import('@/pages/agent')),
errorElement: <FallbackComponent />,
},
{
path: Routes.AgentTemplates,
layout: false,
Component: lazy(() => import('@/pages/agents/agent-templates')),
errorElement: <FallbackComponent />,
},
{
path: Routes.Files,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.Files,
Component: lazy(() => import('@/pages/files')),
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.DatasetBase,
layout: false,
Component: lazy(() => import('@/layouts/next')),
children: [
{
path: Routes.DatasetBase,
element: <Navigate to={Routes.Dataset} replace />,
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.DatasetBase,
layout: false,
Component: lazy(() => import('@/pages/dataset')),
children: [
{
path: `${Routes.Dataset}/:id`,
Component: lazy(() => import('@/pages/dataset/dataset')),
},
{
path: `${Routes.DatasetBase}${Routes.DatasetTesting}/:id`,
Component: lazy(() => import('@/pages/dataset/testing')),
},
{
path: `${Routes.DatasetBase}${Routes.KnowledgeGraph}/:id`,
Component: lazy(() => import('@/pages/dataset/knowledge-graph')),
},
{
path: `${Routes.DatasetBase}${Routes.DataSetOverview}/:id`,
Component: lazy(() => import('@/pages/dataset/dataset-overview')),
},
{
path: `${Routes.DatasetBase}${Routes.DataSetSetting}/:id`,
Component: lazy(() => import('@/pages/dataset/dataset-setting')),
},
],
errorElement: <FallbackComponent />,
},
{
path: `${Routes.DataflowResult}`,
layout: false,
Component: lazy(() => import('@/pages/dataflow-result')),
errorElement: <FallbackComponent />,
},
{
path: `${Routes.ParsedResult}/chunks`,
layout: false,
Component: lazy(
() =>
import('@/pages/chunk/parsed-result/add-knowledge/components/knowledge-chunk'),
),
errorElement: <FallbackComponent />,
},
{
path: Routes.Chunk,
layout: false,
children: [
{
path: Routes.Chunk,
Component: lazy(() => import('@/pages/chunk')),
children: [
{
path: `${Routes.ChunkResult}/:id`,
Component: lazy(() => import('@/pages/chunk/chunk-result')),
},
{
path: `${Routes.ResultView}/:id`,
Component: lazy(() => import('@/pages/chunk/result-view')),
},
],
},
],
errorElement: <FallbackComponent />,
},
{
path: Routes.Chunk,
layout: false,
Component: lazy(() => import('@/pages/chunk')),
errorElement: <FallbackComponent />,
},
{
path: '/user-setting',
Component: lazy(() => import('@/pages/user-setting')),
layout: false,
children: [
{
path: '/user-setting',
element: <Navigate to={`/user-setting${Routes.DataSource}`} replace />,
},
{
path: '/user-setting/profile',
Component: lazy(() => import('@/pages/user-setting/profile')),
},
{
path: '/user-setting/locale',
Component: lazy(() => import('@/pages/user-setting/setting-locale')),
},
{
path: '/user-setting/model',
Component: lazy(() => import('@/pages/user-setting/setting-model')),
},
{
path: '/user-setting/team',
Component: lazy(() => import('@/pages/user-setting/setting-team')),
},
{
path: `/user-setting${Routes.Api}`,
Component: lazy(() => import('@/pages/user-setting/setting-api')),
},
{
path: `/user-setting${Routes.Mcp}`,
Component: lazy(() => import('@/pages/user-setting/mcp')),
},
{
path: `/user-setting${Routes.DataSource}`,
Component: lazy(() => import('@/pages/user-setting/data-source')),
},
],
errorElement: <FallbackComponent />,
},
{
path: `/user-setting${Routes.DataSource}${Routes.DataSourceDetailPage}`,
Component: lazy(
() => import('@/pages/user-setting/data-source/data-source-detail-page'),
),
layout: false,
errorElement: <FallbackComponent />,
},
{
path: Routes.Admin,
Component: lazy(() => import('@/pages/admin/layouts/root-layout')),
errorElement: <FallbackComponent />,
children: [
{
path: Routes.Admin,
Component: lazy(() => import('@/pages/admin/login')),
},
{
path: Routes.Admin,
Component: lazy(
() => import('@/pages/admin/layouts/authorized-layout'),
),
children: [
{
path: `${Routes.AdminUserManagement}/:id`,
Component: lazy(() => import('@/pages/admin/user-detail')),
},
{
Component: lazy(
() => import('@/pages/admin/layouts/navigation-layout'),
),
children: [
{
path: Routes.AdminServices,
Component: lazy(() => import('@/pages/admin/service-status')),
},
{
path: Routes.AdminUserManagement,
Component: lazy(() => import('@/pages/admin/users')),
},
{
path: Routes.AdminSandboxSettings,
Component: lazy(() => import('@/pages/admin/sandbox-settings')),
},
...(IS_ENTERPRISE
? [
{
path: Routes.AdminWhitelist,
Component: lazy(() => import('@/pages/admin/whitelist')),
},
{
path: Routes.AdminRoles,
Component: lazy(() => import('@/pages/admin/roles')),
},
{
path: Routes.AdminMonitoring,
Component: lazy(() => import('@/pages/admin/monitoring')),
},
]
: []),
],
},
],
},
],
} satisfies RouteObject,
];
const routers = createBrowserRouter(routeConfig, {
basename: import.meta.env.VITE_BASE_URL || '/',
});
export { routers };