mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-01-31 15:45:08 +08:00
feat: Implement pluggable multi-provider sandbox architecture (#12820)
## Summary Implement a flexible sandbox provider system supporting both self-managed (Docker) and SaaS (Aliyun Code Interpreter) backends for secure code execution in agent workflows. **Key Changes:** - ✅ Aliyun Code Interpreter provider using official `agentrun-sdk>=0.0.16` - ✅ Self-managed provider with gVisor (runsc) security - ✅ Arguments parameter support for dynamic code execution - ✅ Database-only configuration (removed fallback logic) - ✅ Configuration scripts for quick setup Issue #12479 ## Features ### 🔌 Provider Abstraction Layer **1. Self-Managed Provider** (`agent/sandbox/providers/self_managed.py`) - Wraps existing executor_manager HTTP API - gVisor (runsc) for secure container isolation - Configurable pool size, timeout, retry logic - Languages: Python, Node.js, JavaScript - ⚠️ **Requires**: gVisor installation, Docker, base images **2. Aliyun Code Interpreter** (`agent/sandbox/providers/aliyun_codeinterpreter.py`) - SaaS integration using official agentrun-sdk - Serverless microVM execution with auto-authentication - Hard timeout: 30 seconds max - Credentials: `AGENTRUN_ACCESS_KEY_ID`, `AGENTRUN_ACCESS_KEY_SECRET`, `AGENTRUN_ACCOUNT_ID`, `AGENTRUN_REGION` - Automatically wraps code to call `main()` function **3. E2B Provider** (`agent/sandbox/providers/e2b.py`) - Placeholder for future integration ### ⚙️ Configuration System - `conf/system_settings.json`: Default provider = `aliyun_codeinterpreter` - `agent/sandbox/client.py`: Enforces database-only configuration - Admin UI: `/admin/sandbox-settings` - Configuration validation via `validate_config()` method - Health checks for all providers ### 🎯 Key Capabilities **Arguments Parameter Support:** All providers support passing arguments to `main()` function: ```python # User code def main(name: str, count: int) -> dict: return {"message": f"Hello {name}!" * count} # Executed with: arguments={"name": "World", "count": 3} # Result: {"message": "Hello World!Hello World!Hello World!"} ``` **Self-Describing Providers:** Each provider implements `get_config_schema()` returning form configuration for Admin UI **Error Handling:** Structured `ExecutionResult` with stdout, stderr, exit_code, execution_time ## Configuration Scripts Two scripts for quick Aliyun sandbox setup: **Shell Script (requires jq):** ```bash source scripts/configure_aliyun_sandbox.sh ``` **Python Script (interactive):** ```bash python3 scripts/configure_aliyun_sandbox.py ``` ## Testing ```bash # Unit tests uv run pytest agent/sandbox/tests/test_providers.py -v # Aliyun provider tests uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v # Integration tests (requires credentials) uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v # Quick SDK validation python3 agent/sandbox/tests/verify_sdk.py ``` **Test Coverage:** - 30 unit tests for provider abstraction - Provider-specific tests for Aliyun - Integration tests with real API - Security tests for executor_manager ## Documentation - `docs/develop/sandbox_spec.md` - Complete architecture specification - `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration from legacy sandbox - `agent/sandbox/tests/QUICKSTART.md` - Quick start guide - `agent/sandbox/tests/README.md` - Testing documentation ## Breaking Changes ⚠️ **Migration Required:** 1. **Directory Move**: `sandbox/` → `agent/sandbox/` - Update imports: `from sandbox.` → `from agent.sandbox.` 2. **Mandatory Configuration**: - SystemSettings must have `sandbox.provider_type` configured - Removed fallback default values - Configuration must exist in database (from `conf/system_settings.json`) 3. **Aliyun Credentials**: - Requires `AGENTRUN_*` environment variables (not `ALIYUN_*`) - `AGENTRUN_ACCOUNT_ID` is now required (Aliyun primary account ID) 4. **Self-Managed Provider**: - gVisor (runsc) must be installed for security - Install: `go install gvisor.dev/gvisor/runsc@latest` ## Database Schema Changes ```python # SystemSettings.value: CharField → TextField api/db/db_models.py: Changed for unlimited config length # SystemSettingsService.get_by_name(): Fixed query precision api/db/services/system_settings_service.py: startswith → exact match ``` ## Files Changed ### Backend (Python) - `agent/sandbox/providers/base.py` - SandboxProvider ABC interface - `agent/sandbox/providers/manager.py` - ProviderManager - `agent/sandbox/providers/self_managed.py` - Self-managed provider - `agent/sandbox/providers/aliyun_codeinterpreter.py` - Aliyun provider - `agent/sandbox/providers/e2b.py` - E2B provider (placeholder) - `agent/sandbox/client.py` - Unified client (enforces DB-only config) - `agent/tools/code_exec.py` - Updated to use provider system - `admin/server/services.py` - SandboxMgr with registry & validation - `admin/server/routes.py` - 5 sandbox API endpoints - `conf/system_settings.json` - Default: aliyun_codeinterpreter - `api/db/db_models.py` - TextField for SystemSettings.value - `api/db/services/system_settings_service.py` - Exact match query ### Frontend (TypeScript/React) - `web/src/pages/admin/sandbox-settings.tsx` - Settings UI - `web/src/services/admin-service.ts` - Sandbox service functions - `web/src/services/admin.service.d.ts` - Type definitions - `web/src/utils/api.ts` - Sandbox API endpoints ### Documentation - `docs/develop/sandbox_spec.md` - Architecture spec - `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration guide - `agent/sandbox/tests/QUICKSTART.md` - Quick start - `agent/sandbox/tests/README.md` - Testing guide ### Configuration Scripts - `scripts/configure_aliyun_sandbox.sh` - Shell script (jq) - `scripts/configure_aliyun_sandbox.py` - Python script ### Tests - `agent/sandbox/tests/test_providers.py` - 30 unit tests - `agent/sandbox/tests/test_aliyun_codeinterpreter.py` - Provider tests - `agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py` - Integration tests - `agent/sandbox/tests/verify_sdk.py` - SDK validation ## Architecture ``` Admin UI → Admin API → SandboxMgr → ProviderManager → [SelfManaged|Aliyun|E2B] ↓ SystemSettings ``` ## Usage ### 1. Configure Provider **Via Admin UI:** 1. Navigate to `/admin/sandbox-settings` 2. Select provider (Aliyun Code Interpreter / Self-Managed) 3. Fill in configuration 4. Click "Test Connection" to verify 5. Click "Save" to apply **Via Configuration Scripts:** ```bash # Aliyun provider export AGENTRUN_ACCESS_KEY_ID="xxx" export AGENTRUN_ACCESS_KEY_SECRET="yyy" export AGENTRUN_ACCOUNT_ID="zzz" export AGENTRUN_REGION="cn-shanghai" source scripts/configure_aliyun_sandbox.sh ``` ### 2. Restart Service ```bash cd docker docker compose restart ragflow-server ``` ### 3. Execute Code in Agent ```python from agent.sandbox.client import execute_code result = execute_code( code='def main(name: str) -> dict: return {"message": f"Hello {name}!"}', language="python", timeout=30, arguments={"name": "World"} ) print(result.stdout) # {"message": "Hello World!"} ``` ## Troubleshooting ### "Container pool is busy" (Self-Managed) - **Cause**: Pool exhausted (default: 1 container in `.env`) - **Fix**: Increase `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` to 5+ ### "Sandbox provider type not configured" - **Cause**: Database missing configuration - **Fix**: Run config script or set via Admin UI ### "gVisor not found" - **Cause**: runsc not installed - **Fix**: `go install gvisor.dev/gvisor/runsc@latest && sudo cp ~/go/bin/runsc /usr/local/bin/` ### Aliyun authentication errors - **Cause**: Wrong environment variable names - **Fix**: Use `AGENTRUN_*` prefix (not `ALIYUN_*`) ## Checklist - [x] All tests passing (30 unit tests + integration tests) - [x] Documentation updated (spec, migration guide, quickstart) - [x] Type definitions added (TypeScript) - [x] Admin UI implemented - [x] Configuration validation - [x] Health checks implemented - [x] Error handling with structured results - [x] Breaking changes documented - [x] Configuration scripts created - [x] gVisor requirements documented Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Zhichang Yu
261
agent/sandbox/tests/MIGRATION_GUIDE.md
Normal file
261
agent/sandbox/tests/MIGRATION_GUIDE.md
Normal file
@ -0,0 +1,261 @@
|
||||
# Aliyun Code Interpreter Provider - 使用官方 SDK
|
||||
|
||||
## 重要变更
|
||||
|
||||
### 官方资源
|
||||
- **Code Interpreter API**: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
|
||||
- **官方 SDK**: https://github.com/Serverless-Devs/agentrun-sdk-python
|
||||
- **SDK 文档**: https://docs.agent.run
|
||||
|
||||
## 使用官方 SDK 的优势
|
||||
|
||||
从手动 HTTP 请求迁移到官方 SDK (`agentrun-sdk`) 有以下优势：
|
||||
|
||||
### 1. **自动签名认证**
|
||||
- SDK 自动处理 Aliyun API 签名（无需手动实现 `Authorization` 头）
|
||||
- 支持多种认证方式：AccessKey、STS Token
|
||||
- 自动读取环境变量
|
||||
|
||||
### 2. **简化的 API**
|
||||
```python
|
||||
# 旧实现（手动 HTTP 请求）
|
||||
response = requests.post(
|
||||
f"{DATA_ENDPOINT}/sandboxes/{sandbox_id}/execute",
|
||||
headers={"X-Acs-Parent-Id": account_id},
|
||||
json={"code": code, "language": "python"}
|
||||
)
|
||||
|
||||
# 新实现（使用 SDK）
|
||||
sandbox = CodeInterpreterSandbox(template_name="python-sandbox", config=config)
|
||||
result = sandbox.context.execute(code="print('hello')")
|
||||
```
|
||||
|
||||
### 3. **更好的错误处理**
|
||||
- 结构化的异常类型 (`ServerError`)
|
||||
- 自动重试机制
|
||||
- 详细的错误信息
|
||||
|
||||
## 主要变更
|
||||
|
||||
### 1. 文件重命名
|
||||
|
||||
| 旧文件名 | 新文件名 | 说明 |
|
||||
|---------|---------|------|
|
||||
| `aliyun_opensandbox.py` | `aliyun_codeinterpreter.py` | 提供商实现 |
|
||||
| `test_aliyun_provider.py` | `test_aliyun_codeinterpreter.py` | 单元测试 |
|
||||
| `test_aliyun_integration.py` | `test_aliyun_codeinterpreter_integration.py` | 集成测试 |
|
||||
|
||||
### 2. 配置字段变更
|
||||
|
||||
#### 旧配置（OpenSandbox）
|
||||
```json
|
||||
{
|
||||
"access_key_id": "LTAI5t...",
|
||||
"access_key_secret": "...",
|
||||
"region": "cn-hangzhou",
|
||||
"workspace_id": "ws-xxxxx"
|
||||
}
|
||||
```
|
||||
|
||||
#### 新配置（Code Interpreter）
|
||||
```json
|
||||
{
|
||||
"access_key_id": "LTAI5t...",
|
||||
"access_key_secret": "...",
|
||||
"account_id": "1234567890...", // 新增：阿里云主账号ID（必需）
|
||||
"region": "cn-hangzhou",
|
||||
"template_name": "python-sandbox", // 新增：沙箱模板名称
|
||||
"timeout": 30 // 最大 30 秒（硬限制）
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 关键差异
|
||||
|
||||
| 特性 | OpenSandbox | Code Interpreter |
|
||||
|------|-------------|-----------------|
|
||||
| **API 端点** | `opensandbox.{region}.aliyuncs.com` | `agentrun.{region}.aliyuncs.com` (控制面) |
|
||||
| **API 版本** | `2024-01-01` | `2025-09-10` |
|
||||
| **认证** | 需要 AccessKey | 需要 AccessKey + 主账号ID |
|
||||
| **请求头** | 标准签名 | 需要 `X-Acs-Parent-Id` 头 |
|
||||
| **超时限制** | 可配置 | **最大 30 秒**（硬限制） |
|
||||
| **上下文** | 不支持 | 支持上下文（Jupyter kernel） |
|
||||
|
||||
### 4. API 调用方式变更
|
||||
|
||||
#### 旧实现（假设的 OpenSandbox）
|
||||
```python
|
||||
# 单一端点
|
||||
API_ENDPOINT = "https://opensandbox.cn-hangzhou.aliyuncs.com"
|
||||
|
||||
# 简单的请求/响应
|
||||
response = requests.post(
|
||||
f"{API_ENDPOINT}/execute",
|
||||
json={"code": "print('hello')", "language": "python"}
|
||||
)
|
||||
```
|
||||
|
||||
#### 新实现（Code Interpreter）
|
||||
```python
|
||||
# 控制面 API - 管理沙箱生命周期
|
||||
CONTROL_ENDPOINT = "https://agentrun.cn-hangzhou.aliyuncs.com/2025-09-10"
|
||||
|
||||
# 数据面 API - 执行代码
|
||||
DATA_ENDPOINT = "https://{account_id}.agentrun-data.cn-hangzhou.aliyuncs.com"
|
||||
|
||||
# 创建沙箱（控制面）
|
||||
response = requests.post(
|
||||
f"{CONTROL_ENDPOINT}/sandboxes",
|
||||
headers={"X-Acs-Parent-Id": account_id},
|
||||
json={"templateName": "python-sandbox"}
|
||||
)
|
||||
|
||||
# 执行代码（数据面）
|
||||
response = requests.post(
|
||||
f"{DATA_ENDPOINT}/sandboxes/{sandbox_id}/execute",
|
||||
headers={"X-Acs-Parent-Id": account_id},
|
||||
json={"code": "print('hello')", "language": "python", "timeout": 30}
|
||||
)
|
||||
```
|
||||
|
||||
### 5. 迁移步骤
|
||||
|
||||
#### 步骤 1: 更新配置
|
||||
|
||||
如果您之前使用的是 `aliyun_opensandbox`：
|
||||
|
||||
**旧配置**:
|
||||
```json
|
||||
{
|
||||
"name": "sandbox.provider_type",
|
||||
"value": "aliyun_opensandbox"
|
||||
}
|
||||
```
|
||||
|
||||
**新配置**:
|
||||
```json
|
||||
{
|
||||
"name": "sandbox.provider_type",
|
||||
"value": "aliyun_codeinterpreter"
|
||||
}
|
||||
```
|
||||
|
||||
#### 步骤 2: 添加必需的 account_id
|
||||
|
||||
在 Aliyun 控制台右上角点击头像，获取主账号 ID：
|
||||
1. 登录 [阿里云控制台](https://ram.console.aliyun.com/manage/ak)
|
||||
2. 点击右上角头像
|
||||
3. 复制主账号 ID（16 位数字）
|
||||
|
||||
#### 步骤 3: 更新环境变量
|
||||
|
||||
```bash
|
||||
# 新增必需的环境变量
|
||||
export ALIYUN_ACCOUNT_ID="1234567890123456"
|
||||
|
||||
# 其他环境变量保持不变
|
||||
export ALIYUN_ACCESS_KEY_ID="LTAI5t..."
|
||||
export ALIYUN_ACCESS_KEY_SECRET="..."
|
||||
export ALIYUN_REGION="cn-hangzhou"
|
||||
```
|
||||
|
||||
#### 步骤 4: 运行测试
|
||||
|
||||
```bash
|
||||
# 单元测试（不需要真实凭据）
|
||||
pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v
|
||||
|
||||
# 集成测试（需要真实凭据）
|
||||
pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v -m integration
|
||||
```
|
||||
|
||||
## 文件变更清单
|
||||
|
||||
### ✅ 已完成
|
||||
|
||||
- [x] 创建 `aliyun_codeinterpreter.py` - 新的提供商实现
|
||||
- [x] 更新 `sandbox_spec.md` - 规范文档
|
||||
- [x] 更新 `admin/services.py` - 服务管理器
|
||||
- [x] 更新 `providers/__init__.py` - 包导出
|
||||
- [x] 创建 `test_aliyun_codeinterpreter.py` - 单元测试
|
||||
- [x] 创建 `test_aliyun_codeinterpreter_integration.py` - 集成测试
|
||||
|
||||
### 📝 可选清理
|
||||
|
||||
如果您想删除旧的 OpenSandbox 实现：
|
||||
|
||||
```bash
|
||||
# 删除旧文件（可选）
|
||||
rm agent/sandbox/providers/aliyun_opensandbox.py
|
||||
rm agent/sandbox/tests/test_aliyun_provider.py
|
||||
rm agent/sandbox/tests/test_aliyun_integration.py
|
||||
```
|
||||
|
||||
**注意**: 保留旧文件不会影响新功能，只是代码冗余。
|
||||
|
||||
## API 参考
|
||||
|
||||
### 控制面 API（沙箱管理）
|
||||
|
||||
| 端点 | 方法 | 说明 |
|
||||
|------|------|------|
|
||||
| `/sandboxes` | POST | 创建沙箱实例 |
|
||||
| `/sandboxes/{id}/stop` | POST | 停止实例 |
|
||||
| `/sandboxes/{id}` | DELETE | 删除实例 |
|
||||
| `/templates` | GET | 列出模板 |
|
||||
|
||||
### 数据面 API（代码执行）
|
||||
|
||||
| 端点 | 方法 | 说明 |
|
||||
|------|------|------|
|
||||
| `/sandboxes/{id}/execute` | POST | 执行代码（简化版） |
|
||||
| `/sandboxes/{id}/contexts` | POST | 创建上下文 |
|
||||
| `/sandboxes/{id}/contexts/{ctx_id}/execute` | POST | 在上下文中执行 |
|
||||
| `/sandboxes/{id}/health` | GET | 健康检查 |
|
||||
| `/sandboxes/{id}/files` | GET/POST | 文件读写 |
|
||||
| `/sandboxes/{id}/processes/cmd` | POST | 执行 Shell 命令 |
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q: 为什么要添加 account_id？
|
||||
|
||||
**A**: Code Interpreter API 需要在请求头中提供 `X-Acs-Parent-Id`（阿里云主账号ID）进行身份验证。这是 Aliyun Code Interpreter API 的必需参数。
|
||||
|
||||
### Q: 30 秒超时限制可以绕过吗？
|
||||
|
||||
**A**: 不可以。这是 Aliyun Code Interpreter 的**硬限制**，无法通过配置或请求参数绕过。如果代码执行时间超过 30 秒，请考虑：
|
||||
1. 优化代码逻辑
|
||||
2. 分批处理数据
|
||||
3. 使用上下文保持状态
|
||||
|
||||
### Q: 旧的 OpenSandbox 配置还能用吗？
|
||||
|
||||
**A**: 不能。OpenSandbox 和 Code Interpreter 是两个不同的服务，API 不兼容。必须迁移到新的配置格式。
|
||||
|
||||
### Q: 如何获取阿里云主账号 ID？
|
||||
|
||||
**A**:
|
||||
1. 登录阿里云控制台
|
||||
2. 点击右上角的头像
|
||||
3. 在弹出的信息中可以看到"主账号ID"
|
||||
|
||||
### Q: 迁移后会影响现有功能吗？
|
||||
|
||||
**A**:
|
||||
- **自我管理提供商（self_managed）**: 不受影响
|
||||
- **E2B 提供商**: 不受影响
|
||||
- **Aliyun 提供商**: 需要更新配置并重新测试
|
||||
|
||||
## 相关文档
|
||||
|
||||
- [官方文档](https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter)
|
||||
- [sandbox 规范](../docs/develop/sandbox_spec.md)
|
||||
- [测试指南](./README.md)
|
||||
- [快速开始](./QUICKSTART.md)
|
||||
|
||||
## 技术支持
|
||||
|
||||
如有问题，请：
|
||||
1. 查看官方文档
|
||||
2. 检查配置是否正确
|
||||
3. 查看测试输出中的错误信息
|
||||
4. 联系 RAGFlow 团队
|
||||
178
agent/sandbox/tests/QUICKSTART.md
Normal file
178
agent/sandbox/tests/QUICKSTART.md
Normal file
@ -0,0 +1,178 @@
|
||||
# Aliyun OpenSandbox Provider - 快速测试指南
|
||||
|
||||
## 测试说明
|
||||
|
||||
### 1. 单元测试（不需要真实凭据）
|
||||
|
||||
单元测试使用 mock，**不需要**真实的 Aliyun 凭据，可以随时运行。
|
||||
|
||||
```bash
|
||||
# 运行 Aliyun 提供商的单元测试
|
||||
pytest agent/sandbox/tests/test_aliyun_provider.py -v
|
||||
|
||||
# 预期输出：
|
||||
# test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_provider_initialization PASSED
|
||||
# test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_initialize_success PASSED
|
||||
# ...
|
||||
# ========================= 48 passed in 2.34s ==========================
|
||||
```
|
||||
|
||||
### 2. 集成测试（需要真实凭据）
|
||||
|
||||
集成测试会调用真实的 Aliyun API，需要配置凭据。
|
||||
|
||||
#### 步骤 1: 配置环境变量
|
||||
|
||||
```bash
|
||||
export ALIYUN_ACCESS_KEY_ID="LTAI5t..." # 替换为真实的 Access Key ID
|
||||
export ALIYUN_ACCESS_KEY_SECRET="..." # 替换为真实的 Access Key Secret
|
||||
export ALIYUN_REGION="cn-hangzhou" # 可选，默认为 cn-hangzhou
|
||||
```
|
||||
|
||||
#### 步骤 2: 运行集成测试
|
||||
|
||||
```bash
|
||||
# 运行所有集成测试
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v -m integration
|
||||
|
||||
# 运行特定测试
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check -v
|
||||
```
|
||||
|
||||
#### 步骤 3: 预期输出
|
||||
|
||||
```
|
||||
test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_initialize_provider PASSED
|
||||
test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check PASSED
|
||||
test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code PASSED
|
||||
...
|
||||
========================== 10 passed in 15.67s ==========================
|
||||
```
|
||||
|
||||
### 3. 测试场景
|
||||
|
||||
#### 基础功能测试
|
||||
|
||||
```bash
|
||||
# 健康检查
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check -v
|
||||
|
||||
# 创建实例
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_create_python_instance -v
|
||||
|
||||
# 执行代码
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code -v
|
||||
|
||||
# 销毁实例
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_destroy_instance -v
|
||||
```
|
||||
|
||||
#### 错误处理测试
|
||||
|
||||
```bash
|
||||
# 代码执行错误
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code_with_error -v
|
||||
|
||||
# 超时处理
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code_timeout -v
|
||||
```
|
||||
|
||||
#### 真实场景测试
|
||||
|
||||
```bash
|
||||
# 数据处理工作流
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_data_processing_workflow -v
|
||||
|
||||
# 字符串操作
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_string_manipulation -v
|
||||
|
||||
# 多次执行
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_multiple_executions_same_instance -v
|
||||
```
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q: 没有凭据怎么办？
|
||||
|
||||
**A:** 运行单元测试即可，不需要真实凭据：
|
||||
```bash
|
||||
pytest agent/sandbox/tests/test_aliyun_provider.py -v
|
||||
```
|
||||
|
||||
### Q: 如何跳过集成测试？
|
||||
|
||||
**A:** 使用 pytest 标记跳过：
|
||||
```bash
|
||||
# 只运行单元测试，跳过集成测试
|
||||
pytest agent/sandbox/tests/ -v -m "not integration"
|
||||
```
|
||||
|
||||
### Q: 集成测试失败怎么办？
|
||||
|
||||
**A:** 检查以下几点：
|
||||
|
||||
1. **凭据是否正确**
|
||||
```bash
|
||||
echo $ALIYUN_ACCESS_KEY_ID
|
||||
echo $ALIYUN_ACCESS_KEY_SECRET
|
||||
```
|
||||
|
||||
2. **网络连接是否正常**
|
||||
```bash
|
||||
curl -I https://opensandbox.cn-hangzhou.aliyuncs.com
|
||||
```
|
||||
|
||||
3. **是否有 OpenSandbox 服务权限**
|
||||
- 登录阿里云控制台
|
||||
- 检查是否已开通 OpenSandbox 服务
|
||||
- 检查 AccessKey 权限
|
||||
|
||||
4. **查看详细错误信息**
|
||||
```bash
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v -s
|
||||
```
|
||||
|
||||
### Q: 测试超时怎么办？
|
||||
|
||||
**A:** 增加超时时间或检查网络：
|
||||
```bash
|
||||
# 使用更长的超时
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v --timeout=60
|
||||
```
|
||||
|
||||
## 测试命令速查表
|
||||
|
||||
| 命令 | 说明 | 需要凭据 |
|
||||
|------|------|---------|
|
||||
| `pytest agent/sandbox/tests/test_aliyun_provider.py -v` | 单元测试 | ❌ |
|
||||
| `pytest agent/sandbox/tests/test_aliyun_integration.py -v` | 集成测试 | ✅ |
|
||||
| `pytest agent/sandbox/tests/ -v -m "not integration"` | 仅单元测试 | ❌ |
|
||||
| `pytest agent/sandbox/tests/ -v -m integration` | 仅集成测试 | ✅ |
|
||||
| `pytest agent/sandbox/tests/ -v` | 所有测试 | 部分需要 |
|
||||
|
||||
## 获取 Aliyun 凭据
|
||||
|
||||
1. 访问 [阿里云控制台](https://ram.console.aliyun.com/manage/ak)
|
||||
2. 创建 AccessKey
|
||||
3. 保存 AccessKey ID 和 AccessKey Secret
|
||||
4. 设置环境变量
|
||||
|
||||
⚠️ **安全提示：**
|
||||
- 不要在代码中硬编码凭据
|
||||
- 使用环境变量或配置文件
|
||||
- 定期轮换 AccessKey
|
||||
- 限制 AccessKey 权限
|
||||
|
||||
## 下一步
|
||||
|
||||
1. ✅ **运行单元测试** - 验证代码逻辑
|
||||
2. 🔧 **配置凭据** - 设置环境变量
|
||||
3. 🚀 **运行集成测试** - 测试真实 API
|
||||
4. 📊 **查看结果** - 确保所有测试通过
|
||||
5. 🎯 **集成到系统** - 使用 admin API 配置提供商
|
||||
|
||||
## 需要帮助？
|
||||
|
||||
- 查看 [完整文档](README.md)
|
||||
- 检查 [sandbox 规范](../../../../../docs/develop/sandbox_spec.md)
|
||||
- 联系 RAGFlow 团队
|
||||
213
agent/sandbox/tests/README.md
Normal file
213
agent/sandbox/tests/README.md
Normal file
@ -0,0 +1,213 @@
|
||||
# Sandbox Provider Tests
|
||||
|
||||
This directory contains tests for the RAGFlow sandbox provider system.
|
||||
|
||||
## Test Structure
|
||||
|
||||
```
|
||||
tests/
|
||||
├── pytest.ini # Pytest configuration
|
||||
├── test_providers.py # Unit tests for all providers (mocked)
|
||||
├── test_aliyun_provider.py # Unit tests for Aliyun provider (mocked)
|
||||
├── test_aliyun_integration.py # Integration tests for Aliyun (real API)
|
||||
└── sandbox_security_tests_full.py # Security tests for self-managed provider
|
||||
```
|
||||
|
||||
## Test Types
|
||||
|
||||
### 1. Unit Tests (No Credentials Required)
|
||||
|
||||
Unit tests use mocks and don't require any external services or credentials.
|
||||
|
||||
**Files:**
|
||||
- `test_providers.py` - Tests for base provider interface and manager
|
||||
- `test_aliyun_provider.py` - Tests for Aliyun provider with mocked API calls
|
||||
|
||||
**Run unit tests:**
|
||||
```bash
|
||||
# Run all unit tests
|
||||
pytest agent/sandbox/tests/test_providers.py -v
|
||||
pytest agent/sandbox/tests/test_aliyun_provider.py -v
|
||||
|
||||
# Run specific test
|
||||
pytest agent/sandbox/tests/test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_initialize_success -v
|
||||
|
||||
# Run all unit tests (skip integration)
|
||||
pytest agent/sandbox/tests/ -v -m "not integration"
|
||||
```
|
||||
|
||||
### 2. Integration Tests (Real Credentials Required)
|
||||
|
||||
Integration tests make real API calls to Aliyun OpenSandbox service.
|
||||
|
||||
**Files:**
|
||||
- `test_aliyun_integration.py` - Tests with real Aliyun API calls
|
||||
|
||||
**Setup environment variables:**
|
||||
```bash
|
||||
export ALIYUN_ACCESS_KEY_ID="LTAI5t..."
|
||||
export ALIYUN_ACCESS_KEY_SECRET="..."
|
||||
export ALIYUN_REGION="cn-hangzhou" # Optional, defaults to cn-hangzhou
|
||||
export ALIYUN_WORKSPACE_ID="ws-..." # Optional
|
||||
```
|
||||
|
||||
**Run integration tests:**
|
||||
```bash
|
||||
# Run only integration tests
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v -m integration
|
||||
|
||||
# Run all tests including integration
|
||||
pytest agent/sandbox/tests/ -v
|
||||
|
||||
# Run specific integration test
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check -v
|
||||
```
|
||||
|
||||
### 3. Security Tests
|
||||
|
||||
Security tests validate the security features of the self-managed sandbox provider.
|
||||
|
||||
**Files:**
|
||||
- `sandbox_security_tests_full.py` - Comprehensive security tests
|
||||
|
||||
**Run security tests:**
|
||||
```bash
|
||||
# Run all security tests
|
||||
pytest agent/sandbox/tests/sandbox_security_tests_full.py -v
|
||||
|
||||
# Run specific security test
|
||||
pytest agent/sandbox/tests/sandbox_security_tests_full.py -k "test_dangerous_imports" -v
|
||||
```
|
||||
|
||||
## Test Commands
|
||||
|
||||
### Quick Test Commands
|
||||
|
||||
```bash
|
||||
# Run all sandbox tests (unit only, fast)
|
||||
pytest agent/sandbox/tests/ -v -m "not integration" --tb=short
|
||||
|
||||
# Run tests with coverage
|
||||
pytest agent/sandbox/tests/ -v --cov=agent.sandbox --cov-report=term-missing -m "not integration"
|
||||
|
||||
# Run tests and stop on first failure
|
||||
pytest agent/sandbox/tests/ -v -x -m "not integration"
|
||||
|
||||
# Run tests in parallel (requires pytest-xdist)
|
||||
pytest agent/sandbox/tests/ -v -n auto -m "not integration"
|
||||
```
|
||||
|
||||
### Aliyun Provider Testing
|
||||
|
||||
```bash
|
||||
# 1. Run unit tests (no credentials needed)
|
||||
pytest agent/sandbox/tests/test_aliyun_provider.py -v
|
||||
|
||||
# 2. Set up credentials for integration tests
|
||||
export ALIYUN_ACCESS_KEY_ID="your-key-id"
|
||||
export ALIYUN_ACCESS_KEY_SECRET="your-secret"
|
||||
export ALIYUN_REGION="cn-hangzhou"
|
||||
|
||||
# 3. Run integration tests (makes real API calls)
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v
|
||||
|
||||
# 4. Test specific scenarios
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code -v
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios -v
|
||||
```
|
||||
|
||||
## Understanding Test Results
|
||||
|
||||
### Unit Test Output
|
||||
|
||||
```
|
||||
agent/sandbox/tests/test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_initialize_success PASSED
|
||||
agent/sandbox/tests/test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_create_instance_python PASSED
|
||||
...
|
||||
========================== 48 passed in 2.34s ===========================
|
||||
```
|
||||
|
||||
### Integration Test Output
|
||||
|
||||
```
|
||||
agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check PASSED
|
||||
agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_create_python_instance PASSED
|
||||
agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code PASSED
|
||||
...
|
||||
========================== 10 passed in 15.67s ===========================
|
||||
```
|
||||
|
||||
**Note:** Integration tests will be skipped if credentials are not set:
|
||||
```
|
||||
agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check SKIPPED
|
||||
...
|
||||
========================== 48 skipped, 10 passed in 0.12s ===========================
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Integration Tests Fail
|
||||
|
||||
1. **Check credentials:**
|
||||
```bash
|
||||
echo $ALIYUN_ACCESS_KEY_ID
|
||||
echo $ALIYUN_ACCESS_KEY_SECRET
|
||||
```
|
||||
|
||||
2. **Check network connectivity:**
|
||||
```bash
|
||||
curl -I https://opensandbox.cn-hangzhou.aliyuncs.com
|
||||
```
|
||||
|
||||
3. **Verify permissions:**
|
||||
- Make sure your Aliyun account has OpenSandbox service enabled
|
||||
- Check that your AccessKey has the required permissions
|
||||
|
||||
4. **Check region:**
|
||||
- Verify the region is correct for your account
|
||||
- Try different regions: cn-hangzhou, cn-beijing, cn-shanghai, etc.
|
||||
|
||||
### Tests Timeout
|
||||
|
||||
If tests timeout, increase the timeout in the test configuration or run with a longer timeout:
|
||||
```bash
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v --timeout=60
|
||||
```
|
||||
|
||||
### Mock Tests Fail
|
||||
|
||||
If unit tests fail, it's likely a code issue, not a credentials issue:
|
||||
1. Check the test error message
|
||||
2. Review the code changes
|
||||
3. Run with verbose output: `pytest -vv`
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new providers:
|
||||
|
||||
1. **Create unit tests** in `test_{provider}_provider.py` with mocks
|
||||
2. **Create integration tests** in `test_{provider}_integration.py` with real API calls
|
||||
3. **Add markers** to distinguish test types
|
||||
4. **Update this README** with provider-specific testing instructions
|
||||
|
||||
Example:
|
||||
```python
|
||||
@pytest.mark.integration
|
||||
def test_new_provider_real_api():
|
||||
"""Test with real API calls."""
|
||||
# Your test here
|
||||
```
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
In CI/CD pipelines:
|
||||
|
||||
```yaml
|
||||
# Run unit tests only (fast, no credentials)
|
||||
pytest agent/sandbox/tests/ -v -m "not integration"
|
||||
|
||||
# Run integration tests if credentials available
|
||||
if [ -n "$ALIYUN_ACCESS_KEY_ID" ]; then
|
||||
pytest agent/sandbox/tests/test_aliyun_integration.py -v -m integration
|
||||
fi
|
||||
```
|
||||
19
agent/sandbox/tests/__init__.py
Normal file
19
agent/sandbox/tests/__init__.py
Normal file
@ -0,0 +1,19 @@
|
||||
#
|
||||
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
"""
|
||||
Sandbox provider tests package.
|
||||
"""
|
||||
33
agent/sandbox/tests/pytest.ini
Normal file
33
agent/sandbox/tests/pytest.ini
Normal file
@ -0,0 +1,33 @@
|
||||
[pytest]
|
||||
# Pytest configuration for sandbox tests
|
||||
|
||||
# Test discovery patterns
|
||||
python_files = test_*.py
|
||||
python_classes = Test*
|
||||
python_functions = test_*
|
||||
|
||||
# Markers for different test types
|
||||
markers =
|
||||
integration: Tests that require external services (Aliyun API, etc.)
|
||||
unit: Fast tests that don't require external services
|
||||
slow: Tests that take a long time to run
|
||||
|
||||
# Test paths
|
||||
testpaths = .
|
||||
|
||||
# Minimum version
|
||||
minversion = 7.0
|
||||
|
||||
# Output options
|
||||
addopts =
|
||||
-v
|
||||
--strict-markers
|
||||
--tb=short
|
||||
--disable-warnings
|
||||
|
||||
# Log options
|
||||
log_cli = false
|
||||
log_cli_level = INFO
|
||||
|
||||
# Coverage options (if using pytest-cov)
|
||||
# addopts = --cov=agent.sandbox --cov-report=html --cov-report=term
|
||||
436
agent/sandbox/tests/sandbox_security_tests_full.py
Normal file
436
agent/sandbox/tests/sandbox_security_tests_full.py
Normal file
@ -0,0 +1,436 @@
|
||||
#
|
||||
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
import base64
|
||||
import os
|
||||
import textwrap
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from enum import Enum
|
||||
from typing import Dict, Optional
|
||||
|
||||
import requests
|
||||
from pydantic import BaseModel
|
||||
|
||||
API_URL = os.getenv("SANDBOX_API_URL", "http://localhost:9385/run")
|
||||
TIMEOUT = 15
|
||||
MAX_WORKERS = 5
|
||||
|
||||
|
||||
class ResultStatus(str, Enum):
|
||||
SUCCESS = "success"
|
||||
PROGRAM_ERROR = "program_error"
|
||||
RESOURCE_LIMIT_EXCEEDED = "resource_limit_exceeded"
|
||||
UNAUTHORIZED_ACCESS = "unauthorized_access"
|
||||
RUNTIME_ERROR = "runtime_error"
|
||||
PROGRAM_RUNNER_ERROR = "program_runner_error"
|
||||
|
||||
|
||||
class ResourceLimitType(str, Enum):
|
||||
TIME = "time"
|
||||
MEMORY = "memory"
|
||||
OUTPUT = "output"
|
||||
|
||||
|
||||
class UnauthorizedAccessType(str, Enum):
|
||||
DISALLOWED_SYSCALL = "disallowed_syscall"
|
||||
FILE_ACCESS = "file_access"
|
||||
NETWORK_ACCESS = "network_access"
|
||||
|
||||
|
||||
class RuntimeErrorType(str, Enum):
|
||||
SIGNALLED = "signalled"
|
||||
NONZERO_EXIT = "nonzero_exit"
|
||||
|
||||
|
||||
class ExecutionResult(BaseModel):
|
||||
status: ResultStatus
|
||||
stdout: str
|
||||
stderr: str
|
||||
exit_code: int
|
||||
detail: Optional[str] = None
|
||||
resource_limit_type: Optional[ResourceLimitType] = None
|
||||
unauthorized_access_type: Optional[UnauthorizedAccessType] = None
|
||||
runtime_error_type: Optional[RuntimeErrorType] = None
|
||||
|
||||
|
||||
class TestResult(BaseModel):
|
||||
name: str
|
||||
passed: bool
|
||||
duration: float
|
||||
expected_failure: bool = False
|
||||
result: Optional[ExecutionResult] = None
|
||||
error: Optional[str] = None
|
||||
validation_error: Optional[str] = None
|
||||
|
||||
|
||||
def encode_code(code: str) -> str:
|
||||
return base64.b64encode(code.encode("utf-8")).decode("utf-8")
|
||||
|
||||
|
||||
def execute_single_test(name: str, code: str, language: str, arguments: dict, expect_fail: bool = False) -> TestResult:
|
||||
"""Execute a single test case"""
|
||||
payload = {
|
||||
"code_b64": encode_code(textwrap.dedent(code)),
|
||||
"language": language,
|
||||
"arguments": arguments,
|
||||
}
|
||||
|
||||
test_result = TestResult(name=name, passed=False, duration=0, expected_failure=expect_fail)
|
||||
|
||||
really_processed = False
|
||||
try:
|
||||
while not really_processed:
|
||||
start_time = time.perf_counter()
|
||||
|
||||
resp = requests.post(API_URL, json=payload, timeout=TIMEOUT)
|
||||
resp.raise_for_status()
|
||||
response_data = resp.json()
|
||||
if response_data["exit_code"] == -429: # too many request
|
||||
print(f"[{name}] Reached request limit, retring...")
|
||||
time.sleep(0.5)
|
||||
continue
|
||||
really_processed = True
|
||||
|
||||
print("-------------------")
|
||||
print(f"{name}:\n{response_data}")
|
||||
print("-------------------")
|
||||
|
||||
test_result.duration = time.perf_counter() - start_time
|
||||
test_result.result = ExecutionResult(**response_data)
|
||||
|
||||
# Validate test result expectations
|
||||
validate_test_result(name, expect_fail, test_result)
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
test_result.duration = time.perf_counter() - start_time
|
||||
test_result.error = f"Request failed: {str(e)}"
|
||||
test_result.result = ExecutionResult(
|
||||
status=ResultStatus.PROGRAM_RUNNER_ERROR,
|
||||
stdout="",
|
||||
stderr=str(e),
|
||||
exit_code=-999,
|
||||
detail="request_failed",
|
||||
)
|
||||
|
||||
return test_result
|
||||
|
||||
|
||||
def validate_test_result(name: str, expect_fail: bool, test_result: TestResult):
|
||||
"""Validate if the test result meets expectations"""
|
||||
if not test_result.result:
|
||||
test_result.passed = False
|
||||
test_result.validation_error = "No result returned"
|
||||
return
|
||||
|
||||
test_result.passed = test_result.result.status == ResultStatus.SUCCESS
|
||||
# General validation logic
|
||||
if expect_fail:
|
||||
# Tests expected to fail should return a non-success status
|
||||
if test_result.passed:
|
||||
test_result.validation_error = "Expected failure but actually succeeded"
|
||||
else:
|
||||
# Tests expected to succeed should return a success status
|
||||
if not test_result.passed:
|
||||
test_result.validation_error = f"Unexpected failure (status={test_result.result.status})"
|
||||
|
||||
|
||||
def get_test_cases() -> Dict[str, dict]:
|
||||
"""Return test cases (code, whether expected to fail)"""
|
||||
return {
|
||||
"1 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"2 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"3 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"4 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"5 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"6 Infinite loop: Should be forcibly terminated": {
|
||||
"code": """
|
||||
def main():
|
||||
while True:
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"7 Normal test: Python without dependencies": {
|
||||
"code": """
|
||||
def main():
|
||||
return {"data": "hello, world"}
|
||||
""",
|
||||
"should_fail": False,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"8 Normal test: Python with pandas, should pass without any error": {
|
||||
"code": """
|
||||
import pandas as pd
|
||||
|
||||
def main():
|
||||
data = {'Name': ['Alice', 'Bob', 'Charlie'],
|
||||
'Age': [25, 30, 35]}
|
||||
df = pd.DataFrame(data)
|
||||
""",
|
||||
"should_fail": False,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"9 Normal test: Nodejs without dependencies, should pass without any error": {
|
||||
"code": """
|
||||
const https = require('https');
|
||||
|
||||
async function main(args) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = https.get('https://example.com/', (res) => {
|
||||
let data = '';
|
||||
|
||||
res.on('data', (chunk) => {
|
||||
data += chunk;
|
||||
});
|
||||
|
||||
res.on('end', () => {
|
||||
clearTimeout(timeout);
|
||||
console.log('Body:', data);
|
||||
resolve(data);
|
||||
});
|
||||
});
|
||||
|
||||
const timeout = setTimeout(() => {
|
||||
req.destroy(new Error('Request timeout after 10s'));
|
||||
}, 10000);
|
||||
|
||||
req.on('error', (err) => {
|
||||
clearTimeout(timeout);
|
||||
console.error('Error:', err.message);
|
||||
reject(err);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { main };
|
||||
""",
|
||||
"should_fail": False,
|
||||
"arguments": {},
|
||||
"language": "nodejs",
|
||||
},
|
||||
"10 Normal test: Nodejs with axios, should pass without any error": {
|
||||
"code": """
|
||||
const axios = require('axios');
|
||||
|
||||
async function main(args) {
|
||||
try {
|
||||
const response = await axios.get('https://example.com/', {
|
||||
timeout: 10000
|
||||
});
|
||||
console.log('Body:', response.data);
|
||||
} catch (error) {
|
||||
console.error('Error:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { main };
|
||||
""",
|
||||
"should_fail": False,
|
||||
"arguments": {},
|
||||
"language": "nodejs",
|
||||
},
|
||||
"11 Dangerous import: Should fail due to os module import": {
|
||||
"code": """
|
||||
import os
|
||||
|
||||
def main():
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"12 Dangerous import from subprocess: Should fail due to subprocess import": {
|
||||
"code": """
|
||||
from subprocess import Popen
|
||||
|
||||
def main():
|
||||
pass
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"13 Dangerous call: Should fail due to eval function call": {
|
||||
"code": """
|
||||
def main():
|
||||
eval('os.system("echo hello")')
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"14 Dangerous attribute access: Should fail due to shutil.rmtree": {
|
||||
"code": """
|
||||
import shutil
|
||||
|
||||
def main():
|
||||
shutil.rmtree('/some/path')
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"15 Dangerous binary operation: Should fail due to unsafe concatenation leading to eval": {
|
||||
"code": """
|
||||
def main():
|
||||
dangerous_string = "os." + "system"
|
||||
eval(dangerous_string + '("echo hello")')
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"16 Dangerous function definition: Should fail due to user-defined eval function": {
|
||||
"code": """
|
||||
def eval_function():
|
||||
eval('os.system("echo hello")')
|
||||
|
||||
def main():
|
||||
eval_function()
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
"17 Memory exhaustion(256m): Should fail due to exceeding memory limit(try to allocate 300m)": {
|
||||
"code": """
|
||||
def main():
|
||||
x = ['a' * 1024 * 1024] * 300 # 300MB
|
||||
""",
|
||||
"should_fail": True,
|
||||
"arguments": {},
|
||||
"language": "python",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def print_test_report(results: Dict[str, TestResult]):
|
||||
print("\n=== 🔍 Test Report ===")
|
||||
|
||||
max_name_len = max(len(name) for name in results)
|
||||
|
||||
for name, result in results.items():
|
||||
status = "✅" if result.passed else "❌"
|
||||
if result.expected_failure:
|
||||
status = "⚠️" if result.passed else "✓" # Expected failure case
|
||||
|
||||
print(f"{status} {name.ljust(max_name_len)} {result.duration:.2f}s")
|
||||
|
||||
if result.error:
|
||||
print(f" REQUEST ERROR: {result.error}")
|
||||
if result.validation_error:
|
||||
print(f" VALIDATION ERROR: {result.validation_error}")
|
||||
|
||||
if result.result and not result.passed:
|
||||
print(f" STATUS: {result.result.status}")
|
||||
if result.result.stderr:
|
||||
print(f" STDERR: {result.result.stderr[:200]}...")
|
||||
if result.result.detail:
|
||||
print(f" DETAIL: {result.result.detail}")
|
||||
|
||||
passed = sum(1 for r in results.values() if ((not r.expected_failure and r.passed) or (r.expected_failure and not r.passed)))
|
||||
failed = len(results) - passed
|
||||
|
||||
print("\n=== 📊 Statistics ===")
|
||||
print(f"✅ Passed: {passed}")
|
||||
print(f"❌ Failed: {failed}")
|
||||
print(f"📌 Total: {len(results)}")
|
||||
|
||||
|
||||
def main():
|
||||
print(f"🔐 Starting sandbox security tests (API: {API_URL})")
|
||||
print(f"🚀 Concurrent threads: {MAX_WORKERS}")
|
||||
|
||||
test_cases = get_test_cases()
|
||||
results = {}
|
||||
|
||||
with ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
|
||||
futures = {}
|
||||
for name, detail in test_cases.items():
|
||||
# ✅ Log when a task is submitted
|
||||
print(f"✅ Task submitted: {name}")
|
||||
time.sleep(0.4)
|
||||
future = executor.submit(execute_single_test, name, detail["code"], detail["language"], detail["arguments"], detail["should_fail"])
|
||||
futures[future] = name
|
||||
|
||||
print("\n=== 🚦 Test Progress ===")
|
||||
for i, future in enumerate(as_completed(futures)):
|
||||
name = futures[future]
|
||||
print(f" {i + 1}/{len(test_cases)} completed: {name}")
|
||||
try:
|
||||
results[name] = future.result()
|
||||
except Exception as e:
|
||||
print(f"⚠️ Test {name} execution exception: {str(e)}")
|
||||
results[name] = TestResult(name=name, passed=False, duration=0, error=f"Execution exception: {str(e)}")
|
||||
|
||||
print_test_report(results)
|
||||
|
||||
if any(not r.passed and not r.expected_failure for r in results.values()):
|
||||
exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
329
agent/sandbox/tests/test_aliyun_codeinterpreter.py
Normal file
329
agent/sandbox/tests/test_aliyun_codeinterpreter.py
Normal file
@ -0,0 +1,329 @@
|
||||
#
|
||||
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
"""
|
||||
Unit tests for Aliyun Code Interpreter provider.
|
||||
|
||||
These tests use mocks and don't require real Aliyun credentials.
|
||||
|
||||
Official Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
|
||||
Official SDK: https://github.com/Serverless-Devs/agentrun-sdk-python
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from agent.sandbox.providers.base import SandboxProvider
|
||||
from agent.sandbox.providers.aliyun_codeinterpreter import AliyunCodeInterpreterProvider
|
||||
|
||||
|
||||
class TestAliyunCodeInterpreterProvider:
|
||||
"""Test AliyunCodeInterpreterProvider implementation."""
|
||||
|
||||
def test_provider_initialization(self):
|
||||
"""Test provider initialization."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
assert provider.access_key_id == ""
|
||||
assert provider.access_key_secret == ""
|
||||
assert provider.account_id == ""
|
||||
assert provider.region == "cn-hangzhou"
|
||||
assert provider.template_name == ""
|
||||
assert provider.timeout == 30
|
||||
assert not provider._initialized
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Template")
|
||||
def test_initialize_success(self, mock_template):
|
||||
"""Test successful initialization."""
|
||||
# Mock health check response
|
||||
mock_template.list.return_value = []
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
result = provider.initialize(
|
||||
{
|
||||
"access_key_id": "LTAI5tXXXXXXXXXX",
|
||||
"access_key_secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
|
||||
"account_id": "1234567890123456",
|
||||
"region": "cn-hangzhou",
|
||||
"template_name": "python-sandbox",
|
||||
"timeout": 20,
|
||||
}
|
||||
)
|
||||
|
||||
assert result is True
|
||||
assert provider.access_key_id == "LTAI5tXXXXXXXXXX"
|
||||
assert provider.access_key_secret == "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
|
||||
assert provider.account_id == "1234567890123456"
|
||||
assert provider.region == "cn-hangzhou"
|
||||
assert provider.template_name == "python-sandbox"
|
||||
assert provider.timeout == 20
|
||||
assert provider._initialized
|
||||
|
||||
def test_initialize_missing_credentials(self):
|
||||
"""Test initialization with missing credentials."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
# Missing access_key_id
|
||||
result = provider.initialize({"access_key_secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"})
|
||||
assert result is False
|
||||
|
||||
# Missing access_key_secret
|
||||
result = provider.initialize({"access_key_id": "LTAI5tXXXXXXXXXX"})
|
||||
assert result is False
|
||||
|
||||
# Missing account_id
|
||||
provider2 = AliyunCodeInterpreterProvider()
|
||||
result = provider2.initialize({"access_key_id": "LTAI5tXXXXXXXXXX", "access_key_secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"})
|
||||
assert result is False
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Template")
|
||||
def test_initialize_default_config(self, mock_template):
|
||||
"""Test initialization with default config."""
|
||||
mock_template.list.return_value = []
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
result = provider.initialize({"access_key_id": "LTAI5tXXXXXXXXXX", "access_key_secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", "account_id": "1234567890123456"})
|
||||
|
||||
assert result is True
|
||||
assert provider.region == "cn-hangzhou"
|
||||
assert provider.template_name == ""
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
|
||||
def test_create_instance_python(self, mock_sandbox_class):
|
||||
"""Test creating a Python instance."""
|
||||
# Mock successful instance creation
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.sandbox_id = "01JCED8Z9Y6XQVK8M2NRST5WXY"
|
||||
mock_sandbox_class.return_value = mock_sandbox
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
provider._initialized = True
|
||||
provider._config = MagicMock()
|
||||
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
assert instance.provider == "aliyun_codeinterpreter"
|
||||
assert instance.status == "READY"
|
||||
assert instance.metadata["language"] == "python"
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
|
||||
def test_create_instance_javascript(self, mock_sandbox_class):
|
||||
"""Test creating a JavaScript instance."""
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.sandbox_id = "01JCED8Z9Y6XQVK8M2NRST5WXY"
|
||||
mock_sandbox_class.return_value = mock_sandbox
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
provider._initialized = True
|
||||
provider._config = MagicMock()
|
||||
|
||||
instance = provider.create_instance("javascript")
|
||||
|
||||
assert instance.metadata["language"] == "javascript"
|
||||
|
||||
def test_create_instance_not_initialized(self):
|
||||
"""Test creating instance when provider not initialized."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Provider not initialized"):
|
||||
provider.create_instance("python")
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
|
||||
def test_execute_code_success(self, mock_sandbox_class):
|
||||
"""Test successful code execution."""
|
||||
# Mock sandbox instance
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.context.execute.return_value = {
|
||||
"results": [{"type": "stdout", "text": "Hello, World!"}, {"type": "result", "text": "None"}, {"type": "endOfExecution", "status": "ok"}],
|
||||
"contextId": "kernel-12345-67890",
|
||||
}
|
||||
mock_sandbox_class.return_value = mock_sandbox
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
provider._initialized = True
|
||||
provider._config = MagicMock()
|
||||
|
||||
result = provider.execute_code(instance_id="01JCED8Z9Y6XQVK8M2NRST5WXY", code="print('Hello, World!')", language="python", timeout=10)
|
||||
|
||||
assert result.stdout == "Hello, World!"
|
||||
assert result.stderr == ""
|
||||
assert result.exit_code == 0
|
||||
assert result.execution_time > 0
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
|
||||
def test_execute_code_timeout(self, mock_sandbox_class):
|
||||
"""Test code execution timeout."""
|
||||
from agentrun.utils.exception import ServerError
|
||||
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.context.execute.side_effect = ServerError(408, "Request timeout")
|
||||
mock_sandbox_class.return_value = mock_sandbox
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
provider._initialized = True
|
||||
provider._config = MagicMock()
|
||||
|
||||
with pytest.raises(TimeoutError, match="Execution timed out"):
|
||||
provider.execute_code(instance_id="01JCED8Z9Y6XQVK8M2NRST5WXY", code="while True: pass", language="python", timeout=5)
|
||||
|
||||
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
|
||||
def test_execute_code_with_error(self, mock_sandbox_class):
|
||||
"""Test code execution with error."""
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.context.execute.return_value = {
|
||||
"results": [{"type": "stderr", "text": "Traceback..."}, {"type": "error", "text": "NameError: name 'x' is not defined"}, {"type": "endOfExecution", "status": "error"}]
|
||||
}
|
||||
mock_sandbox_class.return_value = mock_sandbox
|
||||
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
provider._initialized = True
|
||||
provider._config = MagicMock()
|
||||
|
||||
result = provider.execute_code(instance_id="01JCED8Z9Y6XQVK8M2NRST5WXY", code="print(x)", language="python")
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert len(result.stderr) > 0
|
||||
|
||||
def test_get_supported_languages(self):
|
||||
"""Test getting supported languages."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
languages = provider.get_supported_languages()
|
||||
|
||||
assert "python" in languages
|
||||
assert "javascript" in languages
|
||||
|
||||
def test_get_config_schema(self):
|
||||
"""Test getting configuration schema."""
|
||||
schema = AliyunCodeInterpreterProvider.get_config_schema()
|
||||
|
||||
assert "access_key_id" in schema
|
||||
assert schema["access_key_id"]["required"] is True
|
||||
|
||||
assert "access_key_secret" in schema
|
||||
assert schema["access_key_secret"]["required"] is True
|
||||
|
||||
assert "account_id" in schema
|
||||
assert schema["account_id"]["required"] is True
|
||||
|
||||
assert "region" in schema
|
||||
assert "template_name" in schema
|
||||
assert "timeout" in schema
|
||||
|
||||
def test_validate_config_success(self):
|
||||
"""Test successful configuration validation."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
is_valid, error_msg = provider.validate_config({"access_key_id": "LTAI5tXXXXXXXXXX", "account_id": "1234567890123456", "region": "cn-hangzhou"})
|
||||
|
||||
assert is_valid is True
|
||||
assert error_msg is None
|
||||
|
||||
def test_validate_config_invalid_access_key(self):
|
||||
"""Test validation with invalid access key format."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
is_valid, error_msg = provider.validate_config({"access_key_id": "INVALID_KEY"})
|
||||
|
||||
assert is_valid is False
|
||||
assert "AccessKey ID format" in error_msg
|
||||
|
||||
def test_validate_config_missing_account_id(self):
|
||||
"""Test validation with missing account ID."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
is_valid, error_msg = provider.validate_config({})
|
||||
|
||||
assert is_valid is False
|
||||
assert "Account ID" in error_msg
|
||||
|
||||
def test_validate_config_invalid_region(self):
|
||||
"""Test validation with invalid region."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
is_valid, error_msg = provider.validate_config(
|
||||
{
|
||||
"access_key_id": "LTAI5tXXXXXXXXXX",
|
||||
"account_id": "1234567890123456", # Provide required field
|
||||
"region": "us-west-1",
|
||||
}
|
||||
)
|
||||
|
||||
assert is_valid is False
|
||||
assert "Invalid region" in error_msg
|
||||
|
||||
def test_validate_config_invalid_timeout(self):
|
||||
"""Test validation with invalid timeout (> 30 seconds)."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
is_valid, error_msg = provider.validate_config(
|
||||
{
|
||||
"access_key_id": "LTAI5tXXXXXXXXXX",
|
||||
"account_id": "1234567890123456", # Provide required field
|
||||
"timeout": 60,
|
||||
}
|
||||
)
|
||||
|
||||
assert is_valid is False
|
||||
assert "Timeout must be between 1 and 30 seconds" in error_msg
|
||||
|
||||
def test_normalize_language_python(self):
|
||||
"""Test normalizing Python language identifier."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
assert provider._normalize_language("python") == "python"
|
||||
assert provider._normalize_language("python3") == "python"
|
||||
assert provider._normalize_language("PYTHON") == "python"
|
||||
|
||||
def test_normalize_language_javascript(self):
|
||||
"""Test normalizing JavaScript language identifier."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
assert provider._normalize_language("javascript") == "javascript"
|
||||
assert provider._normalize_language("nodejs") == "javascript"
|
||||
assert provider._normalize_language("JavaScript") == "javascript"
|
||||
|
||||
|
||||
class TestAliyunCodeInterpreterInterface:
|
||||
"""Test that Aliyun provider correctly implements the interface."""
|
||||
|
||||
def test_aliyun_provider_is_abstract(self):
|
||||
"""Test that AliyunCodeInterpreterProvider is a SandboxProvider."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
assert isinstance(provider, SandboxProvider)
|
||||
|
||||
def test_aliyun_provider_has_abstract_methods(self):
|
||||
"""Test that AliyunCodeInterpreterProvider implements all abstract methods."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
|
||||
assert hasattr(provider, "initialize")
|
||||
assert callable(provider.initialize)
|
||||
|
||||
assert hasattr(provider, "create_instance")
|
||||
assert callable(provider.create_instance)
|
||||
|
||||
assert hasattr(provider, "execute_code")
|
||||
assert callable(provider.execute_code)
|
||||
|
||||
assert hasattr(provider, "destroy_instance")
|
||||
assert callable(provider.destroy_instance)
|
||||
|
||||
assert hasattr(provider, "health_check")
|
||||
assert callable(provider.health_check)
|
||||
|
||||
assert hasattr(provider, "get_supported_languages")
|
||||
assert callable(provider.get_supported_languages)
|
||||
353
agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py
Normal file
353
agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py
Normal file
@ -0,0 +1,353 @@
|
||||
#
|
||||
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
"""
|
||||
Integration tests for Aliyun Code Interpreter provider.
|
||||
|
||||
These tests require real Aliyun credentials and will make actual API calls.
|
||||
To run these tests, set the following environment variables:
|
||||
|
||||
export AGENTRUN_ACCESS_KEY_ID="LTAI5t..."
|
||||
export AGENTRUN_ACCESS_KEY_SECRET="..."
|
||||
export AGENTRUN_ACCOUNT_ID="1234567890..." # Aliyun primary account ID (主账号ID)
|
||||
export AGENTRUN_REGION="cn-hangzhou" # Note: AGENTRUN_REGION (SDK will read this)
|
||||
|
||||
Then run:
|
||||
pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v
|
||||
|
||||
Official Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
|
||||
"""
|
||||
|
||||
import os
|
||||
import pytest
|
||||
from agent.sandbox.providers.aliyun_codeinterpreter import AliyunCodeInterpreterProvider
|
||||
|
||||
|
||||
# Skip all tests if credentials are not provided
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not all(
|
||||
[
|
||||
os.getenv("AGENTRUN_ACCESS_KEY_ID"),
|
||||
os.getenv("AGENTRUN_ACCESS_KEY_SECRET"),
|
||||
os.getenv("AGENTRUN_ACCOUNT_ID"),
|
||||
]
|
||||
),
|
||||
reason="Aliyun credentials not set. Set AGENTRUN_ACCESS_KEY_ID, AGENTRUN_ACCESS_KEY_SECRET, and AGENTRUN_ACCOUNT_ID.",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def aliyun_config():
|
||||
"""Get Aliyun configuration from environment variables."""
|
||||
return {
|
||||
"access_key_id": os.getenv("AGENTRUN_ACCESS_KEY_ID"),
|
||||
"access_key_secret": os.getenv("AGENTRUN_ACCESS_KEY_SECRET"),
|
||||
"account_id": os.getenv("AGENTRUN_ACCOUNT_ID"),
|
||||
"region": os.getenv("AGENTRUN_REGION", "cn-hangzhou"),
|
||||
"template_name": os.getenv("AGENTRUN_TEMPLATE_NAME", ""),
|
||||
"timeout": 30,
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider(aliyun_config):
|
||||
"""Create an initialized Aliyun provider."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
initialized = provider.initialize(aliyun_config)
|
||||
if not initialized:
|
||||
pytest.skip("Failed to initialize Aliyun provider. Check credentials, account ID, and network.")
|
||||
return provider
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
class TestAliyunCodeInterpreterIntegration:
|
||||
"""Integration tests for Aliyun Code Interpreter provider."""
|
||||
|
||||
def test_initialize_provider(self, aliyun_config):
|
||||
"""Test provider initialization with real credentials."""
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
result = provider.initialize(aliyun_config)
|
||||
|
||||
assert result is True
|
||||
assert provider._initialized is True
|
||||
|
||||
def test_health_check(self, provider):
|
||||
"""Test health check with real API."""
|
||||
result = provider.health_check()
|
||||
|
||||
assert result is True
|
||||
|
||||
def test_get_supported_languages(self, provider):
|
||||
"""Test getting supported languages."""
|
||||
languages = provider.get_supported_languages()
|
||||
|
||||
assert "python" in languages
|
||||
assert "javascript" in languages
|
||||
assert isinstance(languages, list)
|
||||
|
||||
def test_create_python_instance(self, provider):
|
||||
"""Test creating a Python sandbox instance."""
|
||||
try:
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
assert instance.provider == "aliyun_codeinterpreter"
|
||||
assert instance.status in ["READY", "CREATING"]
|
||||
assert instance.metadata["language"] == "python"
|
||||
assert len(instance.instance_id) > 0
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Instance creation failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_execute_python_code(self, provider):
|
||||
"""Test executing Python code in the sandbox."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# Execute simple code
|
||||
result = provider.execute_code(
|
||||
instance_id=instance.instance_id,
|
||||
code="print('Hello from Aliyun Code Interpreter!')\nprint(42)",
|
||||
language="python",
|
||||
timeout=30, # Max 30 seconds
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Hello from Aliyun Code Interpreter!" in result.stdout
|
||||
assert "42" in result.stdout
|
||||
assert result.execution_time > 0
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Code execution test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_execute_python_code_with_arguments(self, provider):
|
||||
"""Test executing Python code with arguments parameter."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# Execute code with arguments
|
||||
result = provider.execute_code(
|
||||
instance_id=instance.instance_id,
|
||||
code="""def main(name: str, count: int) -> dict:
|
||||
return {"message": f"Hello {name}!" * count}
|
||||
""",
|
||||
language="python",
|
||||
timeout=30,
|
||||
arguments={"name": "World", "count": 2}
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Hello World!Hello World!" in result.stdout
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Arguments test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_execute_python_code_with_error(self, provider):
|
||||
"""Test executing Python code that produces an error."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# Execute code with error
|
||||
result = provider.execute_code(instance_id=instance.instance_id, code="raise ValueError('Test error')", language="python", timeout=30)
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert len(result.stderr) > 0 or "ValueError" in result.stdout
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Error handling test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_execute_javascript_code(self, provider):
|
||||
"""Test executing JavaScript code in the sandbox."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("javascript")
|
||||
|
||||
# Execute simple code
|
||||
result = provider.execute_code(instance_id=instance.instance_id, code="console.log('Hello from JavaScript!');", language="javascript", timeout=30)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Hello from JavaScript!" in result.stdout
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"JavaScript execution test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_execute_javascript_code_with_arguments(self, provider):
|
||||
"""Test executing JavaScript code with arguments parameter."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("javascript")
|
||||
|
||||
# Execute code with arguments
|
||||
result = provider.execute_code(
|
||||
instance_id=instance.instance_id,
|
||||
code="""function main(args) {
|
||||
const { name, count } = args;
|
||||
return `Hello ${name}!`.repeat(count);
|
||||
}""",
|
||||
language="javascript",
|
||||
timeout=30,
|
||||
arguments={"name": "World", "count": 2}
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Hello World!Hello World!" in result.stdout
|
||||
|
||||
# Clean up
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"JavaScript arguments test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_destroy_instance(self, provider):
|
||||
"""Test destroying a sandbox instance."""
|
||||
try:
|
||||
# Create instance
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# Destroy instance
|
||||
result = provider.destroy_instance(instance.instance_id)
|
||||
|
||||
# Note: The API might return True immediately or async
|
||||
assert result is True or result is False
|
||||
except Exception as e:
|
||||
pytest.skip(f"Destroy instance test failed: {str(e)}. API might not be available yet.")
|
||||
|
||||
def test_config_validation(self, provider):
|
||||
"""Test configuration validation."""
|
||||
# Valid config
|
||||
is_valid, error = provider.validate_config({"access_key_id": "LTAI5tXXXXXXXXXX", "account_id": "1234567890123456", "region": "cn-hangzhou", "timeout": 30})
|
||||
assert is_valid is True
|
||||
assert error is None
|
||||
|
||||
# Invalid access key
|
||||
is_valid, error = provider.validate_config({"access_key_id": "INVALID_KEY"})
|
||||
assert is_valid is False
|
||||
|
||||
# Missing account ID
|
||||
is_valid, error = provider.validate_config({})
|
||||
assert is_valid is False
|
||||
assert "Account ID" in error
|
||||
|
||||
def test_timeout_limit(self, provider):
|
||||
"""Test that timeout is limited to 30 seconds."""
|
||||
# Timeout > 30 should be clamped to 30
|
||||
provider2 = AliyunCodeInterpreterProvider()
|
||||
provider2.initialize(
|
||||
{
|
||||
"access_key_id": os.getenv("AGENTRUN_ACCESS_KEY_ID"),
|
||||
"access_key_secret": os.getenv("AGENTRUN_ACCESS_KEY_SECRET"),
|
||||
"account_id": os.getenv("AGENTRUN_ACCOUNT_ID"),
|
||||
"timeout": 60, # Request 60 seconds
|
||||
}
|
||||
)
|
||||
|
||||
# Should be clamped to 30
|
||||
assert provider2.timeout == 30
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
class TestAliyunCodeInterpreterScenarios:
|
||||
"""Test real-world usage scenarios."""
|
||||
|
||||
def test_data_processing_workflow(self, provider):
|
||||
"""Test a simple data processing workflow."""
|
||||
try:
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# Execute data processing code
|
||||
code = """
|
||||
import json
|
||||
data = [{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]
|
||||
result = json.dumps(data, indent=2)
|
||||
print(result)
|
||||
"""
|
||||
result = provider.execute_code(instance_id=instance.instance_id, code=code, language="python", timeout=30)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Alice" in result.stdout
|
||||
assert "Bob" in result.stdout
|
||||
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Data processing test failed: {str(e)}")
|
||||
|
||||
def test_string_manipulation(self, provider):
|
||||
"""Test string manipulation operations."""
|
||||
try:
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
code = """
|
||||
text = "Hello, World!"
|
||||
print(text.upper())
|
||||
print(text.lower())
|
||||
print(text.replace("World", "Aliyun"))
|
||||
"""
|
||||
result = provider.execute_code(instance_id=instance.instance_id, code=code, language="python", timeout=30)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "HELLO, WORLD!" in result.stdout
|
||||
assert "hello, world!" in result.stdout
|
||||
assert "Hello, Aliyun!" in result.stdout
|
||||
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"String manipulation test failed: {str(e)}")
|
||||
|
||||
def test_context_persistence(self, provider):
|
||||
"""Test code execution with context persistence."""
|
||||
try:
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
# First execution - define variable
|
||||
result1 = provider.execute_code(instance_id=instance.instance_id, code="x = 42\nprint(x)", language="python", timeout=30)
|
||||
assert result1.exit_code == 0
|
||||
|
||||
# Second execution - use variable
|
||||
# Note: Context persistence depends on whether the contextId is reused
|
||||
result2 = provider.execute_code(instance_id=instance.instance_id, code="print(f'x is {x}')", language="python", timeout=30)
|
||||
|
||||
# Context might or might not persist depending on API implementation
|
||||
assert result2.exit_code == 0
|
||||
|
||||
provider.destroy_instance(instance.instance_id)
|
||||
except Exception as e:
|
||||
pytest.skip(f"Context persistence test failed: {str(e)}")
|
||||
|
||||
|
||||
def test_without_credentials():
|
||||
"""Test that tests are skipped without credentials."""
|
||||
# This test should always run (not skipped)
|
||||
if all(
|
||||
[
|
||||
os.getenv("AGENTRUN_ACCESS_KEY_ID"),
|
||||
os.getenv("AGENTRUN_ACCESS_KEY_SECRET"),
|
||||
os.getenv("AGENTRUN_ACCOUNT_ID"),
|
||||
]
|
||||
):
|
||||
assert True # Credentials are set
|
||||
else:
|
||||
assert True # Credentials not set, test still passes
|
||||
423
agent/sandbox/tests/test_providers.py
Normal file
423
agent/sandbox/tests/test_providers.py
Normal file
@ -0,0 +1,423 @@
|
||||
#
|
||||
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
"""
|
||||
Unit tests for sandbox provider abstraction layer.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch
|
||||
import requests
|
||||
|
||||
from agent.sandbox.providers.base import SandboxProvider, SandboxInstance, ExecutionResult
|
||||
from agent.sandbox.providers.manager import ProviderManager
|
||||
from agent.sandbox.providers.self_managed import SelfManagedProvider
|
||||
|
||||
|
||||
class TestSandboxDataclasses:
|
||||
"""Test sandbox dataclasses."""
|
||||
|
||||
def test_sandbox_instance_creation(self):
|
||||
"""Test SandboxInstance dataclass creation."""
|
||||
instance = SandboxInstance(
|
||||
instance_id="test-123",
|
||||
provider="self_managed",
|
||||
status="running",
|
||||
metadata={"language": "python"}
|
||||
)
|
||||
|
||||
assert instance.instance_id == "test-123"
|
||||
assert instance.provider == "self_managed"
|
||||
assert instance.status == "running"
|
||||
assert instance.metadata == {"language": "python"}
|
||||
|
||||
def test_sandbox_instance_default_metadata(self):
|
||||
"""Test SandboxInstance with None metadata."""
|
||||
instance = SandboxInstance(
|
||||
instance_id="test-123",
|
||||
provider="self_managed",
|
||||
status="running",
|
||||
metadata=None
|
||||
)
|
||||
|
||||
assert instance.metadata == {}
|
||||
|
||||
def test_execution_result_creation(self):
|
||||
"""Test ExecutionResult dataclass creation."""
|
||||
result = ExecutionResult(
|
||||
stdout="Hello, World!",
|
||||
stderr="",
|
||||
exit_code=0,
|
||||
execution_time=1.5,
|
||||
metadata={"status": "success"}
|
||||
)
|
||||
|
||||
assert result.stdout == "Hello, World!"
|
||||
assert result.stderr == ""
|
||||
assert result.exit_code == 0
|
||||
assert result.execution_time == 1.5
|
||||
assert result.metadata == {"status": "success"}
|
||||
|
||||
def test_execution_result_default_metadata(self):
|
||||
"""Test ExecutionResult with None metadata."""
|
||||
result = ExecutionResult(
|
||||
stdout="output",
|
||||
stderr="error",
|
||||
exit_code=1,
|
||||
execution_time=0.5,
|
||||
metadata=None
|
||||
)
|
||||
|
||||
assert result.metadata == {}
|
||||
|
||||
|
||||
class TestProviderManager:
|
||||
"""Test ProviderManager functionality."""
|
||||
|
||||
def test_manager_initialization(self):
|
||||
"""Test ProviderManager initialization."""
|
||||
manager = ProviderManager()
|
||||
|
||||
assert manager.current_provider is None
|
||||
assert manager.current_provider_name is None
|
||||
assert not manager.is_configured()
|
||||
|
||||
def test_set_provider(self):
|
||||
"""Test setting a provider."""
|
||||
manager = ProviderManager()
|
||||
mock_provider = Mock(spec=SandboxProvider)
|
||||
|
||||
manager.set_provider("self_managed", mock_provider)
|
||||
|
||||
assert manager.current_provider == mock_provider
|
||||
assert manager.current_provider_name == "self_managed"
|
||||
assert manager.is_configured()
|
||||
|
||||
def test_get_provider(self):
|
||||
"""Test getting the current provider."""
|
||||
manager = ProviderManager()
|
||||
mock_provider = Mock(spec=SandboxProvider)
|
||||
|
||||
manager.set_provider("self_managed", mock_provider)
|
||||
|
||||
assert manager.get_provider() == mock_provider
|
||||
|
||||
def test_get_provider_name(self):
|
||||
"""Test getting the current provider name."""
|
||||
manager = ProviderManager()
|
||||
mock_provider = Mock(spec=SandboxProvider)
|
||||
|
||||
manager.set_provider("self_managed", mock_provider)
|
||||
|
||||
assert manager.get_provider_name() == "self_managed"
|
||||
|
||||
def test_get_provider_when_not_set(self):
|
||||
"""Test getting provider when none is set."""
|
||||
manager = ProviderManager()
|
||||
|
||||
assert manager.get_provider() is None
|
||||
assert manager.get_provider_name() is None
|
||||
|
||||
|
||||
class TestSelfManagedProvider:
|
||||
"""Test SelfManagedProvider implementation."""
|
||||
|
||||
def test_provider_initialization(self):
|
||||
"""Test provider initialization."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
assert provider.endpoint == "http://localhost:9385"
|
||||
assert provider.timeout == 30
|
||||
assert provider.max_retries == 3
|
||||
assert provider.pool_size == 10
|
||||
assert not provider._initialized
|
||||
|
||||
@patch('requests.get')
|
||||
def test_initialize_success(self, mock_get):
|
||||
"""Test successful initialization."""
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 200
|
||||
mock_get.return_value = mock_response
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
result = provider.initialize({
|
||||
"endpoint": "http://test-endpoint:9385",
|
||||
"timeout": 60,
|
||||
"max_retries": 5,
|
||||
"pool_size": 20
|
||||
})
|
||||
|
||||
assert result is True
|
||||
assert provider.endpoint == "http://test-endpoint:9385"
|
||||
assert provider.timeout == 60
|
||||
assert provider.max_retries == 5
|
||||
assert provider.pool_size == 20
|
||||
assert provider._initialized
|
||||
mock_get.assert_called_once_with("http://test-endpoint:9385/healthz", timeout=5)
|
||||
|
||||
@patch('requests.get')
|
||||
def test_initialize_failure(self, mock_get):
|
||||
"""Test initialization failure."""
|
||||
mock_get.side_effect = Exception("Connection error")
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
result = provider.initialize({"endpoint": "http://invalid:9385"})
|
||||
|
||||
assert result is False
|
||||
assert not provider._initialized
|
||||
|
||||
def test_initialize_default_config(self):
|
||||
"""Test initialization with default config."""
|
||||
with patch('requests.get') as mock_get:
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 200
|
||||
mock_get.return_value = mock_response
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
result = provider.initialize({})
|
||||
|
||||
assert result is True
|
||||
assert provider.endpoint == "http://localhost:9385"
|
||||
assert provider.timeout == 30
|
||||
|
||||
def test_create_instance_python(self):
|
||||
"""Test creating a Python instance."""
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
instance = provider.create_instance("python")
|
||||
|
||||
assert instance.provider == "self_managed"
|
||||
assert instance.status == "running"
|
||||
assert instance.metadata["language"] == "python"
|
||||
assert instance.metadata["endpoint"] == "http://localhost:9385"
|
||||
assert len(instance.instance_id) > 0 # Verify instance_id exists
|
||||
|
||||
def test_create_instance_nodejs(self):
|
||||
"""Test creating a Node.js instance."""
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
instance = provider.create_instance("nodejs")
|
||||
|
||||
assert instance.metadata["language"] == "nodejs"
|
||||
|
||||
def test_create_instance_not_initialized(self):
|
||||
"""Test creating instance when provider not initialized."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Provider not initialized"):
|
||||
provider.create_instance("python")
|
||||
|
||||
@patch('requests.post')
|
||||
def test_execute_code_success(self, mock_post):
|
||||
"""Test successful code execution."""
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 200
|
||||
mock_response.json.return_value = {
|
||||
"status": "success",
|
||||
"stdout": '{"result": 42}',
|
||||
"stderr": "",
|
||||
"exit_code": 0,
|
||||
"time_used_ms": 100.0,
|
||||
"memory_used_kb": 1024.0
|
||||
}
|
||||
mock_post.return_value = mock_response
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
result = provider.execute_code(
|
||||
instance_id="test-123",
|
||||
code="def main(): return {'result': 42}",
|
||||
language="python",
|
||||
timeout=10
|
||||
)
|
||||
|
||||
assert result.stdout == '{"result": 42}'
|
||||
assert result.stderr == ""
|
||||
assert result.exit_code == 0
|
||||
assert result.execution_time > 0
|
||||
assert result.metadata["status"] == "success"
|
||||
assert result.metadata["instance_id"] == "test-123"
|
||||
|
||||
@patch('requests.post')
|
||||
def test_execute_code_timeout(self, mock_post):
|
||||
"""Test code execution timeout."""
|
||||
mock_post.side_effect = requests.Timeout()
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
with pytest.raises(TimeoutError, match="Execution timed out"):
|
||||
provider.execute_code(
|
||||
instance_id="test-123",
|
||||
code="while True: pass",
|
||||
language="python",
|
||||
timeout=5
|
||||
)
|
||||
|
||||
@patch('requests.post')
|
||||
def test_execute_code_http_error(self, mock_post):
|
||||
"""Test code execution with HTTP error."""
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 500
|
||||
mock_response.text = "Internal Server Error"
|
||||
mock_post.return_value = mock_response
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
with pytest.raises(RuntimeError, match="HTTP 500"):
|
||||
provider.execute_code(
|
||||
instance_id="test-123",
|
||||
code="invalid code",
|
||||
language="python"
|
||||
)
|
||||
|
||||
def test_execute_code_not_initialized(self):
|
||||
"""Test executing code when provider not initialized."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Provider not initialized"):
|
||||
provider.execute_code(
|
||||
instance_id="test-123",
|
||||
code="print('hello')",
|
||||
language="python"
|
||||
)
|
||||
|
||||
def test_destroy_instance(self):
|
||||
"""Test destroying an instance (no-op for self-managed)."""
|
||||
provider = SelfManagedProvider()
|
||||
provider._initialized = True
|
||||
|
||||
# For self-managed, destroy_instance is a no-op
|
||||
result = provider.destroy_instance("test-123")
|
||||
|
||||
assert result is True
|
||||
|
||||
@patch('requests.get')
|
||||
def test_health_check_success(self, mock_get):
|
||||
"""Test successful health check."""
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 200
|
||||
mock_get.return_value = mock_response
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
result = provider.health_check()
|
||||
|
||||
assert result is True
|
||||
mock_get.assert_called_once_with("http://localhost:9385/healthz", timeout=5)
|
||||
|
||||
@patch('requests.get')
|
||||
def test_health_check_failure(self, mock_get):
|
||||
"""Test health check failure."""
|
||||
mock_get.side_effect = Exception("Connection error")
|
||||
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
result = provider.health_check()
|
||||
|
||||
assert result is False
|
||||
|
||||
def test_get_supported_languages(self):
|
||||
"""Test getting supported languages."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
languages = provider.get_supported_languages()
|
||||
|
||||
assert "python" in languages
|
||||
assert "nodejs" in languages
|
||||
assert "javascript" in languages
|
||||
|
||||
def test_get_config_schema(self):
|
||||
"""Test getting configuration schema."""
|
||||
schema = SelfManagedProvider.get_config_schema()
|
||||
|
||||
assert "endpoint" in schema
|
||||
assert schema["endpoint"]["type"] == "string"
|
||||
assert schema["endpoint"]["required"] is True
|
||||
assert schema["endpoint"]["default"] == "http://localhost:9385"
|
||||
|
||||
assert "timeout" in schema
|
||||
assert schema["timeout"]["type"] == "integer"
|
||||
assert schema["timeout"]["default"] == 30
|
||||
|
||||
assert "max_retries" in schema
|
||||
assert schema["max_retries"]["type"] == "integer"
|
||||
|
||||
assert "pool_size" in schema
|
||||
assert schema["pool_size"]["type"] == "integer"
|
||||
|
||||
def test_normalize_language_python(self):
|
||||
"""Test normalizing Python language identifier."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
assert provider._normalize_language("python") == "python"
|
||||
assert provider._normalize_language("python3") == "python"
|
||||
assert provider._normalize_language("PYTHON") == "python"
|
||||
assert provider._normalize_language("Python3") == "python"
|
||||
|
||||
def test_normalize_language_javascript(self):
|
||||
"""Test normalizing JavaScript language identifier."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
assert provider._normalize_language("javascript") == "nodejs"
|
||||
assert provider._normalize_language("nodejs") == "nodejs"
|
||||
assert provider._normalize_language("JavaScript") == "nodejs"
|
||||
assert provider._normalize_language("NodeJS") == "nodejs"
|
||||
|
||||
def test_normalize_language_default(self):
|
||||
"""Test language normalization with empty/unknown input."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
assert provider._normalize_language("") == "python"
|
||||
assert provider._normalize_language(None) == "python"
|
||||
assert provider._normalize_language("unknown") == "unknown"
|
||||
|
||||
|
||||
class TestProviderInterface:
|
||||
"""Test that providers correctly implement the interface."""
|
||||
|
||||
def test_self_managed_provider_is_abstract(self):
|
||||
"""Test that SelfManagedProvider is a SandboxProvider."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
assert isinstance(provider, SandboxProvider)
|
||||
|
||||
def test_self_managed_provider_has_abstract_methods(self):
|
||||
"""Test that SelfManagedProvider implements all abstract methods."""
|
||||
provider = SelfManagedProvider()
|
||||
|
||||
# Check all abstract methods are implemented
|
||||
assert hasattr(provider, 'initialize')
|
||||
assert callable(provider.initialize)
|
||||
|
||||
assert hasattr(provider, 'create_instance')
|
||||
assert callable(provider.create_instance)
|
||||
|
||||
assert hasattr(provider, 'execute_code')
|
||||
assert callable(provider.execute_code)
|
||||
|
||||
assert hasattr(provider, 'destroy_instance')
|
||||
assert callable(provider.destroy_instance)
|
||||
|
||||
assert hasattr(provider, 'health_check')
|
||||
assert callable(provider.health_check)
|
||||
|
||||
assert hasattr(provider, 'get_supported_languages')
|
||||
assert callable(provider.get_supported_languages)
|
||||
78
agent/sandbox/tests/verify_sdk.py
Normal file
78
agent/sandbox/tests/verify_sdk.py
Normal file
@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Quick verification script for Aliyun Code Interpreter provider using official SDK.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
print("=" * 60)
|
||||
print("Aliyun Code Interpreter Provider - SDK Verification")
|
||||
print("=" * 60)
|
||||
|
||||
# Test 1: Import provider
|
||||
print("\n[1/5] Testing provider import...")
|
||||
try:
|
||||
from agent.sandbox.providers.aliyun_codeinterpreter import AliyunCodeInterpreterProvider
|
||||
|
||||
print("✓ Provider imported successfully")
|
||||
except ImportError as e:
|
||||
print(f"✗ Import failed: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Test 2: Check provider class
|
||||
print("\n[2/5] Testing provider class...")
|
||||
provider = AliyunCodeInterpreterProvider()
|
||||
assert hasattr(provider, "initialize")
|
||||
assert hasattr(provider, "create_instance")
|
||||
assert hasattr(provider, "execute_code")
|
||||
assert hasattr(provider, "destroy_instance")
|
||||
assert hasattr(provider, "health_check")
|
||||
print("✓ Provider has all required methods")
|
||||
|
||||
# Test 3: Check SDK imports
|
||||
print("\n[3/5] Testing SDK imports...")
|
||||
try:
|
||||
# Check if agentrun SDK is available using importlib
|
||||
if (
|
||||
importlib.util.find_spec("agentrun.sandbox") is None
|
||||
or importlib.util.find_spec("agentrun.utils.config") is None
|
||||
or importlib.util.find_spec("agentrun.utils.exception") is None
|
||||
):
|
||||
raise ImportError("agentrun SDK not found")
|
||||
|
||||
# Verify imports work (assign to _ to indicate they're intentionally unused)
|
||||
from agentrun.sandbox import CodeInterpreterSandbox, TemplateType, CodeLanguage
|
||||
from agentrun.utils.config import Config
|
||||
from agentrun.utils.exception import ServerError
|
||||
_ = (CodeInterpreterSandbox, TemplateType, CodeLanguage, Config, ServerError)
|
||||
|
||||
print("✓ SDK modules imported successfully")
|
||||
except ImportError as e:
|
||||
print(f"✗ SDK import failed: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Test 4: Check config schema
|
||||
print("\n[4/5] Testing configuration schema...")
|
||||
schema = AliyunCodeInterpreterProvider.get_config_schema()
|
||||
required_fields = ["access_key_id", "access_key_secret", "account_id"]
|
||||
for field in required_fields:
|
||||
assert field in schema
|
||||
assert schema[field]["required"] is True
|
||||
print(f"✓ All required fields present: {', '.join(required_fields)}")
|
||||
|
||||
# Test 5: Check supported languages
|
||||
print("\n[5/5] Testing supported languages...")
|
||||
languages = provider.get_supported_languages()
|
||||
assert "python" in languages
|
||||
assert "javascript" in languages
|
||||
print(f"✓ Supported languages: {', '.join(languages)}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("All verification tests passed! ✓")
|
||||
print("=" * 60)
|
||||
print("\nNote: This provider now uses the official agentrun-sdk.")
|
||||
print("SDK Documentation: https://github.com/Serverless-Devs/agentrun-sdk-python")
|
||||
print("API Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter")
|
||||
Reference in New Issue
Block a user