mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-01-31 07:36:46 +08:00
feat: Add OceanBase Performance Monitoring and Health Check Integration (#12886)
## Description This PR implements comprehensive OceanBase performance monitoring and health check functionality as requested in issue #12772. The implementation follows the existing ES/Infinity health check patterns and provides detailed metrics for operations teams. ## Problem Currently, RAGFlow lacks detailed health monitoring for OceanBase when used as the document engine. Operations teams need visibility into: - Connection status and latency - Storage space usage - Query throughput (QPS) - Slow query statistics - Connection pool utilization ## Solution ### 1. Enhanced OBConnection Class (`rag/utils/ob_conn.py`) Added comprehensive performance monitoring methods: - `get_performance_metrics()` - Main method returning all performance metrics - `_get_storage_info()` - Retrieves database storage usage - `_get_connection_pool_stats()` - Gets connection pool statistics - `_get_slow_query_count()` - Counts queries exceeding threshold - `_estimate_qps()` - Estimates queries per second - Enhanced `health()` method with connection status ### 2. Health Check Utilities (`api/utils/health_utils.py`) Added two new functions following ES/Infinity patterns: - `get_oceanbase_status()` - Returns OceanBase status with health and performance metrics - `check_oceanbase_health()` - Comprehensive health check with detailed metrics ### 3. API Endpoint (`api/apps/system_app.py`) Added new endpoint: - `GET /v1/system/oceanbase/status` - Returns OceanBase health status and performance metrics ### 4. Comprehensive Unit Tests (`test/unit_test/utils/test_oceanbase_health.py`) Added 340+ lines of unit tests covering: - Health check success/failure scenarios - Performance metrics retrieval - Error handling and edge cases - Connection pool statistics - Storage information retrieval - QPS estimation - Slow query detection ## Metrics Provided - **Connection Status**: connected/disconnected - **Latency**: Query latency in milliseconds - **Storage**: Used and total storage space - **QPS**: Estimated queries per second - **Slow Queries**: Count of queries exceeding threshold - **Connection Pool**: Active connections, max connections, pool size ## Testing - All unit tests pass - Error handling tested for connection failures - Edge cases covered (missing tables, connection errors) - Follows existing code patterns and conventions ## Code Statistics - **Total Lines Changed**: 665+ lines - **New Code**: ~600 lines - **Test Coverage**: 340+ lines of comprehensive tests - **Files Modified**: 3 - **Files Created**: 1 (test file) ## Acceptance Criteria Met ✅ `/system/oceanbase/status` API returns OceanBase health status ✅ Monitoring metrics accurately reflect OceanBase running status ✅ Clear error messages when health checks fail ✅ Response time optimized (metrics cached where possible) ✅ Follows existing ES/Infinity health check patterns ✅ Comprehensive test coverage ## Related Files - `rag/utils/ob_conn.py` - OceanBase connection class - `api/utils/health_utils.py` - Health check utilities - `api/apps/system_app.py` - System API endpoints - `test/unit_test/utils/test_oceanbase_health.py` - Unit tests Fixes #12772 --------- Co-authored-by: Daniel <daniel@example.com>
This commit is contained in:
20
.github/workflows/tests.yml
vendored
20
.github/workflows/tests.yml
vendored
@ -336,9 +336,13 @@ jobs:
|
||||
- name: Collect ragflow log
|
||||
if: ${{ !cancelled() }}
|
||||
run: |
|
||||
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-es
|
||||
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log
|
||||
sudo rm -rf docker/ragflow-logs
|
||||
if [ -d docker/ragflow-logs ]; then
|
||||
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-es
|
||||
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
|
||||
else
|
||||
echo "No docker/ragflow-logs directory found; skipping log collection"
|
||||
fi
|
||||
sudo rm -rf docker/ragflow-logs || true
|
||||
|
||||
- name: Stop ragflow:nightly
|
||||
if: always() # always run this step even if previous steps failed
|
||||
@ -482,9 +486,13 @@ jobs:
|
||||
- name: Collect ragflow log
|
||||
if: ${{ !cancelled() }}
|
||||
run: |
|
||||
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-infinity
|
||||
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log
|
||||
sudo rm -rf docker/ragflow-logs
|
||||
if [ -d docker/ragflow-logs ]; then
|
||||
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-infinity
|
||||
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
|
||||
else
|
||||
echo "No docker/ragflow-logs directory found; skipping log collection"
|
||||
fi
|
||||
sudo rm -rf docker/ragflow-logs || true
|
||||
- name: Stop ragflow:nightly
|
||||
if: always() # always run this step even if previous steps failed
|
||||
run: |
|
||||
|
||||
Reference in New Issue
Block a user