Commit Graph

64 Commits

Author SHA1 Message Date
0b40eb3e90 Test: Add tests for chunk API endpoints (#8616)
### What problem does this PR solve?

- Add comprehensive test suite for chunk operations including:
  - Test files for create, list, retrieve, update, and delete chunks
  - Authorization tests
  - Batch operations tests
- Update test configurations and common utilities
- Validate `important_kwd` and `question_kwd` fields are lists in
chunk_app.py
- Reorganize imports and clean up duplicate code

### Type of change

- [x] Add test cases
2025-07-02 09:49:08 +08:00
d11cfd4e45 Fix: Add input validation to chunk creation endpoint (#8516)
### What problem does this PR solve?

- Include optional `tag_feas` field if present in request
- Add input validation for `important_kwd` and `question_kwd` to ensure
they are lists
- #8462

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-26 17:46:00 +08:00
d2b346cf9e Feat: When Delete Chunk Will Also Delete Chunk Related Image (#7656)
### What problem does this PR solve?

When Delete Chunk Will Also Delete Chunk Related Image

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2025-05-15 11:53:47 +08:00
2ccec93d71 Feat: support cross-lang search. (#7557)
### What problem does this PR solve?

#7376
#4503
#5710 
#7470

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-09 15:32:02 +08:00
4f40f685d9 Code refactor (#5371)
### What problem does this PR solve?

#5173

### Type of change

- [x] Refactoring
2025-02-26 15:40:52 +08:00
11e6d84d46 Fix: 'Chunk not found!' error in team-sharing knowledge base. (#5361)
### What problem does this PR solve?

As issue #3268 mentioned, "Chun not found!" exception will occur,
especially during the teamwork of knowledge bases.

### The reason of this bug

"tenants" are the people on current_user's team, including the team
owner itself. The old one only checks the first "tenant", tenants[0],
which will cause error when anyone editing the chunk that is not in
tenants[0]'s knowledge base.

My modification won't introduce new errors while iterate all the tenant
then retrieve knowledge bases of each.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 10:24:35 +08:00
dd0ebbea35 Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
7d909d4d1b Add doc meta data. (#4442)
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-10 19:06:59 +08:00
c5da3cdd97 Tagging (#4426)
### What problem does this PR solve?

#4367

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-09 17:07:21 +08:00
d42e78bce2 Fix bugs in chunk api (#4293)
### What problem does this PR solve?

Fix bugs in chunk api #4149

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-12-30 19:01:44 +08:00
a3eeb5de32 Fix: Q&A chunk modification (#4227)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-25 19:11:16 +08:00
c8b1a564aa Replaced md5 with xxhash64 for chunk id (#4009)
### What problem does this PR solve?

Replaced md5 with xxhash64 for chunk id

### Type of change

- [x] Refactoring
2024-12-12 17:47:39 +08:00
03f00c9e6f Rename page_num_list, top_list, position_list (#3940)
### What problem does this PR solve?

Rename page_num_list, top_list, position_list to page_num_int, top_int,
position_int

### Type of change

- [x] Refactoring
2024-12-10 16:32:58 +08:00
1254ecf445 Added static check at PR CI (#3921)
### What problem does this PR solve?

Added static check at PR CI

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2024-12-08 21:23:51 +08:00
0d68a6cd1b Fix errors detected by Ruff (#3918)
### What problem does this PR solve?

Fix errors detected by Ruff

### Type of change

- [x] Refactoring
2024-12-08 14:21:12 +08:00
8037dc7b76 Fix chunk available issue (#3877)
### What problem does this PR solve?

Close #3873

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-05 16:49:43 +08:00
56f473b680 Feat: Add question parameter to edit chunk modal (#3875)
### What problem does this PR solve?

Close #3873

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-05 14:51:19 +08:00
5ae33184d5 Fix chunk position issue. (#3867)
### What problem does this PR solve?

Close #3864

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-05 10:53:26 +08:00
74b28ef1b0 Add pagerank to KB. (#3809)
### What problem does this PR solve?

#3794

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-03 14:30:35 +08:00
d00297a763 Fix chunk creation using Infinity (#3763)
### What problem does this PR solve?

1. Store error type in Infinity
2. position list value read from Infinity isn't correct.

Fix issue: #3729

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-30 00:10:14 +08:00
91f1814a87 Fix error response (#3719)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-11-28 18:56:10 +08:00
bc701d7b4c Edit chunk shall update instead of insert it (#3709)
### What problem does this PR solve?

Edit chunk shall update instead of insert it. Close #3679 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-28 13:00:38 +08:00
9f3141804f Fix chunk enable/disable issue (#3579)
### What problem does this PR solve?

#3576

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-22 12:25:42 +08:00
8930bfcff8 Fix bugs (#3535)
### What problem does this PR solve?

1. system monitor icon and text missing
2. Team knowledge base can't be search

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-20 20:55:42 +08:00
dec9b3e540 Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
1e90a1bf36 Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
4caf932808 fix bug about fetching knowledge graph (#3394)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 12:29:15 +08:00
f4c52371ab Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
185c6a0c71 Unified API response json schema (#3170)
### What problem does this PR solve?

Unified API response json schema

### Type of change

- [x] Refactoring
2024-11-05 11:02:31 +08:00
89d5b2414e fix SILICONFLOW rerank error (#2980)
### What problem does this PR solve?

#2977

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-23 10:12:39 +08:00
226bdd6e99 add auto keywords and auto-question (#2965)
### What problem does this PR solve?

#2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 13:12:49 +08:00
01acc3fd5a fix duplicated llm name betweeen different suppliers (#2477)
### What problem does this PR solve?

#2465

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-18 16:09:22 +08:00
4730145696 debug backend API for TAB 'search' (#2389)
### What problem does this PR solve?
#2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:51:20 +08:00
333608a1d4 add search TAB backend api (#2375)
### What problem does this PR solve?
 #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 19:49:18 +08:00
6b3a40be5c Format file format from Windows/dos to Unix (#1949)
### What problem does this PR solve?

Related source file is in Windows/DOS format, they are format to Unix
format.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-08-15 09:17:36 +08:00
152072f900 Add graphrag (#1793)
### What problem does this PR solve?

#1594

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-02 18:51:14 +08:00
264303ba98 fix delete selected chunks display wrong (#1612)
### What problem does this PR solve?
This PR solves the problem that when you delete selected chunks, the
chunks can be deleted but Chunk Number doesn't change. Now you delete
one chunk, the Chunk Number is reduced by one, delete two chunks, the
Chunk Number is reduced by two...
#900


### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: seaver <zhudan187@qq.com>
2024-07-19 16:25:50 +08:00
c0090a1b4f fix function create to slove chunk number wrong (#1604)
### What problem does this PR solve?
fix function create to slove the problem of creating a chunk and
increasing the chunk number by 2.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue) #900

---------

Signed-off-by: seaver <zhudan187@qq.com>
2024-07-19 13:36:01 +08:00
92e9320657 upgrade laws parser of docx (#1332)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-07-01 15:50:24 +08:00
e99e8b93fb fix:failed to Create new Chunk in database #1159 (#1214)
### What problem does this PR solve?

fix bug: [1159](https://github.com/infiniflow/ragflow/issues/1159)
using embd which user configured at knowledgebase when create new chunk
in database

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-20 09:43:39 +08:00
0b90aab22c fix: using embd which user configured at knowledgebase (#1163)
### What problem does this PR solve?
as title
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-19 14:44:25 +08:00
614defec21 add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
46454362d7 fix raptor bugs (#928)
### What problem does this PR solve?

#922 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 11:01:20 +08:00
20b57144b0 syntax error (#924)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 08:20:32 +08:00
d7744f5870 Refactor method name (#760)
### What problem does this PR solve?

#757

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-14 14:48:15 +08:00
8c07992b6c refine code (#595)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 19:13:33 +08:00
9d60a84958 refactor code (#583)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 13:19:54 +08:00
890561703b Add bce-embedding and fastembed (#383)
### What problem does this PR solve?


Issue link:#326

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-16 16:42:19 +08:00
c39b751600 conversation API backend update (#360)
### What problem does this PR solve?


Issue link:#345

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-15 14:43:44 +08:00
bc6828c45f add quota config in dialog conf (#339)
### What problem does this PR solve?

Issue link:#337

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-12 14:58:03 +08:00