Commit Graph

75 Commits

Author SHA1 Message Date
c813c1ff4c Made task_executor async to speedup parsing (#5530)
### What problem does this PR solve?

Made task_executor async to speedup parsing

### Type of change

- [x] Performance Improvement
2025-03-03 18:59:49 +08:00
1a41b92f77 More robust community report. (#5328)
### What problem does this PR solve?

#5289
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-25 12:58:10 +08:00
ecf5f6976f Make node merging parallel. (#5324)
### What problem does this PR solve?

#5314

### Type of change

- [x] Performance Improvement
2025-02-25 12:02:44 +08:00
39b96849a9 Fix window size issue of ES. (#5175)
### What problem does this PR solve?

#5152

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-20 12:54:29 +08:00
ef95f08c48 Remove redandent code. (#5121)
### What problem does this PR solve?

#5107

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-19 15:46:52 +08:00
84b4b38cbb Remove <think> for exeSql component. (#5069)
### What problem does this PR solve?

#5061
#5067

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-18 13:39:37 +08:00
f46448d04c Remove <think> for KG extraction. (#5027)
### What problem does this PR solve?

#4946

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 14:06:06 +08:00
7c90b87715 Fix window size of ES issue. (#5026)
### What problem does this PR solve?

#5015

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 12:48:56 +08:00
f29da49893 Fix keyerror issue while rebuilding graph. (#5022)
### What problem does this PR solve?

#4995

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 12:02:44 +08:00
1287558f24 Fix xinference chat role order issue. (#4898)
### What problem does this PR solve?

#4831

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-12 13:15:23 +08:00
0d3ed37b48 Make the update script shorter. (#4854)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-10 18:18:49 +08:00
448fa1c4d4 Robust for abnormal response from LLMs. (#4747)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-06 17:34:53 +08:00
2a07eb69a7 Fix too long context issue. (#4735)
### What problem does this PR solve?

#4728

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-06 11:37:23 +08:00
283d036cba Fitin for infinity. (#4722)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-02-05 16:47:05 +08:00
6f30397bb5 Infinity adapt to graphrag. (#4663)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-27 18:35:18 +08:00
42f7261509 Fix param error. (#4645)
### What problem does this PR solve?

#4633

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-26 10:40:02 +08:00
71c132f76d Make infinity adapt (#4635)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-24 17:45:04 +08:00
86892959a0 Rebuild graph when it's out of time. (#4607)
### What problem does this PR solve?

#4543

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-01-23 17:26:20 +08:00
dd0ebbea35 Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
c5da3cdd97 Tagging (#4426)
### What problem does this PR solve?

#4367

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-09 17:07:21 +08:00
8fb18f37f6 Code refactor. (#4291)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-12-30 18:38:51 +08:00
b635002666 Fix duplicated communitiy (#4187)
### What problem does this PR solve?

#4180

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-23 17:07:12 +08:00
6379a934ff Fix redis get error. (#4140)
### What problem does this PR solve?

#4126
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-20 10:39:50 +08:00
fddac1345d Fix raptor resuable issue. (#4063)
### What problem does this PR solve?

#4045

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-17 15:28:35 +08:00
cb6e9ce164 Cache the result from llm for graphrag and raptor (#4051)
### What problem does this PR solve?

#4045

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-17 09:48:03 +08:00
03f00c9e6f Rename page_num_list, top_list, position_list (#3940)
### What problem does this PR solve?

Rename page_num_list, top_list, position_list to page_num_int, top_int,
position_int

### Type of change

- [x] Refactoring
2024-12-10 16:32:58 +08:00
1254ecf445 Added static check at PR CI (#3921)
### What problem does this PR solve?

Added static check at PR CI

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2024-12-08 21:23:51 +08:00
0d68a6cd1b Fix errors detected by Ruff (#3918)
### What problem does this PR solve?

Fix errors detected by Ruff

### Type of change

- [x] Refactoring
2024-12-08 14:21:12 +08:00
90ffcb4ddb Fix graphrag + infinity bugs (#3681)
### What problem does this PR solve?

Fix graphrag + infinity bugs

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 12:45:43 +08:00
0891a393d7 Let ThreadPool exit gracefully. (#3653)
### What problem does this PR solve?

#3646

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-26 16:31:07 +08:00
58a2200b80 Fix: search issue for graphrag (#3546)
### What problem does this PR solve?

#3538

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-21 11:13:29 +08:00
2044bb0039 Fix bugs (#3502)
### What problem does this PR solve?

1. Remove unused code
2. Fix type mismatch, in nlp search and infinity search interface
3. Fix chunk list, get all chunks of this user.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-20 20:52:23 +08:00
bcb6f7168f [bug]import typing.cast for leiden.py (#3493)
### What problem does this PR solve?

leiden alg throws exception for lack func cast definition

### Type of change

- Bug Fix (non-breaking change which fixes an issue)
2024-11-19 18:42:30 +08:00
dec9b3e540 Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
4413683898 Introduced beartype (#3460)
### What problem does this PR solve?

Introduced [beartype](https://github.com/beartype/beartype) for runtime
type-checking.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 17:38:17 +08:00
a1d01a1b2f enlarge the default token length of RAPTOR summarization (#3454)
### What problem does this PR solve?

#3426

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 10:15:26 +08:00
1e90a1bf36 Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
30f6421760 Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?

Use consistent log file names, introduced initLogger

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-14 17:13:48 +08:00
a2a5631da4 Rework logging (#3358)
Unified all log files into one.

### What problem does this PR solve?

Unified all log files into one.

### Type of change

- [x] Refactoring
2024-11-12 17:35:13 +08:00
f4c52371ab Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
88072b1e90 fix: double brace issue (#3328)
### What problem does this PR solve?

#3299

### Type of change

- [x] Performance Improvement
2024-11-11 12:07:02 +08:00
ff2bbb487f fix index of range (#3279)
### What problem does this PR solve?

#3273

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 09:58:11 +08:00
8d9238db14 fix es search parameter error (#3169)
### What problem does this PR solve?

#3151

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-04 09:53:41 +08:00
ac26d09a59 Feature/feat1017 (#2872)
### What problem does this PR solve?

1. fix: mid map show error in knowledge graph, juse because
```@antv/g6```version changed
2. feat: concurrent threads configuration support in graph extractor
3. fix: used tokens update failed for tenant
4. feat: timeout configuration support for llm
5. fix: regex error in graph extractor
6. feat: qwen rerank(```gte-rerank```) support
7. fix: timeout deal in knowledge graph index process. Now chat by
stream output, also, it is configuratable.
8. feat: ```qwen-long``` model configuration

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-21 12:11:08 +08:00
d6db805885 Refactoring entity_resolution (#2692)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-09-30 17:18:09 +08:00
ae5a877ed4 Simplify the usage of dict (#2681)
### What problem does this PR solve?
Simplify the usage of dictionaries

### Type of change

- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-30 16:54:25 +08:00
9945988e44 format mind_map_extractor code (#2686)
### What problem does this PR solve?

format mind_map_extractor code

### Type of change

- [x] Refactoring
2024-09-30 16:29:15 +08:00
604061c4a5 Fix mutable default argument (#2635)
### What problem does this PR solve?

The default value of Python function parameters cannot be mutable.
Modifying this parameter inside the function will permanently change the
default value

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-29 10:16:00 +08:00
34abcf7704 style: fix typo and format code (#2618)
### What problem does this PR solve?

- Fix typo
- Remove unused import
- Format code

### Type of change

- [x] Other (please describe): typo and format
2024-09-27 13:17:25 +08:00
4c0b79c4f6 remove repeat func (#2619)
### What problem does this PR solve?

- remove repeat func

### Type of change

- [x] Other (please describe): remove repeat func
2024-09-27 13:15:26 +08:00