pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 12:32:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,580 Commits 3 Branches 38 Tags
918d5a9ff8cafe6ffe2280ad68354b2bb794a773
Commit Graph

242 Commits

Author SHA1 Message Date
Billy Bao
cf7fdd274b Feat: add gmail connector (#11549) 
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-28 13:09:40 +08:00
Levi
12979a3f21 feat: improve metadata handling in connector service (#11421) 
### What problem does this PR solve?

- Update sync data source to handle metadata properly

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
 2025-11-26 19:55:48 +08:00
Jonah Hartmann
2fd5ac1031 Feat: Add Webdav storage as data source (#11422) 
### What problem does this PR solve?

This PR adds webdav storage as data source for data sync service.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-26 14:14:42 +08:00
Yongteng Lei
d1744aaaf3 Feat: add datasource Dropbox (#11488) 
### What problem does this PR solve?

Add datasource Dropbox.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-25 09:40:03 +08:00
Levi
f0a14f5fce Add Moodle data source integration (#11325) 
### What problem does this PR solve?

This PR adds a native Moodle connector to sync content (courses,
resources, forums, assignments, pages, books) into RAGFlow.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
 2025-11-21 19:58:49 +08:00
Yongteng Lei
13e212c856 Feat: add Jira connector (#11285) 
### What problem does this PR solve?

Add Jira connector.

<img width="978" height="925" alt="image"
src="https://github.com/user-attachments/assets/78bb5c77-2710-4569-a76e-9087ca23b227"
/>

---

<img width="1903" height="489" alt="image"
src="https://github.com/user-attachments/assets/193bc5c5-f751-4bd5-883a-2173282c2b96"
/>

---

<img width="1035" height="925" alt="image"
src="https://github.com/user-attachments/assets/1a0aec19-30eb-4ada-9283-61d1c915f59d"
/>

---

<img width="1905" height="601" alt="image"
src="https://github.com/user-attachments/assets/3dde1062-3f27-4717-8e09-fd5fd5e64171"
/>

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-17 09:38:04 +08:00
buua436
e8f1a245a6 Feat:update check_embedding api (#11254) 
### What problem does this PR solve?
pr: 
#10854
change:
update check_embedding api

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-13 18:48:25 +08:00
Yongteng Lei
908450509f Feat: add fault-tolerant mechanism to RAPTOR (#11206) 
### What problem does this PR solve?

Add fault-tolerant mechanism to RAPTOR.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-13 18:48:07 +08:00
Jin Hai
296476ab89 Refactor function name (#11210) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-12 19:00:15 +08:00
Yongteng Lei
8ae562504b Fix: GraphRAG and RAPTOR tasks do not affect document status (#11194) 
### What problem does this PR solve?

GraphRAG and RAPTOR tasks do not affect document status.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-12 12:03:41 +08:00
Kevin Hu
c30ffb5716 Fix: ollama model list issue. (#11175) 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-11 19:46:41 +08:00
Yongteng Lei
9213568692 Feat: add mechanism to check cancellation in Agent (#10766) 
### What problem does this PR solve?

Add mechanism to check cancellation in Agent.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-11 17:36:48 +08:00
Kevin Hu
f441f8ffc2 Fix: waitForResponse component. (#11172) 
### What problem does this PR solve?

#10056

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
 2025-11-11 16:58:47 +08:00
Yongteng Lei
df16a80f25 Feat: add initial Google Drive connector support (#11147) 
### What problem does this PR solve?

This feature is primarily ported from the
[Onyx](https://github.com/onyx-dot-app/onyx) project with necessary
modifications. Thanks for such a brilliant project.

Minor: consistently use `google_drive` rather than `google_driver`.

<img width="566" height="731" alt="image"
src="https://github.com/user-attachments/assets/6f64e70e-881e-42c7-b45f-809d3e0024a4"
/>

<img width="904" height="830" alt="image"
src="https://github.com/user-attachments/assets/dfa7d1ef-819a-4a82-8c52-0999f48ed4a6"
/>

<img width="911" height="869" alt="image"
src="https://github.com/user-attachments/assets/39e792fb-9fbe-4f3d-9b3c-b2265186bc22"
/>

<img width="947" height="323" alt="image"
src="https://github.com/user-attachments/assets/27d70e96-d9c0-42d9-8c89-276919b6d61d"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-10 19:15:02 +08:00
Kevin Hu
d207291217 Fix: add download stats to kb logs. (#11112) 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-10 13:28:07 +08:00
Lynn
d016a06fd5 Feat/monitor task (#11116) 
### What problem does this PR solve?

Show task executor.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-10 12:51:39 +08:00
Kevin Hu
dd1c8c5779 Feat: add auto parse to connector. (#11099) 
### What problem does this PR solve?

#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-07 16:49:29 +08:00
Kevin Hu
34283d4db4 Feat: add data source to pipleline logs . (#11075) 
### What problem does this PR solve?

#10953

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-07 11:43:59 +08:00
Jin Hai
af98763e27 Admin: add 'show version' (#11079) 
### What problem does this PR solve?

```
admin> show version;
show_version
+-----------------------+
| version               |
+-----------------------+
| v0.21.0-241-gc6cf58d5 |
+-----------------------+
admin> \q
Goodbye!

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-06 19:24:46 +08:00
Yongteng Lei
0cd8024c34 Feat: RAPTOR handle cancel gracefully (#11074) 
### What problem does this PR solve?

RAPTOR handle cancel gracefully.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-06 17:18:03 +08:00
Kevin Hu
3bd1fefe1f Feat: debug sync data. (#11073) 
### What problem does this PR solve?

#10953 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-06 16:48:04 +08:00
Yongteng Lei
23b81eae77 Feat: GraphRAG handle cancel gracefully (#11061) 
### What problem does this PR solve?

 GraghRAG handle cancel gracefully. #10997.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-06 16:12:20 +08:00
Kevin Hu
adbb8319e0 Fix: add fields for logs. (#11039) 
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-06 09:49:57 +08:00
Jin Hai
f98b24c9bf Move api.settings to common.settings (#11036) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-06 09:36:38 +08:00
Kevin Hu
dddf766470 Feat: start data sync service. (#11026) 
### What problem does this PR solve?

#10953 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-05 15:43:15 +08:00
Kevin Hu
b86e07088b Fix: escape multi-steps issues. (#11016) 
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-05 14:51:00 +08:00
Jin Hai
1a9215bc6f Move some vars to globals (#11017) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 14:14:38 +08:00
Jin Hai
96c015fb85 Fix and refactor imports (#11010) 
### What problem does this PR solve?

1. Move EMBEDDING_CFG to common.globals
2. Fix error imports
3. Move signal handles to common/signal_utils.py

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 11:07:54 +08:00
Jin Hai
bab3fce136 Move some constants to common (#11004) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 08:01:39 +08:00
Kevin Hu
4bbbf92331 Refa: link connector to KB. (#10991) 
### What problem does this PR solve?

#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-04 20:13:52 +08:00
Jin Hai
880a6a0428 Move some enumerate type to constants.py (#10998) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 19:25:25 +08:00
Yongteng Lei
465a140727 Feat: refine Confluence connector (#10994) 
### What problem does this PR solve?

Refine Confluence connector.
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2025-11-04 17:29:11 +08:00
Jin Hai
16d2be623c Minor tweaks (#10987) 
### What problem does this PR solve?

1. Rename identifier name
2. Fix some return statement
3. Fix some typos

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 14:15:31 +08:00
Jin Hai
1e45137284 Move 'timeout' to common folder (#10983) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 11:51:12 +08:00
Jin Hai
378bdfccfc Refactor log utils (#10973) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 20:25:02 +08:00
Kevin Hu
3e5a39482e Feat: Support multiple data sources synchronizations (#10954) 
### What problem does this PR solve?
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-03 19:59:18 +08:00
Jin Hai
076d811086 Introduce common/config_utils.py (#10968) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 17:25:06 +08:00
Jin Hai
d008a4df9f Move base64_image related functions to common directory (#10957) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 15:20:46 +08:00
Jin Hai
360f5c1179 Move token related functions to common (#10942) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 08:50:05 +08:00
Jin Hai
44f2d6f5da Move 'get_project_base_directory' to common directory (#10940) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-02 21:05:28 +08:00
Zhichang Yu
73144e278b Don't release full image (#10654) 
### What problem does this PR solve?

Introduced gpu profile in .env
Added Dockerfile_tei
fix datrie
Removed LIGHTEN flag

### Type of change

- [x] Documentation Update
- [x] Refactoring
 2025-10-23 23:02:27 +08:00
Yongteng Lei
2d491188b8 Refa: improve flow of GraphRAG and RAPTOR (#10709) 
### What problem does this PR solve?

Improve flow of GraphRAG and RAPTOR.

### Type of change

- [x] Refactoring
 2025-10-22 09:29:20 +08:00
Jin Hai
deb81810e9 Update message printout when start ingestion server (#10677) 
### What problem does this PR solve?

```
     ____                                  __     _                                                                  
    /  _/   ____    ____ _  ___    _____  / /_   (_)  ____    ____           _____  ___    _____ _   __  ___    _____
    / /    / __ \  / __ `/ / _ \  / ___/ / __/  / /  / __ \  / __ \         / ___/ / _ \  / ___/| | / / / _ \  / ___/
 _/ /    / / / / / /_/ / /  __/ (__  ) / /_   / /  / /_/ / / / / /        (__  ) /  __/ / /    | |/ / /  __/ / /    
/___/   /_/ /_/  \__, /  \___/ /____/  \__/  /_/   \____/ /_/ /_/        /____/  \___/ /_/     |___/  \___/ /_/     
                /____/          
```

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-10-21 09:38:20 +08:00
Liu An
8af769de41 Fix: add toc_kwd field and update page_num_int type (#10596) 
### What problem does this PR solve?

- Added new field 'toc_kwd' to infinity_mapping.json for table of
contents keyword support
- Changed page_num_int from integer to array type in task_executor.py to
handle multiple page numbers

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-10-16 12:47:24 +08:00
Kevin Hu
f92a45dcc4 Feat: let toc run asynchronizly... (#10513) 
### What problem does this PR solve?

#10436 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-14 14:14:52 +08:00
Yongteng Lei
65c3f0406c Fix: maintain backward compatibility for KB tasks (#10508) 
### What problem does this PR solve?

Maintain backward compatibility for KB tasks

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-10-13 11:53:48 +08:00
Kevin Hu
7d2f65671f Feat: debugging toc part. (#10486) 
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-11 18:45:21 +08:00
Kevin Hu
0d8791936e Feat: TOC retrieval (#10456) 
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-10 17:07:55 +08:00
Jin Hai
d931c33ced Fix typos: retrievaler -> retriever (#10372) 
### What problem does this PR solve?

Fix typos

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-10-10 09:17:36 +08:00
Kevin Hu
cbf04ee470 Feat: Use data pipeline to visualize the parsing configuration of the knowledge base (#10423) 
### What problem does this PR solve?

#9869

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: 纷繁下的无奈 <zhileihuang@126.com>
Co-authored-by: huangzl <huangzl@shinemo.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Wilmer <33392318@qq.com>
Co-authored-by: Adrian Weidig <adrianweidig@gmx.net>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: Liu An <asiro@qq.com>
Co-authored-by: buua436 <66937541+buua436@users.noreply.github.com>
Co-authored-by: BadwomanCraZY <511528396@qq.com>
Co-authored-by: cucusenok <31804608+cucusenok@users.noreply.github.com>
Co-authored-by: Russell Valentine <russ@coldstonelabs.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Billy Bao <newyorkupperbay@gmail.com>
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: TensorNull <129579691+TensorNull@users.noreply.github.com>
Co-authored-by: TensorNull <tensor.null@gmail.com>
Co-authored-by: TeslaZY <TeslaZY@outlook.com>
Co-authored-by: Ajay <160579663+aybanda@users.noreply.github.com>
Co-authored-by: AB <aj@Ajays-MacBook-Air.local>
Co-authored-by: 天海蒼灆 <huangaoqin@tecpie.com>
Co-authored-by: He Wang <wanghechn@qq.com>
Co-authored-by: Atsushi Hatakeyama <atu729@icloud.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Mohamed Mathari <155896313+melmathari@users.noreply.github.com>
Co-authored-by: Mohamed Mathari <nocodeventure@Mac-mini-van-Mohamed.fritz.box>
Co-authored-by: Stephen Hu <stephenhu@seismic.com>
Co-authored-by: Shaun Zhang <zhangwfjh@users.noreply.github.com>
Co-authored-by: zhimeng123 <60221886+zhimeng123@users.noreply.github.com>
Co-authored-by: mxc <mxc@example.com>
Co-authored-by: Dominik Novotný <50611433+SgtMarmite@users.noreply.github.com>
Co-authored-by: EVGENY M <168018528+rjohny55@users.noreply.github.com>
Co-authored-by: mcoder6425 <mcoder64@gmail.com>
Co-authored-by: lemsn <lemsn@msn.com>
Co-authored-by: lemsn <lemsn@126.com>
Co-authored-by: Adrian Gora <47756404+adagora@users.noreply.github.com>
Co-authored-by: Womsxd <45663319+Womsxd@users.noreply.github.com>
Co-authored-by: FatMii <39074672+FatMii@users.noreply.github.com>
 2025-10-09 12:36:19 +08:00