Commit Graph

1635 Commits

Author SHA1 Message Date
568322aeaf fix(rag): fix error in viewing document chunk and cannot start task_executor server (#3481)
### What problem does this PR solve?

1. Fix error in viewing document chunk

<img width="1677" alt="Pasted Graphic"
src="https://github.com/user-attachments/assets/acd84cde-f38c-4190-b135-5e5139ae2613">

Viewing document chunk details in a BeartypeCallHintParamViolation
error.

Traceback (most recent call last):
File "ragflow/.venv/lib/python3.12/site-packages/flask/app.py", line
880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
File "ragflow/.venv/lib/python3.12/site-packages/flask/app.py", line
865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
# type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ragflow/.venv/lib/python3.12/site-packages/flask_login/utils.py",
line 290, in decorated_view
    return current_app.ensure_sync(func)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ragflow/api/apps/chunk_app.py", line 311, in knowledge_graph
sres = settings.retrievaler.search(req, search.index_name(tenant_id),
kb_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<@beartype(rag.nlp.search.Dealer.search) at 0x3381fd800>", line
39, in search
beartype.roar.BeartypeCallHintParamViolation: Method
rag.nlp.search.Dealer.search() parameter
idx_names='ragflow_0e1e67f431d711ef98fc00155d29195d' violates type hint
list[str], as str 'ragflow_0e1e67f431d711ef98fc00155d29195d' not
instance of list.
2024-11-19 11:30:29,817 ERROR 91013 Method
rag.nlp.search.Dealer.search() parameter
idx_names='ragflow_0e1e67f431d711ef98fc00155d29195d' violates type hint
list[str], as str 'ragflow_0e1e67f431d711ef98fc00155d29195d' not
instance of list.
Traceback (most recent call last):
  File "ragflow/api/apps/chunk_app.py", line 60, in list_chunk
sres = settings.retrievaler.search(query, search.index_name(tenant_id),
kb_ids, highlight=True)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<@beartype(rag.nlp.search.Dealer.search) at 0x3381fd800>", line
39, in search
beartype.roar.BeartypeCallHintParamViolation: Method
rag.nlp.search.Dealer.search() parameter
idx_names='ragflow_0e1e67f431d711ef98fc00155d29195d' violates type hint
list[str], as str 'ragflow_0e1e67f431d711ef98fc00155d29195d' not
instance of list.


because in nlp/search.py,the idx_names is only list

<img width="1098" alt="Pasted Graphic 2"
src="https://github.com/user-attachments/assets/4998cb1e-94bc-470b-b2f4-41ecb5b08f8a">

but the DocStoreConnection.search method accept list or str
<img width="1175" alt="Pasted Graphic 3"
src="https://github.com/user-attachments/assets/ee918b4a-87a5-42c9-a6d2-d0db0884b875">


and his implements also list and str
es_conn.py

<img width="1121" alt="Pasted Graphic 4"
src="https://github.com/user-attachments/assets/3e6dc030-0a0d-416c-8fd4-0b4cfd576f8c">

infinity_conn.py

<img width="1221" alt="Pasted Graphic 5"
src="https://github.com/user-attachments/assets/44edac2b-6b81-45b0-a3fc-cb1c63219015">

2. Fix cannot star task_executor server with Unresolved reference
'Mapping'
<img width="1283" alt="Pasted Graphic 6"
src="https://github.com/user-attachments/assets/421f17b8-d0a5-46d3-bc4d-d05dc9dfc934">

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-19 14:36:10 +08:00
31decadd8e feat: Let the top navigation bar open in a tab when you right click on it #3018 (#3486)
### What problem does this PR solve?

feat: Let the top navigation bar open in a tab when you right click on
it #3018

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-19 14:20:00 +08:00
dec9b3e540 Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
d0f94a42ff Refactor ragflow server (#3482)
### What problem does this PR solve?

1. Fix document error
2. Move start server out of if __main__ branch.

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-19 12:31:20 +08:00
ed0d47fc8a feat: "Open link in new tab" doesn't work for profile image #3018 (#3480)
### What problem does this PR solve?

feat:  "Open link in new tab" doesn't work for profile image #3018

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-19 11:40:59 +08:00
aa9a16e073 Fix document error (#3456)
### What problem does this PR solve?

1. Update README.md
2. Fix error description in docker/.env

### Type of change

- [x] Documentation Update

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-19 11:31:11 +08:00
eef84a86bf feat: Disable clicking the Next button while uploading files in RunDrawer #3355 (#3477)
### What problem does this PR solve?

feat: Disable clicking the Next button while uploading files in
RunDrawer #3355

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-19 11:03:39 +08:00
ed72d1100b minor format updates (#3471)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-18 19:19:28 +08:00
f4e9dae33a Updated upgrade guide to avoid confusion (#3469)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-18 19:07:05 +08:00
50f7b7e0a3 feat: Capture task executor null value exception #3409 (#3468)
### What problem does this PR solve?

feat: Capture task executor null value exception #3409

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 19:06:52 +08:00
01c2712941 feat: Display task executor tooltip with json-view #3409 (#3467)
### What problem does this PR solve?

feat: Display task executor  tooltip with json-view #3409

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 18:36:18 +08:00
4413683898 Introduced beartype (#3460)
### What problem does this PR solve?

Introduced [beartype](https://github.com/beartype/beartype) for runtime
type-checking.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 17:38:17 +08:00
3824c1fec0 feat: Show task_executor heartbeat #3409 (#3461)
### What problem does this PR solve?

feat: Show task_executor heartbeat #3409
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 17:23:49 +08:00
4b3eeaa6ef Added LocalAI support for rerank models (#3446)
### What problem does this PR solve?

Hi there!
LocalAI added support of rerank models
https://localai.io/features/reranker/

I've implemented LocalAIRerank class (typically copied it from
OpenAI_APIRerank class).
Also, LocalAI model response with 500 error code if len of "documents"
is less than 2 in similarity check.
So I've added the second "document" on RERANK model connection check in
`api/apps/llm_app.py`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-18 12:05:52 +08:00
70cd5c1599 Remove unused code (#3448)
### What problem does this PR solve?

1. Remove unused code.
2. Move some codes from settings to constants

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-18 12:05:38 +08:00
f9643adc43 Adjusted heartbeat format (#3459)
### What problem does this PR solve?

Adjusted heartbeat format

### Type of change

- [x] Refactoring
2024-11-18 12:03:28 +08:00
7b9e0723d6 build(deps): bump cross-spawn from 7.0.3 to 7.0.5 in /web (#3455)
Bumps [cross-spawn](https://github.com/moxystudio/node-cross-spawn) from
7.0.3 to 7.0.5.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/moxystudio/node-cross-spawn/blob/master/CHANGELOG.md">cross-spawn's
changelog</a>.</em></p>
<blockquote>
<h3><a
href="https://github.com/moxystudio/node-cross-spawn/compare/v7.0.4...v7.0.5">7.0.5</a>
(2024-11-07)</h3>
<h3>Bug Fixes</h3>
<ul>
<li>fix escaping bug introduced by backtracking (<a
href="640d391fde">640d391</a>)</li>
</ul>
<h3><a
href="https://github.com/moxystudio/node-cross-spawn/compare/v7.0.3...v7.0.4">7.0.4</a>
(2024-11-07)</h3>
<h3>Bug Fixes</h3>
<ul>
<li>disable regexp backtracking (<a
href="https://redirect.github.com/moxystudio/node-cross-spawn/issues/160">#160</a>)
(<a
href="5ff3a07d9a">5ff3a07</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="085268352d"><code>0852683</code></a>
chore(release): 7.0.5</li>
<li><a
href="640d391fde"><code>640d391</code></a>
fix: fix escaping bug introduced by backtracking</li>
<li><a
href="bff0c87c8b"><code>bff0c87</code></a>
chore: remove codecov</li>
<li><a
href="a7c6abc6fe"><code>a7c6abc</code></a>
chore: replace travis with github workflows</li>
<li><a
href="9b9246e096"><code>9b9246e</code></a>
chore(release): 7.0.4</li>
<li><a
href="5ff3a07d9a"><code>5ff3a07</code></a>
fix: disable regexp backtracking (<a
href="https://redirect.github.com/moxystudio/node-cross-spawn/issues/160">#160</a>)</li>
<li><a
href="9521e2da94"><code>9521e2d</code></a>
chore: fix tests in recent node js versions</li>
<li><a
href="97ded399e9"><code>97ded39</code></a>
chore: convert package lock</li>
<li><a
href="d52b6b9da4"><code>d52b6b9</code></a>
chore: remove unused argument (<a
href="https://redirect.github.com/moxystudio/node-cross-spawn/issues/156">#156</a>)</li>
<li><a
href="5d843849e1"><code>5d84384</code></a>
chore: add travis jobs on ppc64le (<a
href="https://redirect.github.com/moxystudio/node-cross-spawn/issues/142">#142</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/moxystudio/node-cross-spawn/compare/v7.0.3...v7.0.5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cross-spawn&package-manager=npm_and_yarn&previous-version=7.0.3&new-version=7.0.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 11:19:44 +08:00
a1d01a1b2f enlarge the default token length of RAPTOR summarization (#3454)
### What problem does this PR solve?

#3426

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 10:15:26 +08:00
dc05f43eee Minor: Fixed a broken link (#3451)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-16 20:22:53 +08:00
77bdeb32bd Added current task into task executor's hearbeat (#3444)
### What problem does this PR solve?

Added current task into task executor's hearbeat

### Type of change

- [x] Refactoring
2024-11-15 22:55:41 +08:00
af18217d78 feat: Add banner #3221 (#3442)
### What problem does this PR solve?

feat: Add banner #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-15 19:18:23 +08:00
4ed5ca2666 handle_task catch all exception (#3441)
### What problem does this PR solve?

handle_task catch all exception
Report heartbeats

### Type of change

- [x] Refactoring
2024-11-15 18:51:09 +08:00
1e90a1bf36 Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
ac033b62cf fix tika-server issue (#3439)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-15 16:57:01 +08:00
cb3b9d7ada refine the message of queuing a task (#3437)
### What problem does this PR solve?



### Type of change

- [x] Refactoring
2024-11-15 15:59:54 +08:00
ca9e97d2f2 Enlarge the term weight difference (#3435)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-11-15 15:41:50 +08:00
6d451dbe06 Updated retrieval testing UI (#3433)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-15 15:17:34 +08:00
e0659a4f0e feat: Add RunDrawer #3355 (#3434)
### What problem does this PR solve?

feat: Translation test run form #3355
feat: Wrap QueryTable with Collapse #3355
feat: If the required fields are not filled in, the submit button will
be grayed out. #3355
feat: Add RunDrawer #3355

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-15 15:17:23 +08:00
a854bc22d1 Rework task executor heartbeat (#3430)
### What problem does this PR solve?

Rework task executor heartbeat, and print in console.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-15 14:43:55 +08:00
48e060aa53 rm es query escape chars (#3428)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-15 13:19:07 +08:00
47abfc32d4 Remove unused settings (#3427)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 13:18:16 +08:00
a1ba228bc2 fix: empty token bug (#3424)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-15 10:33:03 +08:00
996c94a8e7 Move clk100k_base tokenizer to docker image (#3411)
### What problem does this PR solve?

Move the tiktoken of cl100k_base into docker image

issue: #3338 

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-15 10:18:40 +08:00
220aaddc62 fix: synonym bug (#3423)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-15 10:14:51 +08:00
6878d23a57 Print configs when startup RAGFlow server (#3414)
### What problem does this PR solve?

Print configs at the RAGFlow startup phase.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

```
2024-11-14 21:27:53,090 INFO     962231 Current configs, from /home/weilongma/Documents/development/ragflow/conf/service_conf.yaml:
2024-11-14 21:27:53,090 INFO     962231 ragflow: {'host': '0.0.0.0', 'http_port': 9380}
2024-11-14 21:27:53,090 INFO     962231 mysql: {'name': 'rag_flow', 'user': 'root', 'password': 'infini_rag_flow', 'host': 'mysql', 'port': 5455, 'max_connections': 100, 'stale_timeout': 30}
2024-11-14 21:27:53,090 INFO     962231 minio: {'user': 'rag_flow', 'password': 'infini_rag_flow', 'host': 'minio:9000'}
2024-11-14 21:27:53,090 INFO     962231 es: {'hosts': 'http://es01:1200', 'username': 'elastic', 'password': 'infini_rag_flow'}
2024-11-14 21:27:53,090 INFO     962231 redis: {'db': 1, 'password': 'infini_rag_flow', 'host': 'redis:6379'}
```

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 09:29:40 +08:00
df9d054551 Updated descriptions of agent APIs (#3407)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2024-11-14 18:44:37 +08:00
30c1f7ee29 make variables access robuster (#3406)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-11-14 18:28:41 +08:00
e4c4fdabbd Update version display on web UI (#3405)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-14 17:51:21 +08:00
30f6421760 Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?

Use consistent log file names, introduced initLogger

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-14 17:13:48 +08:00
ab4384e011 Updates on parsing progress, including more detailed time cost inform… (#3402)
### What problem does this PR solve?

#3401 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-14 16:28:10 +08:00
201bbef7c0 Print version when RAGFlow server startup (#3393)
### What problem does this PR solve?

Print version when RAGFlow server startup

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-14 15:51:30 +08:00
95d21e5d9f fix: standardize property name from 'chat' to 'chat_id' (#3383)
### What problem does this PR solve?

This PR addresses the inconsistency in property naming within the
codebase by renaming the 'chat' property to 'chat_id' in the session.py
file. This change aims to align the naming convention with other parts
of the application that refer to chat identifiers as 'chat_id', thereby
improving code clarity and maintainability.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 15:49:03 +08:00
c5368c7745 resolve halt while starting up (#3397)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 13:20:17 +08:00
0657a09e2c Update llm_factories.json (#3396)
### What problem does this PR solve?
Added: Claude-3-5-sonnet-20241022 version. 

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-11-14 13:00:16 +08:00
4caf932808 fix bug about fetching knowledge graph (#3394)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 12:29:15 +08:00
400fc3f5e9 Add enviroment validation at server startup phase (#3388)
### What problem does this PR solve?

1. Validate the Python version should >= 3.11
2. Download nltk data

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
Co-authored-by: jinhai <haijin.chn@gmail.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2024-11-14 11:01:08 +08:00
e44e3a67b0 adapt to lower case cohere (#3392)
### What problem does this PR solve?

#3384

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 10:18:25 +08:00
9d395ab74e Added doc for switching elasticsearch to infinity (#3370)
### What problem does this PR solve?

Added doc for switching elasticsearch to infinity

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2024-11-14 00:08:55 +08:00
83c6b1f308 set DLA active for KG (#3386)
### What problem does this PR solve?

### Type of change


- [x] Refactoring
2024-11-13 16:59:19 +08:00
7ab9715b0e refine the error message (#3382)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-11-13 16:20:59 +08:00