pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix:missing embedding vector on Tokenizer (#10964)

Browse Source 
### What problem does this PR solve?
issue:
[#10890](https://github.com/infiniflow/ragflow/issues/10890)
change:
missing embedding vector on Tokenizer
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
This commit is contained in:
buua436
2025-11-03 19:17:05 +08:00
committed by GitHub
parent 2d83c64eed
commit fd4aa79c07
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
rag/flow/tokenizer/tokenizer.py
View File

@ -63,6 +63,8 @@ class Tokenizer(ProcessBase):
texts = [] texts = []
for c in chunks: for c in chunks:
txt = "" txt = ""
if isinstance(self._param.fields, str):
self._param.fields=[self._param.fields]
for f in self._param.fields: for f in self._param.fields:
f = c.get(f) f = c.get(f)
if isinstance(f, str): if isinstance(f, str):