--- name: doc-reader-01fd83063ac5 description: Read any common document/data file — PDF, Word (.docx), Excel (.xlsx/.xls), PowerPoint (.pptx), images (OCR), CSV/TSV, plain text, JSON/YAML/TOML, HTML/XML, and most source-code files. Use the `read_document` tool. category: tool --- # Universal Document Reader ## Purpose Return extracted text from any supported file in a single unified JSON envelope. The tool dispatches by file extension — you always call the same tool regardless of format. ### Supported formats | Category | Extensions | Notes | |---|---|---| | PDF | `.pdf` | Text pages extracted in ms; scanned/image pages fall back to OCR | | Word | `.docx` | Paragraphs + table cells | | Excel | `.xlsx`, `.xls` | All sheets, first 100 rows per sheet as preview | | PowerPoint | `.pptx` | Slide text content | | Images | `.png/.jpg/.jpeg/.gif/.bmp/.webp/.tiff` | OCR only | | CSV / TSV | `.csv`, `.tsv` | Raw text with encoding fallback | | Plain text | `.txt/.md/.log/.rst` | Encoding fallback | | Config | `.json/.yaml/.yml/.toml/.ini/.cfg/.env` | Raw text | | Markup | `.html/.htm/.xml` | Raw text (no HTML stripping) | | Source code | `.py/.js/.ts/.tsx/.go/.rs/.java/.cpp/.c/.sql/.sh/...` | Raw text | | Unknown extension | anything else | Best-effort read as UTF-8/GBK text | **Blocked** (rejected at `/upload`): executables (`.exe/.dll/.so/...`) and archives (`.zip/.tar/...`). Ask the user to unpack archives locally first. ## Usage **Always call the tool directly — do not run Python from bash.** ``` read_document(file_path="uploads/paper.pdf") read_document(file_path="uploads/annual_report.pdf", pages="1-10") read_document(file_path="uploads/contract.docx") read_document(file_path="uploads/sales.xlsx") read_document(file_path="uploads/deck.pptx") read_document(file_path="uploads/chart.png") # image → OCR read_document(file_path="uploads/config.yaml") read_document(file_path="uploads/notes.md") ``` The `pages` parameter only applies to PDF; other formats ignore it. ## Return envelope All formats share this shape: ```json { "status": "ok", "file": "paper.pdf", "format": "pdf", "char_count": 52000, "truncated": true, "text": "..." } ``` Format-specific extra fields: | Format | Extra keys | |---|---| | `pdf` | `total_pages`, `pages_read`, `ocr_pages`, `ocr_engine`, `ocr_quality`, `skipped_pages` | | `docx` | `paragraphs`, `tables` | | `excel` | `sheets` (array of `{name, rows, cols}`) | | `pptx` | `slides` | | `text` | `encoding`, `size` | Content longer than 15000 chars is truncated; for PDFs use the `pages` parameter to read slices. ## Workflows ### Paper / report summary ``` 1. read_document(file_path="paper.pdf") → full text 2. Extract abstract, methodology, conclusion → summarize ``` ### Contract review ``` 1. read_document(file_path="contract.docx") → paragraphs + tables 2. Flag key clauses (termination, liability, payment, IP) ``` ### Spreadsheet quick-look ``` 1. read_document(file_path="sales.xlsx") → all sheet previews 2. If user wants trade journal analysis specifically, pivot to `analyze_trade_journal` tool instead (see trade-journal skill). ``` ### Chart / screenshot / scanned PDF ``` 1. read_document(file_path="scan.png") → OCR text 2. If OCR returns empty, tell the user; don't fabricate. ``` ## OCR Configuration The `read_document` tool automatically uses OCR for PDF pages with insufficient extractable text. ### OCR Threshold Use `min_text_per_page` to control when OCR is triggered (default: 50 characters): ```python read_document("scanned_report.pdf", min_text_per_page=10) # More aggressive OCR read_document("mixed_pdf.pdf", min_text_per_page=100) # Less aggressive OCR ``` ### OCR Engine Configuration Two OCR engines are built in — no extra packages needed beyond the engine SDK: | Engine | Type | Requires | Install | |--------|------|----------|---------| | `rapid` | Local (offline) | `rapidocr_onnxruntime` | `pip install rapidocr_onnxruntime` | | `llm-vision` | Cloud | A vision-capable LLM model + API key | No extra install — uses your existing LLM provider config | The `llm-vision` engine works with **any OpenAI-compatible vision model** (GPT-4o, Qwen-VL, Gemini, Claude, GLM-4V, etc.). It reuses your existing `LANGCHAIN_PROVIDER` / `LANGCHAIN_MODEL_NAME` / API key configuration — no separate provider mapping needed. If you explicitly set `VIBE_TRADING_OCR_ENGINE=llm-vision`, your model choice is trusted; a real API error from the provider is clearer feedback than a heuristic guess. To override the model used for OCR (without changing your agent's main model): ``` VIBE_TRADING_OCR_LLM_MODEL=qwen3.7-plus ``` Set `VIBE_TRADING_OCR_ENGINE` to select the engine: - `auto` (default): use local engines only, never cloud (privacy: document pages never leave the machine) - `rapid`: force RapidOCR (local, ONNX) - `llm-vision`: force LLM vision OCR (cloud — pages are sent to your configured LLM provider) - `none`: disable OCR entirely ### Response Fields PDF responses include OCR metadata: - `ocr_engine`: name of the OCR engine used (e.g. "rapid", "llm-vision") or `null` - `ocr_pages`: number of pages processed via OCR - `skipped_pages`: number of pages skipped (no OCR engine available) - `ocr_quality`: object with `quality_flag` (`good`/`degraded`/`no_ocr_engine`/`no_ocr_needed`), `ocr_pages`, and `text_density` (chars per page) ## Notes - **Encoding fallback** order for text: utf-8 → utf-8-sig → gbk → gb2312 → big5 → latin-1. - **OCR** uses the configured engine (RapidOCR for local, or LLM vision for cloud). If no engine is available, image/scanned files return empty `text` with a `note` field — tell the user to install `rapidocr-onnxruntime` or set `VIBE_TRADING_OCR_ENGINE=llm-vision` with a vision-capable model. - **Excel previews** are limited to 100 rows per sheet to stay in budget. If the user needs full data (e.g. trade journals), call `analyze_trade_journal` instead. - **Source-code files** are returned raw; do not re-format or re-indent. --- Source attribution: HKUDS Original project: https://github.com/HKUDS/Vibe-Trading License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Mega AI Hub. ## Upstream license notice MIT License Copyright (c) 2026 Gabriel Moreira Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # Mega AI Hub resource metadata Creator / publisher: HKUDS Original project: https://github.com/HKUDS/Vibe-Trading Attribution basis: Upstream repository explicitly recorded in package mirror path Fields of work: Content & Media, Data & Analytics, Education & Research, Marketing, Product Management Roles: Content Marketer, Data Analyst, Product Manager, Researcher, Technical Writer Specialties: API Tools: Tool not specified Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: 81914cea654e7b2b1752ce5d442532bb53218b1b407eddf66f5a6e6d02a007cb