Large Document Processing

Documents

large-document-processing

Large Document Processing is an Agent Skill for documents published in the majiayu000/claude-skill-registry repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Process large documents (200+ pages) with structure preservation, intelligent parsing, and memory-efficient handling. Use when working with complex formatted documents, multi-level hierarchies, or when you need to extract structured data from large files like PDFs, DOCX, or text files.

View on GitHub ↗

QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

Open your project in Codex.
Copy and paste the prompt below.
Review files and risks before approving any action.

Prompt to paste

Inspect the Agent Skill at https://github.com/majiayu000/claude-skill-registry/blob/HEAD/skills/data/large-document-processing/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/large-document-processing, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Large Document Processing

Overview

A comprehensive skill for processing large documents (200+ pages) with structure preservation, intelligent parsing, and memory-efficient handling. Designed for documents with complex formatting, hierarchical structures, and multi-level indentation.

Capabilities

Multi-format Support: DOCX, PDF, and text files
Structure Preservation: Maintains document hierarchy, indentation, and formatting
Memory Efficiency: Chunked processing to handle very large documents
Intelligent Parsing: Recognizes headings, lists, dictionary entries, and semantic boundaries
Progress Tracking: Real-time processing status and error recovery
Metadata Extraction: Comprehensive document analysis and statistics

Core Components

1. Advanced Document Parser

Parse complex document structures while preserving formatting and hierarchy.

Key Features:

Hierarchical structure detection (levels 1-10)
Formatting preservation (bold, italic, fonts, sizes)
Page-by-page processing for memory efficiency
Intelligent content classification
Multi-language support with accent character handling

2. Implementation Pattern

from .large_document_processor import LargeDocumentProcessor, ProcessingConfig

# Configure processing
config = ProcessingConfig(
    chunk_size_pages=50,
    parallel_workers=4,
    preserve_formatting=True
)

# Initialize processor
processor = LargeDocumentProcessor(config)

# Process document
results = processor.process_large_document(
    input_file="large_document.docx",
    output_dir="output/processed"
)

3. Intelligent Text Chunking

from .intelligent_chunker import IntelligentTextChunker, ChunkType

chunker = IntelligentTextChunker(
    max_chunk_size=1024,
    overlap_ratio=0.15,
    preserve_sentences=True
)

chunks = chunker.chunk_document(text, ChunkType.SEMANTIC)

Output Formats

Structured JSON: Complete document hierarchy and metadata
Plain text: Clean extracted text with optional formatting markers
Chunked data: AI-ready text segments with overlap and metadata
Statistics report: Processing metrics and quality analysis

Best Practices

Memory Management: Use chunked processing for documents >100MB
Parallel Processing: Leverage multiple workers for batch operations
Structure Validation: Verify hierarchy detection accuracy
Progress Tracking: Provide user feedback for long-running operations

Dependencies

python-docx: DOCX file processing
PyMuPDF: Advanced PDF processing
Pillow: Image processing for embedded content
pathlib: Cross-platform path handling