# Text File

> Text File is an Agent Skill for documents published in the agentscope-ai/agentscope-samples repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Guidelines for handling text files

- Skill name: `text-file`
- Category: [Documents](https://www.openparable.com/skill/documents/1)
- Source: https://github.com/agentscope-ai/agentscope-samples/blob/HEAD/alias/src/alias/agent/agents/_built_in_skill/data/text/SKILL.md
- Repository: agentscope-ai/agentscope-samples (327 GitHub stars when collected)

## SKILL.md

# Text Files Handling Specifications

## Goals
- Safely read text files without memory exhaustion.
- Accurately detect encoding to avoid garbled characters.
- Identify underlying patterns (e.g., Log formats, Markdown structure, delimiters).
- Efficiently extract or search for specific information within large volumes of text.

## Encoding & Detection

- Encoding Strategy:
  - Default to `utf-8`.
  - If it fails, try `utf-8-sig` (for files with BOM), `gbk/gb18030` (for Chinese context), or `latin-1`.
  - Use `chardet` or similar logic if encoding is unknown and first few bytes look non-standard.
- Line Endings: Be aware of `\n` (Unix), `\r\n` (Windows), and `\r` (Legacy Mac) when counting lines or splitting.

## Inspection

- Preview: Read the first 10-20 lines to determine:
  - Content Type: Is it a log, code, prose, or a semi-structured list?
  - Uniformity: Does every line follow the same format?
- Metadata: Check total file size before reading. If >50MB, treat as a "large file" and avoid full loading.

## Querying & Reading (Large Files)

- Streaming: For files exceeding memory or >50MB:
  - Use `with open(path) as f: for line in f:` to process line-by-line.
  - Never use `.read()` or `.readlines()` on large files.
- Random Sampling: To understand a huge file's structure, read the first N lines, the middle N lines (using `f.seek()`), and the last N lines.
- Pattern Matching: Use Regular Expressions (Regex) for targeted extraction instead of complex string slicing.
- Grep-like Search: If searching for a keyword, iterate through lines and only store/return matching lines + context.

## Data Quality

- Truncation Warning: If only a portion of the file is read, clearly state: "Displaying first X lines of Y total lines."
- Empty Lines/Comments: Decide early whether to ignore blank lines or lines starting with specific comment characters (e.g., `#`, `//`).

## Best Practices

- Resource Safety: Always use context managers (`with` statement) to ensure file handles are closed.
- Memory Consciousness: For logs and large TXT, prioritize "find and extract" over "load and filter."
- Regex Optimization: Compile regex patterns if they are used repeatedly in a loop over millions of lines.
- Validation: After reading, verify the content isn't binary (e.g., PDF or EXE renamed to .txt) by checking for null bytes or a high density of non-ASCII characters.
- Progress Logging: For long-running text processing, log progress every 100k lines or 10% of file size.

---

Source: https://www.openparable.com/skill/documents/text-file (OpenParable). Cite the canonical URL above.