umap-tsne-analysis

UMAP and t-SNE Analysis

Prerequisites

Run the following before the first analysis to install all required R packages:

Rscript scripts/install_dependencies.R

Alternative manual installation:

Rscript -e "install.packages(c('optparse','data.table','Rtsne','umap','ggplot2','vegan','R.utils'), repos='https://cloud.r-project.org')"

Note: R.utils is only required when --timeout > 0, but pre-installing it avoids environment drift across runs. testthat is installed by scripts/install_dependencies.R as the development test dependency.

The skill cannot run until these packages are installed. In new or bare R environments, always run the prerequisite step first.

---

When to Read External Files

---

Usage

Rscript scripts/main.R \
  --input_file ./otu_table.csv \
  --group_file ./group_info.csv \
  --output_dir ./output/ \
  --method both \
  --seed 42

---

Arguments

---

Dependency Baseline

The skill was validated with the exact package baseline recorded in dependencies.lock.tsv.

Use this file as the reproducibility baseline when validating a new environment.

---

Input Format

Abundance / OTU Matrix (input_file)

Features as rows, samples as columns, CSV/TSV-like tabular file with feature ID in the first column.

OTU_ID,S1,S2,S3,S4
OTU_1,10,3,0,5
OTU_2,2,8,1,0
OTU_3,0,0,6,9

Group File (group_file)

Tabular file with at least two columns: sample ID and group label.

SampleID,Group
S1,Control
S2,Control
S3,Treatment
S4,Treatment

Requirements:

• At least 2 groups with at least 2 samples per group are required.
• All sample IDs in the group file must exist in the matrix columns.
• Single-group inputs will produce a SKILL_INVALID_PARAMETER error because dimensionality reduction without group contrast produces uninterpretable plots.

---

Output Files

---

Workflow

Step 1: Validate Input

• Check that matrix file and group file exist
• Resolve sample ID and group columns
• Validate at least 2 groups and at least 2 samples per group
• Ensure all group-file sample IDs exist in the matrix
• Remove samples with zero total abundance after alignment if needed

Step 2: Prepare Matrix

• Convert input table into numeric matrix
• Align matrix columns to sample order from the group file
• Transpose matrix so rows become samples and columns become features

Step 3: Run Dimensionality Reduction

• Run t-SNE if --method tsne or --method both
• Run UMAP if --method umap or --method both
• Apply fixed random seed for reproducibility

Step 4: Generate Visualizations

• Plot sample embeddings
• Color points by group
• Draw group ellipses when enabled
• Save PDF outputs

---

Methods

t-SNE

t-SNE is a non-linear dimensionality reduction method that preserves local neighborhood structure. It is useful for identifying local sample clustering patterns.

UMAP

UMAP is a manifold learning method that aims to preserve both local and some global structure. It is often faster than t-SNE and can produce stable low-dimensional embeddings when parameters are chosen appropriately.

Normalization

When --normalize TRUE, the script uses vegan::decostand() with the selected --norm_method before UMAP. This is helpful for abundance-style ecological matrices.

---

Agent Response Contract

After a successful run, report:

1. Method(s) run (tsne, umap, or both)
2. Sample count and group count processed
3. Key parameters used (perplexity for t-SNE, n_neighbors for UMAP)
4. Group separation quality (describe visible clustering from coordinate ranges if accessible)
5. Artifact paths: coordinate CSV(s) and plot PDF(s) produced

---

Examples

Basic Usage

Rscript scripts/main.R \
  -i otu_table.csv \
  -g group_info.csv \
  -o ./output \
  -m both

Only t-SNE

Rscript scripts/main.R \
  -i otu_table.csv \
  -g group_info.csv \
  -o ./output \
  -m tsne \
  --perplexity 10

Only UMAP with Custom Group Column

Rscript scripts/main.R \
  -i otu_table.csv \
  -g metadata.csv \
  -o ./output \
  -m umap \
  --sample_id_col SampleID \
  --group_col Treatment \
  --n_neighbors 15

---

Error Handling

Common Errors

IF error persists, READ: references/troubleshooting.md

Troubleshooting note: In environments where packages are not yet installed, SKILL_PACKAGE_NOT_FOUND will fire before file-validation errors. Install dependencies first, then re-run to expose any file-related errors.

---

Input Validation

This skill accepts:

1. An abundance or OTU-style feature matrix (CSV/TSV, features as rows, samples as columns)
2. A group file with at least two groups (CSV/TSV, sample IDs and group labels)

If the user's request does not involve UMAP or t-SNE dimensionality reduction for group separation visualization — for example, asking to run differential expression testing, process single-cell RNA-seq with specialized pipelines, perform clustering without a group file, or impute missing values — do not proceed with the workflow. Instead respond:

"UMAP and t-SNE Analysis is designed to perform sample-level dimensionality reduction and visualization on abundance or OTU-style matrices. Your request appears to be outside this scope. Please provide a feature matrix and group file for UMAP/t-SNE, or use a more appropriate tool for differential expression testing, single-cell analysis, or clustering."

---

Testing

Test with Sample Data

Rscript scripts/install_dependencies.R

Rscript scripts/main.R --help

Rscript scripts/main.R \
  -i tests/data/otu_table.csv \
  -g tests/data/group_info.csv \
  -o tests/output/ \
  -m both

Rscript tests/test_skill.R

Rscript tests/run_smoke_test.R

Validation Commands

ls -la tests/output/
ls -la tests/output/table
ls -la tests/output/plot
ls -la tests/output/data
wc -l tests/output/table/tsne_coordinates.csv
wc -l tests/output/table/umap_coordinates.csv

The canonical sample data live in tests/data/. Use those files for examples, smoke tests, and regression checks. The canonical output layout is output_dir/table, output_dir/plot, and output_dir/data.

---

Implementation Checklist

• CLI parsing with optparse
• set.seed() for reproducibility
• requireNamespace() dependency checks
• Dependency bootstrap script
• Session info recording
• File reading instructions in SKILL.md
• Modular script structure
• Error handling with SKILL_* codes
• Test data provided in tests/data/
• Version-pinned dependency baseline in dependencies.lock.tsv
• Automated testthat coverage for validation and plotting edge cases
• Scripts in scripts/ directory
• References in references/ directory

---

Last updated: 2026-04-27 | Version: 1.1.0