sample-correlation-analysis

Source: https://github.com/aipoch/medical-research-skills

Correlation Analysis

Use this skill to run correlation analysis on two variables from a tabular data file.

Use This Skill When

• You need Pearson or Spearman correlation between two variables stored either as columns or as first-column row labels.
• You need a command-line workflow with parameter validation.
• You need standardized output files in CSV or TXT format.

Primary Command

Rscript scripts/main.R \
  --data_file <input_file> \
  --method <pearson|spearman> \
  --x_var <variable_name> \
  --y_var <variable_name> \
  --output_dir <output_dir>

Prerequisites

• Rscript is available in the shell.
• Required R packages: optparse, data.table.
• Install missing packages with Rscript -e 'install.packages(c("optparse", "data.table"), repos="https://cloud.r-project.org")'.

Core Arguments

Argument	Required	Description
--data_file	Yes	Input data file in CSV, TXT, or TSV format
--method	No	Correlation method: pearson or spearman. Default pearson
--x_var	No	First variable name. It can match a column name or a first-column row label. Default variable1
--y_var	No	Second variable name. It can match a column name or a first-column row label. Default variable2
--output_dir	No	Output directory, default ./Correlation_Results
--alternative	No	two.sided, less, or greater. Default two.sided
--conf_level	No	Confidence level between 0 and 1, default 0.95
--output_format	No	csv or txt, default csv
--output_prefix	No	Output filename prefix, default correlation

Input Requirements

• The input file must contain both target variables.
• Both variables must contain numeric values.
• If the first column stores variable names and the remaining columns are samples, the script automatically reads variables by row label.
• Rows with missing values in either variable are excluded.
• At least 3 complete observation pairs are needed.

Example input:

variable1,variable2
10.2,8.5
11.5,9.2
9.8,7.9

Minimal Workflow

1. Confirm the input file exists and variable names are correct.
2. Run scripts/main.R with the requested method and variable names.
3. Check the output directory for result files under table/.

If you omit --data_file, the script exits with SKILL_MISSING_INPUT.

Outputs

Expected output structure:

<output_dir>/
├── table/
├── figure/
└── data/

Primary result file:

• table/<output_prefix>_<method>.csv
• table/<output_prefix>_<method>.txt

Result fields include:

• method
• correlation
• statistic
• p_value
• conf_low
• conf_high
• sample_size
• x_variable
• y_variable
• variable_orientation

Choose the Method

• Use pearson for linear relationships between continuous variables.
• Use spearman for monotonic relationships, non-normal data, or outlier-prone data.

Read These Files When Needed

Need	File
Statistical details and assumptions	references/algorithm.md
More CLI examples	references/cli-guide.md
Error diagnosis	references/troubleshooting.md
Main execution entry point	scripts/main.R
Sample test data	tests/data/

Quick Examples

Pearson:

Rscript scripts/main.R \
  --data_file tests/data/sample_correlation_1.csv \
  --method pearson \
  --x_var variable1 \
  --y_var variable2 \
  --output_dir tests/output_pearson

Spearman:

Rscript scripts/main.R \
  --data_file tests/data/sample_correlation_3.csv \
  --method spearman \
  --x_var "Activated CD8 T cell" \
  --y_var "Central memory CD8 T cell" \
  --output_dir tests/output_spearman

Validation

Rscript scripts/main.R --help

Rscript scripts/main.R \
  --data_file tests/data/sample_correlation_1.csv \
  --method pearson \
  --x_var variable1 \
  --y_var variable2 \
  --output_dir tests/validation_output

After running analysis, verify that tests/validation_output/table/correlation_pearson.csv exists.

Common Errors

• SKILL_FILE_NOT_FOUND: Input file path is wrong or inaccessible.
• SKILL_MISSING_COLUMNS: One or both requested variables are missing.
• SKILL_INVALID_DATA: Input data is malformed or unsuitable for analysis.
• SKILL_INVALID_PARAMETER: An argument value is invalid.
• SKILL_INSUFFICIENT_DATA: Too few complete observations remain after filtering.
• SKILL_DEPENDENCY_MISSING: A required R package such as optparse or data.table is unavailable.

If the issue is not obvious, read references/troubleshooting.md.