insights-clustering

Insights Clustering and Analysis

Overview

This skill covers clustering and analyzing insights from ServiceNow operational data to identify patterns and generate actionable recommendations:

Grouping incidents, cases, and conversations by topic similarity to discover recurring themes
Performing sentiment analysis across interactions to identify service quality trends
Analyzing resolution patterns to find best practices and knowledge gaps
Detecting emerging issue clusters before they become major problems
Generating point-of-view (POV) summaries with data-backed recommendations
Building trend reports that connect operational patterns to business impact

When to use: When performing periodic service review analysis, investigating a spike in support volume, preparing data-driven improvement recommendations, identifying proactive problem management candidates, or generating management insights from operational data.

Prerequisites

Roles: admin, analytics_admin, itil, or problem_manager
Plugins: com.glide.interaction (Agent Workspace), com.snc.problem_management (Problem Management) recommended
Access: Read access to incident, sn_customerservice_case, interaction, sys_cs_conversation, kb_knowledge, problem tables
Data: At least 30 days of incident, case, or interaction data for meaningful clustering
Related Skills: genai/agent-miner for interaction mining, knowledge/gap-analysis for KB gap identification

Procedure

Step 1: Define Analysis Scope

Determine the data sources, time range, and focus area for clustering.

ANALYSIS SCOPE DEFINITION:

Data Sources:
- [ ] Incidents (incident table)
- [ ] Customer Cases (sn_customerservice_case)
- [ ] Agent Interactions (interaction)
- [ ] Virtual Agent Conversations (sys_cs_conversation)
- [ ] Problem Records (problem)

Time Range: [start_date] to [end_date]
Focus Area: [All / Specific category / Assignment group / Service]
Minimum Cluster Size: [threshold, e.g., 5 records]

ANALYSIS OBJECTIVES:
1. [Primary objective, e.g., "Identify top incident drivers"]
2. [Secondary objective, e.g., "Find knowledge gaps"]
3. [Tertiary objective, e.g., "Assess customer sentiment trends"]

Step 2: Extract Incident Data for Clustering

Pull incident records with text and classification fields.

MCP Approach:

Tool: SN-Query-Table
Parameters:
  table_name: incident
  query: sys_created_on>=javascript:gs.daysAgo(30)^state!=8^ORDERBYDESCsys_created_on
  fields: sys_id,number,short_description,description,category,subcategory,priority,impact,urgency,state,close_code,close_notes,assignment_group,cmdb_ci,business_service,resolved_at,opened_at,resolution_code
  limit: 1000

REST Approach:

GET /api/now/table/incident
  ?sysparm_query=sys_created_on>=javascript:gs.daysAgo(30)^state!=8
  &sysparm_fields=sys_id,number,short_description,description,category,subcategory,priority,impact,urgency,state,close_code,close_notes,assignment_group,cmdb_ci,business_service,resolved_at,opened_at
  &sysparm_limit=1000
  &sysparm_display_value=true

Step 3: Extract Case and Interaction Data

Pull additional data sources for cross-channel analysis.

MCP Approach:

Tool: SN-Query-Table
Parameters:
  table_name: sn_customerservice_case
  query: sys_created_on>=javascript:gs.daysAgo(30)^ORDERBYDESCsys_created_on
  fields: sys_id,number,short_description,description,category,subcategory,priority,state,resolution_code,close_notes,contact,account,product
  limit: 500

Tool: SN-Query-Table
Parameters:
  table_name: interaction
  query: sys_created_on>=javascript:gs.daysAgo(30)^state=closed
  fields: sys_id,number,short_description,channel,category,close_code,assignment_group,duration
  limit: 500

Step 4: Perform Topic Clustering

Group records by topic similarity using text analysis of short descriptions and descriptions.

Clustering Approach:

Category-Based Grouping: Group by existing category/subcategory taxonomy
Text Similarity: Cluster short_description fields by keyword and phrase overlap
CI-Based Grouping: Group by affected configuration item or business service
Resolution-Based: Group by close_code and resolution patterns

MCP Approach (Category Aggregation):

Tool: SN-Query-Table
Parameters:
  table_name: incident
  query: sys_created_on>=javascript:gs.daysAgo(30)
  fields: category,subcategory
  limit: 2000

Topic Cluster Output:

=== TOPIC CLUSTERS ===
Analysis Period: [date range]
Total Records Analyzed: [count]
Clusters Identified: [count]

CLUSTER 1: [Topic Name] - [count] records ([%] of total)
  Keywords: [top keywords]
  Categories: [categories represented]
  Sample Records:
  - [INC#]: [short_description]
  - [INC#]: [short_description]
  Trend: [increasing/stable/decreasing] vs prior period

CLUSTER 2: [Topic Name] - [count] records ([%] of total)
  ...

Step 5: Analyze Sentiment Patterns

Assess sentiment from customer comments and interaction transcripts.

MCP Approach:

Tool: SN-Query-Table
Parameters:
  table_name: sys_journal_field
  query: element_id=[RECORD_SYS_ID]^element=comments
  fields: value,sys_created_on,sys_created_by
  limit: 20

Sentiment Classification Framework:

Signal	Positive	Neutral	Negative
Language	"thank you", "great", "resolved"	"ok", "understood", factual	"frustrated", "unacceptable", "still broken"
Frequency	Single contact	Follow-up after resolution	Multiple escalations
Reopens	None	1 reopen	2+ reopens
Survey Score	4-5	3	1-2

Sentiment Distribution:

SENTIMENT ANALYSIS:
Positive: [count] ([%])
Neutral: [count] ([%])
Negative: [count] ([%])

NEGATIVE SENTIMENT DRIVERS:
1. [Topic/Category]: [count] negative interactions
   Common complaints: [themes]
2. [Topic/Category]: [count] negative interactions
   Common complaints: [themes]

SENTIMENT TREND (vs Prior Period):
Positive: [+/- change]
Negative: [+/- change]

Step 6: Analyze Resolution Patterns

Identify common resolution approaches and their effectiveness.

MCP Approach:

Tool: SN-Query-Table
Parameters:
  table_name: incident
  query: sys_created_on>=javascript:gs.daysAgo(30)^state=6
  fields: category,subcategory,close_code,resolution_code,close_notes
  limit: 500

Resolution Pattern Analysis:

RESOLUTION PATTERNS:

CLUSTER: [Topic Name]
Total Incidents: [count]
Resolution Methods:
  1. [Resolution type] - [count] ([%]) - Avg Resolution Time: [time]
  2. [Resolution type] - [count] ([%]) - Avg Resolution Time: [time]
  3. [Resolution type] - [count] ([%]) - Avg Resolution Time: [time]

MOST EFFECTIVE RESOLUTION:
[Resolution type] resolves [%] of incidents in this cluster with
an average resolution time of [time] and [%] reopen rate.

KNOWLEDGE GAP:
[Count] incidents in this cluster had no matching KB article.
[Count] incidents were resolved by workaround (no permanent fix).

Step 7: Detect Emerging Clusters

Identify new or growing issue patterns that may indicate emerging problems.

MCP Approach:

Tool: SN-Query-Table
Parameters:
  table_name: incident
  query: sys_created_on>=javascript:gs.daysAgo(7)
  fields: short_description,category,subcategory,cmdb_ci,business_service,priority
  limit: 500

Compare recent volume against baseline:

EMERGING CLUSTER DETECTION:

ALERT: [Cluster Name]
Current Volume (7 days): [count]
Baseline (avg 7-day): [count]
Increase: [%]
Affected CIs: [list]

PATTERN: [Description of the emerging pattern]
POTENTIAL IMPACT: [Assessment of what this could become]
RECOMMENDED ACTION: [Create problem record / Investigate / Monitor]

Step 8: Cross-Channel Correlation

Correlate patterns across incidents, cases, and interactions.

CROSS-CHANNEL CORRELATION:

Topic: [Cluster Name]
Incidents: [count] | Cases: [count] | Interactions: [count]
Total Volume: [sum]

CHANNEL DISTRIBUTION:
  Self-Service: [%]
  Phone: [%]
  Chat: [%]
  Email: [%]
  Walk-up: [%]

INSIGHT: [Description of cross-channel pattern, e.g., "Users first attempt
self-service, fail, then call. Knowledge article KB0012345 is outdated."]

Step 9: Generate POV Summary

Create an actionable point-of-view summary with data-backed recommendations.

=== INSIGHTS POV SUMMARY ===
Analysis Period: [date range]
Data Analyzed: [total records] across [channels]
Generated: [date]

EXECUTIVE SUMMARY:
[2-3 sentence summary of key findings and their business impact]

TOP INSIGHT CLUSTERS:

1. [CLUSTER NAME] - [Priority: HIGH/MEDIUM/LOW]
   Volume: [count] records ([%] of total, [trend] vs prior period)
   Impact: [business impact description]
   Sentiment: [predominant sentiment]
   Root Pattern: [underlying cause or pattern]

   RECOMMENDATION:
   [Specific, actionable recommendation with expected outcome]

   ESTIMATED IMPACT:
   - Reduction in volume: [%] if addressed
   - Time savings: [hours/month]
   - Satisfaction improvement: [projected]

2. [CLUSTER NAME] - [Priority: HIGH/MEDIUM/LOW]
   ...

CROSS-CUTTING THEMES:
- [Theme that spans multiple clusters]
- [Systemic issue identified across data sources]

KNOWLEDGE GAPS IDENTIFIED:
| Topic | Volume | Existing KB | Gap Description |
|-------|--------|-------------|-----------------|
| [topic] | [count] | [None/Outdated] | [What is needed] |

PROACTIVE PROBLEM CANDIDATES:
| Pattern | Volume | Trend | Suggested PRB |
|---------|--------|-------|---------------|
| [pattern] | [count] | [increasing] | [problem description] |

ACTION PLAN:
| # | Action | Owner | Priority | Timeline |
|---|--------|-------|----------|----------|
| 1 | [Action] | [Team] | High | [timeframe] |
| 2 | [Action] | [Team] | Medium | [timeframe] |
| 3 | [Action] | [Team] | Low | [timeframe] |

DATA QUALITY NOTES:
- [Any data quality issues that may affect analysis accuracy]
- [Assumptions made in the clustering process]

Step 10: Create Problem Records for Top Clusters

For clusters that warrant proactive investigation, create problem records.

MCP Approach:

Tool: SN-Create-Record
Parameters:
  table_name: problem
  data:
    short_description: "Proactive: [Cluster description] - [count] related incidents"
    description: |
      CLUSTER ANALYSIS:
      This problem was created based on insights clustering analysis
      identifying [count] related incidents over [time period].

      PATTERN:
      [Description of the recurring pattern]

      SAMPLE INCIDENTS:
      - [INC#]: [short_description]
      - [INC#]: [short_description]
      - [INC#]: [short_description]

      IMPACT:
      [Business impact assessment]

      SUGGESTED INVESTIGATION:
      [Where to start the root cause investigation]
    category: [category]
    priority: [priority based on cluster analysis]
    assignment_group: [relevant team]

Tool Usage

Tool	Purpose	When to Use
SN-Query-Table	Extract incident, case, and interaction data	Primary data collection
SN-Get-Record	Get detailed record for deep analysis	Investigating specific records
SN-Natural-Language-Search	Find related records by description similarity	Cross-referencing patterns
SN-Create-Record	Create problem records from cluster findings	Acting on insights

Best Practices

Use at least 30 days of data for stable clustering results
Combine structured and unstructured data -- categories alone miss nuance; text alone misses context
Set minimum cluster thresholds -- ignore clusters with fewer than 5 records to reduce noise
Compare against prior periods -- trends matter more than absolute numbers
Validate with SMEs -- have domain experts review cluster labels and recommendations
Exclude noise records -- filter out test tickets, duplicates, and auto-generated records
Weight by impact -- a cluster of 10 P1 incidents is more important than 50 P4 incidents
Track cluster evolution -- re-run clustering monthly to see which patterns grow or shrink
Connect to business outcomes -- translate technical patterns into business impact language
Act on findings -- clustering without action is just analytics theater

Troubleshooting

Issue	Cause	Resolution
Too many small clusters	Threshold too low or data too diverse	Increase minimum cluster size; restrict analysis scope
Clusters too broad	Insufficient text granularity	Include description field, not just short_description; add subcategory
Sentiment analysis inaccurate	Limited customer-facing comments	Focus on interaction transcripts; use reopen count as sentiment proxy
No emerging clusters found	Stable environment or insufficient baseline	Extend baseline period; lower detection threshold
Cross-channel data mismatch	Different fields and categories across tables	Normalize category mappings before clustering; use text-based clustering
Volume too large for analysis	Too many records in time range	Narrow time range or filter to specific category/group

Examples

Example 1: Monthly IT Service Review

Scenario: Generate monthly insights for IT service management review.

INSIGHTS POV - IT Services - February 2026

TOP 3 CLUSTERS:
1. VPN Connectivity Issues (187 incidents, +45% vs Jan)
   Driver: Recent network configuration change
   Action: Roll back VPN split-tunnel config change

2. Password Lockouts (142 incidents, stable)
   Driver: MFA policy change confusing users
   Action: Update KB article; add VA topic for guided MFA setup

3. Printer Issues - Building C (68 incidents, NEW cluster)
   Driver: New printer fleet deployment incomplete
   Action: Complete printer driver deployment; update print server config

Example 2: Customer Service Trend Analysis

Scenario: Analyze customer case trends for quarterly business review.

CUSTOMER INSIGHTS - Q1 2026

SENTIMENT TREND: Negative sentiment increased 12% QoQ
PRIMARY DRIVER: Order processing delays (cluster of 234 cases)

CLUSTER ANALYSIS:
- Order Delays: 234 cases (28%) - Negative sentiment 78%
- Billing Disputes: 156 cases (19%) - Negative sentiment 65%
- Product Questions: 312 cases (37%) - Neutral sentiment 80%
- Account Access: 89 cases (11%) - Mixed sentiment

RECOMMENDATION: Order processing system needs capacity upgrade.
Current processing time exceeds SLA in 32% of orders during peak hours.

Example 3: Proactive Problem Identification

Scenario: Detect emerging patterns for proactive problem management.

EMERGING PATTERN DETECTED:

Application: SAP Finance Module
7-Day Volume: 23 incidents (baseline: 8/week)
Increase: 188%
Affected Users: Finance department, 3 locations

Pattern: "SAP report timeout" and "SAP slow response" keywords
appearing with increasing frequency since March 12.

Correlated Event: Database maintenance performed March 11.

RECOMMENDATION: Create problem record PRB0005678.
Investigate database performance post-maintenance.
Potential index rebuild needed.

Related Skills

genai/agent-miner - Mine agent interactions for automation opportunities
knowledge/gap-analysis - Identify knowledge base coverage gaps
knowledge/gap-grouping - Group knowledge gaps by theme
itsm/problem-analysis - Root cause analysis for identified problems
reporting/trend-analysis - Trend analysis and visualization
csm/sentiment-analysis - Customer sentiment analysis

About this skill

Overview

Prerequisites

Procedure

Step 1: Define Analysis Scope

Step 2: Extract Incident Data for Clustering

Step 3: Extract Case and Interaction Data

Step 4: Perform Topic Clustering

Step 5: Analyze Sentiment Patterns

Step 6: Analyze Resolution Patterns

Step 7: Detect Emerging Clusters

Step 8: Cross-Channel Correlation

Step 9: Generate POV Summary

Step 10: Create Problem Records for Top Clusters

Tool Usage

Best Practices

Troubleshooting

Examples

Example 1: Monthly IT Service Review

Example 2: Customer Service Trend Analysis

Example 3: Proactive Problem Identification

Related Skills