SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillssynapse-migration

synapse-migration

Data & analytics

View repository

Updated 10-16 06:47

About this skill

Telemetry — MANDATORY. Every api.fabric.microsoft.com call must carry x-ms-fabric-skill: synapse-migration (az rest: --headers "x-ms-fabric-skill=synapse-migration"), including every LRO poll, fabric_lro and retry. Snippets omit it — add it anyway.

OFFLINE LAKEHOUSE ARTIFACT FAST PATH — DECIDE FIRST. For a no-live-call request with complete supplied inputs and contracts, load only the fast path; do not load migration-planning-reference.md, dedicated-pool-conversion.md, or implementation scripts. Use one generator for artifacts and final-byte hashes, then one verifier. Do not inspect generated files or run post-pass spot checks. For every optional non-null source default, set missingValueBehavior to UseSourceDefault and restore that default before validation. Do not use this path for a Warehouse target; incomplete and large-procedure requests use their own routes.

CRITICAL NOTES

  1. To find workspace details (including its ID) from a workspace name: list all workspaces, then use JMESPath filtering.
  2. To find item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace, then use JMESPath filtering.
  3. Treat a source Dedicated SQL Pool as read-only. Do not run source DDL, DML, stored procedures, or create sample objects.
  4. Dedicated Pool-to-Lakehouse data movement is out of scope. Migrate schema and executable code artifacts only; do not copy source rows or run row-level/business-result equivalence queries. The separate Warehouse route may move data only from a separately approved restored export copy under its own consent gate and restored-source contract.
  5. For Dedicated Pool-to-Lakehouse requests, complete gap assessment, explicit mapping approval, manifest creation, conversion, deployment, and validation in order. Read dedicated-pool-to-lakehouse.md once as the router, then load only the current phase reference.
  6. Stored-procedure 1:1, N:1, and N:N mappings never apply to views. Views remain schema objects; procedures become traceable Spark SQL notebooks with executable %%sql transformation cells.
  7. Dependent ADF/Synapse callers are conditional scope, not automatic scope. Discover them during assessment, require a separate complete-closure approval, and migrate them only after notebook readiness. Route standalone pipeline requests to pipeline-migration.
  8. Before executing a specialized Dedicated Pool workflow, read the matching contract in dedicated-pool-execution-contracts.md. Its validation, hashing, request-body, resume, and completion-language requirements are mandatory.
  9. Keep Lakehouse and Warehouse paths separate. If the target is unclear, ask before loading path-specific resources. Lakehouse conversion never moves source rows; Warehouse data migration requires its own explicit consent gate and restored-source contract.
  10. For a Fabric Warehouse target, explicitly state that the complete DISTRIBUTION and CLUSTERED COLUMNSTORE INDEX declarations are removed because Fabric manages distribution, storage, and indexing; a bare mention of either source clause is insufficient.
  11. Run setup DDL and CETAS only against the separately approved restored export copy after proving it is not the original production source.
Synapse Analytics to Microsoft Fabric Migration

Core Workflow

  1. Discover the Synapse workspace and classify every source artifact.
  2. Choose targets using workload-routing.md.
  3. Assess gaps before conversion, including runtime, library, connectivity, T-SQL, security, and capacity constraints.
  4. Obtain approval for behavior-changing decisions, target placement, and every stored-procedure-to-notebook mapping.
  5. Create or update migration-manifest.json and persist approval, per-object status, hashes, and checkpoints atomically.
  6. Convert and deploy in dependency order, loading only the current phase resource.
  7. Validate persisted definitions and artifacts without executing generated notebooks or pipelines unless a separately approved workflow explicitly requires execution.
  8. Report completed, blocked, skipped, and pending objects without exposing credentials or sensitive values.

For full-workspace orchestration, read migration-orchestrator.md. For a Dedicated Pool-to-Lakehouse migration, use dedicated-pool-to-lakehouse.md as the phase router. For a Warehouse target, load only the matching dw-* resource.

Scope Routing

Request Action Load
Full Synapse workspace migration Run phased discovery and migration migration-orchestrator.md
Dedicated SQL Pool schema/code to Lakehouse Run the strict assessment-to-validation workflow. For guidance-only notebook publication, explicitly state: "The stored procedures were converted from T-SQL to Spark SQL before publication." dedicated-pool-to-lakehouse.md
Guidance-only Dedicated Pool incremental deployment hardening Answer directly from the focused checklist; do not search scripts or load phase resources. State: "For columns with special characters, enable Delta columnMapping selectively and only with approval." dedicated-pool-execution-contracts.md
Large or complex stored procedure For a complete offline contract, read only the audit resource once, use one run-local generator and one verifier, do not inspect implementation scripts or generated outputs, and stop after successful verification dedicated-pool-large-procedure-audit.md
Spark pool Convert configuration and libraries to a Fabric Environment spark-pool-migration.md
Lake Database or HMS Create or bind the target Lakehouse lake-database-migration.md or external-hms-migration.md
Notebook or Spark Job Definition Port code, bindings, parameters, and runtime settings spark-item-migration.md
Linked Service Choose a Fabric Data Connection for external services or a OneLake Shortcut for ADLS Gen2/Blob connectivity-migration.md
Approved dependent procedure callers Run the internal closure-preserving sub-flow after notebook readback dedicated-pool-dependent-pipelines.md
Standalone pipeline migration Delegate pipeline-migration
Dedicated SQL Pool to Fabric Warehouse Run the source extraction, DDL compatibility, approval, optional data, security, and validation route dw-source-and-extraction.md, then the matching dw-* phase resource
Standalone Fabric Warehouse administration Delegate sqldw-cli
Synapse Link Use Fabric Mirroring; not covered here —

Always report the Lake Database/HMS phase. If none is discovered, mark it NotApplicable (none discovered) while retaining the required target Lakehouse binding.

Load References On Demand

Do not preload all files. Load only the references needed for the current phase:

Need Reference
Routing, phase order, API audiences, quick examples, troubleshooting, or post-migration handoff workload-routing.md
Dedicated Pool execution variants and response contracts dedicated-pool-execution-contracts.md
Dedicated Pool discovery dedicated-pool-discovery.md
Gap assessment and mapping approval dedicated-pool-gap-assessment.md
Procedure conversion and notebook shape dedicated-pool-conversion.md
Schema/notebook deployment dedicated-pool-deployment.md
Dedicated Pool validation dedicated-pool-validation.md
Runtime/library compatibility feature-parity.md and library-compatibility.md
mssparkutils and connector refactoring utility-api-mapping.md, connector-refactoring.md, and code-patterns.md
Capacity planning capacity-sizing.md
Dedicated Pool-to-Warehouse source discovery and extraction dw-source-and-extraction.md
Warehouse DDL compatibility and deployment dw-ddl-compatibility.md
Separately approved Warehouse data migration dw-data-migration.md
Warehouse security and validation dw-security-validation.md
Validation, reporting, and production readiness validation-testing.md, migration-report.md, and security-governance.md

Use COMMON-CLI.md only for shared authentication and az rest mechanics not already covered by the phase resource. Use COMMON-CORE.md only for shared Fabric REST patterns.

Must

  • Preserve every stored-procedure input contract, source default, source decision, and approved target mapping.
  • Block conversion until gap dispositions, notebook names, dependency grouping, and target workspace placement are explicitly approved.
  • Use SqlPackage/DMVs for source discovery, Fabric REST for item management, and Fabric Livy statements for Lakehouse schema and Delta execution.
  • Run the maintained validators required by the selected execution contract; never replace them with ad hoc JSON or cell-shape checks.
  • Choose the Lakehouse or Warehouse target before loading implementation resources; never apply Warehouse data-movement rules to Lakehouse artifact conversion.
  • Before persisting any dependent-pipeline source definition, recursively inspect it for secret-bearing or uncertain properties; block the complete selected closure without writing the artifact or mutating a target when any are found.
  • In every dependent-pipeline response, including guidance-only and pre-approval explanations, state: "Use a TridentNotebook activity with exact target identities typeProperties.notebookId and typeProperties.workspaceId, and pass inputs through typeProperties.notebookParameters." Then state: "After deployment, read back each pipeline with POST /v1/workspaces/{workspaceId}/items/{dataPipelineId}/getDefinition and validate its canonical hash and structural bindings."
  • For every Linked Service response, first state both replacement classes: Fabric Data Connections for external databases/services and OneLake Shortcuts for ADLS Gen2/Blob storage; then identify which class applies.
  • Replace mssparkutils with notebookutils, Linked Services with the correct Fabric connection class, and spark.read.synapsesql() with a supported OneLake or JDBC pattern.
  • Externalize workspace and item IDs through parameters or Variable Libraries.
  • Keep phase status concise, emit bounded heartbeats for long operations, and persist per-object evidence in the manifest.

Prefer

  • OneLake Shortcuts over copying data.
  • Fabric Environments for reproducible libraries and Spark settings.
  • Incremental, dependency-ordered migration over big-bang cutover.
  • Parameterized notebooks and Variable Libraries for environment promotion.
  • Lakehouse SQL Endpoint for compatible Serverless SQL Pool read workloads.

Avoid

  • Do not execute generated notebooks or pipelines as part of artifact migration.
  • Do not broaden a selected dependent-pipeline closure or route that internal sub-flow to pipeline-migration.
  • Do not use sqlcmd against the target Fabric Warehouse; use the registered SQL endpoint tooling required by the Warehouse resource.
  • Do not preserve Synapse distribution/storage hints, PolyBase DDL, Linked Service connection strings, or pool-level library installation patterns verbatim.
  • Do not use target notebooks to orchestrate the migration itself.
  • Do not hardcode workspace IDs, item IDs, credentials, connection strings, or environment-specific paths.

Examples

Prompt: "Migrate this Dedicated SQL Pool DACPAC to a Fabric Lakehouse and convert the procedures."

Response behavior: Assess gaps first, present and obtain approval for the complete procedure-to-notebook mapping and workspace placement, create the manifest, convert approved schema/code artifacts without moving source rows, validate persisted definitions, and report the resolved Lakehouse and artifact evidence.

Prompt: "Migrate this ADF pipeline to Fabric."

Response behavior: Route to pipeline-migration. Keep it here only when the pipeline is an explicitly approved dependent caller discovered inside an active Dedicated Pool-to-Lakehouse migration.

Prompt: "Migrate this Dedicated SQL Pool to Fabric Warehouse."

Response behavior: Keep this inside synapse-migration, assess DDL compatibility, remove unsupported distribution and columnstore declarations, obtain provisioning consent, and ask separately before any restored-source data migration.

For code transformations and edge cases, load workload-routing.md and the phase-specific resource rather than expanding the main instructions.