AWS can automate Hive-to-Iceberg migration—but the agent must not automate your cutover decision
S3 Tables now handles planning, copying and incremental sync. Its companion agent skill correctly treats approval, validation and completion as separate control points.
AWS has launched a managed path for converting Glue-registered Hive tables into managed Apache Iceberg tables in S3 Tables. The service validates, converts and copies data while the source remains available, then supports incremental synchronization before cutover. AWS also published an Agent Toolkit skill that can orchestrate the workflow—but the useful part is not merely automation. It is the explicit separation between planning, approval, validation and completion.
Start with the launch limits
The initial service is narrower than “migrate any data lake.” AWS says the source must be a Glue-registered Hive table stored as Parquet, and the destination must be an S3 table bucket in the same account and Region. The migration creates a second, managed Iceberg table; it does not convert the source in place. The destination can be configured for compression, target file size and snapshot management, while small files are compacted during the copy.
That scope should be the first eligibility gate. The companion skill says Iceberg sources, raw files, non-Parquet formats, cross-account moves and self-run CTAS, Glue ETL or Spark migrations are outside this workflow. It also identifies Lake Formation-registered tables and partitions outside the table location as blockers that require stopping rather than silently changing governance settings.
Treat the plan as a change request
S3 Tables can produce compatibility reports before copying data and can pause in a pendingReview state. The AWS skill requires an agent to show the source, destination, write paths, readers, saved queries and permissions before proceeding. It also requires explicit approval before changes to IAM, KMS or buckets—and separate approval for policies that replace incompatible values with nulls or drop unmatched fields.
That is the right operating model. A schema-conversion job can report technical compatibility without knowing whether a dropped field breaks a finance dashboard or whether a direct-to-S3 writer bypasses Iceberg metadata. Platform teams should make the generated migration plan part of their normal review trail, not treat an agent’s successful preflight as authorization to proceed.
Validate before completing
AWS allows the source to keep receiving writes during the initial copy, followed by repeatable sync operations. Before completion, the skill calls for zero failed objects, object-count reconciliation, row-count parity, per-partition counts, spot checks of converted values and review of every schema conversion. Only after those checks—and another explicit approval—should the migration be completed and the destination promoted to normal read-write operation.
Completion is a meaningful boundary because the skill says synchronization is refused afterward. The practical runbook is therefore: inventory writers and consumers, start with plan review, inspect compatibility reports, approve the copy, sync until current, validate through an Iceberg-aware query engine, move workloads deliberately, and complete only when rollback to the still-available source is no longer the preferred recovery path.
The managed service removes custom migration infrastructure. It does not remove the need to own schema decisions, permission changes and cutover timing. AWS’s agent skill is strongest when it preserves those human decisions instead of hiding them.
sources
- Amazon S3 Tables now provide a simple, cost-effective path for data lake modernizationaws.amazon.com
- Migrating tables to S3 Tablesdocs.aws.amazon.com
- AWS Agent Toolkit: migrating Hive tables to S3 Tables skillraw.githubusercontent.com
comments · 0