ETL Migration
Accelerated SSIS ETL Migration to Azure Data Factory
A leading multinational manufacturer of food products needed to migrate integrations developed in SSIS to Azure Data Factory (ADF) for one of its pet care businesses. The customer required an accelerated migration solution with minimum disruption to the live applications and enablement of disaster recovery.
Client Challenges and Requirements
- SSIS (IaaS) to Azure Data Factory (PaaS)
- Re-engineering approach (convert SSIS packages to native ADF components)
- Enabling Disaster Recovery (DR) – Run mock failover DR test
- Cloud Infrastructure setup inputs
- Solutioning challenges considering customer’s private network policies
- Migrating live batch with ongoing code updates
- Need for custom DR solution
Bitwise Solution
- Infra assistance: In-depth analysis of infra topology to create optimal solution architecture compatible with customer’s cloud infra policies and limitations
- Auto conversion: Use of Bitwise ETL Converter for as-is migration of SSIS control flows to native ADF code
- Smart solutioning: Efforts reduced by re-designing existing batch with metadata driven approach making it easier to maintain and support
- Reduced Azure costs: Eliminated dataflows in ADF by relocating transformations in them to input SQLs. This also decluttered cloud infrastructure architecture
- Automated data validations using Bitwise accelerator for comparing output feeds from both ETL tools resulted in quicker integration testing, also helping business users to sign-off on migration with confidence
- Value add in existing design by uncovering system design flaws and fixing them
- Custom DR solution to meet customer requirement for 24h RTO, 4h RPO
Tools & Technologies We Used
ADF
SQL-MI
SSIS
Azure Services
SSMS
Custom utilities
Key Results
100% successful migration with reduced effort and cost through automation
DR enabled solution with mock failover tests
Established standard coding practices for customer’s migrations to ADF
Estimated $170K annual savings in Azure costs with optimized application design
