Introduction
From Scattered to Structured Data Landscape
Client
Financial Services
Client since
Services
Solutions
Technologies
The problem
A scattered data landscape causing discrepancies in reports
The initial challenge at our client was apparent – a scattered data landscape causing discrepancies in reports due to multiple versions spread across different spaces. The audience-centric approach further fueled duplication of reports, creating static filters based on business lines instead of universally applicable dynamic filters. This resulted in varying datasets for the same report, multiplying the risk of discrepancies and undermining key performance indicators (KPIs). Maintenance of reports became a daunting task, lacking clear demarcation between development, testing, and production phases. We took up the challenge and started by indicating what success would look like.
How we solved it
Creating the right indicators for success
In response to these challenges, a strategic plan emerged, emphasizing the migration from scattered on-premises and Cloudera databases to a streamlined structure on AWS. The migration strategy focused on three key success indicators: creating high-performing and future-proof data models, improving reporting environment manageability, and ensuring data clarity and quality.
- Documenting as-is and define to-be: A meticulous documentation process began by collaborating closely with data owners. An extensive inventory of existing reports and datasets, along with their usage patterns, provided a comprehensive understanding of the reporting landscape. This step enabled the identification of outdated components and the merging of similar reports, addressing the challenge of overgrowth.
- Design the new infrastructure: The migration strategy involved the design and implementation of a new reporting infrastructure. A development pipeline was established with workspaces per development phase, differentiating between development, testing, and production. Reports and datasets were separated to enhance maintainability.
- Data modelling and dataset migration: The data models were revamped during the AWS migration, adhering to best practices such as star modeling and utilizing technical keys. Datasets were created in workspaces tied to specific departments, ensuring consistency across reports and accurate data and calculations.
- Reporting migration: Reports with similar subjects were consolidated during the reporting migration phase, addressing the issue of report overgrowth. Filters were introduced to empower users to customize information based on their preferences.
- Continuous validation by business: Throughout the migration process, constant communication with key end-users and data owners ensured a smooth transition. The validation by business users, a crucial step, was ongoing to maintain alignment with business requirements.
Creating the right indicators for success
In response to these challenges, a strategic plan emerged, emphasizing the migration from scattered on-premises and Cloudera databases to a streamlined structure on AWS. The migration strategy focused on three key success indicators: creating high-performing and future-proof data models, improving reporting environment manageability, and ensuring data clarity and quality.
- Documenting as-is and define to-be: A meticulous documentation process began by collaborating closely with data owners. An extensive inventory of existing reports and datasets, along with their usage patterns, provided a comprehensive understanding of the reporting landscape. This step enabled the identification of outdated components and the merging of similar reports, addressing the challenge of overgrowth.
- Design the new infrastructure: The migration strategy involved the design and implementation of a new reporting infrastructure. A development pipeline was established with workspaces per development phase, differentiating between development, testing, and production. Reports and datasets were separated to enhance maintainability.
- Data modelling and dataset migration: The data models were revamped during the AWS migration, adhering to best practices such as star modeling and utilizing technical keys. Datasets were created in workspaces tied to specific departments, ensuring consistency across reports and accurate data and calculations.
- Reporting migration: Reports with similar subjects were consolidated during the reporting migration phase, addressing the issue of report overgrowth. Filters were introduced to empower users to customize information based on their preferences.
- Continuous validation by business: Throughout the migration process, constant communication with key end-users and data owners ensured a smooth transition. The validation by business users, a crucial step, was ongoing to maintain alignment with business requirements.
The results
A structured Data Landscape
The migration yielded transformative outcomes. Workspaces are now meticulously managed, with a dedicated deployment pipeline for datasets and reports, maintaining separation between development, testing, and production. A consistent central dataset is used across all reports, and improved workspace structure enhances business confidence and understanding of the reporting landscape. In essence, the Power BI dataset and report migration not only resolved existing issues but also established a well-organized system for efficiency and clarity.
Ready to transform your Data Landscape? Reach out to learn how Datashift can support you on this journey.
Key Learnings
More case studies
We deliver impact where it matters most.
Unlocking Viewer Insights with a Scalable Data Platform
Streamz is the Flemish video-on-demand platform offering local and international series and films through a subscription-based model. Streamz focuses on delivering high-quality content tailored to local audiences, supported by a strategic partnership with Paramount.
