Skip to content
For professionalsBook a call
Case studiesData engineering and reporting

Breaking the 14-hour reporting barrier

Truffle brought specialist expertise to a six-month reporting challenge, with a redesigned pipeline projected to cut processing time by 96%.

By Truffle teamPublished

Read the story Download PDF
Original Truffle geometric artwork in pink and purple.
Complex data. Clear direction.
Business need
Contractor safety reporting
Specialist
Achmad, data engineer
Engineering work
June 2026, early in the engagement

Problem

Critical reporting. An incomplete picture.

The customer's contractor safety team depended on work-hours data it could trust. Yet daily processing took about 14 hours, and dashboard totals had failed to match the source for roughly six months.

The problem had persisted despite previous attempts by multiple teams. Behind a single Power BI dashboard lay several connected failures, demanding expertise across the entire system to tackle both speed and accuracy.

    Solution

    The expertise to see the whole picture.

    Truffle data engineer Achmad stepped into an unfamiliar environment and took on the challenge from source to dashboard. Working across Azure Data Factory, Azure SQL, and Power BI, he uncovered the bottleneck and the hidden gaps behind the unreliable totals.

    Professionals working together in a workshop.
    Expertise across the whole reporting chain.
    • Rebuilt for speedReplaced one-project-at-a-time processing with parallel execution, validated through triggered runs.
    • Uncovered the hidden gapsFixed SQL logic that silently excluded work hours and traced broken site mappings and archived projects.
    • Revealed a major blind spotIdentified about 13.71 million contractor work hours excluded by a pre-2019 cutoff.
    Of mismatched work-hour totals
    6 months

    Approximate period when dashboard totals did not match the contractor platform.

    Historical contractor work hours
    13.71M

    Approximate total excluded by the dashboard date cutoff.

    Technical findings

    Inside the reporting challenge

    • One project at a time

      Rebuilt the pipeline to process projects in parallel.

      Fixed
    • Records dropped by SQL

      Corrected and validated SQL handling of blank archive-status values so affected records were no longer excluded.

      Fixed
    • Broken project mappings

      Traced the disconnected source linking projects to sites.

      Identified
    • Historical work-hour cutoff

      Identified the pre-2019 date cutoff excluding historical work hours.

      Identified
    • Archived project records

      Mapped archived projects whose work hours were not reaching the dashboard.

      Identified

    Reward

    Less waiting. More clarity. New momentum.

    The redesigned pipeline creates a path to same-day reporting, with a projected 96% reduction in processing time. The customer also gained the insight to address months of reporting discrepancies.

    Projected processing-time savings

    Projected processing time saved per run
    13.5 hours
    Projected reduction in processing time
    96%
    Approximate daily processing time
    14 hours
    Projected full processing run
    25-30 min
    • A shorter wait for dataA projected 13.5 hours saved per run brings a 25-30-minute processing window within reach.
    • Clarity that drives progressThe investigation replaced unexplained mismatches with specific causes the customer could act on.