Statcode
Services

Problems We Solve

Every organization has its own unique challenges, below are some common ones we've helped solve. Click "View All" to begin

Incomplete or Low-Quality Administrative Data

Cause:

Collected for operational—not evaluative—purposes; may lack consistency or detail.

Solution:

  • Conduct a data audit early in the process.
  • Supplement with other data sources like surveys or interviews.
  • Partner with data custodians to improve accuracy and accessibility.

Limited Access to Secondary Data

Cause:

Restrictions due to privacy, ownership, or licensing issues.

Solution:

  • Establish data-sharing agreements upfront.
  • Use open data alternatives or aggregate-level datasets where possible.
  • Seek Institutional Review Board (IRB) guidance if ethics are involved.

Survey Fatigue & Low Response Rates

Cause:

Over-surveyed populations, unclear incentives, or long questionnaires.

Solution:

  • Keep surveys short and relevant.
  • Use mixed methods (e.g., online + phone).
  • Provide incentives or partner with trusted community leaders.

Misaligned Data With Evaluation Goals

Cause:

Data doesn't answer the core implementation or impact questions.

Solution:

  • Define evaluation questions and logic model before data collection.
  • Choose or adapt tools that match those goals.
  • Use process evaluations to bridge context and goals.

Time Lag in Outcome Data

Cause:

Some effects take months or years to manifest (e.g., education, health).

Solution:

  • Use intermediate indicators as proxies.
  • Conduct longitudinal or follow-up evaluations.
  • Combine quantitative data with qualitative insights.

Context-Specific Findings Limit Generalizability

Cause:

Evaluation reflects a single time/place/policy context.

Solution:

  • Document contextual variables thoroughly.
  • Use translation process evaluations to adapt findings.
  • Plan pilot replication studies before scaling.

Data Overload Without Insight

Cause:

Large volumes of data collected without a clear analytic framework.

Solution:

  • Use a theory of change or evaluation matrix to structure analysis.
  • Visualize key metrics and trends with dashboards or summary reports.
  • Prioritize actionable insights over exhaustive reporting.

Data is Hard to Interpret

Cause:

Overly complex charts or raw data with no structure.

Solution:

  • Use clear, familiar visual forms (bar, line, scatter).
  • Apply visual hierarchy and simplify layouts using Gestalt principles.

Audience Misreads the Message

Cause:

Assumes viewers know how to read the visualization or interpret the data context.

Solution:

  • Add explanatory titles, annotations, and visual cues that highlight the key takeaway directly on the chart.

Charts are Overloaded with Information

Cause:

Trying to answer too many questions in one visualization.

Solution:

  • Focus on one insight per chart.
  • Break down complex topics into small multiples or sequenced visuals.

Data Lacks Context

Cause:

No benchmarks, comparisons, or prior data for reference.

Solution:

  • Add baselines, targets, historical comparisons, or peer data to provide meaning and relevance.

Stakeholders Resist Using Data

Cause:

Data feels irrelevant or imposed, not aligned with their work or goals.

Solution:

  • Co-create metrics and visuals with stakeholders.
  • Use language and framing that connects to their priorities.

Too Many Tools, Not Enough Clarity

Cause:

Fragmented tech stack with inconsistent outputs.

Solution:

  • Standardize tools across teams.
  • Choose ones that are intuitive and accessible for both analysts and non-technical users.

Data is Siloed Across Departments

Cause:

Lack of data integration and cross-functional communication.

Solution:

  • Create a centralized database or dashboard.
  • Assign data stewards or cross-team leads to bridge silos.

Dashboard Fatigue

Cause:

Static, repetitive, and overly detailed dashboards that lose relevance.

Solution:

  • Customize dashboards for each audience.
  • Use alerts, highlights, or narratives to make changes stand out.

Visuals Don't Match Your Brand

Cause:

No visual identity guidelines for data communications.

Solution:

  • Develop a style guide for colors, fonts, and chart types.
  • Align visuals with your organization's tone and values.

Flashy Graphics Obscure Meaning

Cause:

Prioritizing aesthetics or novelty over clarity.

Solution:

  • Use simple, clean 2D graphics.
  • Remove chart junk and distractions.
  • Focus on what matters most.

Visuals are Not Accessible

Cause:

Lack of awareness of accessibility needs (color, format, structure).

Solution:

  • Use high-contrast, colorblind-friendly palettes.
  • Include alt text, descriptive labels, and readable typography.

Visuals Don't Lead to Action

Cause:

Charts show data, but no clear recommendation or next step.

Solution:

  • Add callouts for decisions, flags for issues, or action prompts directly linked to the data.

Too Many KPIs, Not Enough Focus

Cause:

Misalignment between data collection and strategic goals.

Solution:

  • Select 3–5 core metrics.
  • Organize KPIs into primary (strategic) and secondary (supporting) layers.

Weekly Reports Go Unread

Cause:

Reports are dense, dry, and disconnected from current decisions.

Solution:

  • Use engaging formats (infographics, dashboards, video explainers).
  • Keep it visual and narrative-driven.

Staff Lacks Data Literacy

Cause:

No training or support in reading and using charts.

Solution:

  • Run regular workshops, data jams, or office hours.
  • Empower staff with simple interpretation guides.

Visuals Lack Consistency

Cause:

No shared templates or visual standards.

Solution:

  • Create a design system with templates for common charts and dashboards to ensure consistency and efficiency.

Data Isn't Up-to-Date

Cause:

Manual reporting or fragmented data sources.

Solution:

  • Automate data updates with ETL tools.
  • Sync dashboards to real-time or frequent data refreshes.

Reports Lack Emotional Engagement

Cause:

Over-focus on numbers, ignoring human impact.

Solution:

  • Add qualitative stories, quotes, or images alongside data.
  • Humanize insights to drive empathy and connection.

Leaders Ignore Data for Gut Instinct

Cause:

Data doesn't feel actionable or trustworthy.

Solution:

  • Deliver insights with confidence using the "Flash Roll" — a prepared, fluent pitch showing you've seen this before and know what to do.

Data is Only Used for Compliance

Cause:

Culture sees data as something for reporting, not learning.

Solution:

  • Shift to a learning culture.
  • Celebrate data-driven decisions.
  • Use visuals to provoke curiosity and experimentation, not just track KPIs.

Slow dbt Runs

Cause:

Inefficient SQL models or lack of model materialization strategy.

Solution:

  • Optimize SQL logic, use incremental or ephemeral models where appropriate, and refactor long-running models.

Hard-to-Debug Broken Pipelines

Cause:

Poor documentation and unclear model dependencies.

Solution:

  • Use dbt's built-in documentation and DAG (Directed Acyclic Graph) to visualize and trace dependencies.

Model Sprawl (Too Many Models)

Cause:

Lack of naming conventions or folder organization.

Solution:

  • Apply consistent naming conventions and a modular folder structure (like staging → intermediate → marts).

Stale or Inaccurate Data

Cause:

Missing freshness checks or tests.

Solution:

  • Implement dbt tests (e.g., unique, not_null) and freshness checks for source data.

Low Stakeholder Trust in Data

Cause:

Lack of data testing or transparency.

Solution:

  • Use dbt tests, generate documentation, and communicate data lineage to business users.

Inconsistent Metric Definitions

Cause:

Metrics defined in multiple tools or places.

Solution:

  • Centralize metric logic using dbt's metrics layer or reusable CTEs in models.

Deployment Fails Without Clear Reason

Cause:

Missing CI/CD or local testing workflows.

Solution:

  • Set up a CI pipeline with dbt Cloud or GitHub Actions to validate models before deployment.

Overwritten or Lost Model Changes

Cause:

Lack of version control practices.

Solution:

  • Use Git with pull requests and code reviews for all model changes.

No Visibility Into Model Lineage

Cause:

Unused dbt documentation or no DAG review.

Solution:

  • Regularly generate and share dbt docs to visualize lineage and model relationships.

Duplicate Logic Across Models

Cause:

Copy-pasting SQL instead of using Jinja macros or sources.

Solution:

  • Abstract logic into reusable macros or common CTEs.

Unclear Data Ownership

Cause:

Shared responsibility across teams with no documentation.

Solution:

  • Define ownership and model tags in dbt_project.yml.

Cost Overruns in the Warehouse

Cause:

Non-optimized queries or full table refreshes.

Solution:

  • Use incremental models, limit unnecessary joins, and audit warehouse costs.

Team Doesn't Understand dbt Workflows

Cause:

Poor onboarding or documentation.

Solution:

  • Create internal training materials and enforce code standards via dbt style guides.

Hard to Onboard New Team Members

Cause:

Missing project-level documentation.

Solution:

  • Document folder structure, model purposes, and run instructions in a central README.

Inconsistent Run Environments

Cause:

Differences between dev and prod environments.

Solution:

  • Use dbt profiles to define and test environment-specific configurations.

Schedule Drift (Jobs Not Running as Expected)

Cause:

Misconfigured scheduler or pipeline orchestration.

Solution:

  • Use dbt Cloud scheduling or integrate with tools like Airflow, Prefect, or Dagster.

Data Updates Aren't Traceable

Cause:

No logging or audit trails for dbt runs.

Solution:

  • Enable dbt artifacts, use logging, and version output tables.

Long Turnaround Time for Changes

Cause:

Bottlenecks in testing or approval processes.

Solution:

  • Automate tests and CI pipelines to accelerate development cycles.

Lack of Monitoring for Failed Models

Cause:

No alerting set up for dbt runs.

Solution:

  • Set up alerts via dbt Cloud, Slack integrations, or use orchestration alerts.

Misalignment with Business KPIs

Cause:

Technical teams not syncing with business needs.

Solution:

  • Include business stakeholders in model review processes and translate metrics into accessible terms.

Inconsistent Column Formats

Cause:

Data collected from multiple sources with differing standards.

Solution:

  • Standardize formats using scripts (e.g., Pandas, SQL) early in the ETL pipeline.

Missing Values

Cause:

Incomplete data entry, sensor failure, or system errors.

Solution:

  • Impute with statistical methods, drop rows, or flag and route for human review.

Duplicated Records

Cause:

Data ingestion from overlapping sources.

Solution:

  • Use drop_duplicates() or ROW_NUMBER() in SQL with clear deduplication logic.

Typos and Misspellings

Cause:

Manual entry or OCR errors.

Solution:

  • Use fuzzy matching or dictionaries to normalize string values.

Inconsistent Date and Time Formats

Cause:

Systems using different locales or formatting standards.

Solution:

  • Parse all dates to ISO-8601 or standard datetime formats using libraries like dateutil.

Mixed Data Types in a Single Column

Cause:

Poor data validation at collection.

Solution:

  • Coerce data types and create error logs for manual review.

Outliers Distorting Analyses

Cause:

Errors in measurement, entry, or edge cases.

Solution:

  • Use statistical thresholds (IQR, Z-score) to flag, investigate, and address.

Encoding Issues (e.g., UTF-8 vs ANSI)

Cause:

Mismatched encoding across files or sources.

Solution:

  • Standardize file encodings at ingestion and convert with appropriate libraries.

Column Names with Inconsistent Naming

Cause:

Manual column entry or lack of naming standards.

Solution:

  • Rename columns consistently and use naming conventions (snake_case, camelCase).

Blank vs Null vs 0 Confusion

Cause:

Semantic differences across systems.

Solution:

  • Define and enforce a missing value policy, and clearly document how each value is treated.

Mismatched IDs or Foreign Keys

Cause:

Disparate databases or legacy systems.

Solution:

  • Use mapping tables or fuzzy joins, and audit data lineage.

Truncated or Overflowed Data

Cause:

Field size limitations in source systems.

Solution:

  • Increase field length during ingestion and validate against original source.

Units of Measure Inconsistencies

Cause:

Manual entry or international data sources.

Solution:

  • Convert units to a common standard (e.g., metric vs imperial) using conversion tables.

Non-normalized Categorical Data

Cause:

Free-form text entries.

Solution:

  • Use controlled vocabularies, lookup tables, or drop-down input forms upstream.

Unnecessary Whitespace or Hidden Characters

Cause:

Copy-paste or OCR errors.

Solution:

  • Strip whitespace and remove non-printable characters with regex or string methods.

Lack of Metadata or Data Dictionaries

Cause:

Informal data collection processes.

Solution:

  • Document data structure, meaning, and provenance using data catalog tools.

Case Sensitivity Errors

Cause:

Systems treat "ABC" and "abc" differently.

Solution:

  • Normalize text case early in preprocessing.

Conflicting Sources of Truth

Cause:

No master data management strategy.

Solution:

  • Establish source of truth and reconciliation logic using version control or data contracts.

Poorly Formatted or Nested JSON/XML

Cause:

Inconsistent API responses or malformed files.

Solution:

  • Use schema validation, flattening utilities, and recursive parsing tools.

Manual, Repeatable Tasks

Cause:

No automation in data pipeline.

Solution:

  • Automate using dbt, Python scripts, or low-code ETL tools like Airbyte or Fivetran.

Frequently Asked Questions