Forecasting Demand With Hierarchical Data: Reconciliation Made Simple

In modern supply chain management, data is rarely a single, flat stream. A national retailer does not just look at total sales; they examine sales by region, then by store, then by category, and finally at the individual SKU level. This nested structure creates a mathematical challenge where independent forecasts at different levels rarely sum up correctly. If the store-level forecast predicts a demand for 100 units but the sum of its individual SKU forecasts equals 120, the supply chain faces a synchronization failure that leads to either stockouts or bloated inventory costs.

Hierarchical forecasting is the discipline of ensuring that these multi-level predictions remain coherent. By applying reconciliation techniques, data scientists can force the aggregate parts to equal the whole while often improving the accuracy of individual nodes. In this guide, we will move beyond basic time-series modeling to explore the mechanics of reconciliation, the trade-offs between different aggregation strategies, and the implementation of optimal combination methods using modern computation frameworks.

The structure of hierarchical time series

A hierarchy in time series data is typically represented as a tree or a directed acyclic graph. At the root, we have the total aggregate (Level 0). Below that, the data branches into child nodes based on attributes like geography or product taxonomy. For example, a global manufacturer might branch from Total Revenue to North America and Europe, and then further branch into specific product lines. The critical characteristic of these structures is the summation constraint: the value of a parent node at time t must equal the sum of its children at time t.

We distinguish between hierarchical and grouped time series. In a strict hierarchy, the levels have a unique, nested relationship (e.g., a city is in exactly one state). In grouped time series, also known as crossed hierarchies, the groupings can overlap. A product might belong to a 'Electronics' category and also a 'High-Margin' group. While the math for reconciliation is similar for both, crossed hierarchies introduce significantly more complexity in the mapping matrix, often requiring sparse matrix optimizations to handle the computational load.

Mathematically, we represent the hierarchy using a summing matrix, often denoted as S. If we have a vector of bottom-level observations b_t, the full vector of all observations across all levels y_t is calculated as y_t = S * b_t. The matrix S is a binary matrix of dimensions n x m, where n is the total number of nodes and m is the number of bottom-level nodes. Understanding this linear relationship is the foundation for every reconciliation algorithm we will discuss.

Traditional reconciliation strategies

The most intuitive approach is the bottom-up method. In this scenario, models are trained only for the lowest level (e.g., individual SKUs). The forecasts for all higher levels are then generated by simply summing these base-level predictions. The primary advantage is that it captures the specific idiosyncratic behavior of individual items, such as local promotions or stockouts. However, the bottom-up approach often suffers from a high signal-to-noise ratio at the lowest level, which can lead to volatile and inaccurate aggregate forecasts.

Conversely, the top-down method involves forecasting only the total aggregate and then distributing that forecast down to the lower levels based on historical proportions. While this provides a very stable forecast for the total business, it fails to account for changing dynamics at the bottom. If one specific store begins growing faster than others, a static top-down proportion will systematically under-forecast that store. Various flavors of top-down methods exist, such as using the average of historical proportions or the proportions of historical averages, but all share the same structural weakness regarding local variance.

A third middle-ground approach is the middle-out method, which is common in large retail hierarchies. You might choose to forecast at the 'Category' level because the data is clean and seasonal patterns are clear, then sum up for the total and distribute down for the SKUs. While these methods are easy to explain to stakeholders, they are suboptimal because they discard information available at other levels of the hierarchy.

The mechanics of optimal reconciliation

Optimal reconciliation treats the initial, unreconciled forecasts (known as base forecasts) as noisy estimates that need to be adjusted to satisfy the summation constraints. The goal is to find a set of reconciled forecasts that are as close as possible to the base forecasts while remaining coherent. This is framed as a linear regression problem. We represent the reconciled forecasts y_hat as y_hat = S * P * y_base, where P is a matrix that maps the base forecasts into the bottom level.

The choice of the P matrix determines the reconciliation flavor. The Ordinary Least Squares (OLS) approach assumes that the errors in the base forecasts are independent and identically distributed (i.i.d.). However, this is rarely true in time series; errors at the top level are usually much larger in magnitude than errors at the bottom. To account for this, we use Weighted Least Squares (WLS), where the weights are derived from the variances of the base forecast errors.

The most robust modern method is Minimum Trace (MinT) reconciliation. MinT estimates the full covariance matrix of the base forecast errors, W. By minimizing the trace of the covariance matrix of the reconciled errors, MinT provides the best linear unbiased estimator for the hierarchy. In practice, estimating a full n x n covariance matrix can be unstable, so we often use shrinkage estimators or assume a proportional variance structure to keep the computation tractable.

Reconciliation is not just about making numbers match; it is about leveraging the cross-sectional correlations in your data to reduce the variance of every individual forecast.

Comparing reconciliation methods

When selecting a method, practitioners must weigh computational cost against the expected gain in accuracy. For very large hierarchies with hundreds of thousands of nodes, calculating the MinT optimal P matrix can be memory-intensive. In these cases, WLS with structural scaling (where weights are derived from the S matrix itself) offers a good compromise between speed and performance.

In 2026, the standard toolset for these operations includes the HierarchicalForecast library in Python or the fable package in R. These libraries automate the construction of the S matrix and provide optimized solvers for MinT. When working with these tools, it is crucial to ensure your data is indexed correctly; a single missing node in the hierarchy will lead to a singular matrix, causing the reconciliation step to fail.

Handling intermittency and non-negativity

A common problem in hierarchical forecasting, particularly in spare parts or luxury retail, is intermittency. At the bottom level, many time periods may show zero sales. Standard reconciliation methods are linear and do not naturally respect non-negativity constraints. It is entirely possible for a MinT reconciliation to produce a negative forecast for a specific SKU to satisfy the aggregate constraints, which is physically impossible for physical inventory.

To solve this, we often employ constrained optimization or post-processing. One method is to use a non-negative least squares (NNLS) solver during the reconciliation step. While this adds to the latency of the forecasting pipeline, it ensures the outputs are actionable for warehouse systems. Another approach is to perform reconciliation in a transformed space (like a log transform), though this complicates the summation logic because exp(a + b) does not equal exp(a) + exp(b).

For intermittent data, the choice of the base model is also vital. Using a Poisson or Negative Binomial distribution for the base forecasts, followed by a reconciliation step that respects the mean-variance relationship of those distributions, yields better results than standard Gaussian assumptions. If your bottom level is 90% zeros, reconciliation acts more as a signal-smoothing filter than a traditional aggregator.

Scaling to millions of series

As hierarchies grow to millions of series, the P matrix becomes too large to store in memory. The computational complexity of MinT is roughly O(n^3) in the worst case, where n is the number of nodes. To scale, we must use the sparse structure of the S matrix. Since each column of S only contains a few non-zero entries (corresponding to the path from the leaf to the root), sparse linear algebra can reduce the complexity significantly.

Distributed computing frameworks like Spark or Ray are often used to generate the base forecasts in parallel. The reconciliation step, however, is a synchronization point that requires data from all nodes. A common architectural pattern is to generate base forecasts on worker nodes, then perform a 'reduce' operation to a central high-memory node that solves the reconciliation equation, and finally broadcast the reconciled values back to the workers for downstream use.

Latency is a major factor in real-time pricing or dynamic inventory systems. If your forecast must be updated hourly, you may not have the luxury of a full MinT optimization. In these environments, structural scaling or simple bottom-up approaches are often preferred, with a full optimal reconciliation performed weekly or monthly to reset the system's bias.

Common mistakes in hierarchical implementation

• Ignoring missing nodes: If a new store opens and is not added to the S matrix, the aggregate forecast will be systematically biased downward.
• Mixing units: Reconciling a hierarchy that mixes revenue (dollars) and volume (units) leads to nonsense results. Ensure all nodes in a tree share the same unit of measure.
• Neglecting temporal hierarchy: Only reconciling spatially (across products) while ignoring temporal consistency (days must sum to weeks) can lead to conflicting planning decisions.
• Over-fitting the covariance: Using a sample covariance matrix for MinT without shrinkage when you have short history will lead to highly unstable reconciled forecasts.
• Using reconciliation as a silver bullet: Reconciliation can improve poor forecasts, but it cannot fix base models that lack fundamental features like seasonality or price elasticity.

Temporal reconciliation: The third dimension

While most of this article focuses on cross-sectional hierarchies (geography, products), the same principles apply to temporal hierarchies. A daily forecast should sum to a weekly forecast, which should sum to a monthly forecast. In many business contexts, the monthly forecast is more accurate because it smooths out day-of-the-week effects, while the daily forecast is necessary for labor scheduling.

Cross-temporal reconciliation is the state-of-the-art approach that handles both dimensions simultaneously. This creates a massive 3D grid of constraints. Implementing this usually involves a two-step process: first, reconcile the spatial hierarchy at each time granularly, and then reconcile the temporal strings. Advanced solvers can now handle this in a single step, ensuring that the 'Total Sales for January' is equal to the sum of all 'Daily Sales for SKUs' in that month.

The benefit of temporal reconciliation is particularly visible in capacity planning. If your shipping department plans by the week but your warehouse picks by the hour, having a coherent view across those timeframes prevents the 'bullwhip effect' where small variances at the hourly level create massive, unnecessary adjustments at the weekly level.

What to practice this week

1. Map a small hierarchy using a toy dataset to manually construct the S matrix and verify that y = S * b holds.
2. Generate base forecasts using a simple model like AutoARIMA or ExponentialSmoothing for all nodes in your hierarchy.
3. Implement a bottom-up reconciliation and measure the Mean Absolute Scaled Error (MASE) at the aggregate level versus a direct forecast of the aggregate.
4. Use a library like HierarchicalForecast to apply MinT with 'shrinkage' and compare the results to the bottom-up approach.
5. Check for negative values in your reconciled forecasts and implement a simple clipping or NNLS adjustment to handle them.
6. Review your data pipeline to ensure that new nodes (new products or locations) are automatically integrated into the hierarchy structure.