Robert Tetlow has published a fascinating research paper in the International Journal of Central Banking on policy robustness with the Fed’s FRB/US model. Perhaps the most important part of the paper is his careful documentation of the enormous shifts in the coefficients and the equations of FRB/US over time. This chart from the paper illustrates these shifts. The solid black line plots changes in the estimated sacrifice ratio of unemployment to changes in inflation implied by the model over 64 vintages.

Given these large shifts in the model it is not surprising that monetary policy rules calculated with the model shift over time. This is why robustness studies are so important.

A typical robustness study looks at different models. It takes policy rules that work well in one model and tries them out in other models.  If the rules also work well in other models, then the rules are considered robust. Volker Wieland’s macro model data base with 61 different models is ideal for such robustness studies.

Tetlow takes a related but different approach to robustness. Rather considering different models, he looks at the different vintages of FRB/US that have been used for policy work since the mid-1990s. He then compares a bunch of different policy rules derived from each model-vintage including a model-specific Taylor rule as a “benchmark for comparison.”

For the case where the policy makers know the model and calculate the rule based on that specific model, he finds that the Taylor rule “renders a very good performance with losses that are lower than nearly all of the alternatives” or “does pretty well on average” depending on the time period examined.

But he then looks at how rules calculated for one model-vintage work in a different model-vintage. Because of the huge shifts in the model, the policy rules change a lot with vintages. This chart from the paper shows how model-specific (optimized) Taylor rule coefficients change as the model-vintage changes.

 Tetlow finds that performance with such a model-specific Taylor rule is about in the middle of the pack in terms of robustness, but he also finds that some rules like “pure inflation-targeting rules” are not robust,” that “adding an instrument smoothing term…contributes little to the robustness,” and that “notwithstanding problems of mismeasurement of output gaps, it generally pays for policy to feedback on some measure of excess demand.”

Of course, the model-specific rules in Tetlow’s study are not designed to be robust. An important question for future research is whether rules designed with robustness in mind would be more robust than the model-specific rules that Tetlow examines.

I note that the IJCB, which is an important outlet for serious policy-relevant applied research on central banking, is having its 10^th anniversary this year. John Williams is now the managing editor and before him were Ben Bernanke, Frank Smets, and me.