A common source of frustration in PPNR balance model development is the discrepancy in coefficients and error estimates when a model is specified using two common SAS commands: PROC AUTOREG and PROC ARIMA. As we describe herein, our solution is not to use a single SAS command, but to adjust both SAS command defaults to minimize the discrepancy.
“PROC AUTOREG,” “PROC ARIMA,” or both?
SAS provides two distinct approaches for developing time series models: the PROC AUTOREG command, which generates an autoregressive form of a generalized least squares (GLS) model, and the PROC ARIMA command, which generates an Autoregressive Integrated Moving Average model (or ARIMAX models, which is an ARIMA model with eXogenous factors). Both GLS and ARIMA models are time series models and both are appropriate for PPNR balance modeling. PROC ARIMA is able to incorporate exogenous factors, as well as lag (‘p’), moving average (‘q’), and difference (‘d’) terms.
An ARIMA model built using PROC ARIMA with only ’p‘ and ’d‘ terms (i.e., no moving average measures) and specified using a consistent method (see the next section for detail) can be identically replicated by a PROC AUTOREG statement.
PROC ARIMA and PROC AUTOREG have strengths and weaknesses — we find it necessary to leverage both:
- PROC ARIMA is more useful in determining if a model makes practical sense — whether it will meet business expectations. PROC ARIMA has powerful exploration features to allow the modeler to determine the appropriate differencing and lag structures of dependent variables (DVs) and independent variables (IVs). It also has a very powerful forecasting engine, allowing the modeler to produce dynamic “backcasts” as well as forecasts.
- PROC AUTOREG is more useful in determining if a model makes technical sense — whether it passes statistical tests. With a few simple commands, the model developer can both specify the model and run all necessary statistical tests (e.g., verify unit root, study and resolve serial correlation, determine whether errors are homoscedastic or otherwise calculate robust standard errors, etc). However, it cannot produce dynamic “backcasts”, which makes evaluation of historic goodness of fit difficult.
Both commands are necessary tools in the modeler’s toolbox and both are critical to properly research, evaluate, and document a stress testing PPNR model.
Reconcile by Deviating from Default Options
SAS procedures have default options that are applied unless actively overridden. Many times, these default options work well enough that the model developer can be indifferent. The defaults in PROC AUTOREG and PROC ARIMA, however, can generate disconnects in model results for PPNR balance models.
The default algorithms (“methods”) PROC ARIMA and PROC AUTOREG used to specify coefficients and estimate standard errors differ slightly:
Usually the default methods converge on results with coefficients that are within 3%–5%, which typically results in negligible differences in backcasts and forecasts:
However, in roughly one in ten portfolios we model, the results of PROC AUTOREG and PROC ARIMA with default methods produce materially different results:
To get around this constraint, we recommend all models developed using PROC AUTOREG and PROC ARIMA use the Maximum Likelihood optimization method. This method produces more consistent (nearly identical) coefficients and standard errors when using either the PROC AUTOREG or PROC ARIMA statements:
Changing the method is straightforward, simply by inserting “method = ML” into model development commands. See the sample pseudocode below to identify where this needs to be placed.
While there are some small tradeoffs choosing Maximum Likelihood versus other estimation methods, we find these are outweighed by the power of quickly and easily producing consistent models across different commands.
We welcome your feedback and are happy to continue the conversation about this article or other Treasury and Risk viewpoints. Please reach out to the head of Novantas Global Treasury and Risk, Pete Gilchrist at email@example.com; or the head of Novantas PPNR Modeling and Forecasting, Jonathan “Wes” West at firstname.lastname@example.org.