Project development decisions are influenced by a multitude of factors but, if an acceptable legal and political structure can be engineered, then the decision is generally governed by the result of a complex tapestry of spreadsheet coding, referred to as 'the financial model'. The financial model is often a black box, with just a single able user, and consequently often receives little interrogation. But because the financial model is central to all facets of a project financing the risk being introduced by the model can become the most significant risk of all.

The financial model is crucial because so many decisions are made by reference to its outcome. It incorporates the key components of the finance and project documents along with the parties' view of the operational, technical, and econometric parameters. This model-centric approach usually results in hundreds, often thousands of assumptions flowing through millions of cells of spreadsheet coding, often discounted over decades, culminating in just a handful of coverage ratios or equity return measures. These results, often simply quoted to two decimal places, decide the outcome of important structuring and often relationship critical decisions.

When relying on this framework the financier needs to be aware of two significant risks – ignoring the effect of compounding input errors and the level of Model Risk introduced by the quality of the analytical framework – and ensure that they are understood and ultimately mitigated before executing the deal.

By properly addressing these risks a financer will have a better understanding of the level of risk versus uncertainty inherent in the economic analysis. The distinction between these concepts was concisely drawn by Frank Night, the pre-eminent and often controversial economist: "If you don't know for sure what will happen, but you know the odds, that is risk: If you don't even know the odds that is uncertainty."

When undertaking a typical financing the financier will go to great length to convert uncertainty to risk and then ultimately structure the project so the amount of uncovered risk is within the acceptable limit for the financial institution and the pricing being applied. Knowing the odds allows the application of Monte Carlo simulation, a highly effective computational approach to calculate the overall error (Risk) involved.

Understanding the odds

Before accepting, say a 3.5% per year growth rate for the patronage of an inter-city toll road a good financier performs a high degree of due-diligence and scrutiny in order to satisfy themselves, their credit committee and syndicate banks that it is the most likely value within their risk aversion bracket. There is however an inherent level of error attached to this base case assumption, which may be, say, 0.5% per year. This error can be illustrated by a statistical distribution with the base case assuming the most likely, mean, value:

Plot 1: The Base Case assumption is the most likely value amongst a whole range of values, each with a different likelihood of occurring.

Typical financial analysis assumes a 1-dimensional approach by selecting a more conservative figure, say 2.20% per year and assessing the model output, which, let's say is the minimum debt service cover ratio (MDSCR). But why not run all reasonable values of the Growth Rate and record a 'profile' for the MDSCR rather than just the value at two points?

This approach provides the analyst with a data series from which they can report a number of useful statistics. For example "the MDSCR has a mean value of 150bp ± 10bp within a 95% confidence interval". Instantly this has provided the decision maker with another dimension of information to assist in making their structuring decisions.

The logical progression of this method is to apply a meaningful statistical distribution to all of the key inputs, with distributions in later years having increasingly larger spreads to reflect larger uncertainty. To compute this efficiently the user will require an efficient and flexible financial model and access to a simulation package such as Crystal Ball (www.decisioneering.com).

Plot 2: The MDSCR is one value amongst a distribution. By assessing the spread of this distribution lenders can increase their understanding of the risks they are taking.

This methodology encourages project teams to not only focus upon the base case value of a particular variable, which is important in its own right, but also on the variability of each input. This analysis demonstrates analytical rigor and also provides the user of the financial analysis with an additional dimension of information of which traditional analysis only allows a glimpse.

Tornado Analysis

An efficient way to focus resources is to prepare a 'Tornado' plot. Tornado analysis involves flexing each input individually, by the same amount, and reporting the impact on a selected output. This then allows the inputs to be 'ranked' according to which one has the most impact. The analyst can then define the assumptions as of either primary, secondary or tertiary importance. Previous experience has shown some inputs to have an unexpectedly high bearing on the deal. For example, it may be more important for a project team to negotiate an additional three months off of the EPC rather than spend weeks reducing financing fees by 10bp. This allocation of resources is crucial to an efficient project financing.

Introducing Model Risk

Of all of the many facets that make up a project financing, the preparation, presentation, and continual management of the financial model is arguably the most pivotal. However, for many transactions in both the private and the public sector, in established and emerging markets alike it is an aspect that can introduce more uncertainty than it resolves.

Traditionally viewed amongst financiers as a training ground for junior staff this role is one which demands rigour and, in a time-critical environment, experience.

When performed well the task attracts little attention, the modeller works on a few more deals and then moves on to a different aspect of the financing. Although beneficial for providing an individual with a well-rounded appreciation of the effort and skill required to complete a project financing this approach inherently means that the most pivotal role in the transaction falls to an individual with little to no experience in analysing project financings.

An approach based on learning on the job, combined with career rotation and, often extreme, time pressure, which is at the heart of model risk.

"Can you do addition?" the White Queen asked. "What's one and one and one and one and one and one and one and one and one and one?" "I don't know," said Alice. "I lost count!" (Lewis Carroll, Through the Looking Glass)

Compared to many aspects of applied finance the mathematical principles associated with assessing the cashflows of a typical project are straight forward. Generally, transactions with complex contractual arrangements, tariff mechanisms and financing structures can be represented without the need to apply anything more than a good understanding of high school mathematics. When an investment bank is trying to win a new client or a developer is raising urgent funds, Project Finance models need to be:

• prepared rapidly,
• readily sense-checked,
• work first time; and
• be capable of instant analysis.

In addition they also need to be able to evolve in line with negotiations.

As in other aspects of project financing it is fair to say that the devil lies in the detail. The way in which calculations are composed, represented and nested together forms the basis of a sound project model and when not considered properly, the model could be an unacceptable liability.

Poor design and over-complication

A common shortcoming of financial models is that users try and reduce the number of formulae by increasing the complexity of single lines of code. All too often this results in formulae that take too long to review and understand rendering the model impractical. The direct implication is that users will have difficulty establishing confidence in the model and ultimately not be able to rely upon it.

A few years ago a US investment firm, announced its forecast of a capital gains distribution for participants in one established US fund. A short time later the forecast was altered to a loss. Embarrassingly a spreadsheet entry had been given the wrong sign, which meant that it needed to change the public forecast from a $2 billion profit to a $100 million loss.

Experienced analysts had prepared the model for the financing; the assumptions had also been checked as the deal evolved. The model projected around $200 million in profit and $20 million in costs – however at the eleventh hour it was found that some very long and unnecessarily complicated formulae had a bracket in the wrong position.

This is not an uncommon occurrence, since in my experience I have found that the majority of financial models and other appraisal frameworks which are billed as corporate standard are wrong. They are wrong by virtue of the fact that they are an ill-constructed black box, which is virtually impossible to audit let alone create confidence.

Divide the time invested in a model into three equal segments. The first is spent designing the model and talking to the end users. The middle third spent coding and the final third spent testing and finalising presentation.

Work within the limits of design and reason

Another example of where a model is used without proper considerations for its capability is in sensitivity and breakeven analysis. To use a well designed and properly populated model to estimate the impact on the project economics of a ± 20% movement in a key project assumption is perfectly acceptable.

Testing a 50% cut in production, in isolation, would likely be beyond the bounds of most frameworks. Such a movement would require other aspects of the model to be re-coded in much more detail to properly measure the sensitivity of the cashflows. For example, a prolonged 50% cut in production would require step functions to be factored into the cost of labour.

In recent times we have seen incredibly, but unnecessarily, complex financial models submitted to financiers for financing of several oil and gas transactions. In the oil and gas industry the term P90 broadly denotes a Reserve with a 90% probability of economic extraction, while P50 represents 50%. When switching between P90 and P50, these models often simply changed the volumes of oil and gas being produced. In reality however, these two operational phases required different capital expenditure profiles and higher operating and development expenses. A framework to accommodate these two different phases requires careful design and implementation. In these particular transactions the financial model not only caused frustration but significant delay and incurred audit costs 4-5 times higher than necessary.

Care should be taken to use the model within acceptable limitations and to avoid overcomplicating the framework in order to only slightly increase the accuracy of a one-off sensitivity.
One institution at which I worked encouraged employees to include both CPI and floating interest rate breakeven measures in credit submissions. It is of course extremely improbable that a CPI index might move up from 2.5% per year to 9% per year for a prolonged period of time without the base interest rate also moving. To avoid misrepresenting the project's strengths and weaknesses, the analysis submitted must be reasonable and within the bounds of the financial model's capability.

A model audit is not an insurance policy

Usually a condition precedent to financial close is that the financial model is reviewed by a suitably qualified and independent third party. This is a role commonly adopted by the well-known accounting and actuarial firms. This approach is generally expensive and does not provide any tangible degree of insurance. This role should be viewed as additional quality control rather than providing a party that could be sued in the event that remedial action is required. To fulfil this service, these parties employ dedicated teams in order to ensure that the models are properly scrutinised prior to sign off.

A common misconception is that the model audit is a tick in the box that happens at the end of the transaction. This process often takes up significant amount of management time and unless prepared with absolute rigour usually highlights material errors in the code – so arrangers would be advised to bring in auditors sooner rather than later.
Have the financial model audited as part of the initial project development and then once the deal is finalised it can be refreshed just prior to financial close. This helps to avoid embarrassing, potentially costly, mistakes from presenting problems at the last minute.

Nick Crawley is the principal of Navigator Project Finance, a Sydney-based consultancy specialising in financial modelling and risk analysis for project and structured financing. www.navigatorpf.com