A discussion about forecast errors

Forecasting continues to be a hot topic. My recent interviews with Steve Morlidge continues to be very popular. Also, ‘Franz the Frog’ sparked some interesting discussions behind the scenes. Given the strong interest in these topics, I reached out to a friend who has spent a lot of time and effort driving solid forecasting processes.

Please meet Ulrich Pilsl. He provides a different perspective. Ulrich currently works as an Interim Manager in Munich. He spent over 14 years at Softlab / BMW Group (later Cirquent / NTT Data Group). As a member of the executive board, he held different senior executive positions including CFO of a consulting subsidiary and as the Head of Controlling & Business Administration.

Christoph Papenfuss: Forecasting is a key focus area for many finance professionals. But many organizations are struggling to obtain an objective view of the future. What are some of the key problems?

Ulrich Pilsl: The biggest problem I see is complexity. Many companies have bloated processes that are too detailed. It simply takes too much time and people have a hard time differentiating between what is important and what is not. There is no clear focus. Also, management tends to have a hard time managing the process. My advice is to simplify and to get rid of excessive detail. More detail does not create more accurate forecasts. On the contrary: the more detail, the less accurate forecasts tend to be for the above mentioned reasons.

Christoph Papenfuss: What is the main problem with inaccurate forecasts?

Ulrich Pilsl: Inaccurate forecasts lead to a serious confidence problem. Shareholders don’t like surprises. It gets worse when surprises are caused by poor forecasting efforts.

Christoph Papenfuss: Are positive and negative errors equally problematic? Let’s take a look at a typical sales or business forecast. Some people tend to create very conservative forecasts and often end up outperforming. Isn’t this better than creating a very ambitious forecast and then coming in lower?

Ulrich Pilsl: This is an interesting but common situation. First of all, positive and negative errors are equally problematic. Both type of errors can create serious management challenges apart from the already discussed confidence problems. In regards to this specific situation, one might be tempted to say that it is a good thing for a sales person to continuously beat his or her forecast. However, this can create some serious challenges. Let’s take a look at a consulting company. Low sales forecasts indicated low resource requirements. Hiring efforts might be slowed down and the business might quickly end up in a situation where they do not have enough talent available. Business is lost. Customers might loose confidence in us as a trust-worthy business partner. I therefore strongly believe that both negative and positive errors require serious attention.

Christoph Papenfuss: What should the Controller do to help minimize forecast errors?

Ulrich Pilsl: The Controlling department should show some ‘tough love’. They have to challenge the departments to deliver realistic forecasts. We found that it is critical to provide suggestions and to jointly develop scenarios with the business managers. Finance basically acts as a tough but fair coach in the process. This continues in the the monthly and weekly management meetings: We openly discussed the forecast results and challenged the numbers. It is obviously the job of the Business Controller to moderate this process. Last but not least, we found that it sometimes makes sense to create top-down adjustments that reflect upside and downside risk.

Christoph Papenfuss: Based on your experience, does it make sense to measure forecast accuracy? If yes, how often and at what level did you measure accuracy?

Ulrich Pilsl: It depends on the organization. This reminds me of a quote by my former manager who said: “Most companies are over-controlled but under-managed.” A team that understands the value of a forecast will usually deliver solid forecasts. Measuring forecast accuracy won’t necessarily improve it. I do believe, though, that it makes sense to measure it if the organization has challenges with the forecast process. Especially in the case of a management team that does not see the value in the forecast. It might make sense to add an accuracy target to the annual objectives. We had a variable goal “internal quality”. This allowed us to substantially change the mindset of some managers. The goal was set once per year.

Christoph Papenfuss: How do you utilize forecast accuracy measures? Should you communicate the numbers to the organization or is this something that should stay within the walls of the finance department?

Ulrich Pilsl: In my opinion, it does make sense to communicate forecast accuracy to the management team. But it makes no sense to communicate it to the whole organization. The aim is to improve forecast quality and not to blame the management in the organization.

Christoph Papenfuss: What can Finance do to help create a culture where people are happy to create meaningful and objective forecasts?

Ulrich Pilsl: Finance simply has to be the role of a coach and consultant for the business. It is our role to educate and to support the business.

An imaginary conversation with a weather frog

European folklore believed that frogs kept in a glass would be able to forecast the weather. People filled some water in the glass to keep the amphibian happy, and then added a small ladder. A climbing frog would indicate good weather, whereas a frog hanging out in the water would show bad weather. This belief especially stuck with people in the German speaking countries where weather forecasters are typically called ‘Weather Frogs’ (Wetterfrosch). Well, weather forecasters do one thing well: forecasting. They are the true masters and I thought that we could get some insights from one of them. It is frog migration season in Bavaria and I happened to have found one who is willing to talk to me. Please meet Franz the Frog. Franz resides in Bavaria, Germany where people recognize him as a trusted master forecaster.



Christoph: How is life as a master forecaster? Your pictures are every on the side of the roads these days. You must be pretty busy? Spring is known for its volatile weather.

Franz the Frog: All cool here in the pond. Thanks for asking. We were quite busy up until yesterday. That’s when we finished our quarterly forecast. I am looking forward to jumping around for the next few weeks.

Christoph: But wait a second! It’s a volatile climate out there. How can you just sit there, jump around and not forecast whenever things change?

Franz the Frog: Dude, I hear ya’. But the big boss here in the pond decided that 2-3 forecasts per year are totally fine. Plus we have so much other stuff to do. Also, do you realize how much work we have to do to complete a forecast?

A weather frog

Christoph: Sorry, this was news to me.  Then tell me, why exactly is this so much work?

Franz the Frog: Helllllooooooo!!!!! Each forecast starts with us looking at weather patterns for the past five years. Just simply gathering the data that is stored all over our pond takes us forever. Our big boss in the pond also wants us to create detailed variance reports for those years. That takes about a month. To create the actual forecast, we turn over every single leaf in our pond. And we record every little rain drop. That takes a lot of time. Get it?

Christoph: Oh…I see…a lot of detail. But a lot of detail should result in higher accuracy right? My wife Jen complained about your reliability the other day. She claimed that she would not even ‘pack a suitcase for vacation’ using your information. Errr…please don’t shoot the messenger.

Franz the Frog: Watch it, buddy! I will stick my tongue out here in a second. What do you expect? We get paid by our big boss in the pond. If the boss is happy, the flies are happy and we are happy. The boss decided that it’s best for us to provide a forecast that tells you guys exactly what you want to hear. Last year we saw rain coming. Your wife complained that she did not want any rain that particular week. She said:’Ahh, this stupid forecast. Rain is driving me crazy. It should be sunny!!!’. That got my big boss in the pond really upset. And guess what happened: no pay. That made our decision easy: we would eliminate a lot of pain and frustration if we simply forecast what everybody wants to hear.

Christoph: Oh…ok. That is so not cool. Let’s change topics. What type of tools do you guys use to do your forecasts? I mean, we’re in the year 2011 so  I suspect that you guys have some cool tools…like that PC game Frogger?

Franz the Frog: Frogger rules!!! If you like Frogger, you will be happy to hear that we are still using the same platform. No changes. Here, take a look: We have special leaves from a searose that was created in Redmond, WA’. Those leaves allow us to play with the data that we collect. The nice thing is that everybody in our pond has a ton of those searoses flying around. And the other guys love those leaves. The all create their own versions. But do me a favor and DO NOT talk to the green guy over there: he has to collect all the leaves at quarter-end. He hates his job. A bunch of my colleagues sometimes play a joke on him and swap out leaves or hide them. Others change the carefully thought-out leaf structures by ripping a holes in them or by chewing on them. You should see his face when the leaves don’t stack!!! Haha…RRRiibbitt.. Hilarious.

Christoph: Holy tadpole! That sounds like a tough job. Can you trust the data then?

Franz the Frog: Probably not. But hey…that’s the way the pond has been for a long time. It worked in the past it should work in the future, right? We have been around for thousands of years.  And the big boss is happy and we’re getting paid—what’s not to like?