If there is a lack of reliable data on the occurrence of specific events or knowledge on the consequences of decisions, there is a problem of uncertainty. Better decisions can hardly be achieved through the trained use of statistics or their transparent communication. Instead, the central question is how individual consumers can reduce uncertainty in their decision-making situation. Two scenarios are central here:

How can uncertainty be reduced (quickly, practically) for everyday problems in which consumers are left to their own devices?

How can uncertainty be reduced (quickly, practically) for everyday problems for which an expert provides advice to the consumer?

Why is it difficult to provide decision support for problems of uncertainty?

Decision problems of uncertainty are characterised by a lack of reliable data. This effectively rules out the direct selection of the best decision option. The support consists of identifying key strategies to reduce uncertainty. What do I need to ask to reduce the choice of potential information or options? What do I need to look for? What do I need to consider to sort out inappropriate options that do not meet the minimum requirements?

In contrast to consumers, experts in a particular subject area are able to identify objective shortfalls in the standard of a decision problem on the basis of fewer heuristic features. With the help of an analysis of specific consumer decision situations, possible expert heuristics are distilled into decision trees. These summarize the experts' gut feeling based on their experiences and provide consumers with a robust expertise that enables them, similar to the expert, to separate the wheat from the chaff. 

This is not only important for issues where consumers are left to their own devices. Potential decision heuristics can also be combined in decision trees for consulting situations: Here it is a matter of asking the consultant the most important questions in order to be able to assess this situation robustly. 

Fast-and-Frugal Trees (FFTs) are suitable decision trees that can be transparent, comprehensible to consumers and of high quality at the same time. These FFTs represent a sequence of features to be examined (Martignon et al., 2008). There is always only one branch (stop) or one arrives at the next test feature, but there are no further branches (see example below). This distinguishes the FFTs from the usual decision trees. Only the last feature in the chain has two branches. 

It has been shown that FFTs enable fast and reliable decisions in various decision situations under uncertainty, e.g. in psychiatry, anaesthesiology, but also in the financial world (Aikman et al., 2014; Green & Mehr, 1997; Jenny et al., 2013). FFTs can be presented both digitally (e.g. app, website) and analogously to consumers (e.g. on posters or in brochures) in the form of a graphically illustrated, simple tree structure. This makes them an evidence-based instrument for decision support that is easy to implement. In the RisikoAtlas project it was developed and implemented for the first time for everyday consumer practice. The use of FFTs is also helpful because their application trains skills. The use of FFTs facilitates the internalisation of key characteristics for problems and stimulates critical thinking.

The order of features in an FFT is critical and must be determined in advance. There are both manual and more complex approaches using machine learning methods. Once statistically determined, this combination of features allows consumers to robustly classify decision options (e.g., whether an informed decision is possible) by independently examining those features.

How to construct a decision tree for a consumer problem - the FFT method of expert feature validity

A. What do you need?

For the evidence-based development of FFTs, all approaches (including the FFT method of expert-based feature validity) require base data consisting of three parts: Characteristics of the problem, problem cases and the respective case assessment.

Part 1 – Characteristics of the problem 

First, it is necessary to clarify what the problem is and to define the concrete decision or assessment on which information should be provided. What is the decision tree supposed to deliver? Under this aspect, potential features  are researched with the help of experts (e.g. workshops), colleagues, laypersons and specialist literature (trade journals, white papers, government reports and experience reports). Potential features are all those characteristics of the problem situation that could possibly be an indicator of a good or bad decision regarding the problem. It may also be worthwhile to include new features such as one's own assumptions or intuitions. A list of potential features should then have been established.

Each potential feature must be comprehensible and testable by a layperson. Ideally, the list should summarize similar features, especially if there are too many of them. It is fair to say that expert supported feature selection is the most important tool in advance, particularly when it comes to cost-effective development. After all, each additional feature requires more cases in order to allow robust development. As a rule of thumb, you can basically calculate 20 to 50 cases for each feature. Each case requires effort: Each case must be individually coded for all features and an assessment must be obtained. If you need support during this process, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

Part 2 - Problem cases

Once you have made a selection of potential features, you need to find out how often and under what circumstances they occur in the real world. For this you collect material of typical decision situations, e.g. real purchase offers, videos of real consulting situations or real informational services. 

If such case material of typical decision situations is not available, the FFT method of expert-based feature validity is the method of choice. Instead of the natural combinations of features in real cases, all possible virtual profiles of potential features are combined. Each combination of characteristics represents a profile and therefore a case.

Part 3 -  Case assessment

For each case in your data basis, you must know or determine whether the target criterion is met or not. In the case of health information, for example, a positive assessment would be the target criterion if it enables an informed decision, otherwise it would be a negative assessment. Without this basis of already determined profiles, no model for future decision support is possible. One approach would be to test each profile or case, i.e. determine how it turned out. This involves considerable experimental effort. The alternative is the "view of the expert", which the model approach presented here was aimed at right from the start. Several independent experts evaluate each individual case (i.e. each profile, each combination of features) with regard to the objective of the development, e.g: "Does this health information allow an informed decision?

B. How do you proceed?

With the FFT method of expert-based feature validity, the significance of potential features is tested directly through expert assessments right from the start. Normalized frequency formats (... of every 100) are used to estimate the presence of each feature in relation to positive and negative target conditions. Resulting measures - positive predictive value, negative predictive value, false omission rate, false recognition rate, sensitivity, specificity, feature prevalence - are evaluated to minimize the number of target features. In addition, the frequency of occurrence of the target object is determined with experts. If you need assistance with the procedure, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here. 

This selection can be further reduced by testing laypersons on how successfully they evaluate the individual features. If you aim for six characteristics, this means that you always have to generate 2 to the power of 6 = 64 different combinations. Each feature can either be present or absent (or above or below a certain value limit). Since the experts also estimated feature prevalences, the associated probabilities can be used to estimate the profile frequency, i.e. how often certain combinations occur. This is crucial in order to weight the occurrence of the profiles in the data set realistically.

For all profiles, the expert evaluations are "collected" in a further study. Three experts receive every feature profile. This means that the expert's view can only be modeled using features familiar to them. This is qualitatively weaker than, for example, the FFT method of case-based feature validity. 
The decision tree is modeled on the basis of these feature assessment profiles.

The pipeline for development can be summarized in a simplified illustration:

Modeling from tree development and cross-validation can be performed manually, but in the sense of effective modeling it is easier with the open source solution R. In addition to the FFTrees package (Phillips et al., 2017), you can also download a web solution by Evaldas Jablonskis and Uwe Czienskowski from  http://www.adaptivetoolbox.net/Library/Trees/TreesHome#/. If you need assistance with this, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

A Fast-and-Frugal Tree (FFT) is modeled using the portion of cases selected as training data; often 33% or 50% of cases. This FFT has a certain quality in terms of tracking down the target feature (assessment). This means it will miss cases in the real world and cause false alarms in others. To quantify this quality, either a statistical cross-validation can be performed (the determined decision tree is applied on randomly repeated cases; test data cases), or it can be applied once to a collection of cases with assessments that were put aside before modeling. Alternatively, a completely new sample of cases with feature encodings and ratings (out-of-sample) can be collected to which the decision tree is applied (additional effort).

Which quality is sufficient depends very much on the types of errors and the costs associated with the error. Finally, the model must be tested in practice with laypersons. Here a randomised controlled study is useful in which the decision intentions of consumers who are given the decision tree are compared with those who have nothing or a standard information sheet. If you need assistance with quality or evaluation, please consult the final report of the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

• Aikman, D., Galesic, M., Gigerenzer, G., Kapadia, S., Katsikopoulos, K. V., Kothiyal, A., ... & Neumann, T. (2014). Taking uncertainty seriously: Simplicity versus complexity in financial regulation. Bank of England Financial Stability Paper, 28.
• Green, L., & Mehr, D. R. (1997). What alters physicians' decisions to admit to the coronary care unit?. Journal of Family Practice, 45(3), 219–226.
• Jablonskis, E., & Czienskowski, U. (2017). Decision trees online. http://www.adaptivetoolbox.net/Library/Trees/TreesHome#/
• Jenny, M. A., Pachur, T., Williams, S. L., Becker, E., & Margraf, J. (2013). Simple rules for detecting depression. Journal of Applied Research in Memory and Cognition, 2(3), 149–157.
• Luan, S., Schooler, L. J., & Gigerenzer, G. (2011). A signal-detection analysis of fast-and-frugal trees. Psychological Review, 118(2), 316.
• Martignon, L., Katsikopoulos, K. V., & Woike, J. K. (2008). Categorization with limited resources: A family of simple heuristics. Journal of Mathematical Psychology, 52(6), 352–361.

If you would like to adopt a consumer topic from our website, you can do so in the following three ways: 

1. You are using a digital copy. Either you directly save an illustration or download our PDF, or you integrate the illustration via Link(a href) or iframe.
2. You take your analogue copy and print out our PDF. The resolution and vector-based graphic is suitable for posters and brochures.
3. You recommend the app and refer to the Risikokompass from the PlayStore and AppStore.

If you would like to develop your own model, please consult the final report on the RiskAtlas project from July 2020 or contact us. Contact details can be found here.

When using the instruments, please mention the funding agency, which is the German Federal Ministry of Justice and Consumer Protection, and the Harding Centre for Risk Literacy as the responsible developers.

Logos can be downladed here.

FFT-method case validity

FFT-method model

FFT-Methode getesteter Merkmalsvalidität

Method Natural Frequency Tree (NFT)

Method risk reading assistant for your browser

Even in times of digitalisation, most of the money from private households is still invested with bank and/or financial advisors. In order for you to receive a product that suits you and that allows you as a customer to invest after assessing possible returns and risks, a high quality of advice is crucial. But what is quality advice? How can you recognize whether quality standards are violated by your financial advisor? Our decision tree enables you to check exactly that.

For whom is the graphic suitable?

The decision tree should provide support to all consumers who can invest money, regardless of the purpose of propagation. It is intended to enable consumers to carry out a check immediately after the investment consultation before making an investment decision during the second appointment, or signing an offer. It is also suitable to a limited extent for being used during a consultation. If shortcomings are recognizable by the end of a consultation, you could address them with the advisor. However, it is not to be expected that you can force a high-quality consultation by asking questions.

You can also check the consultation you have experienced according to additional quality criteria. Please note, however, that no consultation and no checklist is ever perfect. With each additional feature that you check, the risk of an incorrect assessment also increases.

• Meeting minutes are recorded.
• The consultant asks for previous knowledge of financial products..
• The consultant offers various products/options.
• The advisor asks consumers about their experiences with financial products
• The advisor makes a clear distinction between brokerage services and investment advice.
• The advisor asks about the consumer's initial financial situation.
• The advisor asks whether there is a preference for investment advice or a broker relationship.
• The advisor suggests an individual split of the investments between different products
• The consultant explains that the offered product is growing continuously and without fluctuation. (NEGATIVE FEATURE)
• The advisor asks consumers about their investment goals.

Where is the data coming from?

Cases – Which served as a basis?

All 64 combinations of six critical requirements (requirement profiles) and their evaluations served as the basis for training and test data sets.

Target assessment –  How were the requirement profiles pre-estimated?

Each requirement profile was assessed by three experts.

Potential features – Which requirements were considered?

Using literature studies, 60 requirements were identified that potentially constitute quality in investment advice or violations of requirements.

Selection of features and modelling

Using a feature study and statistical analysis, six potential features that are assessable by laypersons were selected and given the highest validity by the experts.

The model

The FFTrees package was used for model identification (Phillips et al., 2017). The ifan algorithm was used to optimize for balanced accuracy.

What is the quality of the data?

The data set was randomly divided into training data sets (two thirds) and test data sets (one third).

The model is of the following quality:
A cross validation of the identified decision tree resulted in the following quality measures: balanced accuracy = 0.87; correct classification of fulfilled minimum quality requirements (share of 14% in the test set) with 0.99. This means that fulfilled minimum requirements were recognised in 99 out of every 100 cases.
Missing requirements were detected in 74 out of every 100 cases. This still reveals many overlooked quality violations.

Potential for development

Continuous further development of the underlying training data due to changes in the market situation.

Empirical evaluation with consumers

All research results on the fundamentals and on the effectiveness of the RiskoAtlas tools in terms of competence enhancement, information search and risk communication will be published together with the project research report on 30 June 2020. If you are interested beforehand, please contact us directly (Felix Rebitschek, rebitschek@mpib-berlin.mpg.de).

Sources

•    Phillips, N. D., Neth, H., Woike, J. K., & Gaissmaier, W. (2017). FFTrees: A toolbox to create, visualize, and evaluate fast-and-frugal decision trees. Judgment and Decision making, 12(4), 344-368.

Last update: 27 November 2019.

Comparing forms of investment with a goal

Comparing forms of investment without a goal

Conflicts of interest in investment advice

Qualität in der Anlageberatung einfordern