If there is a lack of reliable data on the occurrence of specific events or knowledge on the consequences of decisions, there is a problem of uncertainty. Better decisions can hardly be achieved through the trained use of statistics or their transparent communication. Instead, the central question is how individual consumers can reduce uncertainty in their decision-making situation. Two scenarios are central here:

How can uncertainty be reduced (quickly, practically) for everyday problems in which consumers are left to their own devices?

How can uncertainty be reduced (quickly, practically) for everyday problems for which an expert provides advice to the consumer?

Why is it difficult to provide decision support for problems of uncertainty?

Decision problems of uncertainty are characterised by a lack of reliable data. This effectively rules out the direct selection of the best decision option. The support consists of identifying key strategies to reduce uncertainty. What do I need to ask to reduce the choice of potential information or options? What do I need to look for? What do I need to consider to sort out inappropriate options that do not meet the minimum requirements?

In contrast to consumers, experts in a particular subject area are able to identify objective shortfalls in the standard of a decision problem on the basis of fewer heuristic features. With the help of an analysis of specific consumer decision situations, possible expert heuristics are distilled into decision trees. These summarize the experts' gut feeling based on their experiences and provide consumers with a robust expertise that enables them, similar to the expert, to separate the wheat from the chaff. 

This is not only important for issues where consumers are left to their own devices. Potential decision heuristics can also be combined in decision trees for consulting situations: Here it is a matter of asking the consultant the most important questions in order to be able to assess this situation robustly. 

Fast-and-Frugal Trees (FFTs) are suitable decision trees that can be transparent, comprehensible to consumers and of high quality at the same time. These FFTs represent a sequence of features to be examined (Martignon et al., 2008). There is always only one branch (stop) or one arrives at the next test feature, but there are no further branches (see example below). This distinguishes the FFTs from the usual decision trees. Only the last feature in the chain has two branches. 

It has been shown that FFTs enable fast and reliable decisions in various decision situations under uncertainty, e.g. in psychiatry, anaesthesiology, but also in the financial world (Aikman et al., 2014; Green & Mehr, 1997; Jenny et al., 2013). FFTs can be presented both digitally (e.g. app, website) and analogously to consumers (e.g. on posters or in brochures) in the form of a graphically illustrated, simple tree structure. This makes them an evidence-based instrument for decision support that is easy to implement. In the RisikoAtlas project it was developed and implemented for the first time for everyday consumer practice. The use of FFTs is also helpful because their application trains skills. The use of FFTs facilitates the internalisation of key characteristics for problems and stimulates critical thinking.

The order of features in an FFT is critical and must be determined in advance. There are both manual and more complex approaches using machine learning methods. Once statistically determined, this combination of features allows consumers to robustly classify decision options (e.g., whether an informed decision is possible) by independently examining those features.

How to construct a decision tree for a consumer problem - the FFT method of expert feature validity

A. What do you need?

For the evidence-based development of FFTs, all approaches (including the FFT method of expert-based feature validity) require base data consisting of three parts: Characteristics of the problem, problem cases and the respective case assessment.

Part 1 – Characteristics of the problem 

First, it is necessary to clarify what the problem is and to define the concrete decision or assessment on which information should be provided. What is the decision tree supposed to deliver? Under this aspect, potential features  are researched with the help of experts (e.g. workshops), colleagues, laypersons and specialist literature (trade journals, white papers, government reports and experience reports). Potential features are all those characteristics of the problem situation that could possibly be an indicator of a good or bad decision regarding the problem. It may also be worthwhile to include new features such as one's own assumptions or intuitions. A list of potential features should then have been established.

Each potential feature must be comprehensible and testable by a layperson. Ideally, the list should summarize similar features, especially if there are too many of them. It is fair to say that expert supported feature selection is the most important tool in advance, particularly when it comes to cost-effective development. After all, each additional feature requires more cases in order to allow robust development. As a rule of thumb, you can basically calculate 20 to 50 cases for each feature. Each case requires effort: Each case must be individually coded for all features and an assessment must be obtained. If you need support during this process, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

Part 2 - Problem cases

Once you have made a selection of potential features, you need to find out how often and under what circumstances they occur in the real world. For this you collect material of typical decision situations, e.g. real purchase offers, videos of real consulting situations or real informational services. 

If such case material of typical decision situations is not available, the FFT method of expert-based feature validity is the method of choice. Instead of the natural combinations of features in real cases, all possible virtual profiles of potential features are combined. Each combination of characteristics represents a profile and therefore a case.

Part 3 -  Case assessment

For each case in your data basis, you must know or determine whether the target criterion is met or not. In the case of health information, for example, a positive assessment would be the target criterion if it enables an informed decision, otherwise it would be a negative assessment. Without this basis of already determined profiles, no model for future decision support is possible. One approach would be to test each profile or case, i.e. determine how it turned out. This involves considerable experimental effort. The alternative is the "view of the expert", which the model approach presented here was aimed at right from the start. Several independent experts evaluate each individual case (i.e. each profile, each combination of features) with regard to the objective of the development, e.g: "Does this health information allow an informed decision?

B. How do you proceed?

With the FFT method of expert-based feature validity, the significance of potential features is tested directly through expert assessments right from the start. Normalized frequency formats (... of every 100) are used to estimate the presence of each feature in relation to positive and negative target conditions. Resulting measures - positive predictive value, negative predictive value, false omission rate, false recognition rate, sensitivity, specificity, feature prevalence - are evaluated to minimize the number of target features. In addition, the frequency of occurrence of the target object is determined with experts. If you need assistance with the procedure, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here. 

This selection can be further reduced by testing laypersons on how successfully they evaluate the individual features. If you aim for six characteristics, this means that you always have to generate 2 to the power of 6 = 64 different combinations. Each feature can either be present or absent (or above or below a certain value limit). Since the experts also estimated feature prevalences, the associated probabilities can be used to estimate the profile frequency, i.e. how often certain combinations occur. This is crucial in order to weight the occurrence of the profiles in the data set realistically.

For all profiles, the expert evaluations are "collected" in a further study. Three experts receive every feature profile. This means that the expert's view can only be modeled using features familiar to them. This is qualitatively weaker than, for example, the FFT method of case-based feature validity. 
The decision tree is modeled on the basis of these feature assessment profiles.

The pipeline for development can be summarized in a simplified illustration:

Modeling from tree development and cross-validation can be performed manually, but in the sense of effective modeling it is easier with the open source solution R. In addition to the FFTrees package (Phillips et al., 2017), you can also download a web solution by Evaldas Jablonskis and Uwe Czienskowski from  http://www.adaptivetoolbox.net/Library/Trees/TreesHome#/. If you need assistance with this, please consult the final report on the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

A Fast-and-Frugal Tree (FFT) is modeled using the portion of cases selected as training data; often 33% or 50% of cases. This FFT has a certain quality in terms of tracking down the target feature (assessment). This means it will miss cases in the real world and cause false alarms in others. To quantify this quality, either a statistical cross-validation can be performed (the determined decision tree is applied on randomly repeated cases; test data cases), or it can be applied once to a collection of cases with assessments that were put aside before modeling. Alternatively, a completely new sample of cases with feature encodings and ratings (out-of-sample) can be collected to which the decision tree is applied (additional effort).

Which quality is sufficient depends very much on the types of errors and the costs associated with the error. Finally, the model must be tested in practice with laypersons. Here a randomised controlled study is useful in which the decision intentions of consumers who are given the decision tree are compared with those who have nothing or a standard information sheet. If you need assistance with quality or evaluation, please consult the final report of the Risk Atlas project from July 2020 or contact us. Contact details can be found here.

• Aikman, D., Galesic, M., Gigerenzer, G., Kapadia, S., Katsikopoulos, K. V., Kothiyal, A., ... & Neumann, T. (2014). Taking uncertainty seriously: Simplicity versus complexity in financial regulation. Bank of England Financial Stability Paper, 28.
• Green, L., & Mehr, D. R. (1997). What alters physicians' decisions to admit to the coronary care unit?. Journal of Family Practice, 45(3), 219–226.
• Jablonskis, E., & Czienskowski, U. (2017). Decision trees online. http://www.adaptivetoolbox.net/Library/Trees/TreesHome#/
• Jenny, M. A., Pachur, T., Williams, S. L., Becker, E., & Margraf, J. (2013). Simple rules for detecting depression. Journal of Applied Research in Memory and Cognition, 2(3), 149–157.
• Luan, S., Schooler, L. J., & Gigerenzer, G. (2011). A signal-detection analysis of fast-and-frugal trees. Psychological Review, 118(2), 316.
• Martignon, L., Katsikopoulos, K. V., & Woike, J. K. (2008). Categorization with limited resources: A family of simple heuristics. Journal of Mathematical Psychology, 52(6), 352–361.

If you would like to adopt a consumer topic from our website, you can do so in the following three ways: 

1. You are using a digital copy. Either you directly save an illustration or download our PDF, or you integrate the illustration via Link(a href) or iframe.
2. You take your analogue copy and print out our PDF. The resolution and vector-based graphic is suitable for posters and brochures.
3. You recommend the app and refer to the Risikokompass from the PlayStore and AppStore.

If you would like to develop your own model, please consult the final report on the RiskAtlas project from July 2020 or contact us. Contact details can be found here.

When using the instruments, please mention the funding agency, which is the German Federal Ministry of Justice and Consumer Protection, and the Harding Centre for Risk Literacy as the responsible developers.

Logos can be downladed here.

FFT-method case validity

FFT-method model

FFT-Methode getesteter Merkmalsvalidität

Method Natural Frequency Tree (NFT)

Method risk reading assistant for your browser

Bonus programs from health insurance are enjoying great popularity. If you actively take care of your health, almost every health insurance will offer you a reward in the form of points or cash bonuses. It doesn't matter whether you have a gym membership, certain preventive medical check-ups or walk 10,000 steps daily: Health insurances reward a whole range of health-promoting measures. However, not for every rewarded measures the benefits are proven or outweigh the potential harms (e.g. the consequences of early detection examinations forcertain types of cancer). You yourself cannot take advantage of all the rewarded measures, or the measures involve new costs for you (e.g. membership of a fitness studio). You therefore have to actively evaluate this and decide whether or not to participate. Thus, the decision tree helps you to request key information for informed participation in a bonus program in order to increase your personal benefit and limit possible risks.

Who is this decision tree for?

Insured individuals (primarily those of statutory health insurances) who are considering participating in a bonus program by their health insurance.

There is certainly more relevant knowledge that could help you to participate in an informed way:

• Whether seemingly relevant features are used: One could improve the predicted health-promoting lifestyle without actually improving it (keyword gaming)!
• What is actually considered a health-promoting lifestyle?
• A health-promoting lifestyle is a calculation based on assumptions in a mathematical model.
• Calculations of health-promoting lifestyles, like any prediction, are never perfect, but how good is this one?
• More personal responsibility (through monetary incentives) marginalizes in part personal living circumstances, genetic heritage and coincidence in their responsibility for the health of an insured person. This can jeopardise solidarity.

Where is the data coming from?

Cases – Which served as a basis?

All 64 combinations of six critical information (information profiles) and their evaluations served as the basis for training and test data sets.

Target assessment –  How were the information profiles pre-estimated?

Each information profile was assessed by three experts.

Potential features – Which requirements were considered?

Using literature studies, 34 features were identified that are relevant for an informed participation.

Selection of features and modelling

Using a feature study and statistical analysis, six potential features that are assessable by laypersons were selected and given the highest validity by the experts.

The model

The FFTrees package was used for model identification (Phillips et al., 2017). The ifan algorithm was used to optimize for balanced accuracy.

What is the quality of the data?

The data set was randomly divided into training data sets (two thirds) and test data sets (one third).

The model is of the following quality:
A cross validation of the identified decision tree resulted in the following quality measures: balanced accuracy = 0.98; correct classification of limited informed participation (share of 5% in the test set) with 0.99.  This means that in 99 out of 100 cases the fulfillment of the decision tree resembles a profile for which experts assume an informed participation in credit scoring.
Critical information profiles that prevent an informed participation are recognised correctly in 97 out of every 100 of such cases.

Potential for development

Continuous further development of the underlying training data due to changes in the market situation.

Empirical evaluation with consumers

All research results on the fundamentals and on the effectiveness of the RiskoAtlas tools in terms of competence enhancement, information search and risk communication will be published together with the project research report on 30 June 2020. If you are interested beforehand, please contact us directly (Felix Rebitschek, rebitschek@mpib-berlin.mpg.de).

Sources

•    Phillips, N. D., Neth, H., Woike, J. K., & Gaissmaier, W. (2017). FFTrees: A toolbox to create, visualize, and evaluate fast-and-frugal decision trees. Judgment and Decision making, 12(4), 344-368.

Last update: 26 November 2019.
 

Examining digital investment information

Bedarfsgerechte Kreditkartenwahl

Examining health information

Informed telematics rate selection

Vor selektiven Informationen schützen