FFT-method expert validity
For an informed credit score – small number with great effect
Why is it difficult to provide decision support for problems of uncertainty?
How to construct a decision tree for a consumer problem - the FFT method of expert feature validity
A. What do you need?
B. How do you proceed?
- Aikman, D., Galesic, M., Gigerenzer, G., Kapadia, S., Katsikopoulos, K. V., Kothiyal, A., ... & Neumann, T. (2014). Taking uncertainty seriously: Simplicity versus complexity in financial regulation. Bank of England Financial Stability Paper, 28.
- Green, L., & Mehr, D. R. (1997). What alters physicians' decisions to admit to the coronary care unit?. Journal of Family Practice, 45(3), 219–226.
- Jablonskis, E., & Czienskowski, U. (2017). Decision trees online. http://www.adaptivetoolbox.net/Library/Trees/TreesHome#/
- Jenny, M. A., Pachur, T., Williams, S. L., Becker, E., & Margraf, J. (2013). Simple rules for detecting depression. Journal of Applied Research in Memory and Cognition, 2(3), 149–157.
- Luan, S., Schooler, L. J., & Gigerenzer, G. (2011). A signal-detection analysis of fast-and-frugal trees. Psychological Review, 118(2), 316.
- Martignon, L., Katsikopoulos, K. V., & Woike, J. K. (2008). Categorization with limited resources: A family of simple heuristics. Journal of Mathematical Psychology, 52(6), 352–361.
If you have ever wanted to rent an apartment, you will know it: the Schufa certificate. A note with a small percentage indicating the probability that you will meet your payment obligations. It can decide whether or not you will get the apartment. However, your creditworthiness is much more frequently demanded than you realise, because these processes often run in the background. No matter whether you take out a loan, sign a cell phone contract or make a purchase by invoice: The sellers always check whether you are creditworthy. However, the criteria used by credit agencies to classify you are often opaque and difficult for you to understand. In order for you to appear as a self-determined customer, tenant and business partner, an informed participation in the credit scoring is necessary: Which data are decisive, and are they considered accurately? You should be able to identify, verify and correct all information in order to protect your rights to a realistic, scientifically appropriate rating. Our decision tree will help you request key information for an informed participation in credit scoring.
For whom is the graphic suitable?
The decision tree is intended to support all consumers, because every consumer is an economic actor who signs phone or Internet contracts, rents, or shops online. Of particular importance is the decision tree for those whose credit was refused, who were not allowed to pay by invoice or who were refused payment by invoice even though it was offered. Check what information is known about you and whether it is correct.
There is more relevant knowledge that could help you signal your creditworthiness more clearly and participate more informedly:
You can inquire regularly and free of charge with the credit agencies about the assessment of your creditworthiness.
Depending on the provider (credit agency), the calculated creditworthiness can be influenced by:
Number of bank accounts
number of credit cards
Existing leasing contract
Existing installment payment
Existing guarantee of payment
Negative features (e.g. failures to pay) of other households in your housing unit
Place of residence (address, if the provider does not know enough about you)
Other, unknown features and behaviours
Calculations of creditworthiness are, like any prediction, never perfect.
Two important types of errors arise in the calculation of creditworthiness: creditworthiness that is erroneously too bad or erroneously too good.
Errors in the calculation can be due to the quality of the information used or because of the prediction model.
There are identification errors or assignment errors (entity recognition).
There are indications of how reliably the creditworthiness can be calculated.
Does the credit agency use a model which can report back that it is not possible to make a prediction due to lack of data?
Non-use of creditworthiness-relevant features carries a risk of discrimination
The use of seemingly relevant features [there is no plausible explanation for their occurrence in creditworthiness] carries a risk of discrimination.
Indirect access to data that is actually protected entails a risk of discrimination
Characteristics that you can influence: The use of characteristics that cannot be influenced or accounted for [e.g. collective responsibility if negative features of neighbors were used for own creditworthiness] carries a risk of discrimination.
Creditworthiness calculations for specific consumer groups (= rarer combinations of characteristics) are less reliable.
Request data sources of personal information about your characteristics and behaviours!
Where is the data coming from?
Cases – Which served as a basis?
Target assessment – How were the information profiles pre-estimated?
Potential features – Which requirements were considered?
Using literature studies, 50 features were identified that are relevant for an informed participation.
Selection of features and modelling
What is the quality of the data?
The data set was randomly divided into training data sets (two thirds) and test data sets (one third).
The model is of the following quality:
A cross validation of the identified decision tree resulted in the following quality measures: balanced accuracy = 0.97; correct classification of limited informed participation (share of 3% in the test set) with 0.99. This means that in 99 out of 100 cases the fulfillment of the decision tree resembles a profile for which experts assume an informed participation in credit scoring.
Critical information profiles that prevent an informed participation are recognised correctly in 94 out of every 100 of such cases.
Potential for development
Empirical evaluation with consumers
• Phillips, N. D., Neth, H., Woike, J. K., & Gaissmaier, W. (2017). FFTrees: A toolbox to create, visualize, and evaluate fast-and-frugal decision trees. Judgment and Decision making, 12(4), 344-368.
Last update: 27 November 2019.