Risk reading assistant for the browser: RisikoRadar
The power of statistics and how to better understand them
Authors and media who use statistics in texts have a great influence on the perception of these statistics. Risk perception becomes problematic through so-called relative risk expressions.
In 1995, the Committee on Safety of Medicines published a report stating that the risk of thromboembolism increased by 100 % through taking the third-generation pill. This was picked up by the major media and created concerns among large parts of the population. Instead of switching to earlier generations of the pill, thousands of women stopped taking the pill in general. 13,600 abortions in England and Wales can be attributed to this media event (Furedi, 1999).
The doubled the risk of thrombosis was based solely on the fact that two out of every 7,000 women who took the third-generation pill got thrombosis. This was one more woman than among those women who took the second-generation pill. The fact that the increase from 1 to 2 per 7,000 women - the absolute increase in risk - was communicated as 100 % relative increase in risk was a serious violation of the standards of risk communication.
Without absolute risk information, it is not possible to determine how high an initial risk is. How many were affected at first? How high is the baseline risk? In addition, it is not clear how large the increasing or decreasing effect actually is. The actual effect size remains unknown. An example: A new drug may reduce side effects by 50%. However, whether this means that only 4 out of 10 patients are affected instead of 8, or whether this means that only 1 out of 100,000 patients are affected instead of 2, cannot be determined with a relative indication. Knowing the absolute risk information is therefore essentail for being an informed patient.
1. Training authors, scientists, physicians, managers and editors to ensure that texts always provide absolute risk or risk changes in addition to relative information.
2. Training readers. In the context of institutional education (schools, vocational schools, universities), awareness must be raised of the problems in appropriate subject-specific situations.
3. Strengthening and supporting consumers - the RisikoRadar
Most consumers read digital texts. Particularly in the digital environment, many authors are not sufficiently trained or even guided by interests. In order to support consumers so that they are not misled by relative representations of risk, an intervention approach is needed which becomes active as soon as the problem arises.
Consumers use browsers to surf on the Internet. Within a browser, an intervention tool can be installed that becomes active when problematic text passages appear. If a relative representation of risk appears, several interventions are possible. If it stands alone, without an indication of absolute change, there should be a warning that key information is missing. This means: the initial situation (baseline rate) remains hidden and so does the actual size of change (effect size). An illustrative example should be included.
If the relative risk is accompanied by an absolute risk, a correct calculation by the reader is still necessary in order to understand the size of change or the respective missing initial or target situation. The fact box with icon arrays (McDowell et al., 2016, 2019) is, among others, a known tool from risk communication, to compare risks. Thus, the reader can be aided by being providing with a visualisation.
Furedi, A. (1999). Social consequences. The public health implications of the 1995'pill scare'. Human Reproduction Update, 5(6), 621-626.
McDowell, M. E., Gigerenzer, G., Wegwarth, O., & Rebitschek, F. G. (2019). Effect of tabular and icon fact box formats on comprehension of benefits and harms of prostate cancer screening: A randomized trial. Medical Decision Making, 39, 41-56.
McDowell, M., Rebitschek, F. G., Gigerenzer, G., & Wegwarth, O. (2016). A Simple Tool for Communicating the Benefits and Harms of Health Interventions: A Guide for Creating a Fact Box. MDM Policy & Practice, 1(1).
RisikoRadar is a browser extension which processes linguistic contents of internet pages on the basis of certain rules.
The RisikoRadar only works in German!
Browsers do not have capacities for this kind of processing by default. Therefore, installing an additional application on the operating system is necessary:
Installation
1. Download the installer file (Windows, macOS).
Installation under Windows 7–10
2. Double click the downloaded file to open it.
3. Install the application according to the installation instructions (possibly running in the background).
4. Follow the completion of the installation in the Chrome or Firefox store to download the browser extension for the desired browser. If your default browser is not the browser in which you want to use RisikoRadar, you must copy the link (Ctrl+C) and paste it into the desired browser.
5. Add the browser extension directly to your browser in the running Chrome or Firefox store.
Installation under macOS 10.12–10.15
2. Open the downloaded file.
3. Install the application according to the installation instructions (possibly running in the background).
4. Follow the completion of the installation in the Chrome or Firefox store to download the browser extension for the desired browser. If your default browser is not the browser in which you want to use RisikoRadar, you must copy the link (Ctrl+C) and paste it into the desired browser.
5. Add the browser extension directly to your browser in the running Chrome or Firefox store.
Uninstallation can be carried out via the operating system's own program management.
Privacy policy
1. The browser extension only works locally. This means that the websites that the extension reads are not transmitted to a server. The browser extension processes all data on your computer.
2. Here is our privacy statement regarding the browser extention.
FFT-Methode getesteter Merkmalsvalidität
The power of statistics and how to better understand them
Well, statistics are strange. If we read somewhere: "The danger increases by 100%", we become alarmed and worried. If, on the other hand, we hear that the risk for something decreases by 90% if we do one thing or another, then we will most likely seriously consider doing this one thing or another. Such increases or decreases do not say much at first. We should therefore always ask ourselves the following question with relative statistical data: What does this percentage mean? In many texts, only relative figures are given. This can also be a journalistic means of making facts sound more dramatic. Our browser extension "RisikoRadar" is intended to draw your attention to these cases, illustrate absolute risks graphically and support you in breaking the power of statistics.
What is the purpose of the RisikoRadars?
Besides being useful to consumers, it should also train certain abilities when used.
1. The RisikoRadar warns you if you might be misled by an isolated relative risk statistic
2. The RiskRadar helps you to correctly interpret relative risk statistics in conjunction with absolute risk statistics by offering you a supporting visualisation in the form of an icon array fact box.
3. The RisikoRadar makes you more aware of relative risk and change expressions in texts. Thus, you become a more critical reader.
4. You learn what relative and absolute risks are. You transfer this knowledge to situations in which you do not use the RisikoRadar on the Internet.
Installation
1. Download the installer file (Windows, macOS).
Installation under Windows 7–10
2. Double click the downloaded file to open it.
3. Install the application according to the installation instructions (possibly running in the background).
4. Follow the completion of the installation in the Chrome or Firefox store to download the browser extension for the desired browser. If your default browser is not the browser in which you want to use RisikoRadar, you must copy the link (Ctrl+C) and paste it into the desired browser.
5. Add the browser extension directly to your browser in the running Chrome or Firefox store.
Installation under macOS 10.12–10.15
2. Open the downloaded file.
3. Install the application according to the installation instructions (possibly running in the background).
4. Follow the completion of the installation in the Chrome or Firefox store to download the browser extension for the desired browser. If your default browser is not the browser in which you want to use RisikoRadar, you must copy the link (Ctrl+C) and paste it into the desired browser.
5. Add the browser extension directly to your browser in the running Chrome or Firefox store
Uninstallation can be carried out via the operating system's own program management.
The RisikoRadar is based on a rule system, which was developed with training and test data from a body of German texts. Like all models, the rule system can never be perfect, which means that text passages are misinterpreted or critical text passages are overlooked.
Relative expressions that are communicated together with an absolute base level or target level are recognised by the browser extension in 95% of the cases (recall in the test data set). At the same time, only 2 false alarms were triggered for 6,000 sentences. Isolated change expressions (relative risks) are recognised by the browser extension in 94% of the cases. 47 false alarms occurred in 6,000 sentences.
The RisikoRadar only works on the basis of single sentences. This is due to the considerably higher technical effort required to analyse across sentences. However, key information is often found in preceding or following sentences. A warning by the RisikoRadar therefore means first of all that the whole context must be checked again to see whether the author has provided the crucial information after all.
The automatic support that makes the written text more understandable through visualisations with an icon array fact box does not change the intention of the author, however. Thus, it cannot be ruled out that problematic, criminal or dehumanizing contents may be given more credibility. If you happen to come across such a scenario, please contact us.
The browser extension RisikoRadar was technically developed by LangTec.
The text analysis algorithms on which the RisikoRadar is based were developed using a development corpus and linguistic rules.
In a validation on a separate demonstration corpus with a total of 28,111 sentences, 206 constellations of relative and absolute expressions of change (89.2% sensitivity) were recognised out of 231 target sentences. In addition, of 27,880 other sentenes, 27,866 were confirmed to be free of combined relative and absolute expressions of change (99.9% specificity). Regarding the phenomenon of a combined statement of relative and absolute expressions of change, the positive predictive value is 93.6%, which is the probability that the algorithm will correctly recognise such an expression.
In the same corpus, out of 427 target sentences, 365 isolated relative expressions of change were detected (85.5 % sensitivity). Furthermore, of 27,684 other sentences, 27,645 were confirmed to be free of isolated relative expressions of change (99.9% specificity). For isolated relative expressions of change, the positive predictive value is 90.3%, which is the probability that the algorithm correctly recognises such an expression.
The detection is followed by the classification of correct sentence structure elements. This is also not a perfect way to apply linguistic rules to the real digital world.
For combinations of relative and absolute expressions of change, 90.68 % correct structural elements were recognised. In this respect, the proportion of correct recognitions in the practical application drops to a sensitivity of 80.9 %.
For isolated relative expressions of change, 97.37 % correct structural elements were detected. In this respect, the proportion of correct detections in the practical application drops to a sensitivity of 83.2 %.
Empirical evaluation with consumers
All research results on the fundamentals and on the effectiveness of the RiskoAtlas tools in terms of competence enhancement, information search and risk communication will be published together with the project research report on 30 June 2020. If you are interested beforehand, please contact us directly (Felix Rebitschek, rebitschek@mpib-berlin.mpg.de).
Last update: 27 November 2019.