en pl
en pl

Decyzje

Show issue
Year 12//2020 
Volume 2020 
Issue 34

Sensitivity to interventions and the relationship with numeracy

Michał Dzieżyk
SWPS University of Social Sciences and Humanities

Weronika Hetmańczuk
SWPS University of Social Sciences and Humanities

Jakub Traczyk
SWPS University of Social Sciences and Humanities

12//2020 2020 (34) Decyzje

DOI 10.7206/DEC.1733-0092.147

Article contents

Abstract

The main goal of this research was to investigate whether people exhibit algorithm aversion—a tendency to avoid using an imperfect algorithm even if it outperforms human judgments—in the case of estimating students’ percentile scores on a standardized math test. We also explored the relationships between numeracy and algorithm aversion and tested two interventions aimed at reducing algorithm aversion. In two studies, we asked participants to estimate the percentiles of 46 real 15-year-old Polish students on a standardized math test. Participants were offered the opportunity to compare their estimates with the forecasts of an algorithm—a statistical model that predicted real percentile scores based on fi ve explanatory variables (i.e., gender, repeating a class, the number of pages read before the exam, the frequency of playing online games, socioeconomic status). Across two studies, we demonstrated that even though the predictions of the statistical model were closer to students’ percentile scores,
participants were less likely to rely on the statistical model predictions in making forecasts. We also found that higher statistical numeracy was related to a higher reluctance to use the algorithm. In Study 2, we introduced two interventions to reduce algorithm aversion. Depending on the experimental condition, participants either received feedback on statistical model predictions or were provided with a detailed description of the statistical model. We found that people, especially those with higher statistical numeracy, avoided using the imperfect algorithm even though it outperformed human judgments.
Interestingly, a simple intervention that explained how the statistical model
works led to better performance in an estimation task.

References

  1. Ashby, N.J.S. (2017). Numeracy predicts preference consistency : Deliberative search heuristics increase choice consistency for choices from description and experience. Judgement and Decision Making, 12(2), 128–139. [Google Scholar]
  2. Bürkner, P.-C. (2018). Advanced Bayesian Multilevel Modeling with the R Package brms. The R Journal, 10(1), 395–411. https://doi.org/10.32614/RJ-2018-017 [Google Scholar]
  3. Burton, J.W., Stein, M., & Jensen, T.B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239. https://doi.org/10.1002/bdm.2155 [Google Scholar]
  4. Cokely, E.T., Feltz, A., Ghazal, S., Allan, J.N., Petrova, D., & Garcia-Retamero, R. (2018). Decision making skill: From intelligence to numeracy and expertise. In K.A. Ericsson, R.R. Hoffman, [Google Scholar]
  5. A. Kozbelt, & A.M. Williams (Eds.), The Cambridge Handbook of Expertise and Expert Performance (2nd ed., pp. 476–505). Cambridge University Press. [Google Scholar]
  6. Cokely, E.T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin numeracy test. Judgment and Decision Making, 7(1), 25–47. http://journal.sjdm.org/11/11808/jdm11808.html [Google Scholar]
  7. Cokely, E.T., & Kelley, C.M. (2009). Cognitive abilities and superior decision making under risk : A protocol analysis and process model evaluation. Judgement and Decision Making, 4(1), 20–33. http://journal.sjdm.org/81125/jdm81125.pdf [Google Scholar]
  8. Dawes, R.M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571–582. https://doi.org/10.1037/0003-066X.34.7.571 [Google Scholar]
  9. Dawes, R.M., Faust, D., & Meehl, P. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573 [Google Scholar]
  10. Diab, D.L., Pui, S.-Y., Yankelevich, M., & Highhouse, S. (2011). Lay Perceptions of Selection Decision Aids in US and Non-US Samples. International Journal of Selection and Assessment, 19(2), 209–216. https://doi.org/10.1111/j.1468-2389.2011.00548.x [Google Scholar]
  11. Dietvorst, B.J., & Bharti, S. (2020). People Reject Algorithms in Uncertain Decision Domains Because They Have Diminishing Sensitivity to Forecasting Error. Psychological Science, 31(10), 1302–1314. https://doi.org/10.1177/0956797620948841 [Google Scholar]
  12. Dietvorst, B.J., Simmons, J.P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033 [Google Scholar]
  13. Dietvorst, B.J., Simmons, J.P., & Massey, C. (2018). Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them. Management Science, 64(3), 1155–1170. https://doi.org/10.1287/mnsc.2016.2643 [Google Scholar]
  14. Einhorn, H.J. (1986). Accepting Error to Make Less Error. Journal of Personality Assessment, 50(3), 387–395. https://doi.org/10.1207/s15327752jpa5003_8 [Google Scholar]
  15. Esmaeilzadeh, P., Sambasivan, M., Kumar, N., & Nezakati, H. (2015). Adoption of clinical decision support systems in a developing country: Antecedents and outcomes of physician’s threat to perceived professional autonomy. International Journal of Medical Informatics, 84(8), 548–560. https://doi.org/10.1016/j.ijmedinf.2015.03.007 [Google Scholar]
  16. Estrada-Mejia, C., Peters, E., Dieckmann, N.F., Zeelenberg, M., De Vries, M., & Baker, D.P. (2020). Schooling, numeracy, and wealth accumulation: A study involving an agrarian population. Journal of Consumer Affairs, 54(2), 648-674. [Google Scholar]
  17. Estrada-Mejia, C., de Vries, M., & Zeelenberg, M. (2016). Numeracy and wealth. Journal of Economic Psychology, 54, 53–63. https://doi.org/10.1016/j.joep.2016.02.011 [Google Scholar]
  18. Fagerlin, A., Zikmund-Fisher, B.J., Ubel, P.A., Jankovic, A., Derry, H.A., & Smith, D.M. (2007). Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Medical Decision Making, 27(5), 672–680. https://doi.org/10.1177/0272989X07304449 [Google Scholar]
  19. Fildes, R., Goodwin, P., Lawrence, M., & Nikolopoulos, K. (2009). Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply chain planning. International Journal of Forecasting, 25(1), 3–23. https://doi.org/10.1016/j.ijforecast.2008.11.010 [Google Scholar]
  20. Fry, H. (2018). Hello World: How to be Human in the Age of the Machine. Random House. [Google Scholar]
  21. Garcia-Retamero, R., Sobkow, A., Petrova, D., Garrido, D., & Traczyk, J. (2019). Numeracy and Risk Literacy: What Have We Learned so Far? The Spanish Journal of Psychology, 22, E10. https://doi.org/10.1017/sjp.2019.16 [Google Scholar]
  22. Ghazal, S., Cokely, E.T., & Garcia-Retamero, R. (2014). Predicting biases in very highly educated samples: Numeracy and metacognition. Judgment and Decision Making, 9(1), 15–34. [Google Scholar]
  23. Green, G.I., & Hughes, C.T. (1986). Effects of Decision Support Systems Training and Cognitive Style on Decision Process Attributes. Journal of Management Information Systems, 3(2), 83–93. https://doi.org/10.1080/07421222.1986.11517764 [Google Scholar]
  24. Grove, W.M., Zald, D.H., Lebow, B.S., Snitz, B.E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-12.1.19 [Google Scholar]
  25. Highhouse, S. (2008). Stubborn Reliance on Intuition and Subjectivity in Employee Selection. Industrial and Organizational Psychology, 1(3), 333–342. 2008.00058.x https://doi.org/10.1111/j.1754- [Google Scholar]
  26. Inthorn, J., Tabacchi, M. E., & Seising, R. (2015). Having the Final Say: Machine Support of Ethical Decisions of Doctors. In S.P. van Rysewyk & M. Pontier (Eds.), Machine Medical Ethics (pp. 181–206). Springer International Publishing. https://doi.org/10.1007/978-3-319-08108-3_12 [Google Scholar]
  27. Jasper, J.D., Bhattacharya, C., & Corser, R. (2017). Numeracy Predicts More Effortful and Elaborative Search Strategies in a Complex Risky Choice Context: A Process-Tracing Approach. Journal of Behavioral Decision Making, 30(2), 224–235. https://doi.org/10.1002/bdm.1934 [Google Scholar]
  28. Jasper, J.D., Bhattacharya, C., Levin, I.P., Jones, L., & Bossard, E. (2013). Numeracy as a Predictor of Adaptive Risky Decision Making. Journal of Behavioral Decision Making, 26(2), 164–173. https://doi.org/10.1002/bdm.1748 [Google Scholar]
  29. Logg, J.M., Minson, J.A., & Moore, D.A. (2019). Algorithm appreciation : People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90–103. https://doi.org/10.1016/j.obhdp.2018.12.005 [Google Scholar]
  30. Longoni, C., Bonezzi, A., & Morewedge, C.K. (2019). Resistance to Medical Artifi cial Intelligence. Journal of Consumer Research, 46(4), 629–650. https://doi.org/10.1093/jcr/ucz013 [Google Scholar]
  31. Meehl, P. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. University of Minnesota Press. https://doi.org/10.1037/11281-000 [Google Scholar]
  32. Millroth, P., & Juslin, P. (2015). Prospect evaluation as a function of numeracy and probability denominator. Cognition, 138, 1–9. https://doi.org/10.1016/j.cognition.2015.01.014 [Google Scholar]
  33. Nalborczyk, L., Batailler, C., Loevenbruck, H., Vilain, A., & Bürkner, P.-C. (2019). An Introduction to Bayesian Multilevel Models Using brms: A Case Study of Gender Effects on Vowel Variability in Standard Indonesian. Journal of Speech, Language, and Hearing Research, 62(5), 1225–1242. https://doi.org/10.1044/2018_JSLHR-S-18-0006 [Google Scholar]
  34. Önkal, D., Goodwin, P., Thomson, M., Gönül, S., & Pollock, A. (2009). The relative infl uence of advice from human experts and statistical methods on forecast adjustments. Journal of Behavioral Decision Making, 22(4), 390–409. https://doi.org/10.1002/bdm.637 [Google Scholar]
  35. Peters, E. (2012). Beyond Comprehension: The Role of Numeracy in Judgments and Decisions. Current Directions in Psychological Science, 21(1), 31–35. https://doi.org/10.1177/0963721411429960 [Google Scholar]
  36. Peters, E., & Bjalkebring, P. (2015). Multiple numeric competencies: When a number is not just a number. Journal of Personality and Social Psychology, 108(5), 802–822. https://doi.org/10.1037/pspp0000019 [Google Scholar]
  37. Peters, E., Kate, M., Simon, A., Tompkins, M.K., Knoll, M.A.Z., Ardoin, S.P., Shoots-Reinhard, B., & Meara, A.S. (2019). Despite high objective numeracy, lower numeric confi dence relates to worse fi nancial and medical outcomes. Proceedings of the National Academy of Sciences, 116(39), 19386–19391. https://doi.org/10.1073/pnas.1903126116 [Google Scholar]
  38. Peters, E., Västfjäll, D., Slovic, P., Mertz, C.K., Mazzocco, K., & Dickert, S. (2006). Numeracy and Decision Making. Psychological Science, 17(5), 407–413. https://doi.org/10.1111/j.1467-2006.01720.x [Google Scholar]
  39. Petrova, D., Garcia-Retamero, R., Catena, A., Cokely, E.T., Heredia Carrasco, A., Arrebola Moreno, a Retrospective Study of Acute Coronary Syndrome Survival. Annals of Behavioral Medicine, 51(2), 292–306. https://doi.org/10.1007/s12160-016-9853-1 [Google Scholar]
  40. Petrova, D., Garcia-Retamero, R., Catena, A., & van der Pligt, J. (2016). To screen or not to screen: What factors infl uence complex screening decisions? Journal of Experimental Psychology: [Google Scholar]
  41. Petrova, D., Traczyk, J., & Garcia-Retamero, R. (2019). What shapes the probability weighting function? Infl uence of affect, numeric competencies, and information formats. Journal of Behavioral Decision Making, 32(2), 124–139. https://doi.org/10.1002/bdm.2100 [Google Scholar]
  42. Petrova, D., van der Pligt, J., & Garcia-Retamero, R. (2014). Feeling the Numbers: On the Interplay Between Risk, Affect, and Numeracy. Journal of Behavioral Decision Making, 27(3), 191–199. https://doi.org/10.1002/bdm.1803 [Google Scholar]
  43. Prahl, A., Dexter, F., Braun, M.T., & Van Swol, L. (2013). Review of Experimental Studies in Social Psychology of Small Groups When an Optimal Choice Exists and Application to Operating Room Management Decision-Making. Anesthesia & Analgesia, 117(5), 1221–1229. https://doi.org/10.1213/ANE.0b013e3182a0eed1 [Google Scholar]
  44. Schley, D.R., & Peters, E. (2014). Assessing “Economic Value” Symbolic-Number Mappings Predict Risky and Riskless Valuations. Psychological Science, 25(3), 753–761. https://doi.org/10.1177/0956797613515485 [Google Scholar]
  45. Shaffer, V.A., Probst, C.A., Merkle, E.C., Arkes, H.R., & Medow, M.A. (2013). Why Do Patients Derogate Physicians Who Use a Computer-Based Diagnostic Support System? Medical Decision Making, 33(1), 108–118. https://doi.org/10.1177/0272989X12453501 [Google Scholar]
  46. Shoots-Reinhard, B., Erford, B., Romer, D., Evans, A.T., Shoben, A., Klein, E.G., & Peters, E. (2020). Numeracy and memory for risk probabilities and risk outcomes depicted on cigarette warning labels. Health Psychology. https://doi.org/10.1037/hea0000879 [Google Scholar]
  47. Sobków, A., Figol, J., & Traczyk, J. (2020). Zdolności numeryczne jako kluczowe zdolności poznawcze w procesie podejmowania decyzji. Decyzje, 33, 25–53. https://doi.org/10.7206/DEC.1733-0092.139 [Google Scholar]
  48. Sobkow, A., Fulawka, K., Tomczak, P., Zjawiony, P., & Traczyk, J. (2019). Does mental number line training work? The effects of cognitive training on real-life mathematics, numeracy, and decision making. Journal of Experimental Psychology: Applied, 25(3), 372–385. https://doi.org/10.1037/xap0000207 [Google Scholar]
  49. Sobkow, A., Garrido, D., & Garcia-Retamero, R. (2020). Psychological Perspectives on Financial Decision Making. In T. Zaleskiewicz & J. Traczyk (Eds.), Psychological Perspectives on Financial Decision Making. Springer International Publishing. https://doi.org/10.1007/978-3-030-45500-2 [Google Scholar]
  50. Sobkow, A., Olszewska, A., & Traczyk, J. (2020). Multiple numeric competencies predict decision outcomes beyond fl uid intelligence and cognitive refl ection. Intelligence, 80, 101452. https://doi.org/10.1016/j.intell.2020.101452 [Google Scholar]
  51. Sobkow, A., Traczyk, J., Kaufman, S.B., & Nosal, C. (2018). The structure of intuitive abilities and their relationships with intelligence and Openness to Experience. Intelligence, 67, 1–10. https://doi.org/10.1016/j.intell.2017.12.001 [Google Scholar]
  52. Traczyk, J., & Fulawka, K. (2016). Numeracy moderates the infl uence of task-irrelevant affect on probability weighting. Cognition, 151, 37–41. https://doi.org/10.1016/j.cognition.2016.03.002 [Google Scholar]
  53. Traczyk, J., Fulawka, K., Lenda, D., & Zaleskiewicz, T. (2021). Consistency in probability processing as a function of affective context and numeracy. Journal of Behavioral Decision Making, 34(2), 228–246. https://doi.org/10.1002/bdm.2206 [Google Scholar]
  54. Traczyk, J., Lenda, D., Serek, J., Fulawka, K., Tomczak, P., Strizyk, K., Polec, A., Zjawiony, P., & Sobkow, A. (2018). Does Fear Increase Search Effort in More Numerate People? An Experimental Study Investigating Information Acquisition in a Decision From Experience Task. Frontiers in Psychology, 9(1203). https://doi.org/10.3389/fpsyg.2018.01203 [Google Scholar]
  55. Traczyk, J., Sobkow, A., Fulawka, K., Kus, J., Petrova, D., & Garcia-Retamero, R. (2018). Numerate decision makers don’t use more effortful strategies unless it pays: A process tracing investigation of skilled and adaptive strategy selection in risky decision making. Judgment and Decision Making, 13(4), 372–381. http://www.sjdm.org/journal/17/17208/jdm17208.pdf [Google Scholar]
  56. Traczyk, J., Sobkow, A., Matukiewicz, A., Petrova, D., & Garcia-Retamero, R. (2020). The experience-based format of probability improves probability estimates: The moderating role of individual differences in numeracy. International Journal of Psychology, 55(2), 273–281. https://doi.org/10.1002/ijop.12566 [Google Scholar]
  57. Weller, J.A., Dieckmann, N.F., Tusler, M., Mertz, C.K., Burns, W.J., & Peters, E. (2013). Development and Testing of an Abbreviated Numeracy Scale: A Rasch Analysis Approach. Journal of Behavioral Decision Making, 26(2), 198–212. https://doi.org/10.1002/bdm.1751 [Google Scholar]

Full metadata record

Cite this record

APA style

Dzieżyk, Michał & Hetmańczuk, Weronika & Traczyk, Jakub (2020). Sensitivity to interventions and the relationship with numeracy. (2020). Sensitivity to interventions and the relationship with numeracy. Decyzje, 2020(34), 67-90. https://doi.org/10.7206/DEC.1733-0092.147 (Original work published 12//2020AD)

MLA style

Dzieżyk, Michał and Hetmańczuk, Weronika and Traczyk, Jakub. “Sensitivity To Interventions And The Relationship With Numeracy”. 12//2020AD. Decyzje, vol. 2020, no. 34, 2020, pp. 67-90.

Chicago style

Dzieżyk, Michał and Hetmańczuk, Weronika and Traczyk, Jakub. “Sensitivity To Interventions And The Relationship With Numeracy”. Decyzje, Decyzje, 2020, no. 34 (2020): 67-90. doi:10.7206/DEC.1733-0092.147.

Copyright info

copyright info

CC-BY

People also read: