The Evidence-Based Paradigm

sufi

lala
lore
0_dd989_a809bfab_orig.jpg

will you
  • carry on towards the lair of ultimate objectivity
  • consume the enchanted data strawberry
  • scale the algorithmic flower of arboresence
  • soar on yr staff of flying crypto
  • change channel again?
 

toko

Well-known member
im convinced that the real reason evidence based approaches are used is not because they actually lead to better outcomes, but rather because "evidence based" decisions lend itself to bureaucratization/formalization of the decision making process.
 

sufi

lala
im convinced that the real reason evidence based approaches are used is not because they actually lead to better outcomes, but rather because "evidence based" decisions lend itself to bureaucratization/formalization of the decision making process.
this assertion is quantifiably correct
 

toko

Well-known member
this in turn makes it easier to justify to the public/share holders that things are going okay
 

shakahislop

Well-known member
Some things are pretty measurable, or otherwise easy enough to get evidence for. Most stuff is more complex than that. The idea of evidence-based decision-making is the right one and you'd have to be an idiot to disagree with the idea that you make decisions using the best data etc that you have to hand, in theory. However you can't get away from the fact that most important decisions are made by people working in organizations of one kind or another, and the idea that its possible for organizations to be pure evidence-driven machines feels pretty unrealistic when you think about these kinds of structures work.
 

william_kent

Well-known member
Some things are pretty measurable, or otherwise easy enough to get evidence for. Most stuff is more complex than that. The idea of evidence-based decision-making is the right one and you'd have to be an idiot to disagree with the idea that you make decisions using the best data etc that you have to hand, in theory. However you can't get away from the fact that most important decisions are made by people working in organizations of one kind or another, and the idea that its possible for organizations to be pure evidence-driven machines feels pretty unrealistic when you think about these kinds of structures work.

twitter? Musk? how is that going to go?

edit: I worked with "young" people a while ago, and I'm wondering, would they approve?
 

shakahislop

Well-known member
Most people who have engaged with social science research methodology on a moderately sophisticated basis understand very well the epistemological weaknesses of the research methods available to policy makers in most situations. People making decisions almost never understand this and tend to have much more postivist views of the world.
 

william_kent

Well-known member
Most people who have engaged with social science research methodology on a moderately sophisticated basis understand very well the epistemological weaknesses of the research methods available to policy makers in most situations. People making decisions almost never understand this and tend to have much more postivist views of the world.

whoah!

big words!

"epistemological"

define yr terms bro!
 

shakahislop

Well-known member
People making high-level policy-type decisions are pretty varied. They're making decisions about what to do based on a whole range of considerations, depending on what kind of thing you're talking about. 'Evidence', by which I mean data, the opinions of researchers, and studies by in-house departments, monitoring systems, think tanks, universities or whatever, are generally only one piece of the puzzle. The management school of thought that's referred to by the term 'evidence based' is in effect an attempt to increase the importance given to evidence over other factors.
 

sufi

lala
People making high-level policy-type decisions are pretty varied. They're making decisions about what to do based on a whole range of considerations, depending on what kind of thing you're talking about. 'Evidence', by which I mean data, the opinions of researchers, and studies by in-house departments, monitoring systems, think tanks, universities or whatever, are generally only one piece of the puzzle. The management school of thought that's referred to by the term 'evidence based' is in effect an attempt to increase the importance given to evidence over other factors.
the evidence based community
 

shakahislop

Well-known member
One of the most complicated things about it is that for a lot of things the actual evidence base is very weak, though obviously it depends exactly what you're talking about, particularly in terms of predicting how one given action will have any particular consequences. Even something as well studied and divorced from politics as interest rate decisions by central banks - they are still guessing to some extent. There is a lot of evidence and its an area which lends itself well to quantification, and they have a load of very good established data systems and multiple countries to look at over time. But still they are figuring it out as they go along and things that used to be thought of as impossible happen (0 interest rates, negative interest rates etc)
 

padraig (u.s.)

a monkey that will go ape
im convinced that the real reason evidence based approaches are used is not because they actually lead to better outcomes, but rather because "evidence based" decisions lend itself to bureaucratization/formalization of the decision making process.
got into essentially this same argument a couple years ago w/gus and his fellow-traveler who eventually rage quit (don't remember their name - big Tesla fan, v knowledgeable about systems management)

their position iirc was basically that an algorithm isn't necessarily better but that you its process is transparent (will always give the same output given the same input) and easier to change than the opacity and unpredictability of individual human decision-making. that position falls apart for me basically immediately bc it's always humans creating the algorithms, so you never remove the potential flaws of human decision-making - if anything you potentially amplify them by further removing that decision-making power from 1) accessibility for the people it's effecting 2) putting it in the hands of someone who knows how to write algorithms rather than someone who at least nominally knows the field that they're being written for (generally not the same person if it's outside of software engineering etc).

that's not to say they are never useful as tools to aid decision-making, the devil is in exactly how much control is being ceded and to what end.

even when they are used you still wind up with same very human problems. i.e. applicant tracking software (ATS) for helping HR people filter through resumes - sounds like a great way to weed out unqualified applicants, but since applicants know about ATS, they tailor their resumes to game it - there's an entire cottage industry devoted to it - and here we are, full circle back to human decision-making. applicants also hate ATS, with good reason, like I'm pretty sure most people hate the idea of software directly making crucial decisions about their lives.
 

william_kent

Well-known member
One of the most complicated things about it is that for a lot of things the actual evidence base is very weak, though obviously it depends exactly what you're talking about, particularly in terms of predicting how one given action will have any particular consequences. Even something as well studied and divorced from politics as interest rate decisions by central banks - they are still guessing to some extent. There is a lot of evidence and its an area which lends itself well to quantification, and they have a load of very good established data systems and multiple countries to look at over time. But still they are figuring it out as they go along and things that used to be thought of as impossible happen (0 interest rates, negative interest rates etc)

don't even get me started on long range weather forecasts!

( edit: 5 days at best! )
 

sufi

lala
One of the most complicated things about it is that for a lot of things the actual evidence base is very weak, though obviously it depends exactly what you're talking about, particularly in terms of predicting how one given action will have any particular consequences. Even something as well studied and divorced from politics as interest rate decisions by central banks - they are still guessing to some extent. There is a lot of evidence and its an area which lends itself well to quantification, and they have a load of very good established data systems and multiple countries to look at over time. But still they are figuring it out as they go along and things that used to be thought of as impossible happen (0 interest rates, negative interest rates etc)
because economics only works well retrospectively tho? or is that the case for all of this ??
 
Top