Science Journal: Scientists research questions few would ask
This article goes into the decline in standards of research seen of late. I hope the author and publisher won't mind the extensive quoting I have given it here, but given all of the studies thrown in our face these days to support everything from Social Security Reform to changes in worker jobs and benefits understanding the pitfalls of good and bad research programs is starting to become relevant to all of us, whether we can do the research ourselves or not.
Peter Files Staff comments follow the story.
Science Journal: Scientists research questions few would ask
Friday, May 27, 2005
By Sharon Begley, The Wall Street Journal
Not every scientist can discover the double helix, or the cellular basis of memory, or the fundamental building blocks of matter. But fear not. For those who fall short of these lofty goals, another entry in the "publications" section of the ol' c.v. is within your reach.
The proliferation of scientific journals and meetings makes it possible to publish or present papers whose conclusion inspires less "Wow! Who would have guessed?" and more "For this you got a Ph.D.?" In what follows (with thanks to colleagues who passed along their favorites), names have been withheld to protect the silly.
Want job satisfaction? A "careful choice of career is the key," researchers concluded in a paper this spring in the Journal of Economic Psychology. Choosing a career based on a well-lubricated encounter at a bar, it turns out, may not be the most promising route to career satisfaction. People who choose their jobs carefully are more likely to be satisfied with them than those who take a flying leap into the great unknown.
In April, scientists reported in the journal Alcoholism: Clinical & Experimental Research that college students tend to drink much more alcohol than they think. Or, may I suggest, than they like to think. Or than they admit to their parents. Or remember.
Want to reduce problems with medications, such as harmful side effects or drug combinations that will kill you? The solution is at hand: "Communication between primary-care physicians and patients can reduce" such problems and the chance that patients will be harmed. That is especially true if doctors encourage their patients to -- wait for it -- tell them when they experience a bad side effect, concluded a study published in the Archives of Internal Medicine in January. When patients reported an adverse effect, they were more likely to be switched to a different drug than if they never mentioned it. For this, let us be grateful.
In what its sponsors called a "landmark study," scientists found that when your fingers are numb and turning that lovely robin's-egg blue, you make more typing effors. Er, errors. "When employees get chilly," the scientists concluded, "they are not working to their full potential." Achoo!
Investigators working on that finger-in-the-chili case at Wendy's may find inspiration in a study published online in March in the Annals of Emergency Medicine. Every year some 28,000 kids and adults wind up in hospital emergency rooms because some mishap has cut off a finger; one high-risk group is men over 55. Apart from digits lost in workplace accidents, the most common cause of finger amputation in the men is -- drumroll, please -- power tools. So anyone looking suspiciously at, oh, sinks or toasters for their finger-gobbling potential can more profitably focus on chainsaws.
Taking nothing, especially not their readers' intelligence, for granted, the researchers advise men who use power tools to "avoid exposing their fingers to direct contact" with razor-sharp blades spinning at a few thousand rpm. Wise advice, to be sure, although you've got to think that anyone who didn't know this is in for more serious problems than a lost finger.
Just in case you were wondering whether it's a good idea to suck up carcinogens and respiratory poisons when your airways are already crippled, scientific proof is at hand. A study found that asthma worsens the effects of smoking, putting puffers at greater risk for the kinds of lung problems that smoking causes than people without asthma. If you do not have asthma, your airways are in somewhat better shape to withstand a toxic assault. Bottom line: Doctors should urge asthmatics to quit smoking.
Far be it from me to belittle research on forensic science, since I have written about the importance of questioning such conventional wisdom as the reliability of fingerprint evidence and the credibility of confessions. But surely we can do better than a February study in the journal Psychonomic Bulletin & Review that concluded that it's easier to identify someone close to you than someone more than a football-field-length away. At 450 feet, the scientist concludes, "the human visual system starts to lose small details."
If you had found yourself in the nation's capital earlier this month, you might have heard researchers at an American Heart Association conference proclaim that if you work full time and watch television, play videogames or surf the Internet in your off hours, then you are probably not engaging in as much heart-healthy physical activity as full-timers who spend no time with TV, videogames and the computer.
Full-time workers who spend more of their down time in front of a screen also get significantly less exercise than part-time workers who spend the same number of hours glued to one screen or another, but do other things with the rest of their time. (Memo to self: Working full-time eats up ... time.) While the finding fails the "tell us something we didn't know" test, at least it does so with statistical significance: It was based on data from 4,500 people.
My thoughts:
This is the stuff that makes normal people - and I exclude myself in that - want to wring their heads with frustration, then wring their congressman's necks because their air is dirty, their schools are underfunded and the bus and train fare is going up while service is being cut, again, while all these studies seem to be proving the obvious.
Yet, there is a reason for all these studies, it's just a bad one.
Its called Peter's Principle of Political Paralysis without Significance Testing which says "No Politician Shall Approve any Remotely Risky Course of Action Without a Scientifically Conducted Study with Statistically Significant Results to Back Up His Reason for Embarking on Any Course Whatsoever".
That has lately been followed by Peter's Principle of Political Pricing without Model Calibration which says 1: "No Politician in his right mind will levy or change a tax, fee, fare, toll, price, ticket or contribution without having a consultant model the economic and political impact of such a fee on revenue, usage and political fallout of the fee considered." and 2: "Any outcome that cannot be modeled is too risky and radical to try, even if it can't be modeled only because the consultant was too stupid or pig-headed to ask the one or two questions in the input survey that would have allowed it to be modeled in the first place."
These principles illuminate why so many stupid studies are done. Because somewhere a politico was afraid of making a decision so basic that even a looney could have made it properly, but to cover his tail, her or she, hired a consultant or university consulting group to do an outside study to remove the appearance of bias from the completely obvious decision.
The consultant of course needs to get new business and wants to publicize the government client and so writes a paper or press release and viola, the next thing you know we have a story "Consultants at XYZ Research Group working on a grant from the Buttontown, PA mayor's office have learned that men as a whole feel that while buttons are very nice, thank you, despite the possibility of a fly by night injury, ZIPPERS ARE HERE TO STAY!!! n=1,200 z score=painless.
In other words, these kinds of studies have become the politician's, as Hitchcock would have put it, McGuffin, that thing that everybody wants, that you don't care what it is, but it moves the plot along until you get to the end of the movie - or in this case - until the politician gets re-elected.
As a result, sometimes all kinds of studies are done as stepping stones to prove some lynch-pin in an idiotic campaign. Or to prevent a decision being the lynching pin in someone else's campaign:
"BUTTON TOWN MAYOR DOWN ON BUTTONS - REFUSED TO SIGN NO FLY ZONE ORDER FOR BUTTON TOWN", could be a catastrophic negative campaign ad, but, with study in hand, Zippy D. Doodah, the current Mayor of Button Town can now show with a scientifically randomized study, that the citizens of Button Town would have no more traded in their pants zippers for buttons, than they would have traded in their children for a new entertainment center (though he decided not to point out the disturbing result that 19% of the population was ready to make the trade if they were allowed visiting rights.)
The results of an actual study cited in the article that says humans start to lose visual acuity at 450 feet could be recast as part of an environmentalist campaign "New study says Local Vision skills rapidly dropping off due to area toxins." The fact that due to area toxins has been added, is for emphasis a campaign manager would be only to happy to assert. To draw attention to this important vision loss issue. Hmmph.
Then of course there are the research factories, aka our institutions of higher learning.
In the course of teaching and learning, they need to keep researching and publishing and their grad students come up with the ideas that keep the money coming in, and well, to be charitable, I am sure they are all very nice graduate students, but, c'mon, a few, at least some of them must be a tad slow right? A few legacies?
That's where you get, mixed in with thousands if incredibly valuable studies and research projects, topics like the relative satisfaction about careers picked carefully vs while sitting in bars - its a scam to let the grad student spend hundreds of hours sitting in bars interviewing people while boozing off on the grant.
Now that's job satisfaction.
And if the random selection of bars was weighted to favor the area strip clubs, so much the better! For the student. Not that that could ever happen of course - after all what faculty advisor would ever sign off on such a sampling plan, especially if he had to do periodic supervision checks.... Oh, never mind.
Of course what do we know about these studies anyway. n=4,500?
Please! Half the time they've mathematically whittled the sample size down to a bag of skittles through clustered caramel cream egg sampling stratopheric randomistic chaotic string theory based on the pseudo random number generator in a Lotus Spread sheet, a hunk of yellow cheese and a randy hampster named Frederick glued (temporarily) to the F9 (recalc) key so that the iterate random function across the spreadsheet proceeds in hyper somnambulistic quadratic endoplastic formats. Really, don't use a spreadsheet as the sole basis of your random number generator for sample selection. Really. Not if it is important. Really.I mean it. Don't say I didn't warn you. Spread sheet random numbers are often based on a seed, based on what time it is, so the aren't really random are they? That bit about the hampster is a little less funny now isn't it? Don't believe me, populate a spreadsheet full of cells based on the RND function and a sample universe of an even number so it is easy to track. Or an odd, but do it even first to make it easy. Ok. now hit recalc. All those cells repopulating from top left to bottom right. Based on a time seed, the random number formula and your processor speed. Now, look hard. See any patterns. After a while you should. Shake your faith in spreadsheet random number generator??? Thought so. Maybe the newer ones have a way around this.
In fairness, some researchers prefer to draw their samples using a random number table, some M & M's and an egg carton, but we call them conservative fuddy-duddies, or chocoholics, or brilliant, free-thinking pioneers of research depending on how much sharesies they do with the M&M's.
If after all this, the results do not match the anticipated results, selected portions of the sample can be pseudo-randomly thrown out and the preferred comments can be assembled into a pretty binder.
In that case the study is relabeled from quantitative to qualitative and warnings are plainly indicated in tiny print which are readily ignored that results *may* not reflect the population under study because of the known possibility - in a qualitative study - particularly this qualitative study, but why split hairs - of variances between the study population and the population of the universe as a whole - with particular attention to Gamma Hydra IV - they are quite tricky over there. And they don't have much hair to split either.
So, thank you, Sharon, for bringing to light these serious abuses of the industry, I wish I could dig up my copy of the "WMD's sounds good to me Focus Group Results" that I found lying around a dentists' office in Irving, Texas a few years ago. What a penetrating analysis.
Basically it said that the American Public was ready to believe anything that someone with a big smile, a nice suite, and who had a big political office said to them as long as they repeated it with fingers crossed and looked very sincere.
This research brought to you be the Crackpot Staff of The Peter Files
Not to be confused with high quality research done using professional methods in any professional capacity by any persons named Peter known to the operators of this blog.
The Peter Files Blog of Satire and Fun
Peter
Who Reminds You "That if you rely on a sample size of 1 you have only yourself to blame. You need n=2 to blame someone else, n=3 to confuse the matter, n=4 to improve your chances of a getaway, and in most cases, n=>1,200 to get away with it." ===> We love comments about our posts good or bad!
Peter Files Staff comments follow the story.
Science Journal: Scientists research questions few would ask
Friday, May 27, 2005
By Sharon Begley, The Wall Street Journal
Not every scientist can discover the double helix, or the cellular basis of memory, or the fundamental building blocks of matter. But fear not. For those who fall short of these lofty goals, another entry in the "publications" section of the ol' c.v. is within your reach.
The proliferation of scientific journals and meetings makes it possible to publish or present papers whose conclusion inspires less "Wow! Who would have guessed?" and more "For this you got a Ph.D.?" In what follows (with thanks to colleagues who passed along their favorites), names have been withheld to protect the silly.
Want job satisfaction? A "careful choice of career is the key," researchers concluded in a paper this spring in the Journal of Economic Psychology. Choosing a career based on a well-lubricated encounter at a bar, it turns out, may not be the most promising route to career satisfaction. People who choose their jobs carefully are more likely to be satisfied with them than those who take a flying leap into the great unknown.
In April, scientists reported in the journal Alcoholism: Clinical & Experimental Research that college students tend to drink much more alcohol than they think. Or, may I suggest, than they like to think. Or than they admit to their parents. Or remember.
Want to reduce problems with medications, such as harmful side effects or drug combinations that will kill you? The solution is at hand: "Communication between primary-care physicians and patients can reduce" such problems and the chance that patients will be harmed. That is especially true if doctors encourage their patients to -- wait for it -- tell them when they experience a bad side effect, concluded a study published in the Archives of Internal Medicine in January. When patients reported an adverse effect, they were more likely to be switched to a different drug than if they never mentioned it. For this, let us be grateful.
In what its sponsors called a "landmark study," scientists found that when your fingers are numb and turning that lovely robin's-egg blue, you make more typing effors. Er, errors. "When employees get chilly," the scientists concluded, "they are not working to their full potential." Achoo!
Investigators working on that finger-in-the-chili case at Wendy's may find inspiration in a study published online in March in the Annals of Emergency Medicine. Every year some 28,000 kids and adults wind up in hospital emergency rooms because some mishap has cut off a finger; one high-risk group is men over 55. Apart from digits lost in workplace accidents, the most common cause of finger amputation in the men is -- drumroll, please -- power tools. So anyone looking suspiciously at, oh, sinks or toasters for their finger-gobbling potential can more profitably focus on chainsaws.
Taking nothing, especially not their readers' intelligence, for granted, the researchers advise men who use power tools to "avoid exposing their fingers to direct contact" with razor-sharp blades spinning at a few thousand rpm. Wise advice, to be sure, although you've got to think that anyone who didn't know this is in for more serious problems than a lost finger.
Just in case you were wondering whether it's a good idea to suck up carcinogens and respiratory poisons when your airways are already crippled, scientific proof is at hand. A study found that asthma worsens the effects of smoking, putting puffers at greater risk for the kinds of lung problems that smoking causes than people without asthma. If you do not have asthma, your airways are in somewhat better shape to withstand a toxic assault. Bottom line: Doctors should urge asthmatics to quit smoking.
Far be it from me to belittle research on forensic science, since I have written about the importance of questioning such conventional wisdom as the reliability of fingerprint evidence and the credibility of confessions. But surely we can do better than a February study in the journal Psychonomic Bulletin & Review that concluded that it's easier to identify someone close to you than someone more than a football-field-length away. At 450 feet, the scientist concludes, "the human visual system starts to lose small details."
If you had found yourself in the nation's capital earlier this month, you might have heard researchers at an American Heart Association conference proclaim that if you work full time and watch television, play videogames or surf the Internet in your off hours, then you are probably not engaging in as much heart-healthy physical activity as full-timers who spend no time with TV, videogames and the computer.
Full-time workers who spend more of their down time in front of a screen also get significantly less exercise than part-time workers who spend the same number of hours glued to one screen or another, but do other things with the rest of their time. (Memo to self: Working full-time eats up ... time.) While the finding fails the "tell us something we didn't know" test, at least it does so with statistical significance: It was based on data from 4,500 people.
My thoughts:
This is the stuff that makes normal people - and I exclude myself in that - want to wring their heads with frustration, then wring their congressman's necks because their air is dirty, their schools are underfunded and the bus and train fare is going up while service is being cut, again, while all these studies seem to be proving the obvious.
Yet, there is a reason for all these studies, it's just a bad one.
Its called Peter's Principle of Political Paralysis without Significance Testing which says "No Politician Shall Approve any Remotely Risky Course of Action Without a Scientifically Conducted Study with Statistically Significant Results to Back Up His Reason for Embarking on Any Course Whatsoever".
That has lately been followed by Peter's Principle of Political Pricing without Model Calibration which says 1: "No Politician in his right mind will levy or change a tax, fee, fare, toll, price, ticket or contribution without having a consultant model the economic and political impact of such a fee on revenue, usage and political fallout of the fee considered." and 2: "Any outcome that cannot be modeled is too risky and radical to try, even if it can't be modeled only because the consultant was too stupid or pig-headed to ask the one or two questions in the input survey that would have allowed it to be modeled in the first place."
These principles illuminate why so many stupid studies are done. Because somewhere a politico was afraid of making a decision so basic that even a looney could have made it properly, but to cover his tail, her or she, hired a consultant or university consulting group to do an outside study to remove the appearance of bias from the completely obvious decision.
The consultant of course needs to get new business and wants to publicize the government client and so writes a paper or press release and viola, the next thing you know we have a story "Consultants at XYZ Research Group working on a grant from the Buttontown, PA mayor's office have learned that men as a whole feel that while buttons are very nice, thank you, despite the possibility of a fly by night injury, ZIPPERS ARE HERE TO STAY!!! n=1,200 z score=painless.
In other words, these kinds of studies have become the politician's, as Hitchcock would have put it, McGuffin, that thing that everybody wants, that you don't care what it is, but it moves the plot along until you get to the end of the movie - or in this case - until the politician gets re-elected.
As a result, sometimes all kinds of studies are done as stepping stones to prove some lynch-pin in an idiotic campaign. Or to prevent a decision being the lynching pin in someone else's campaign:
"BUTTON TOWN MAYOR DOWN ON BUTTONS - REFUSED TO SIGN NO FLY ZONE ORDER FOR BUTTON TOWN", could be a catastrophic negative campaign ad, but, with study in hand, Zippy D. Doodah, the current Mayor of Button Town can now show with a scientifically randomized study, that the citizens of Button Town would have no more traded in their pants zippers for buttons, than they would have traded in their children for a new entertainment center (though he decided not to point out the disturbing result that 19% of the population was ready to make the trade if they were allowed visiting rights.)
The results of an actual study cited in the article that says humans start to lose visual acuity at 450 feet could be recast as part of an environmentalist campaign "New study says Local Vision skills rapidly dropping off due to area toxins." The fact that due to area toxins has been added, is for emphasis a campaign manager would be only to happy to assert. To draw attention to this important vision loss issue. Hmmph.
Then of course there are the research factories, aka our institutions of higher learning.
In the course of teaching and learning, they need to keep researching and publishing and their grad students come up with the ideas that keep the money coming in, and well, to be charitable, I am sure they are all very nice graduate students, but, c'mon, a few, at least some of them must be a tad slow right? A few legacies?
That's where you get, mixed in with thousands if incredibly valuable studies and research projects, topics like the relative satisfaction about careers picked carefully vs while sitting in bars - its a scam to let the grad student spend hundreds of hours sitting in bars interviewing people while boozing off on the grant.
Now that's job satisfaction.
And if the random selection of bars was weighted to favor the area strip clubs, so much the better! For the student. Not that that could ever happen of course - after all what faculty advisor would ever sign off on such a sampling plan, especially if he had to do periodic supervision checks.... Oh, never mind.
Of course what do we know about these studies anyway. n=4,500?
Please! Half the time they've mathematically whittled the sample size down to a bag of skittles through clustered caramel cream egg sampling stratopheric randomistic chaotic string theory based on the pseudo random number generator in a Lotus Spread sheet, a hunk of yellow cheese and a randy hampster named Frederick glued (temporarily) to the F9 (recalc) key so that the iterate random function across the spreadsheet proceeds in hyper somnambulistic quadratic endoplastic formats. Really, don't use a spreadsheet as the sole basis of your random number generator for sample selection. Really. Not if it is important. Really.I mean it. Don't say I didn't warn you. Spread sheet random numbers are often based on a seed, based on what time it is, so the aren't really random are they? That bit about the hampster is a little less funny now isn't it? Don't believe me, populate a spreadsheet full of cells based on the RND function and a sample universe of an even number so it is easy to track. Or an odd, but do it even first to make it easy. Ok. now hit recalc. All those cells repopulating from top left to bottom right. Based on a time seed, the random number formula and your processor speed. Now, look hard. See any patterns. After a while you should. Shake your faith in spreadsheet random number generator??? Thought so. Maybe the newer ones have a way around this.
In fairness, some researchers prefer to draw their samples using a random number table, some M & M's and an egg carton, but we call them conservative fuddy-duddies, or chocoholics, or brilliant, free-thinking pioneers of research depending on how much sharesies they do with the M&M's.
If after all this, the results do not match the anticipated results, selected portions of the sample can be pseudo-randomly thrown out and the preferred comments can be assembled into a pretty binder.
In that case the study is relabeled from quantitative to qualitative and warnings are plainly indicated in tiny print which are readily ignored that results *may* not reflect the population under study because of the known possibility - in a qualitative study - particularly this qualitative study, but why split hairs - of variances between the study population and the population of the universe as a whole - with particular attention to Gamma Hydra IV - they are quite tricky over there. And they don't have much hair to split either.
So, thank you, Sharon, for bringing to light these serious abuses of the industry, I wish I could dig up my copy of the "WMD's sounds good to me Focus Group Results" that I found lying around a dentists' office in Irving, Texas a few years ago. What a penetrating analysis.
Basically it said that the American Public was ready to believe anything that someone with a big smile, a nice suite, and who had a big political office said to them as long as they repeated it with fingers crossed and looked very sincere.
This research brought to you be the Crackpot Staff of The Peter Files
Not to be confused with high quality research done using professional methods in any professional capacity by any persons named Peter known to the operators of this blog.
The Peter Files Blog of Satire and Fun
Peter
Who Reminds You "That if you rely on a sample size of 1 you have only yourself to blame. You need n=2 to blame someone else, n=3 to confuse the matter, n=4 to improve your chances of a getaway, and in most cases, n=>1,200 to get away with it." ===> We love comments about our posts good or bad!
Please make comments by clicking on the word "Comments"!
Click on the Envelope to forward posts to your friends! Thank you! The staff.
posted by The Peter Files Blog of Comedy : Sunday, May 29, 2005
Home of the Best Doctor Who Podcast in Time and Space
1 Comments:
Think of the thousands of PhD students every year, who, with their faculty advisors, must come up with "original" research topics. It's been getting harder and harder to be USEFULLY original, hence an increasing tendency to come up with frivolous, trivial, obvious, or painfully esoteric topics. Think this situation will get better or worse, as time marches on ?
Post a Comment
<< Home
New Peter Files Blog Selected Comedy Videos!
Day By Day - - Copyright 2007 by Chris Muir, All Rights Reserved