Modern Science Needs Its Version of Sarbanes-Oxley
By Repair Man Jack Posted in Policy — Comments (37) / Email this page » / Leave a comment »
In the late 1990s, Enron Corporation was seen as a leading progressive among business corporations and a refreshing antidote to the depredations of Big Oil™. In 1997, President Bill Clinton had Enron CEO Ken Lay over to the White House to discuss whether or not he should sign the Kyoto Agreement. But even as early as the late 1990s, Enron had only Enron’s interests in mind, and internal corporate documents described the Kyoto Treaty as follows.
"precisely what [Enron has] been lobbying for": "This agreement will be good for Enron stock!!"
Enron fooled and lied to far more people than just Bill Clinton and Al Gore. The level of fraud and deceit they practiced eventually led to their bankruptcy in 2001. Enron tried to protest their innocence. They claimed that the accounting firm of Arthur Anderson rigorously audited all of the corporate books. This held great credibility in many people’s minds. The founders of The Arthur Anderson Firm were famous for demanding the highest standards of accuracy and honesty. Arthur Anderson held great standing in the accounting industry – prior to their collusion with Enron to help lie about Enron’s financial health on mandatory SEC filings.
Eventually, the whole edifice of deceit came crashing down. Enron’s thieves, churls and wag-halters were no longer considered the smartest guys in the room. Their bankruptcy proceedings can be accessed on line by the morbidly curious. Arthur Anderson was destroyed as well, and for a time, no one believed that American Corporations were receiving properly rigorous audits.
A controversial, but necessary, act of legislation was passed through Congress to address the issue. The Sarbanes-Oxley Act of 2002 established the Public Company Accounting Oversight Board to backstop the accounting firms auditing public companies. The law has caused the negative externality of making the compliance activities of corporations more bothersome and expensive, but it has also prevented any major repetitions of the dishonesty displayed by Enron, Global Crossings or WorldCom.
The realm of government funded scientific research has developed a problem with the truth that mirrors the attitudes displayed by Enron in two ways. First, scientists have famously distorted data to make their results more impressive. Secondly, they are self-regulating, through the auspices of peer review, and face a moral hazard problem similar to the problem faced by Arthur Anderson when Enron made it very lucrative for the auditors to ignore existing facts and compose a few facts of their own.
In the case of the infamous Utah University cold fusion experiments, peer-review worked well. Dr. B. Stanley Pons and Dr. Martin Fleischmann made the claim that they had achieved nuclear fusion, in a jar of water, standing at room temperature. The American and European physics community attempted to replicate this in every way imaginable. Upon failing, they challenged the assertion, and debunked the claim. This yeoman’s work saved the United States taxpayers $25 Mil that would otherwise have been appropriated by Congress to continue their research.
In the field of Climate Science, the peer review process has not produced similar success. In 2001, The United Nations IPCC released a study on the state of the Earth’s climate. The study claimed with a high degree of certainty that the Earth was warming at a rate that was significantly greater than the previous historical trends. This study was bolstered by a paper written by Climatologist Michael Mann. This paper contained a graph that was described as “The Hockey Stick Curve.” Relying heavily on Dr. Mann’s work, the IPCC made the public pronouncement “that the 1990s has been the warmest decade and 1998 the warmest year of the millennium”
Mann’s work underwent peer review by eight different climate scientists. The IPCC stood by it as genuine and honest work. However, the veracity of the so-called “Hockey Stick Curve” came under assault from a number of people.
David Deming preceded Dr. Mann as a proponent of using physical proxies as a methodology to pursue estimates of previous climate history. He authored a study that tracked the most recent 150 years of North American climate that attracted the attention of the climate modeling community. He then received the following in an email.
A major person working in the area of climate change and global warming sent me an astonishing email that said “We have to get rid of the Medieval Warm Period.”
In other words, this was a leading climate modeler claiming that the data, as it stood, had to be altered substantially to make the case that global warming was actually happening the way its believers wanted to tell the story. This was raw advocacy, not science.
This interesting perspective on the truth was not restricted to David Deming’s anonymous emailer. Nature magazine describes the attitude of Stanford University Climatologist Stephen Schneider.
In a 1989 Discover magazine article, Schneider discussed the dilemma facing scientists who wanted to draw attention to climate change while remaining true to current scientific knowledge of the subject. “We need to capture the public’s imagination, ”he noted. “That entails getting loads of media coverage, so we have to offer up scary scenarios, make simplified and dramatic statements and make little mention of any doubts we might have.”
This works well in sales and public relations, but doesn’t work so well in the pursuit of actual truth. Mann was severely debunked. It was discovered that feeding randomly generated data sets into Mann’s algorithms would produce the same hockey stick curve that Mann got with his Bristlecone Pine data. Primary debunkers McKitrick and McIntyre describe the mathematical process Mann used to microwave his data.
A conventional PC algorithm centers the data by subtracting the column means of the underlying series. For the AD1400 step highlighted here, this would be the full 1400-1980 interval. Instead, MBH98 Fortran code (ftp://holocene.evsc.edu/pub/MBH98/TREE/ITRDB/NOAMER/pca-noamer) contains an
unusual data transformation prior to PC calculation that has never been reported in print.
Each 4 tree ring series was transformed by subtracting the 1902-1980 mean, then dividing by the 1902-1980 standard deviation and dividing again by the standard deviation of the residuals from fitting a linear trend in the 1902-1980 period. The PCs were then computed using singular value decomposition on the transformed data. (The effects reported here would have been partly mitigated if PCs had been calculated using the covariance or correlation matrix.)
Quite literally, when it became time to fit the temperature curve, 502 years of data got made to disappear. Amazingly, eight different reviewers, all of whom have earned PhDs in mathematically dense subject matters, failed to note the modification of the traditionally accepted axis transformation in Mann’s algorithmic methodology. Even if Mann did something benign and acceptable under the norms of statistical practice, he still should have been required to vouch for its methodological validity and demonstrate its absence of bias.
In the real world inhabited by people not geeky enough to care about esoteric uses of linear algebra to calculate convoluted statistical models, readers scratch their heads and ask why care? So what if a bunch of science wonks blatantly cheat? It won’t cost me any money, will it?
Actually, it just might begin to. Mississippi attorney Gerald Maples has recently filed a class-action lawsuit against twenty-six oil companies. He claims that their CO2 emissions caused anthropogenic global warming, which in turn caused Hurricane Katrina to tear up lots of homes in Southern Mississippi. Maples explains the gravaman of his case.
Maples says he has scientific proof that's been ignored by Congress. The attorney will argue in court that oil companies and their impact on global warming are partly to blame for the excessive pounding south Mississippi absorbed during Katrina.
I, for one, believe Congress needs to stop ignoring all of this brilliant science. They need to exercise oversight on what gets published, because if Michael Mann can redefine what a Principal Components Analysis is to suit his ideological agenda, and then have his work indirectly used to support an utter junk law suit, science needs to be much more carefully regulated.
Dr. Mann would defend himself by accurately stating he has no association with the oleaginous pursuer of emergency medical conveyances in question. He would state further that none of his work has involved any directed statement of moral culpability of any particular sector of the current US economy. In other words, he’d say associating his work with this lawsuit was bogus.
Yet, had Dr. Mann’s work been properly vetted prior to it’s inclusion in IPCC III, Maples would perhaps be peeing the wrong way in a wind tunnel with his lawsuit. Had Dr. Mann and others been in the actual business of rigorous science, rather than in the disingenuous pursuit of “offer(ing) up scary scenarios, make simplified and dramatic statements and make little mention of any doubts we might have”, Maples would possibly have to look elsewhere for a set of deep pockets to fleece on behalf of his clientele.
Anthropogenic Global Warming may really be occurring as we speak. The science on climate change stands far from settled and still provokes vigorous argument and debate. That debate needs to be refereed so that it stays measured and honorable. Like America’s public corporations, this potentially involves trillions of dollars and our future as a nation.
All science, not just climate science, needs to receive more rigorous oversight before it receives one dime of taxpayer funding or is allowed to be published, in the public domain, with any type of official seal of approval. American scientific research needs the equivalent of a Sarbanes-Oxley. The stakes on this are too high to just let our scientists go around believing that they owe no accountability to anyone because they are the smartest guys in the room.
I realize it's poor netiquette to correct ordinary spelling errors, but with proper names I think it's a matter of courtesy to get them right.
And in this case it's the least we can do for a very fine firm that made mistakes in their Houston office, but unjustly got the death penalty from the government. (Enron got everything they deserved, of course, but the market took care of killing them.)
Cold fusion is a great example of peer review and the scientific principle of reproducibility. The problem with debunking AGW is the physical and time scale. It is relatively easy and inexpensive to try to reproduce a bench-top experiment. A global scale and decades time scale create too much room for scientific shenanigans.
Excellent article. Well documented. Science has been acting lately like the 16th century Church. You challenge their assumptions and they want to throw you in jail. How sad.
What is needed for better science is more government intervention...
Wait, what blog am I reading?
-jb
can supposedly read Michael Mann's Fortran Code independently and replicate his work and all reach a consensus that that PCA wasn't rigged, someone without moral hazard has to intervene. Otherwise, science will steadily lose its validity.
James Hansen - Scott THomas Beauchamp with a PhD.
The real question, can this group of eight credentialed professionals reach a consensus on the conclusion, that the last couple of decades are the hottest in a millenium. Well, the answer is here, and is in essence yes, given the limitations on data accuracy.
If A (Mann's particular methods correct) Then B (warmest decade).
Not A (it's generally agreed his methods were improper/incorrect).
That does not equate to Not B. Rail against Mann if you want, but don't forget what the actual point is.
-jb
involved the fact that eight specific peer reviews of Mann failed to report a methodological error that *anybody* who has ever done a PCA on a large data set would have found exceptional. That puts more than just global warming science on trial. If Exxon Mobil did their geology this poorly, they would never have struck oil. When something gets that heavily peer-reviewed and still includes mathematical errors on the undergraduate level of complexity, the peer review process is busted. That has profound implications beyond climatology.
James Hansen - Scott THomas Beauchamp with a PhD.
Let's just extend your argument to where you want to take it, science has been shown invalid. Scrap anything it has explained, scrap all derivative technology, scrap all science teaching and funding. You've done it, you've shown it to be completely lacking, we were all fooled for so long.
Let's replace it with government reviewed and restricted research, preferably with some sort of fairness doctrine so global warming skeptics won't be drowned out by any sort of evidentiary imbalance.
-jb
Even if such work is deemed necessary, is the government the best place to get redress? Won't any rebukes be questioned based on the perceived political motivations of whomever creates the oversight group?
SOX is an interesting case and point, but I don't think it really answers this question.
--
We would also like to know your advice for somebody like my daughter, who's going to graduate in two years, advice that you would give a young person.
SEC. RUMSFELD: Advice for a young person. Study history.
eight specific peer reviews of Mann failed to report a methodological error that *anybody* who has ever done a PCA on a large data set would have found exceptional
I'm amazed at how many people there are who seem to have done a PCA on a large data set, and to whom the misconduct of Mann is so obvious. Actually, I have done PCA's, including a large one on climate data, and agree that you should normalize using the whole data range. Normally, if you used a subset, it wouldn't matter too much unless the subset were atypical. That leads to a paradox in this overheated criticism. It is said that Mann's error in normalising using the 1900's was grievous, because that was an atypical hot period, and falsely showed that it was ... a hot period.
If I recall correctly one of the major complaints about the "Hockey stick graph" is that Mann et al. would not release the source code or data that supported the conclusion. In other words, the paper passed peer review with no supporting evidence. This speaks volumes about the quality of "Climate Science".
for scientific journals never examine source code, and would very rarely scrutinise data not provided in the manuscript. It simply isn't practical to do so, in any field of science. It also bypasses the point of the review, which is to ensure that the text of the paper, accessible to all readers, adequately makes the scientific case.
Now that climate science is being used to adjudicate $mil class action law suits and make decisions on major policy portfolios, "Scout's Honor" will no longer cut it in the laboratory, any more than it cuts it on Wall Street.
James Hansen - Scott THomas Beauchamp with a PhD.
since the "peers" all undoubtedly share the same political leanings as the authors...which, to me, is the crux of the problem. "Peer review" assumes that the reviewers are considering the scientific validity of the paper, but in the case of AGW, ideology trumps scientific accuracy in the eyes of the activist scientists who are churning this stuff out.
...when they see me they'll say, "There goes Loren Wallace,
the greatest thing to ever climb into a race car."
That no-one should be allowed to publish any scientific proposition that hasn't been approved by a bank of government auditors? On pain of...?
I had heard that as well, but had also heard that acusation questioned. Plus, the link to the code is right out on the internet. I will grant you that it may be possible that it got out there in response to the criticisms you mentioned...
James Hansen - Scott THomas Beauchamp with a PhD.
From Wikipedia:
One point of contention relates to McIntyre's requests for Mann to provide him with the data, methods and source code McIntyre needed to "audit" MBH98.[19] Mann provided some data and then stopped. After a long process - in which the National Science Foundation supported Mann - the code was made publicly available [20]. It happened because Congress investigated after an article in the Wall Street Journal [21] detailed criticisms raised by McIntyre.[22] Congress was especially concerned about Mann’s reported refusal to provide data. In June 2005, Congress asked Mann to testify before a special subcommittee. The chairman of the committee (Joe Barton, a prominent global warming skeptic) wrote a letter to Mann requesting he provide his data, including his source code, archives of all data for all of Mann's scientific publications, identities of his present and past scientific collaborators, and details of all funding for any of Mann's ongoing or prior research, including all of the supporting forms and agreements.
Then it will be payback time for all of the companies that have caved in to the AGW hype and hysteria.
I don't know much about scientific research. I also don't know much about SOX. I do however know plenty about government interference, and as you can see by my quote, I don't like it. There are all sorts of standards that the government sets up in mortgages as well. The government has interfered me right out of many loans and I have documented some examples on these pages. So, can you address the concern that someone like myself has with even more government regulation into yet another field?
"The nine most dangerous words in the English language are 'I'm from the government and I'm here to help'"
Ronald Reagan
Science, as you have pointed out, does have a self-correcting mechanism. Others can try to reproduce the results, and note whether it works. This applies to Mann's work as much as any. Independent scientists have done similar temperature reconstructions, no doubt aware of the criticism of Mann's minor errors with normalising, and got similar results. Here, from this site , is a collection of ten analyses of proxy data for the millenium. Michael Mann was involved in three of them; one is the set referred to here. See if you can spot it - the answers are in the link.

Now tell us, how many of those incorporate the same flawed ground temperature readings, which are inappropriate for measuring climactic changes because they are influenced by urban development round them over the years?
This image is a comparison of 10 different published reconstructions of mean temperature changes during the last 1000 years. More recent reconstructions are plotted towards the front and in redder colors, older reconstructions appear towards the back and in bluer colors. An instrumental history of temperature is also shown in black.
Noone has 1000 years of instrumental data.
Sarbanes-Oxley was a terrible law that has stifled investment in our country. It ought to be repealed not repeated, heh.
All science, not just climate science, needs to receive more rigorous oversight before it receives one dime of taxpayer funding or is allowed to be published, in the public domain, with any type of official seal of approval.
Any grant application submitted to NSF, NIH, DARPA, etc. receives an extremely rigorous review by both government employees and academic peers.
The author of this post is suggesting that written material receive government approval prior to publication in a non-government publication. This seems to me like a serious infringement of the first amendment.
By the way, I read many scientific journals, and I have never seen an article with an "official seal of approval."
Although my specialty is not, under any circumstances, climate. I would like to point out a few things.
1. From my understanding, the tendency to discount the importance of the medieval warming period is that it was localized over the north atlantic, and was not a global phenomenon. This in contrast to the last century, which has seen global mean temperatures rising.
2. Scientists are not in any of this for the money (there is none). Some like the teaching, some have a genuine interest in expanding knowledge, and others are in it for the ego boost of being the first to discover something or being right.
In this last case, you'll find the explanation for the tendency not to share data sets and source code. Teams work very hard and expend tremendous resources, drilling ice cores, examining tree rings, etc. to reconstruct proxy data sets. Once the data set is constructed, the analysis begins, and the papers start getting pumped out. Giving away the raw data is like inventing something and not getting the patent on it. Of course, the data should eventually be made publicly available, but give the researchers some time to stake their claim.
Quick notes: your typical university research isn't in it for personal wealth, but he lives and dies on grant money, so there is still an incentive to produce results that will bring in the most grant money.
Starting off your post with an appeal to authority fallacy is not a good sign.
You handwave that the medieval warm period was in the atlantic, but what have you to say about the little ice age? Was that also localized? If so, what evidence is there? The proxies commonly used for temperature don't even show the medieval warm period and the little ice age, do we, so how would we know how localized they were?
Giving away raw data is the only flipping way we have to know whether a given modeler is making everything up, because this isn't like cold fusion where any halfway decent lab can reproduce the conditions of the test and gather his own data.
OK, so much for some quick notes. I'll wrap it up with some They Might Be Giants lyrics
And now that you've tried it, you're back to report
That the spiralling shape was a fraud and a fake
You didn't enjoy it, you never believed it
There won't be a refund, you'll never go back
The spiralling shape will make you go insane
Well, if you're going to suggest we should "follow the money" and consider university/public institution researchers' motives regarding grants, just make sure you also remember those six figure salaries that oil company scientists critical of global warming are making the next time you cite one of them.
Why do you assume only people critical of AGW are paid by oil companies? Why do you attack the person and not the validity of the work, thats not very scientific of you.
your typical university research isn't in it for personal wealth, but he lives and dies on grant money, so there is still an incentive to produce results that will bring in the most grant money.
Research money distributed by government agencies (e.g., NSF, NIH, DARPA) to scientists not contingent upon whether the scientists obtain certain results. The money is distributed before the study. The results are published after the study is done.
Perhaps you are thinking that a scientist who has obtained "favorable" results in the past is more likely to receive grant money the next time around. Do you have any evidence that this happens? Have you ever seen a case where a researcher has been denied funding for obtaining an "unfavorable" result earlier in their career?
Congress needs to stop ignoring all of this brilliant science. They need to exercise oversight on what gets published...
What? You want congress to decide what is science and what isn't? Is this really what you want?
Ok, so now we'll have Hillary Clinton in charge of what gets published? Or Nancy Pelosi? This is really what you want? Dennis Kucinich in charge of the House Censorship Committee?
I don't know about you, but I'd rather scientists watch over themselves. Your analogy of comparing them to having Enron watching over themself is a little misguided. It's more like having the entire business world watching over you. Every other company (scientist) is watching over you, and if you screw up, you're through. Science does an excellent job of rooting out the screwups and the frauds. Just because you don't like the conclusion, doesn't mean it's not true.
This blog is a classic conspiracy theory. Everyone is evil and corrupt. Only the few true scientists see the light, all others are deluded or liars out for gain. The world has been breaking temperature records right and left for the last ten years. Lindzen was absolutely correct ten years ago when he said the evidence was not there. That isn't true now.
A changing climate is going to effect us. Instead of arguing over that, we should be arguing over what to do next (if anything). It is obvious CO2 output is going to continue to go up. China will replace any reductions the western world makes, so a carbon tax seems the wrong approach. It is going to continue to warm.
We should be planning ways to mitigate the effect of warming, on water, agriculture, flooding, disease, and energy needs. We should change flood insurance rules to discourage building (and rebuilding at federal expense) on the coast. We should change water rights to discourage waste. We should be investing in developing new technologies for energy production, solar, wind, nuclear, etc and find ways to improve energy efficiency.

Fire NASA's Hansen. If there ever was political shenanigan's in science and scientific data, this has got to be the epicenter of misleading globull warming information.