Amid information madness, it’s the methods that matter

It’s getting harder to know what to believe.

That statement doesn’t come as a revelation to many reading this blog, as the constant inundation of information makes distillation into beliefs, assertions and forecasts extremely challenging. Just as you’re trying to wrap your head around a set of data, another comes out that contradicts it or evolves it in ways that cause you to scrap or alter what you had been considering.

Take for example the much-discussed malaria drug hydroxychloroquine, which has been touted by US President Donald Trump as a therapeutic drug for Covid-19. Some medical professionals have cited studies saying at the least it doesn’t show significant effectiveness against coronavirus and at most could kill the person taking it.

On 3 June, The New England Journal of Medicine published a study that found that the drug performed no better than a placebo in preventing Covid-19. A day later, another prestigious medical journal, The Lancet, retracted a study that had found that coronavirus patients who took hydroxychloroquine died at a higher rate and had increased heart problems than those who did not take the drug.

Following investigative reporting by The Guardian newspaper into the company that did the study, Surgisphere, The Lancet cited “several concerns were raised with respect to the veracity of the data and analyses conducted by Surgisphere Corporation and its founder and our co-author, Sapan Desai, in our publication.” As a result, The New England Journal of Medicine has retracted a 1 May study on Covid-19 that had Desai as a co-author and was based on Surgisphere data.

So why bring this up here? Is this an announcement that ICIS is launching hydroxychloroquine price assessments? We’re not (I think), so how is this all relevant to what we do at ICIS?

In reading the original outstanding Guardian reporting, one word leapt out at me, and it’s in this paragraph:

“A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology.”

Give yourself a gold star if you guessed “methodology” as the word, although I can understand if you tripped up on the revelations that the employee roll count at Surgisphere included a science fiction writer and an adult-content model.

In compilation of valuable data – and we can all agree that gathering data that could help end this pandemic is quite valuable – how you gather the data points is crucial, and transparent methodologies that the detail the approaches and preparation that go into data gathering take on critical importance.

On the many client and prospect calls I have taken part in during the pandemic, invariably questions surrounding our methodologies come up, usually in the form of:

• So how do you go about creating a price for {fill in the blank} market?
• What parts of the market do you talk to?
• Are your prices just for suppliers? Are they just for buyers? Who is the audience for your prices?
• What is the typical cargo size that you assess for?
• Do you have a hierarchy of data inputs regarding what forms the basis of your assessments? Does anything or any side of the market take precedence?

We ICIS representatives are in a strong position to answer each of those questions by referring to the open methodologies contained on ICIS.com. We want our customers and soon-to-be customers to know how we conduct our business regarding price assessments. We want those contacts to know that we rotate those methodologies through markets for consultation with market participants so that they can suggest changes, and those suggestions have led to new assessments and alterations in our methodology (and we note those as well on our website). We want those contacts to know and show on our website that we put our assessments through the rigors of outside auditing (our Head of Market Reporting, Barbara Ortner, does a wonderful job of explaining the value of that here). Our transparency in those efforts build confidence with contacts and form the baseline for a trusted partnership with them.

In short, our transparent methodologies separate us from the Surgispheres of the world.

Maybe it’s not so much getting harder to know what to believe as it is harder to know what to trust. Companies managing and trading the world’s precious resources increasingly are showing they trust ICIS due to the standards we set for ourselves.