Home Services Dataexpress Icis Api API quick start guide
Home Services ICIS excel plug-in ICIS data express – API API quick start guide

API quick start guide

Jump to
Introduction
How do I …?
Use cases
contact us

Introduction

The ICIS API provides authorised users with a secure interface through which a variety of data sets can be accessed. Using the API, a user can create flexible and customised queries which return data such as reports, associated pricing series, reference data, etc.

This quick start guide contains some examples and ready-to-use recipes to help you build your first customer. Each example can be fully customised for your needs so please refer to the online API documentation to explore the full potential of the service and experiment with interactive examples.

The examples are structured as follows:

  • How to make an ICIS API call: the basics of communicating with the API.
  • How do I…: queries and concepts of general use.
  • Common use cases: tailored queries for the most common scenarios.
 

How to make an ICIS API call

The ICIS API exposes RESTful endpoints to fetch and search data in the repository. The communication format for both request and response is XML, which is exchanged with standard HTTP methods such as GET and POST. It is therefore possible to communicate with the ICIS API using any language capable of issuing HTTP requests.

In the following examples we will use the command line cURL which is installed by default on many operating systems and available for download from https://curl.haxx.se/. One of the numerous cross-platform alternatives is Postman (http://www.getpostman.com) however most modern programming languages have equivalent functions for your final implementation.

Please note that all communication is carried over a secure channel and any mention of HTTP in this document should be interpreted as HTTPS.

Your first ICIS API request: a simple GET

As a first interaction with the API, let’s fetch all the available currencies.

curl -u your_username:your_password https://api.icis.com/v1/entities/ref-data/currency

 

What makes up this cURL request?

 

Authentication

Every request to the ICIS API must contain the authentication header in compliance with the HTTP Basic authentication protocol. cURL will build the necessary header using the credentials specified with the –u switch. In the above example you will simply need to replace your_username and your_password with the authentication details.

If you do not already have an ICIS API subscription, please contact the ICIS Customer Support Centre.

URL

The only other parameter for the command is the URL to post the request to/fetch the response from: https://api.icis.com/v1/entities/ref-data/currency

Note that the simple GET request above can also be issued using a web browser and providing the service credentials when prompted.

Your first ICIS API response

Below is a snippet of the response. Please note: … snip … represents data being omitted.

<atom:feed xmlns:atom="http://www.w3.org/2005/Atom" … snip …>
  <atom:id>http://iddn.icis.com/search</atom:id>
  <atom:title>IDDN Search Results</atom:title>
  <atom:updated>2016-05-03T08:42:23.363499Z</atom:updated>
  <atom:author>ICIS Data Distribution Network (IDDN)</atom:author>
  <atom:generator>IDDN /search Endpoint</atom:generator>
  <atom:link href="/search" rel="self" type="application/atom"/>
  <s:first-result>1</s:first-result>
  <s:max-results>100</s:max-results>
  <s:selected-count>44</s:selected-count>
  <s:returned-count>44</s:returned-count>
  … snip …
  <atom:entry>
    <atom:title>http://iddn.icis.com/ref-data/currency/2</atom:title>
    <atom:id>http://iddn.icis.com/ref-data/currency/2</atom:id>
    <atom:updated>2015-09-08T15:12:20.828Z</atom:updated>
    <atom:relevance-score>60928</atom:relevance-score>
    <atom:content type="application/vnd.icis.iddn.entity+xml">
      <a:currency xmlns:r="http://iddn.icis.com/ns/refdata">
        <c:id>http://iddn.icis.com/ref-data/currency/2</c:id>
        <c:version>2</c:version>
        <c:type>ref-data</c:type>
        <c:created-on>2015-09-08T15:12:20.828Z</c:created-on>
        <c:descriptor href="http://iddn.icis.com/descriptor/currency"/>
        <c:domain href="http://iddn.icis.com/domain/general"/>
		
        <c:released-on>2015-09-08T15:13:00Z</c:released-on>
        <f:code>BRC</f:code>
        <f:name>
          <c:l10n xml:lang="en">Brazilian Cruzeiro</c:l10n>
        </f:name>
        <f:symbol>-</f:symbol>
        <f:location href="http://iddn.icis.com/ref-data/location/74"/>
        <f:active>false</f:active>
      </a:currency>
    </atom:content>
  </atom:entry>
  … snip …
</atom:feed>

What makes up this response?

Without entering into the details of this specific response, it’s easy to notice that the results are returned as an Atom feed. It is worth noting that unless otherwise specified the API will return the first page of 100 results. In this case there are only 44 results so all are returned, but some queries may return in excess of 100 results – in which case the recommended approach would be to issue subsequent requests that make use of the pagination feature to access all results. Alternatively it is possible to extend the size of the results page.

Your second ICIS API request: a simple POST

We can now exercise the /search end point by POSTing a simple search request that retrieves all the series that the user is entitled to.

curl -u your_username:your_password -d "<request xmlns='http://iddn.icis.com/ns/search'><scope><type>series</type></scope></request>" https://api.icis.com/v1/search

What makes up this cURL request?

Compared with the previous request we are now:

  • Consuming a different resource: /search rather than /entities
  • Specifying a body for the HTTP request (see the value of the -d switch, enclosed in double quotes) and using the API query language to apply a filter on the asset type. The very simple query is better visualised below:
<request xmlns='http://iddn.icis.com/ns/search'>
  <scope>
    <type>series</type>
  </scope>
</request>


Note that the data part (body) of the HTTP request has been enclosed in double quotes. Appropriate escaping of internal double quotes will depend on the platform under which cURL is run.

Alternatively, the body can be saved in a file and referenced with the @ option – please refer to the official cURL documentation.

In the reminder of this document the full cURL command line will often be omitted when no special parameters are required and it’s left to the user to build the request as needed.

Your second ICIS API response

<atom:feed xmlns:atom="http://www.w3.org/2005/Atom" … snip … >
  <atom:id>http://iddn.icis.com/search</atom:id>
  <atom:title>IDDN Search Results</atom:title>
  <atom:updated>2016-04-28T09:23:40.173506Z</atom:updated>
  <atom:author>ICIS Data Distribution Network (IDDN)</atom:author>
… snip …
  <atom:entry>
    <atom:title>http://iddn.icis.com/series/general/timeframe-002</atom:title>
    <atom:id>http://iddn.icis.com/series/general/timeframe-002</atom:id>
    <atom:updated>2013-08-20T16:05:22.526Z</atom:updated>
    <atom:relevance-score>43904</atom:relevance-score>
    <atom:content type="application/vnd.icis.iddn.entity+xml">
      <a:rolling-timeframe … snip … >
        <c:id>http://iddn.icis.com/series/general/timeframe-002</c:id>
        <c:version>1</c:version>
        <c:type>series</c:type>
        <c:created-on>2013-08-20T16:05:22.526Z</c:created-on>
        <c:descriptor href="http://iddn.icis.com/descriptor/rolling-timeframe"/>
        <c:domain href="http://iddn.icis.com/domain/general"/>
        <c:released-on>2013-04-01T12:00:00Z</c:released-on>
        <c:entitlement-class>unrestricted</c:entitlement-class>
        <c:terminated>false</c:terminated>
        <f:name>
          <c:l10n xml:lang="en">Next Month</c:l10n>
        </f:name>
        <f:timeframe-unit href="http://iddn.icis.com/ref-data/timeframe-unit/5"/>
        <f:timeframe-offset>1</f:timeframe-offset>
        <f:methodology href="http://iddn.icis.com/ref-data/methodology/35"/>
      </a:rolling-timeframe>
    </atom:content>
  </atom:entry>
… snip …
  <s:total-time>PT0.316992S</s:total-time>
</atom:feed>

The response to our simple query is again an Atom feed, which is the default format for the /search endpoint. The first page of results may not be of particular interest as it is likely to exclusively contain timeframes, but more useful examples are detailed in the remainder of this document.

Note that a series is essentially an entity that provides context for the series-items that point to it, like a sort of container for time series such as prices, indices, production, ships’ positions, etc. Series links to a number of other entities that describe aspects of the data it represents – for example, commodity, unit of measure, currency, location, vessel, etc.

How do I…

This section contains a small selection of generic-use cases that can be applied in many scenarios. For a more exhaustive list, please refer to the online documentation.

How do I retrieve prices between two dates for a known series?

Every entity (price, report, etc) available via the ICIS API has a unique ID which is always in the format of a URI. Note that the ID is not an actual URL, but just a unique string that looks like a URL. Assuming that we already know the ID of the series (more on how to find this in further examples), we can easily construct a search request to obtain the prices between two given dates by specifying the constraints as below:

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <series>http://iddn.icis.com/series/petchem/4021285</series>
  </scope>
  <constraints>
    <compare field="c:series-order" op="ge" value="2016-01-01"/>
    <compare field="c:series-order" op="le" value="2016-05-01"/>
  </constraints>
</request>

The series in the above example is the “LDPE, Assessment, Loading 7-30 days” and the prices are requested for the dates 1 Jan 2016 – 1 May 2016 inclusive.

The response will contain entries like the one below where we have highlighted key information such as the date the price refers to and the low, mid and high prices.

<atom:feed
  … snip …
  <atom:entry>
    <atom:title>http://iddn.icis.com/series-item/petchem/4021285-pricehistory-20160115000000</atom:title>
    <atom:id>http://iddn.icis.com/series-item/petchem/4021285-pricehistory-20160115000000</atom:id>
    <atom:updated>2016-01-15T16:57:34.232477Z</atom:updated>
    <atom:relevance-score>72192</atom:relevance-score>
    <atom:content type="application/vnd.icis.iddn.entity+xml">
      <a:price-range>
        <c:id>http://iddn.icis.com/series-item/petchem/4021285-pricehistory-20160115000000</c:id>
        <c:version>1</c:version>
        <c:type>series-item</c:type>
        <c:created-on>2016-01-15T16:57:34.232477Z</c:created-on>
        <c:descriptor href="http://iddn.icis.com/descriptor/price-range"/>
        <c:domain href="http://iddn.icis.com/domain/petchem"/>
        <c:released-on>2016-01-15T17:30:00Z</c:released-on>
        <c:series href="http://iddn.icis.com/series/petchem/4021285"/>
        <c:series-order>2016-01-15T00:00:00Z</c:series-order>
        <f:assessment-low precision="0">1261</f:assessment-low>
        <f:assessment-high precision="0">1355</f:assessment-high>
        <f:mid precision="1">1308</f:mid>
		
        <f:assessment-low-delta>-127</f:assessment-low-delta>
        <f:assessment-high-delta>-134</f:assessment-high-delta>
      </a:price-range>
    </atom:content>
  </atom:entry>
  … snip …
</atom:feed>

How do I find the series ID for a given commodity?

We want to find all the series for which the commodity name or long-name contains the string polyeth. We include two string-contains search filters with varying field values for each field we are searching.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <type>series</type>
  </scope>
  <constraints>
    <or>
      <string-contains field="f:commodity.f:name" pattern="*polyeth*"/>
      <string-contains field="f:commodity.f:long-name" pattern="*polyeth*"/>
    </or>
  </constraints>
  <view>
    <field>c:id</field>
    <field>name</field>
    <field>commodity.name</field>
    <field>commodity.long-name</field>
  </view>
</request>

Depending on the user entitlements a different number of results will be returned in the format below.

 … snip …
<atom:entry>
  <atom:title>http://iddn.icis.com/series/petchem/4057004</atom:title>
  <atom:id>http://iddn.icis.com/series/petchem/4057004</atom:id>
  <atom:updated>2014-10-18T15:39:55.661Z</atom:updated>
  <atom:relevance-score>35401</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.search.view+xml">
  <s:credit-enabled-petchem-energy-series>
    <s:c_id>http://iddn.icis.com/series/petchem/4057004</s:c_id>
    <s:name xml:lang="en">PE LLDPE Metallocene - C6 Blown Film Spot FD NWE 4Weeks</s:name>
    <s:commodity.name xml:lang="en">LLDPE Metallocene - C6 Blown Film</s:commodity.name>
    <s:commodity.long-name xml:lang="en">Metallocene Linear Low-Density Polyethylene -C6 Blown Film</s:commodity.long-name>
  
  </s:credit-enabled-petchem-energy-series>
  </atom:content>
</atom:entry>
 … snip …

Note that in the above example we have restricted the fields returned by the view, but it is possible to return the entire list of fields associated to an entity by adding the element below to the view section (or simply removing the section entirely):

<field>*</field>

How do I find any news articles citing a particular commodity?

We want to find all the news-articles where the body or headline contains the string Benzene. With the above in mind we construct a search request including two term-search filters with varying field values for each field we are searching.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>news-article</kind>
  </scope>
  <constraints>
    <or>
      <term-search field="f:body" type="free-text" pattern="Benzene" />
      <term-search field="f:headline" type="free-text" pattern="Benzene" />
    </or>
  </constraints>
  <options>
    <order-by key="created-on" direction="descending" xml:lang="en"/>
  </options>
  <view>
    <field>f:headline</field>
    <field>f:source.name</field>
    <field>f:author.display-name</field>
    <field>f:body</field>
  </view>
</request>

Depending on the user entitlements a different number of results will be returned in the following format.

 … snip …
<atom:entry>
  <atom:title>http://iddn.icis.com/asset/petchem/news-article-0019983616</atom:title>
  <atom:id>http://iddn.icis.com/asset/petchem/news-article-0019983616</atom:id>
  <atom:updated>2016-07-13T10:35:11.631242Z</atom:updated>
  <atom:relevance-score>15360</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.search.view+xml">
    <s:news-article>
      <s:headline>
        <c:content>
          <c:l10n xml:lang="en">
            <p>Argentina PS price up 5% in April</p>
          </c:l10n>
        </c:content>
        <c:resources/>
        </s:headline>
        <s:f_source.name xml:lang="en">ICIS News</s:f_source.name>
        <s:f_author.display-name>Marianela Toledo</s:f_author.display-name>
        <s:body>
          <c:content>
            <c:l10n xml:lang="en">
              <p>HOUSTON (ICIS)--Argentina sole polystyrene (PS) producer, Petrobras, is increasing its prices by 5% in April, a source familiar with the matter said on Thursday.</p>
              <p>The price increase is for all of its PS sold into the country, and it tracks delayed domestic currency devaluations, higher feedstock costs and other items, the source said.</p>
              <p>International prices of crystal and high impact polystyrene (HIPS) began to rise by the end of February and through March. Upstream benzene and styrene prices also rose in the US and Asia.</p>
              <p>Argentina’s domestic market has been described as volatile because of economic policy.</p>
            </c:l10n>
          </c:content>
        <c:resources/>
		
      </s:body>
    </s:news-article>
  </atom:content>
</atom:entry>
… snip …

Note that as with the previous example, we have restricted the fields returned by the view.

How do I detect when new data is available via the ICIS API?

A simple way to detect data changes is to set up a polling operation that regularly (eg, every few minutes) sends a search request like the one below, which picks up only entities that have been added to the system after a given point in time:

<request xmlns="http://iddn.icis.com/ns/search">
  <constraints>
    <or>
      <compare field="c:created-on" op="gt" value="2016-10-31T09:00:00.000Z"/>
      <compare field="c:released-on" op="gt" value="2016-10-31T09:00:00.000Z"/>
      <committed-since>2016-10-31T08:59:34.000Z</committed-since>
    </or>
  </constraints>
  <options>
    <include-last-commit-time>true</include-last-commit-time>
    <include-result-types>deleted</include-result-types>
  </options>
</request>

The operator gt (greater than) is applied to the fields c:created-on, which is an automatic database timestamp available for all entities, and c:released-on which is the data entity published date & time. The committed-since date & time value pertains to the transaction log and is included in the request to maintain transaction integrity during polling operations of the API. The actual values of all three of these constraint fields is stored as UTC.

When using the committed-since constraint you should also specify an include-last-commit-time within the options node and store the returned value to use for the committed-since value the next time you poll the API. You can also include another option include-result-types with a value of deleted if you also want to retrieve a list of deleted items since the committed-since date & time.


The request can obviously be customised to restrict it to only certain kinds of assets and/or to return only a specific set of fields.

Note that subscriptions to the ICIS API define a daily usage quota and aggressive polling approaches may result in consuming the available resources sooner than expected.

Some common use cases

This section contains a few ready-to-use recipes, each one presented with a walkthrough of the steps required to achieve the desired result. As usual, the scripts can be customised to address similar cases and should be seen as a starting point.

Retrieve the weekend price on a given date for NBP

We can break this requirement into the following steps:

  1. Identifying the ID of the location representing NBP
  2. Retrieving the list of series for this location
  3. Retrieving the prices for one of the above series

1) Identifying the ID of the location representing NBP

We construct a search request including two string-contains search filters with varying field values for each field we are searching. In this case we are looking for the string NBP in the name and long-name fields.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>location</kind>
  </scope>
  <constraints>
    <or>
      <string-contains field="f:name" pattern="NBP"/>
      <string-contains field="f:long-name" pattern="NBP"/>
    </or>
  </constraints>
</request>

The ID can be extracted from the response: http://iddn.icis.com/ref-data/location/1132

… snip …
<atom:entry>
  <atom:title>http://iddn.icis.com/ref-data/location/1132</atom:title>
  <atom:id>http://iddn.icis.com/ref-data/location/1132</atom:id>
  <atom:updated>2013-08-20T17:39:04.141Z</atom:updated>
  <atom:relevance-score>26841</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <a:location xmlns:r="http://iddn.icis.com/ns/refdata" xmlns:d="http://drdc.icis.com/ns/
      config">
      <c:id>http://iddn.icis.com/ref-data/location/1132</c:id>
      <c:version>1</c:version>
      <c:type>ref-data</c:type>
      <c:created-on>2013-08-20T17:39:04.141Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/location"/>
      <c:domain href="http://iddn.icis.com/domain/general"/>
      <c:released-on>2013-08-20T15:42:37Z</c:released-on>
      <f:location-type href="http://iddn.icis.com/ref-data/location-type/17"/>
      <f:name>
        <c:l10n xml:lang="en">NBP</c:l10n>
      </f:name>
      <f:long-name>
        <c:l10n xml:lang="en">National Balancing Point</c:l10n>
      </f:long-name>
      <f:part-of href="http://iddn.icis.com/ref-data/location/30"/>
    </a:location>
  </atom:content>
</atom:entry>
… snip …

2) Retrieving the list of series for this location

We know the location and we know that we are looking for a weekend price, so we construct the below request, specifying we are searching for series referring to the location http://iddn.icis.com/ref-data/location/1132 and the timeframe weekend.

We could separately identify the ID of the timeframe in the same way we did in step 1 for the location, but for the purpose of this example we’ll take a shortcut and filter inline based on the timeframe name. This isn’t generally considered good practice in that, although infrequently, the name of certain reference data entities can change, thus affecting queries that rely on it – such as the one below. A much safer approach is to use the entity ID in the filters, as done for the location, as this doesn’t change.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <type>series</type>
  </scope>
  <constraints>
    <field-refers-to field="location" href="http://iddn.icis.com/ref-data/location/1132"/>
    <string-contains field="timeframe.name" pattern="*weekend*" />
  </constraints>
  <view>
    <field>name</field>
    <field>family.name</field>
    <field>commodity.name</field>
    <field>location.name</field>
    <field>trade-terms.name</field>
    <field>price-type.name</field>
    <field>price-class.name</field>
    <field>timeframe.name</field>
    <field>observed-period.name</field>
    <field>quote-approach.name</field>
    <field>quote-measurement-style.name</field>
    <field>currency.name</field>
    <field>launch-date.name</field>
    <field>frequency.name</field>
    <field>specification.name</field>
    <field>transaction-type.name</field>
    <field>originator.name</field>
    <field>size-unit.name</field>
    <field>typical-size</field>
    <field>transport.name</field>
    <field>assessed-location.name</field>
    <field>methodology.name</field>
    <field>publication.name</field>
  </view>
</request>

Note that it would be possible to send the request without specifying the view section, in which case the response would contain the links to the other entities. As these would be just a set of opaque URLs, we have made use of the Dot Notation, to navigate through the data graph and retrieve the more user-readable name field for each of the target entities.

The response will be in the following format:

<atom:feed
    … snip …
  <s:returned-count>6</s:returned-count>
    … snip …
    <atom:entry>
    <atom:title>http://iddn.icis.com/series/energy/nbp-prices-weekend</atom:title>
    <atom:id>http://iddn.icis.com/series/energy/nbp-prices-weekend</atom:id>
    <atom:updated>2014-10-18T15:37:28.884Z</atom:updated>
    <atom:relevance-score>58112</atom:relevance-score>
    <atom:content type="application/vnd.icis.iddn.search.view+xml">
      <s:petchem-energy-series>
        <s:name xml:lang="en">NBP Continental Price Assessments</s:name>
        <s:commodity.name xml:lang="en">Natural Gas</s:commodity.name>
        <s:location.name xml:lang="en">NBP</s:location.name>
        <s:trade-terms.name xml:lang="en">DEL</s:trade-terms.name>
        <s:price-type.name xml:lang="en">Outright</s:price-type.name>
        <s:price-class.name xml:lang="en">Physical</s:price-class.name>
        <s:timeframe.name xml:lang="en">Weekend</s:timeframe.name>
        <s:observed-period.name xml:lang="en">Day</s:observed-period.name>
        <s:quote-approach.name xml:lang="en">Assessment</s:quote-approach.name>
        <s:quote-measurement-style.name xml:lang="en">Bid-offer range</s:quotemeasurement-style.name>
        <s:currency.name xml:lang="en">Euro</s:currency.name>
        <s:frequency.name xml:lang="en">daily</s:frequency.name>
        <s:specification.name xml:lang="en">Natural Gas</s:specification.name>
        <s:transaction-type.name xml:lang="en">Spot</s:transaction-type.name>
        <s:originator.name xml:lang="en">ICIS Energy</s:originator.name>
        <s:size-unit.name xml:lang="en">MWh</s:size-unit.name>
        <s:typical-size>1</s:typical-size>
        <s:transport.name xml:lang="en">Pipeline</s:transport.name>
        <s:assessed-location.name xml:lang="en">Europe</s:assessed-location.name>
        <s:methodology.name xml:lang="en">ESGM Continental Price Assessements</s:methodology.name>
        <s:publication.name xml:lang="en">European Spot Gas Markets</s:publication.name>
      </s:petchem-energy-series>
    </atom:content>
</atom:entry>

Assuming we are interested in the series http://iddn.icis.com/series/energy/nbp-prices-weekend, we can now retrieve the prices for a given weekend, let’s say the one starting on 2016-04-23, as assessed on 2016-04-22:

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <series>http://iddn.icis.com/series/energy/nbp-prices-weekend</series>
  </scope>
  <constraints>
    <compare field="start-date" op="eq" value="2016-04-23"/>
    <compare field="c:series-order" op="eq" value="2016-04-22"/>
  </constraints>
  <view>
    <field>c:id</field>
    <field>c:series-order</field>
    <field>assessment-low</field>
    <field>assessment-high</field>
    <field>mid</field>
    <field>start-date</field>
    <field>end-date</field>
  </view>
</request>

The resulting response carries the information we were after:

… snip …
  <atom:entry>
    <atom:title>http://iddn.icis.com/series-item/energy/nbp-prices-weekend-20160422</atom:title>
    <atom:id>http://iddn.icis.com/series-item/energy/nbp-prices-weekend-20160422</atom:id>
    <atom:updated>2016-04-22T19:02:43.267256Z</atom:updated>
    <atom:relevance-score>61440</atom:relevance-score>
    <atom:content type="application/vnd.icis.iddn.search.view+xml">
      <s:price-range>
        <s:c_id>http://iddn.icis.com/series-item/energy/nbp-prices-weekend-20160422</s:c_id>
        <s:c_series-order>2016-04-22T23:59:59Z</s:c_series-order>
        <s:assessment-low precision="3">13.893</s:assessment-low>
        
		<s:assessment-high precision="3">13.904</s:assessment-high>
        <s:mid precision="3">13.899</s:mid>
        <s:start-date>2016-04-23</s:start-date>
        <s:end-date>2016-04-24</s:end-date>
      </s:price-range>
    </atom:content>
  </atom:entry>
… snip …

Download the latest PDF report for the European Gas Market

We can break the requirement into the following steps:

  1. Identify the publication to which the report belongs
  2. Find the required issue
  3. Locate the binary-body referencing the issue
  4. Download the report in PDF form

1) Identify the publication to which the report belongs

Construct a search request to retrieve entities of kind publication containing European Gas Market in the publication name.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>publication</kind>
  </scope>
  <constraints>
    <string-contains field="f:name" pattern="*European gas market*" />
  </constraints>
</request>

The response below returns a single result containing the ID.

<atom:entry>
  <atom:title>http://iddn.icis.com/asset/energy/publication-e-egm</atom:title>
  <atom:id>http://iddn.icis.com/asset/energy/publication-e-egm</atom:id>
  <atom:updated>2015-05-18T21:42:36.063Z</atom:updated>
  <atom:relevance-score>61052</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <a:publication xmlns:r="http://iddn.icis.com/ns/refdata">
      <c:id>http://iddn.icis.com/asset/energy/publication-e-egm</c:id>
      <c:version>5</c:version>
      <c:type>asset</c:type>
      <c:created-on>2015-05-18T21:42:36.063Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/publication"/>
      <c:domain href="http://iddn.icis.com/domain/energy"/>
      <c:released-on>2015-05-18T21:43:00Z</c:released-on>
      <f:name>
        <c:l10n xml:lang="en">European Gas Markets</c:l10n>
      </f:name>
      <f:alias>
        <c:l10n xml:lang="en">EGM</c:l10n>
      </f:alias>
      … snip …
    </a:publication>
  </atom:content>
</atom:entry>

2) Find the required issue

Construct a search request to find the latest published issue under the publication. There will typically be multiple issues so we’ll add a descending order-by clause on the released-on field, and a max-results = 1 clause to get the latest one only.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>issue</kind>
  </scope>
  <constraints>
    <referencing>http://iddn.icis.com/asset/energy/publication-e-egm</referencing>
  </constraints>
  <options>
    <order-by key="released-on" direction="descending"/>
    <max-results>1</max-results>
  </options>
</request>

The ID is returned within the entry record as highlighted below.

<atom:entry>
  <atom:title>http://iddn.icis.com/asset/energy/issue-heren-egm-20160429</atom:title>
  <atom:id>http://iddn.icis.com/asset/energy/issue-heren-egm-20160429</atom:id>
  <atom:updated>2016-04-29T17:26:23.261244Z</atom:updated>
  <atom:relevance-score>78848</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <a:issue xmlns:r="http://iddn.icis.com/ns/refdata">
      <c:id>http://iddn.icis.com/asset/energy/issue-heren-egm-20160429</c:id>
      <c:version>1</c:version>
      <c:type>asset</c:type>
      <c:created-on>2016-04-29T17:26:23.261244Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/issue"/>
      <c:domain href="http://iddn.icis.com/domain/energy"/>
      <c:released-on>2016-04-29T00:00:00Z</c:released-on>
      <f:publication href="http://iddn.icis.com/asset/energy/publication-e-egm"/>
      <f:date>2016-04-29</f:date>
      <f:issue-number>2308</f:issue-number>
      <f:binary-body href="http://iddn.icis.com/binary/energy/heren-egmpdf-20160429"/>
      <f:is-corrected>false</f:is-corrected>
    </a:issue>
  </atom:content>
</atom:entry>

3) Locate the binary-body referencing the issue

Using the ID found above we can now identify the PDF (binary) referencing the issue by checking against the appropriate mime-type.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <type>binary</type>
  </scope>
  <constraints>
    <referencing>http://iddn.icis.com/asset/energy/issue-heren-egm-20160429</referencing>
    <compare field="c:mime-type" op="eq" value="application/pdf"/>
  </constraints>
</request>

The above returns a single result from which we can identify the ID of the binary-body containing the PDF report.

<atom:entry>
  <atom:title>http://iddn.icis.com/binary/energy/heren-egm-pdf-20160429</atom:title>
  <atom:id>http://iddn.icis.com/binary/energy/heren-egm-pdf-20160429</atom:id>
  <atom:updated>2016-04-29T17:26:23.86331Z</atom:updated>
  <atom:relevance-score>90624</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <binary-body xmlns:r="http://iddn.icis.com/ns/refdata" xmlns="http://iddn.icis.com/ns/assets">
      <c:id>http://iddn.icis.com/binary/energy/heren-egm-pdf-20160429</c:id>
      <c:version>2</c:version>
      <c:type>binary</c:type>
      <c:created-on>2016-04-29T17:26:23.86331Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/binary-body"/>
      <c:domain href="http://iddn.icis.com/domain/energy"/>
      <c:released-on>2016-04-29T17:26:00Z</c:released-on>
      <c:mime-type>application/pdf</c:mime-type>
      <c:binary-data-url>/file-store/binary/energy/2016-04-29-17-26-23-73851.bin</c:binary-data-url>
      <c:binary-size>1665674</c:binary-size>
      <f:issue href="http://iddn.icis.com/asset/energy/issue-heren-egm-20160429" isreciprocal-link="true"/>
    </binary-body>
  </atom:content>
  
</atom:entry>

4) Download the report in PDF form

To access the binary part of the binary-body asset we need to use the /binary resource – ie, use a different URL in the HTTP request. We will also want to redirect the output to a file.

curl -u your_username:your_password https://api.icis.com/v1//binary/energy/heren-egm-pdf-20160429>my_first_report.pdf

Retrieve the information available in the latest FTP price file for PE LDPE GP FILM for FOB NWE FULL MARKET RANGE

The e-polye.prc price file is a CSV file with the following fields:

  • Field 1: series number, corresponding to the field c:original-id of the series
  • Field 2: date, corresponding to the field c:series-order in the series-items
  • Field 3: contract period (numeric) – not relevant in this example
  • Field 4: contract period (text) – not relevant in this example
  • Field 5, 6: low and high price, corresponding to f:assessment-low and f:assessment-high in the series-items
  • Field 7: not relevant in this example

To find the information we are after, we can construct a request which includes an or constraint for each series number/original-id in the prc file. To get the latest price for each series we include a series-item clause in the view, with a position/pos attribute = 1 and a start-at attribute = newest.

<request xmlns="http://iddn.icis.com/ns/search">
  <constraints>
    <or>
      <compare field="c:original-id" op="eq" value="4021004"/>
      <compare field="c:original-id" op="eq" value="4021005"/>
      <compare field="c:original-id" op="eq" value="4021006"/>
      <compare field="c:original-id" op="eq" value="4021016"/>
      <compare field="c:original-id" op="eq" value="4021017"/>
      <compare field="c:original-id" op="eq" value="4021018"/>
      … snip …
    </or>
  </constraints>
  <view>
    <field>c:original-id</field>
    <series-item pos="1" start-at="newest">c:series-order</series-item>
    <series-item pos="1" start-at="newest">assessment-high</series-item>
    <series-item pos="1" start-at="newest">assessment-low</series-item>
  </view>
</request>

The response below contains an entry for each of the series specified in the compare clause. From the response it is easy to determine the series-number/original-id, the date/series-order and the assessment-low and assessment-high price values.

<atom:entry>
  <atom:title>http://iddn.icis.com/series/petchem/4021005</atom:title>
  <atom:id>http://iddn.icis.com/series/petchem/4021005</atom:id>
  <atom:updated>2015-07-01T09:07:24.513Z</atom:updated>
  <atom:relevance-score>32000</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.search.view+xml">
    <s:credit-enabled-petchem-energy-series>
      <s:c_original-id>4021005</s:c_original-id>
      <s:series-item.1.c_series-order>2016-04-29T00:00:00Z</s:series-item.1.c_series-order>
      <s:series-item.1.assessment-low precision="0">1360</s:seriesitem.1.assessment-low>
	  
      <s:series-item.1.assessment-high precision="0">1380</s:seriesitem.1.assessment-high>
    </s:credit-enabled-petchem-energy-series>
  </atom:content>
</atom:entry>

Retrieve the information available in the latest FTP price file for Natural Gas NBP

The NBP_Price_Assessments.csv price file contains Contract Price Assessments for Natural Gas related to the locations NBP, Bacton and St Fergus.

Although the walkthrough below illustrates a dynamic and instructional approach to getting to the required data, the actual queries that pull the final data are represented in steps 4 and 5:

  1. Identify the ID of the locations representing NBP, St Fergus and Bacton
  2. Identify the ID of the commodity Natural Gas
  3. Identify the ID of the quote approach Contract Price Assessment
  4. Retrieve the latest prices for NBP
  5. Retrieve the latest prices for St Fergus and Bacton

1) Identify the ID of the locations representing NBP, St Fergus and Bacton

Following a similar procedure to the one described in the previous case Retrieve the weekend price on a given date for NBP, we’ll retrieve the ID of the locations St Fergus and Bacton:

  • NBP: http://iddn.icis.com/ref-data/location/1132
  • St Fergus: http://iddn.icis.com/ref-data/location/1134
  • Bacton: http://iddn.icis.com/ref-data/location/1133

2) Identify the ID of the commodity Natural Gas

As in the previous step, the ID of the commodity can be determined with the following search request:

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>commodity</kind>
  </scope>
  <constraints>
    <or>
      <string-contains field="f:name" pattern="Natural Gas"/>
      <string-contains field="f:long-name" pattern="Natural Gas"/>
    </or>
  </constraints>
</request>

The ID gathered from the response is http://iddn.icis.com/ref-data/commodity/606

3) Identify the ID of the quote approach Contract Price Assessment

The above step can be reused, as below:

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>quote-approach</kind>
    </scope>
    <constraints>
      <or>
        <string-contains field="f:name" pattern="Contract Price Assessment"/>
        <string-contains field="f:long-name" pattern="Contract Price Assessment"/>
      </or>
    </constraints>
</request>

The ID gathered from the response is http://iddn.icis.com/ref-data/quote-approach/7

4) Retrieve the latest prices for NBP

With the appropriate filters on the commodity, location and quote approach, it is easy to extract the latest price assessments, together with the delta, the volatility index, the period range, etc.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <type>series</type>
  </scope>
  <constraints>
    <field-refers-to field="location" href="http://iddn.icis.com/ref-data/location/1132"/>
    <field-refers-to field="commodity" href="http://iddn.icis.com/ref-data/commodity/606"/>
    <field-refers-to field="quote-approach" href="http://iddn.icis.com/ref-data/quote-approach/7"/>
  </constraints>
  <view>
    <series-item pos="1" start-at="newest">c:series-order</series-item>
    <series-item pos="1" start-at="newest">*</series-item>
    <item-delta field="f:mid" from="2" to="1"/>
  </view>
</request>

It is easy to identify in the response the different pieces of information originally available in the FTP file.

… snip …
<atom:entry>
  <atom:title>http://iddn.icis.com/series/energy/nbp-assess-season-1-nbp</atom:title>
  <atom:id>http://iddn.icis.com/series/energy/nbp-assess-season-1-nbp</atom:id>
  <atom:updated>2015-01-07T15:50:00.58Z</atom:updated>
  <atom:relevance-score>57754</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.search.view+xml">
  <s:petchem-energy-series>
    <s:series-item.1.c_series-order>2016-05-06T23:59:59Z</s:series-item.1.c_series-order>
    <s:series-item.1.assessment-low precision="3">36.100</s:series-item.1.assessmentlow>
    <s:series-item.1.assessment-high precision="3">36.200</s:series-item.1.assessmenthigh>
    <s:series-item.1.mid precision="3">36.150</s:series-item.1.mid>
    <s:series-item.1.start-date>2016-10-01</s:series-item.1.start-date>
    <s:series-item.1.end-date>2017-03-31</s:series-item.1.end-date>
    <s:series-item.1.period-label xml:lang="en">Winter 16</s:series-item.1.period-label>
    <s:series-item.1.volatility precision="1">40.2</s:series-item.1.volatility>
    <s:mid>0.05</s:mid>
  </s:petchem-energy-series>
</atom:content>
… snip …

5) Retrieve the latest prices for St Fergus and Bacton

The query below is very similar to the one in step 4, but doesn’t return the price delta and includes the location name in the view, as each entry might belong to any of the two locations. Alternatively, two distinct queries could be sent, one for each location.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <type>series</type>
  </scope>
  <constraints>
    <or>
      <field-refers-to field="location" href="http://iddn.icis.com/ref-data/location/1133"/>
      <field-refers-to field="location" href="http://iddn.icis.com/ref-data/location/1134"/>
    </or>
    <field-refers-to field="commodity" href="http://iddn.icis.com/ref-data/commodity/606"/>
  </constraints>
  
  <view>
    <series-item pos="1" start-at="newest">c:series-order</series-item>
    <series-item pos="1" start-at="newest">c:series.location.name</series-item>
    <series-item pos="1" start-at="newest">*</series-item>
  </view>
</request>

Retrieve all news articles relating to acetone in Western Europe

We can break this requirement into the following four steps:

  1. Identifying the ID of the article-region for Western Europe
  2. Identifying the ID of the article-product for acetone
  3. Search for the associated news-articles
  4. Download any binaries referenced in the news-article

1) Identifying the ID of the article-region for Western Europe

Construct a search request to retrieve entities of kind article-region with the name Western Europe.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>article-region</kind>
  </scope>
  <constraints>
    <string-contains field="name" pattern="Western Europe" />
  </constraints>
</request>

The response below returns a single result containing the ID.

<atom:entry>
  <atom:title>http://iddn.icis.com/taxon/article-region/00111129</atom:title>
  <atom:id>http://iddn.icis.com/taxon/article-region/00111129</atom:id>
  <atom:updated>2015-05-12T15:33:22.632Z</atom:updated>
  <atom:relevance-score>107520</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <article-region xmlns:r="http://iddn.icis.com/ns/refdata" xmlns="http://iddn.icis.com/ns/assets">
      <c:id>http://iddn.icis.com/taxon/article-region/00111129</c:id>
      <c:version>2</c:version>
      <c:type>taxon</c:type>
      <c:created-on>2015-05-12T15:33:22.632Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/article-region"/>
      <c:domain href="http://iddn.icis.com/domain/petchem"/>
      <c:released-on>2014-07-01T12:53:00Z</c:released-on>
      <c:parent href="http://iddn.icis.com/taxon/article-region/00110345"/>
      <f:name>
        <c:l10n xml:lang="en">Western Europe</c:l10n>
      </f:name>
    </article-region>
  </atom:content>
</atom:entry>

2) Identifying the ID of the article-product for acetone

Construct a search request to retrieve entities of kind article-product with the name acetone.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>article-product</kind>
  </scope>
  <constraints>
    <string-contains field="name" pattern="*acetone*" />
  </constraints>
</request>

The ID is returned within the entry record as highlighted below.

<atom:entry>
  <atom:title>http://iddn.icis.com/taxon/article-product/00110146</atom:title>
  <atom:id>http://iddn.icis.com/taxon/article-product/00110146</atom:id>
  <atom:updated>2016-01-13T14:57:36.357075Z</atom:updated>
  <atom:relevance-score>92160</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
    <a:article-product xmlns:r="http://iddn.icis.com/ns/refdata">
      <c:id>http://iddn.icis.com/taxon/article-product/00110146</c:id>
      <c:version>3</c:version>
      <c:type>taxon</c:type>
      <c:created-on>2016-01-13T14:57:36.357075Z</c:created-on>
      <c:descriptor href="http://iddn.icis.com/descriptor/article-product"/>
      <c:domain href="http://iddn.icis.com/domain/general"/>
      <c:released-on>2014-07-01T12:53:00Z</c:released-on>
      <c:parent href="http://iddn.icis.com/taxon/article-product/00110084"/>
      <f:name>
        <c:l10n xml:lang="en">Acetone</c:l10n>
      </f:name>
      <f:commodity href="http://iddn.icis.com/ref-data/commodity/537"/>
      <f:related-article-product href="http://iddn.icis.com/taxon/article-product/00110163"/>
      <f:related-article-product href="http://iddn.icis.com/taxon/article-product/00110188"/>
      <f:original-id>10146</f:original-id>
    </a:article-product>
  </atom:content>
</atom:entry>

3) Search for the associated news-articles

We can now perform the actual search for kind news-articles referring to the IDs found in the previous two searches. We use the field-refers-to constraint for restricting results to a particular article-product, and the classified-as constraint to restrict results to a particular article-region; for more information on using the classified-as constraint please refer to the online documentation.

<request xmlns="http://iddn.icis.com/ns/search">
  <scope>
    <kind>news-article</kind>
  </scope>
  <constraints>
    <classified-as field="article-region" href="http://iddn.icis.com/taxon/article-region/00111129"/>
    <field-refers-to field="article-product" href="http://iddn.icis.com/taxon/article-product/00110146"/>
  </constraints>
  <options>
    <order-by key="released-on" direction="descending" xml:lang="en"/>
  </options>
</request>

The above request returns multiple relevant news-articles in date descending order with the article headline and body in HTML format including any associated binary files referenced in the news-article.

<atom:entry>
  <atom:title>http://iddn.icis.com/asset/petchem/news-article-0019983051</atom:title>
  <atom:id>http://iddn.icis.com/asset/petchem/news-article-0019983051</atom:id>
  <atom:updated>2016-07-11T03:07:08.762317Z</atom:updated>
  <atom:relevance-score>4571</atom:relevance-score>
  <atom:content type="application/vnd.icis.iddn.entity+xml">
  … snip …
    <c:id>http://iddn.icis.com/asset/petchem/news-article-0019983051</c:id>
    <c:version>1</c:version>
    <c:type>asset</c:type>
    <c:created-on>2016-07-11T03:07:08.762317Z</c:created-on>
    <c:descriptor href="http://iddn.icis.com/descriptor/news-article"/>
    <c:domain href="http://iddn.icis.com/domain/petchem"/>
    <c:released-on>2016-03-30T11:21:00Z</c:released-on>
    <f:original-id>9983051</f:original-id>
    <f:headline>
      <c:content>
        <c:l10n xml:lang="en">
          <p>EU acetone imports and exports up in January – Eurostat</p>
        </c:l10n>
      </c:content>
      <c:resources/>
      … snip …
      <f:associated-binary href="http://iddn.icis.com/binary/general/image-001809581"/>
      <f:associated-binary href="http://iddn.icis.com/binary/general/image-001809582"/>
      <f:body>
        <c:content>
          <c:l10n xml:lang="en">
            <p>LONDON (ICIS)--The EU’s imports and exports of acetone increased year on year in January 2016, according to data from the European statistics agency Eurostat on Wednesday.</p>
            <p><img style="HEIGHT: 238px; WIDTH: 349px" border="0" alt="" src="../assets/getAsset.aspx?ItemID=809581"/></p>
            <p>Imports of acetone to the EU increased by nearly 42% year on year in January, while exports of acetone increased by close to 32% year on year.</p>
            <p><img style="HEIGHT: 238px; WIDTH: 349px" border="0" alt="" src="../assets/getAsset.aspx?ItemID=809582"/></p>
            <p>The EU was a net exporter of acetone in both January 2015 and January 2016.</p>
            … snip …
          </c:l10n>
        </c:content>
        <c:resources>
          <c:resource src="../assets/getAsset.aspx?ItemID=809581" href="http://iddn.icis.com/binary/general/image-001809581"/>
          <c:resource src="../assets/getAsset.aspx?ItemID=809582" href="http://iddn.icis.com/binary/general/image-001809582"/>
        </c:resources>
      </f:body>
  … snip …
</atom:entry>

4) Download any binaries referenced in the news-article

The logical relationship between article body image src and the binary href is contained in the resources element at the base of the body. To download the news-article images you will need to perform additional requests to the API to download the referenced binary files.

For each binary referenced within the document:

4.1) Locate the relevant resource mapping entry in the resources section

4.2) To access the binary part of the image asset we need to use the /binary resource – ie, use a different URL in the HTTP request. Therefore using the href value of http://iddn.icis.com/binary/general/image-001809581; we would perform a GET to the URL: https://api.icis.com/v1/binary/general/image-001809582.

Please note that the binary file type can be found in the response header under Content-Type, in this case it is of type image/jpeg.

Further code samples and documentation can be found in the ICIS online API documentation located here.

Speak to ICIS

Now, more than ever, dynamic insights are key to navigating complex, volatile commodity markets. Access to expert insights on the latest industry developments and tracking market changes are vital in making sustainable business decisions.

Want to learn about how we can work together to bring you actionable insight and support your business decisions?

GET IN TOUCH

Need Help?

Need Help?

Speak with ICIS