A reconciliation API is a web service designed to match and link data entities from different datasets. It helps in identifying and merging duplicate or related entities by providing mechanisms to compare and align data from disparate sources. This process is particularly useful for integrating and enriching data, ensuring consistency, and improving the quality of information in databases.

In the context of the Offshore Leaks Database, the reconciliation API allows users to match their own data entities—such as addresses, entities, intermediaries, officers, and other types—against the entities in the Offshore Leaks Database. This facilitates the identification of potential matches and the integration of external data with the rich, investigative data provided by the Offshore Leaks Database.

By using this reconciliation API, users can:

• Automate the matching process to save time and reduce manual effort.
• Ensure that their data aligns with the verified information in the Offshore Leaks Database.
• Enhance their datasets with additional context and details from the Offshore Leaks Database.

This document provides an overview of how to use the Offshore Leaks Database Reconciliation API, including supported types, authentication methods, and example usage scenarios.

Supported Types

The Offshore Leaks Database API supports reconciliation for the following types:

General API

When the reconciliation service endpoint is queried with a HTTP GET request without parameters, the service manifest is returned. The manifest provides a comprehensive overview of its structure and capabilities. It includes details on supported versions, namespaces, default types, and authentication methods. The manifest also outlines the endpoints for entity viewing, preview, property proposals, and suggestions, along with configuration settings such as batch size and property settings.

• Service URL: https://offshoreleaks.icij.org/api/v1/reconcile
• Method: GET
• Spec: CG-FINAL-specs-0.2#service-manifest

For instance:

curl -XGET https://offshoreleaks.icij.org/api/v1/reconcile

Query API

When the reconciliation service endpoint is queried with a HTTP POST request, a reconciliation query is performed. A reconciliation query consists of several fields, designed to help clients search for and identify matching entities within the Offshore Leaks Database. At least one of query or queries must be supplied. All other fields are optional.

• Service URL: https://offshoreleaks.icij.org/api/v1/reconcile
• Method: POST
• Required Params: query or queries
• Spec: CG-FINAL-specs-0.2#reconciliation-queries

For instance, a query for "foo" on the Officer type:

curl -XPOST https://offshoreleaks.icij.org/api/v1/reconcile --header "Content-Type: application/json" -d '{
  "query": "foo",
  "type": "Officer"
}'

A reconciliation query batch can be a set of reconciliation queries indexed by string identifiers. This allows clients to perform multiple reconciliation queries in a single request, each with its own unique identifier.

This comprehensive structure ensures that users can efficiently and accurately find matching entities in the Offshore Leaks Database, enhancing the quality and consistency of their datasets.

For instance queries on the Address type, for "paris" in France (FRA) and "fin" in Monaco (MCO):

curl -XPOST https://offshoreleaks.icij.org/api/v1/reconcile --header "Content-Type: application/json" -d '{
  "type": "Address",
  "queries": {
    "q0": {
      "query": "paris",
      "properties": [
        {
          "pid": "country_codes",
          "v": "FRA"
        }
      ]
    },
    "q1": {
      "query": "fin",
      "properties": [
        {
          "pid": "country_codes",
          "v": "MCO"
        }
      ]
    }
  }
}'

Data extension API

When the reconciliation service endpoint is queried with a HTTP GET request and the extend query parameter, clients can request additional information for entities by specifying the properties they are interested in. To use the extend API, clients should specify the entities and the properties they want to fetch values for. The service will then return the values for those properties.

• Service URL: https://offshoreleaks.icij.org/api/v1/reconcile/properties
• Method: GET, POST
• Required Params: extend
• Spec: CG-FINAL-specs-0.2#data-extension-service

For instance, to get the country codes of two nodes, the following JSON is URL encoded:

{
  "ids": [20131544, 24000001], 
  "properties": [
    { "id": "country_codes" }
  ]
}

Which translates to:

curl -XGET 'https://offshoreleaks.icij.org/api/v1/reconcile?extend=%7B%22ids%22%3A%5B20131544%2C24000001%5D%2C%22properties%22%3A%5B%7B%22id%22%3A%22country_codes%22%7D%5D%7D' 

And results to:

{
  "meta": [
    {
      "id": "country_codes",
      "name": "Country Codes"
    }
  ],
  "rows": {
    "20131544": {
      "country_codes": [],
      "schema": [
        {
          "str": "https://offshoreleaks.icij.org/schema/oldb/entity"
        }
      ]
    },
    "24000001": {
      "country_codes": [
        {
          "str": "BS"
        }
      ],
      "schema": [
        {
          "str": "https://offshoreleaks.icij.org/schema/oldb/address"
        }
      ]
    }
  }
}

Suggest API

To get suggestions for properties, entities, or types, send a HTTP GET request to the respective suggestion URL with a required prefix query parameter.

• Property Suggestion:
  • Service URL: https://offshoreleaks.icij.org/api/v1/reconcile/suggest/property
  • Method: GET
  • Spec: CG-FINAL-specs-0.2#dfn-property
• Entity Suggestion:
  • Service URL: https://offshoreleaks.icij.org/api/v1/reconcile/suggest/entity
  • Method: GET
  • Spec: CG-FINAL-specs-0.2#dfn-entity
• Type Suggestion:
  • Service URL: https://offshoreleaks.icij.org/api/v1/reconcile/suggest/type
  • Method: GET
  • Spec: CG-FINAL-specs-0.2#dfn-type

For instance, for property suggestions:

curl -XGET https://offshoreleaks.icij.org/api/v1/reconcile/suggest/property?prefix=cou

Which will results to:

{
  "result": [
    {
      "id": "country_codes",
      "name": "Country",
      "description": "The country code of the node (\"FRA\", \"HKG\", etc)."
    }
  ]
}

Or for type suggestion:

curl -XGET https://offshoreleaks.icij.org/api/v1/reconcile/suggest/type?prefix=off

Which will results to:

{
  "result": [
    {
      "id": "https://offshoreleaks.icij.org/schema/oldb/officer",
      "name": "Officer"
    }
  ]
}

View API

To view detailed information about an entity, clients should use the provided URL template, replacing {{id}} with the entity's unique identifier. The resulting URL can then be used to fetch and display the entity's detailed information.

• Service URL : https://offshoreleaks.icij.org/api/v1/rest/nodes/{{id}}
• Method: GET
• Spec: CG-FINAL-specs-0.2#dfn-view-template

For instance:

curl -XGET https://offshoreleaks.icij.org/api/v1/rest/nodes/10067217

Preview API

To display a preview of an entity, clients should use the provided URL template, replacing {{id}} with the entity's unique identifier. The resulting URL can then be embedded as an HTML iframe or in any other suitable manner to show the entity's detailed information.

• Service URL: https://offshoreleaks.icij.org/nodes/{{id}}?embed=1
• Method: GET
• Spec: CG-FINAL-specs-0.2#preview-service

For instance, within an iframe (using the recommended size):

<iframe src="https://offshoreleaks.icij.org/nodes/10067217?embed=1" width="575" height="575"></iframe>

Namespaced API

The Offshore Leaks Database Reconciliation API is designed to support various investigations, each accessible through a specific, namespaced endpoint. This design ensures that users can also target their reconciliation efforts to a specific dataset that best fits their needs.

When using the API, you can specify the appropriate namespace for the investigation you are working with. Below are the endpoints, with their own manifest, for each available investigation:

• https://offshoreleaks.icij.org/api/v1/reconcile/bahamas-leaks
• https://offshoreleaks.icij.org/api/v1/reconcile/offshore-leaks
• https://offshoreleaks.icij.org/api/v1/reconcile/panama-papers
• https://offshoreleaks.icij.org/api/v1/reconcile/pandora-papers
• https://offshoreleaks.icij.org/api/v1/reconcile/paradise-papers

For instance, a query for "foo" on the Officer type in the Pandora Papers:

curl -XPOST https://offshoreleaks.icij.org/api/v1/reconcile/pandora-papers --header "Content-Type: application/json" -d '{
  "query": "foo",
  "type": "Officer"
}'

Community & Tools

This API adheres to the Reconciliation Service API specification v0.2, enabling seamless integration with various tools and platforms. This standardization facilitates collaboration within the community by providing a common framework for data matching and linking. Users can leverage existing tools and libraries designed for reconciliation, enhancing their workflows and ensuring compatibility with other data sources and services.

• W3C entity reconciliation community group
• Reconciliation service test bench
• OpenRefine
• OpenRefine Reconciling