CKAN harvester
Retrieve datasets from a CKAN portal.
Parameters
Name | Description | Example |
URL (url) | The base url of the portal | https://www.hri.fi/api/3 |
API Key (api_key) | If the portal requires authentication, your API Key | |
Sort parameter (sort) | Sorting of the CKAN search results. Default: 'relevance asc, metadata_modified desc'. This is a comma-separated string of field names and sort-orderings. | relevance desc |
CKAN Group (group) | Name of the group you want to harvest | education |
Query (query) | A Solr-format query to filter the datasets to harvest | value AND |
Maximum results (limit) | Only retrieve a certain amount of datasets | 5 |
Offset (offset) | Skip a certain amount of datasets before starting to harvest | 3 |
Language (language) | If the portal has multilingual metadata, the code of the language to take the values in. It must be one of the portal's languages. Otherwise, the harvest stops with an error listing the available ones. | fr |
Include private datasets (include_private) | Select this checkbox to also harvest the portal's private datasets. Only the private datasets that the API key can see are harvested. |
Metadata
The harvester fills in the following metadata for each dataset it imports from CKAN. Where a source is empty, the next one listed is used.
These mappings are fixed and can't be changed. The technical IDs are provided for those working with the APIs.
CKAN source | CKAN source in the API | Huwise metadata | Huwise field ID in the API |
The dataset's title, or its name | data.title, data.name | Title | default.title |
The dataset's notes, or the description of the resource Huwise imports | data.notes, resource.description | Description | default.description |
The dataset's tags, or its keywords | data.tags[].display_name, data.keywords | Keywords | default.keyword |
The titles of the dataset's groups, or their names | data.groups[].title, data.groups[].name | Themes | default.theme |
The dataset's license. Licenses containing "Etalab", "ODbL" or "Public domain" are replaced with "Licence Ouverte (Etalab)", "Open Database License (ODbL)" or "Public Domain" | data.license_title | License | default.license |
The harvester's language setting, or an extra field named lang or language | Harvester parameter, data.extras | Data language | default.language |
The dataset's publishers, or an extra field named publisher, or its author | data.publishers, data.extras, data.author | Creator | default.publisher |
The last modification date of the resource Huwise imports, or its creation date | resource.last_modified, resource.created | Display modification date | default.modified |
If the CKAN portal has multilingual metadata, the values are taken in the language set in the harvester's language parameter.