data.json harvester
Use this harvester to retrieve datasets from a /data.json file at the root of a portal.
The data.json norm is promoted by the Project Open Data.
Parameters
Name | Description | Example |
Portal URL (portal_url) | The URL of the portal you want to harvest. Azavea, Ckan, Dkan, ESRI, Junar, Huwise, OpenGov, Socrata portals implements the data.json norm. | https://data.townofcary.org/ |
Number of datasets (number_of_datasets) | If set, only retrieve a certain amount of datasets. | 5 |
Imported data
For each dataset, the harvester imports one file listed in the dataset's distributions. To be imported, a file must:
Be a shapefile, JSON, GeoJSON, CSV, TSV, or Excel (XLS) file, according to the format or media type given in the data.json file
Have a direct download link (downloadURL)
Distributions that point to an API or a web page (accessURL) are ignored. If several files qualify, only one is imported.
If a dataset has no file that meets these conditions, it's imported with its metadata only, without data.
Metadata
The harvester fills in the following metadata for each dataset it imports from the data.json file.
These mappings are fixed and can't be changed. The technical IDs are provided for those working with the APIs.
data.json source | data.json source in the API | Huwise metadata | Huwise field ID in the API |
The dataset's title | ds.title | Title | default.title |
The dataset's description | ds.description | Description | default.description |
The dataset's rights | ds.rights | License | default.license |
The dataset's language | ds.language | Data language | default.language |
The dataset's keywords | ds.keyword | Keywords | default.keyword |
The dataset's landing page | ds.landingPage | Reference | default.references |
The dataset's last modification date | ds.modified | Display modification date | default.modified |
The dataset's themes | ds.theme | Themes | default.theme |
The name of the dataset's publisher | ds.publisher.name | Creator | default.publisher |