The Data Product Descriptor Specification (DPDS) is a community-driven open specification that declaratively defines a data product in all its components using a JSON or YAML descriptor document. It is released under Apache 2.0 license. Contributions from the data community are welcomed!
There is not yet a generally available version of the specification. Version 1.0.0 is in a DRAFT state. We plan to collect community feedback and contributions over the following months to finalize and release it in the GA state. There is not yet a planned date for the release of version 1.0.0 in GA but ideally, we would like to release it by the end of Q1 of 2023.
This is the current working draft of the specification:
For a more detailed introduction to the specification check out our documentation pages.
This site is made with MkDocs and in particular with its extension Material for MkDocs.
Both are great tools that build a static site starting from simple markdown documentation.
They provide many amazing features, perfectly documented in their websites, with step-by-step tutorials to create a new site from scratch.
However, we would like to explain here all the prerequisites needed to build this very site and an overview of the principal customizations implemented.
The first thing to do in order to be able to build this site is to install Material for MkDocs, it will also install all dependencies: MkDocs, Markdown, Pygments and Python Markdown Extensions:
pip install mkdocs-materialIn order to add the date of the last update and creation of a document at the bottom of each page, we enabled the git-revision-date-localized plugin, which can be installed with pip:
pip install mkdocs-git-revision-date-localized-pluginFinally, two Python libraries must be installed to generate the social preview images, both of which are based on Cairo Graphics – Pillow and CairoSVG.
Install pillow and cairosvg with pip:
pip install pillow cairosvgInstall the other dependencies depending on your system:
There are several package managers for Linux with varying availability per distribution.
apt-get install libcairo2-dev libfreetype6-dev libffi-dev libjpeg-dev libpng-dev libz-dev
If you are on Ubuntu you can directly launch the setup.sh script located in script folder! It contains all the previously described steps.
yum install cairo-devel freetype-devel libffi-devel libjpeg-devel libpng-devel zlib-devel
zypper install cairo-devel freetype-devel libffi-devel libjpeg-devel libpng-devel zlib-devel
The easiest way to get up and running with the Cairo Graphics library on Windows is by installing GTK+ since it has Cairo as a dependency. You can also download and install a precompiled GTK runtime.
Make sure Homebrew is installed, which is a modern package manager for macOS. Next, use the following command to install all necessary dependencies.
brew install cairo freetype libffi libjpeg libpng zlib
In order to build the site, the following command must be launched in the root directory:
mkdocs buildIt is not necessary to explicitly build the site before serving or publishing but it can be useful to inspect the site folder it creates to verify resource paths.
It is also useful for sites published without the aid of GitHub Pages since the site folder is the deployment unit.
In order to locally test the site it is possible to launch the following command in the root directory:
mkdocs serveThe site will be hosted at the address configured in mkdcos.yml and almost any change in the docs folder will trigger an automatic republish.
If the site is published through GitHub Pages, as it is in our case, it can be easily deployed with a single command:
mkdocs gh-deploy --forceThe site will soon appear at '<username>.github.io/<repository>'.
If you wish to publish the site using GitHub pages but on a custom domain, this guide contains all the available possibilities.
If you don't want to use GitHub pages, you can always build your site as explained before and upload the content of the site folder to your hosting provider.
Most of the configurations reside in the mkdcos.yml file, located in the root folder.
The site_* and repo properties configure website and repository locations and meta tags for the website.
The nav section defines the structure of the site and the location of each markdown document published.
The theme, markdown_extensions and plugins sections contain all the Material for MkDocs configurations currently used and the extensions required to make them work.
The analytics and consent sections configure the usage of Google Analytics and govern cookie consent.
The social and copyright sections define the content of the footer, copyright and social icons.
The few CSS customizations we use can be found in docs>stylesheets>extra.css.
It contains the color palette we picked and a slight increase in the width of the pages.
The homepage is written directly in HTML since we wanted it to be different from the rest of the site.
It can be found in docs>override and has its own customized style embedded.