using_the_APD.qmd

---
title: "Using the APD: exploring and extracting trait definitions and metadata"
author: "Elizabeth Wenk"
format: 
  html:
    smooth-scroll: true
    toc: true
    toc-expand: 1
    embed-resources: true
---

The AusTraits Plant Dictionary (APD) is a published vocabulary, documenting explicit definitions for more than 500 plant trait concepts. The full dictionary is available at https://w3id.org/APD/, and each trait (and trait groupings) has its own URI.

To faciliate use of this formal vocabulary by researchers, the content is also output as a pairs of tables, `APD_traits.csv` and `APD_categorical_values`, which can be downloaded from [github.com/traitecoevo/APD/tree/master/docs](https://github.com/traitecoevo/APD/tree/master/docs) or Zenodo at DOI: [10.5281/zenodo.8040789](https://doi.org/10.5281/zenodo.8040789).

The examples below indicate how the definitions and metadata for specific traits or trait clusters can be extracted from these files, allowing searches for traits by trait name, trait cluster, characteristic measured, reference, previously used trait labels, or matches to identical or similar trait concepts in other vocabularies or databases.

```{r, warning=FALSE, message=FALSE, echo=TRUE}
library(tidyverse)
library(stringr)
library(kableExtra)

# Todo: update links after branch merged in
APD_traits <- read_csv("https://raw.githubusercontent.com/traitecoevo/APD/master/APD_traits.csv", show_col_types = FALSE)
APD_categorical_values <- read_csv("https://raw.githubusercontent.com/traitecoevo/APD/master/APD_categorical_values.csv", show_col_types = FALSE)

```

Metadata fields (annotation properties) documented for each trait include:

```{r, output=FALSE}

names(APD_traits)
```

```{r, echo=FALSE}

names(APD_traits) %>% 
  as.data.frame() %>% 
  rename(field = 1) %>%
  kableExtra::kable(format = "markdown")
```

## Extracting terms based on specific fields

The following examples show how to extract terms based on specific fields in the APD.

### Structure measured

To determine the possible values for `structure_measured`:

```{r, output=FALSE}

structure_measured <- 
  APD_traits %>%
  dplyr::select(structure_measured) %>%
  dplyr::mutate(structure_measured = (stringr::str_split(structure_measured, "; "))) %>%
  tidyr::unnest_longer("structure_measured") %>%
  dplyr::distinct()
```

```{r, echo=FALSE}

structure_measured  %>%
  kableExtra::kable(format = "markdown")
```

You can then select the subset of traits that relate to a specific structure.

For instance, to extract traits relating to seeds:

```{r, output=FALSE}
seed_traits <- 
  APD_traits %>%
  dplyr::filter(stringr::str_detect(structure_measured, "seed"))
```

The first 10 traits on this list are:

```{r, echo=FALSE}
seed_traits  %>%
  select(trait, label, structure_measured) %>%
  slice(1:10) %>%
  kableExtra::kable(format = "markdown")
```

### Trait groupings

To extract all traits within one of the defined trait groupings.

First, generate a list of `trait_grouping` terms:

```{r, output=FALSE}
trait_groupings <- 
  APD_traits %>%
  dplyr::select(trait_groupings) %>%
  dplyr::mutate(trait_groupings = (stringr::str_split(trait_groupings, "; "))) %>%
  tidyr::unnest_longer("trait_groupings") %>%
  dplyr::distinct()
```

Items 20:29 on this list are:

```{r, echo=FALSE}
trait_groupings %>% 
  dplyr::slice(20:29) %>%
  kableExtra::kable(format = "markdown")
```

Filter the APD for `leaf stomatal complex morphology` traits:

```{r, output=FALSE}
stomatal_traits <- 
  APD_traits %>%
  dplyr::filter(stringr::str_detect(trait_groupings, "leaf stomatal complex morphology trait"))
```

```{r, echo=FALSE}
stomatal_traits  %>%
  select(trait, label, trait_groupings) %>%
  kableExtra::kable(format = "markdown")
```

### Characteristic measured

The term `characteristic_measured` documents "what" is being measured - a mass, length, etc.

Multiple terms may be used, for instance to indicate a trait captures volume and is a ratio:

```{r, output=FALSE}
volume_ratio_traits <- 
  APD_traits %>%
  select(trait, label, characteristic_measured) %>%
  filter(
    str_detect(characteristic_measured, "volume") &
    str_detect(characteristic_measured, "ratio") 
    )
```

```{r, echo=FALSE}

volume_ratio_traits  %>%
  kableExtra::kable(format = "markdown")
```

Or all terms that measure a force:

```{r, output=FALSE}

force_traits <-
  APD_traits %>%
  select(trait, label, characteristic_measured) %>%
  filter(
    str_detect(characteristic_measured, "force") 
    )
```

```{r, echo=FALSE}

force_traits  %>%
  kableExtra::kable(format = "markdown")
```

### Categorical values

The allowed values for categorical traits are output in the second table, `APD_categorical_values`.

You can first extract all metadata fields for the trait from `APD_traits`, then merge in the allowed categorical trait values. Researchers might use this information either when merging together disparate datasets or databases, or to acquire a list of trait values to use when scoring study plants.

For `life_history`:

```{r, output=FALSE}

life_history_values <-
  APD_categorical_values %>%
    filter(trait == "life_history") %>%
    select(-trait, -categorical_trait_identifier, -categorical_trait_synonyms) %>%
    mutate(
      term = "allowed_values_levels",
      value = paste0(allowed_values_levels, ": ", categorical_trait_description)
      ) %>%
    select(term, value)

life_history <- 
  APD_traits %>% 
    filter(trait == "life_history") %>%
    mutate(across(c(1:ncol(APD_traits)), ~ as.character(.x))) %>%
    pivot_longer(cols = 1:ncol(APD_traits)) %>%
    rename(term = name) %>%
    bind_rows(life_history_values) %>%
    filter(!is.na(value))
```

```{r, echo=FALSE}

life_history %>%
  kableExtra::kable(format = "markdown")
```

### Matches to other databases and vocabularies

Informal matches to traits included in databases and unpublished dictionaries are mapped in as `examples`, while matches to published vocabularies and ontologies and mapped using the formal terms, `exact_match`, `close_match`, and `related`\_match\`

To identify all traits that are also in the TRY trait database:

```{r, output=FALSE}

matches_to_TRY <- 
  APD_traits %>%
  select(trait, examples) %>%
  filter(!is.na(examples)) %>%
  mutate(examples = stringr::str_split(examples, "; ")) %>%
  unnest_longer(examples) %>%
  filter(stringr::str_detect(examples, "\\[TRY"))

```

The first 20 of many trait matches:

```{r, echo=FALSE}

matches_to_TRY %>%
  slice(1:20) %>%
  kableExtra::kable(format = "markdown")
```

To obtain matches to other informally published trait matches in dictionary and ontologies, use the following patterns with str_detect:

| Vocabulary / Database    | string to match to |
|--------------------------|--------------------|
| TRY Plant Trait Database | \\\\\[TRY          |
| TOP Thesaurus            | \\\\\[TOP          |
| BIEN                     | \\\\\[BIEN         |
| GIFT                     | \\\\\[GIFT         |
| LEDA                     | \\\\\[LEDA         |
| BROT Database            | \\\\\[BROT         |
| Palm Traits Database     | \\\\\[Palm         |

Slightly different code is required to extract lists of formally published trait definitions for which there are matches:

```{r, output=FALSE}

matches_to_WoodyPlants <- 
  APD_traits %>%
  select(trait, exact_match, close_match, related_match) %>%
  pivot_longer(cols = 2:4) %>%
  filter(!is.na(value)) %>%
  mutate(value = stringr::str_split(value, "; ")) %>%
  unnest_longer(value) %>%
  filter(stringr::str_detect(value, "CO_357"))
```

The first 20 matches to the Woody Plants Ontology are:

```{r, echo=FALSE}

matches_to_WoodyPlants %>%
  slice(1:20) %>%
  kableExtra::kable(format = "markdown")
```

To obtain matches to other formally published trait in ontologies, use the following patterns with str_detect:

| Vocabulary / Database            | string to match to |
|----------------------------------|--------------------|
| Woody Plants Ontology            | CO_357             |
| Plant Trait Ontology (TO)        | obo\\\\/TO\_       |
| Flora Phenotype Ontology (FLOPO) | obo\\\\/FLOPO\_    |
| EnvThes                          | EnvThes            |

### References

As appropriate, references are listed for each trait concept, under `references`.

```{r, output=FALSE}

fluorescence <- 
  APD_traits %>%
  select(trait, label, references) %>%
  filter(str_detect(trait, "fluorescence"))
```

```{r, echo=FALSE}

fluorescence %>% kableExtra::kable(format = "markdown")
```

Alternatively, if you know the doi for a reference andyou can search for traits that reference it.

```{r, output=FALSE}
BT12225 <- 
  APD_traits %>%
  select(trait, label, references) %>%
  filter(str_detect(references, "doi.org/10.1071/BT12225"))
```

The first 15 traits referencing `Pérez-Harguindeguy 2013`:

```{r, echo=FALSE}
BT12225  %>%
  slice(1:15) %>%
  kableExtra::kable(format = "markdown")
```

### Deprecated trait names

For AusTraits users, if you have used an old version of AusTraits, previous names used for trait concepts are documented under `deprecated_trait_name`.

You can extract all trait names that have changed, with 10 of them displayed:

```{r, output=FALSE}
deprecated_trait_names <- 
  APD_traits %>%
  select(trait, label, deprecated_trait_name) %>%
  filter(!is.na(deprecated_trait_name))
```

```{r, echo=FALSE}
deprecated_trait_names %>%
  slice(60:70) %>%
  kableExtra::kable(format = "markdown")

```

Or look up a specific trait name that is no longer used:

```{r, output=FALSE}
deprecated_fire_traits <- 
  APD_traits %>%
  select(trait, label, deprecated_trait_name) %>%
  filter(str_detect(deprecated_trait_name, "fire"))
```

```{r, echo=FALSE}

deprecated_fire_traits %>%
  kableExtra::kable(format = "markdown")
```