# Installation

## First
# install.packages('tidyMicro')
## Or
# devtools::install_github("CharlieCarpenter/tidyMicro")

library(tidyMicro); library(magrittr)

# tidyMicro

The first step is to read in your table(s) of operational taxonomic unit (OTU) counts and the clinical data for your subjects. The tidyMicro package contains 4 OTU tables from premature infants who required mechanical ventilation and had tracheal aspirates samples collected at 7, 14, and 21 days of age (+/- 48 hours). Each infant had bronchopulmonary dysplasia severity classified as mild, moderate, or severe. The OTU tables come from the phylum, class, order, and family level classifications. The corresponding clinical data is also included. The data are published here.

## Loading OTU tables
data(bpd_phy, package = "tidyMicro")
data(bpd_cla, package = "tidyMicro")
data(bpd_ord, package = "tidyMicro")
data(bpd_fam, package = "tidyMicro")

data(bpd_clin, package = "tidyMicro")

The clinical data must have a column of unique sequencing libraries named “Lib” that match the column names of your OTU table(s).

The OTU tables should be formatted as we see below: 1. The first column contains the OTU names with taxonomic rank delineated by “/” 2. The following column names are the unique sequencing libraries 3. The rest of the table is populated with sequencing counts.

OTU_Name T0003 T0004 T0007 T0008 T0011
2 Bacteria 0 1 10 0 14323
3 Acidobacteria/Acidobacteria 0 0 0 0 0
4 Actinobacteria/Acidimicrobiia 0 0 0 0 0
5 Actinobacteria/Actinobacteria 6 48 0 6 1
6 Actinobacteria/Coriobacteriia 5 0 0 0 2
7 Actinobacteria/Nitriliruptoria 0 0 0 0 0

## tidy_micro

Once the OTU table is formatted in this way the tidy_micro function is fairly simple to use. There are three possible ways to merge your OTU table(s) and clinical data into a “micro_set”.

1. Provide a single OTU table and the name separately
2. Provide an unnamed list of your OTU tables and provide the OTU names separately
1. Names provided will be the “Table” column in your final micro_set
1. Order of names and OTU tables should match up
3. Provide a named list of your OTU tables
1. Names of OTU tables will be the “Table” column in your final micro_set
## 1. Single OTU table
micro.set <- tidy_micro(otu_tabs = bpd_cla,    ## OTU Table
tab_names = "Class",   ## OTU Names (Ranks)
clinical = bpd_clin)   ## Clinical Data

## 2. Unnamed List
otu_tabs <- list(bpd_phy, bpd_cla, bpd_ord, bpd_fam)
tab_names <- c("Phylum", "Class", "Order", "Family")

micro.set <- tidy_micro(otu_tabs = otu_tabs,   ## OTU Table
tab_names = tab_names, ## OTU Names (Ranks)
clinical = bpd_clin)   ## Clinical Data

## 3. Named List
otu_tabs <- list(Phylum = bpd_phy, Class = bpd_cla,
Order = bpd_ord, Family = bpd_fam)

micro.set <- tidy_micro(otu_tabs = otu_tabs,  ## OTU Table
clinical = bpd_clin)  ## Clinical Data

The default for this function is to keep only the libraries found in each of the OTU tables’ column names and the sequencing library column of your clinical data. You will receive a warning if some libraries are excluded during the merge. This option can be turned off by setting the option complete_clinical = FALSE.

Below are the first 6 rows and 12 columns of the data frame we created with tidy_micro. The first 4 columns are the OTU table names, the subject/library names, the taxa names, and the sequencing depths (Total). Columns 4-8 are calculated from the given OTU table: bin is a binary variable for presence or absence of taxa in that subject/library, cts is the original count given, clr is centered log-ratio transformed counts, and ra is the relative abundance, $$RA = \frac{Taxa\ Count}{Seq\ Depth}$$, of the taxa. A small amount (1/sequencing depth) is added to each library’s taxa count before the clr transformation in order to avoid $$log(0) = -\infty$$. The following columns contain the supplied clinical data.

Table Lib Taxa Total bin cts clr ra study_id weight gender gestational_age
Phylum T0003 Bacteria 47490 0 0 -10.8545090 0.0000000 0127Y 0.7 M 25
Phylum T0003 Acidobacteria 47490 0 0 -10.8545090 0.0000000 0127Y 0.7 M 25
Phylum T0003 Actinobacteria 47490 1 11 2.3116626 0.0231628 0127Y 0.7 M 25
Phylum T0003 Bacteroidetes 47490 1 57 3.9568171 0.1200253 0127Y 0.7 M 25
Phylum T0003 Candidate-division-SR1 47490 0 0 -10.8545090 0.0000000 0127Y 0.7 M 25
Phylum T0003 Candidate-division-TM7 47490 1 1 -0.0862135 0.0021057 0127Y 0.7 M 25

All of the following functions rely on this format of the data set, so this must be the first step for any use of this pipeline.

# Data Exploration

For simplicity, we will continue using only the sequencing information from day 7.

micro.set %<>% filter(day == 7)

Several standard plots useful for data exploration are available within tidy.micro.

## Taxa Summaries

The function taxa_summary will supply a table of useful descriptive statistics. It will output a table containing group counts, the percent of subjects with $$RA=0$$, the RA mean, RA median, RA standard deviation, RA IQR, and several RA percentiles. You can control the taxa information summarized using the obj argument. You can stratify by categorical variables using the ... argument. If the table argument is left to NULL, taxa_summary will summarize all tables within your micro_set.

taxa_summary(micro.set, table = "Phylum") %>%
knitr::kable()
Table Taxa n Percent_0 Mean SD Median IQR Percentile_5th Percentile_10th Percentile_25th Percentile_75th Percentile_90th Percentile_95th
Phylum Acidobacteria 24 100.00 0.0000000 0.0000000 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.00
Phylum Actinobacteria 24 0.00 0.3341367 1.0115688 0.0257123 0.0765905 0.00 0.00 0.01 0.09 0.45 1.65
Phylum Bacteria 24 33.33 4.4180972 20.2317256 0.0048883 0.0160987 0.00 0.00 0.00 0.02 0.07 5.64
Phylum Bacteroidetes 24 0.00 0.1990559 0.1534557 0.1731676 0.1453758 0.01 0.02 0.10 0.24 0.43 0.48
Phylum Candidate-division-SR1 24 91.67 0.0001979 0.0007042 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.00
Phylum Candidate-division-TM7 24 41.67 0.0040695 0.0058673 0.0029175 0.0042201 0.00 0.00 0.00 0.00 0.01 0.02
Phylum Cyanobacteria 24 87.50 0.0006640 0.0020834 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.00
Phylum Deinococcus-Thermus 24 100.00 0.0000000 0.0000000 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.00
Phylum Firmicutes 24 0.00 70.9380828 39.5968844 96.6334919 56.0562921 0.42 2.49 43.34 99.39 99.67 99.71
Phylum Fusobacteria 24 4.17 0.0311693 0.0276454 0.0194823 0.0274130 0.01 0.01 0.01 0.04 0.07 0.08
Phylum Proteobacteria 24 0.00 5.1187023 20.4554034 0.1416012 0.1792101 0.01 0.02 0.04 0.22 0.88 16.65
Phylum Spirochaetae 24 87.50 0.0008745 0.0026564 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.01
Phylum Synergistetes 24 100.00 0.0000000 0.0000000 0.0000000 0.0000000 0.00 0.00 0.00 0.00 0.00 0.00
Phylum Tenericutes 24 4.17 18.9420583 34.3059209 0.0268098 17.2474957 0.00 0.00 0.01 17.26 83.10 92.16
Phylum Unclassified 24 20.83 0.0128916 0.0177126 0.0061030 0.0094733 0.00 0.00 0.00 0.01 0.04 0.05

## PCA and PCoA Plots

### Principal Components

micro_pca will calculate principle components on the centered log ratio transformation of the taxa counts using the prcomp function from the stats package. Scaling the taxa counts to a unit variance is the default option, and recommended, but this can be changed using scaled = F. The components are then plotted using the ggbiplot function.

We need to specify the OTU table we’d like to work with using the table argument. This is consistent throughout the pipeline. Different tables can contain different the taxonomic ranks for analysis as it does in this walk through (e.g. phylum, family, or genus level data), or the different tables could also reflect other important differences such as sampling site (e.g. skin, nasal, gut, etc).

micro.set %>% micro_pca(table = "Family",   ## Taxonomic table of interest
grp_var = bpd1, ## A factor variable for colors
legend_title = "BPD Severity")      

### Principal Coordinates

Principle components calculated from a dissimilarity matrix are called “principle coordinates”. If we supply a dissimilarity or distance matrix, e.g. a beta diversity, micro_pca will output a principal coordinate plot. Principle coordinate analysis (PCoA) and principle component analysis (PCA) are both excellent exploratory plots, and are the same process mathematically. However, PCoA is more appropriate than PCA when data are missing or when there are fewer subjects than there are dimensions of the feature space, as is often the case in microbiome (or any omics) analysis.

bray_beta <- micro.set %>%  ## Calculating dissimilarity
beta_div(table = "Family")

micro.set %>%
micro_pca(dist = bray_beta, ## Beta diversity
grp_var = bpd1, ## A factor variable for colors
legend_title = "BPD Severity")      
## PCoA plot created

### Longitudinal PCA/PCoA

The following methods were established in microbiome data by K. Williamson and is based off of the Tucker3 loss function. These are tools for dealing with repeated measures within the ordination framework.

long_micro <- tidy_micro(otu_tabs = otu_tabs,  ## OTU tables
clinical = bpd_clin)  ## clinical Data
## Contains 74 libraries from OTU files.
## Summary of sequencing depth:
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
##    8851   24938   33314   36650   43590   97408

#### Three Mode PCA

Three Mode PCA is a way of creating Principle component plots while controlling for correlations between observations. We can interpret them the same as we would any PCA plot. Unfortunately we aren’t seeing much separation between our groups.

The function requires your micro_set to have a column that indicates your subject names and a column that indicates your time points. The function also requires a grouping variable.

long_micro %>%
three_mode(table = "Family", group = bpd1, subject = study_id,
time_var = day, main = "ThreeMode PCA",
subtitle = "3 Time Points", legend_title = "BPD Severity")
## Warning: Subjects are not consistent across time points.
## Only complete cases will be used.
## Found 3 time points and 15 subjects with complete cases.
## Rational ORTHONORMALIZED start
## Tucker3 function value at start is  9.09494701772928e-13
## Tucker3 function value is 9.09494701772928e-13 after 3 iterations
## Fit percentage is 100 %
## Procedure used 0.03 seconds

Notice the warning message that pops up. The function will automatically pull out the subjects that are in every time point and only make plots with their information.

#### 3D PCA

These 3D plots are a method for making PCA or PCoA plots for data with repeated measures. They will help us view trends within each time point, or help us understand when large changes in community composition occur. For instance, we may be able to detect linear trends within a single time point, or see that composition makes a large departure at a specific time point. Unfortunately, we do not see dramatic shifts in composition from week to week in this data set.

In order to make the function work properly we need to supply a subject and time column.

long_micro %>%
pca_3d(table = "Family", time_var = day, subject = study_id,
type = "PCoA", legend.title = "Day")
## Warning: Subjects are not consistent across time points.
## Only complete cases will be used.
## Found 3 time points and 15 subjects with complete cases.
## Tucker3 function value at start is  10716.5212840935
## Tucker3 function value is 0 after 2 iterations
## Fit percentage is 100 %
## Procedure used 0.02 seconds

## Bar Charts

Stacked bar charts of taxa RA are very standard visualizations for microbiome research. We can create bar charts of the raw RA stratified by categorical variable(s) of interest. As before, we need to specify the OTU table we want to plot.

ra_bars(micro.set,         ## Dataset
table = "Phylum",  ## Table we want
bpd1,              ## Variable of interest
ylab = "% RA",
xlab = "BPD",
main = "Stacked Bar Charts")

As you can see, several of these phylum are very low abundance within the cohort, not to mention how quickly our plot legend would overflow if we tried to plot all of our taxa orders or families. Several of our functions include options to aggregate low taxa counts together in order to clean up these plots. 1. Specify how many taxa will be named and used in the stacked bar charts using top_taxa. This option will take $$X$$ taxa with the highest average RA and aggregate all others into an “Other” category. 2. Use the RA option to aggregate all taxa with RA below your cutoff into an “Other” category.

Only one of these can be used at a time. If a particular taxa is of interest you can use specific_taxa to pull any taxa out of the “Other” category if it doesn’t meet the bar set by either top_taxa or RA.

ra_bars(micro.set,          ## Dataset
table = "Phylum",  ## Table we want
bpd1,              ## Variable of interest
top_taxa = 3,
RA = 0,
specific_taxa = c("Actinobacteria", "Bacteroidetes"),
ylab = "% RA", xlab = "BPD", main = "Stacked Bar Charts")

ra_bars is flexible enough to include multiple factors in the ... argument.

ra_bars(micro.set,           ## Dataset
table = "Phylum",    ## Table we want
bpd1, gender,        ## Variables of interest
top_taxa = 6,
ylab = "% RA", xlab = "BPD by Sex",
main = "Stacked Bar Charts") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))

This function can also create subject level bar charts by using “Lib” as your grouping variable.

ra_bars(micro.set,          ## Dataset
table = "Phylum",   ## Table we want
Lib,                ## Variable of interest
top_taxa = 6,
ylab = "% RA", xlab = "Library", main = "Stacked Bar Charts") +
theme(axis.text.x = element_text(angle = 90, hjust = 1))

Or just create a single stacked bar chart for the entire cohort by ignoring the ... argument.

ra_bars(micro.set,            ## Dataset
table = "Phylum",     ## Table we want
top_taxa = 6,
ylab = "% RA", main = "Stacked Bar Charts")

Note that you can manipulate the output plots by adding on geoms as we did in ra_bars. Most plotting functions in this pipeline will return a ggplot that you can manipulate through additional geom just like any other ggplot.

ra_bars(micro.set,            ## Dataset
table = "Phylum",     ## Table we want
top_taxa = 6,
main = "Manipulated Stacked Bar Charts") +

theme_dark() +
coord_flip() +
theme(legend.title = element_text(color = "blue", size = 20),
legend.text = element_text(color = "red"))

## Box Plots

We can make box plots of taxa relative abundance stratified by some categorical variable using the function taxa_boxplot.

staph <- "Firmicutes/Bacilli/Bacillales/Staphylococcaceae"

taxa_boxplot(micro.set,      ## Our dataset
taxa = staph,   ## Taxa we are interested in
bpd1,           ## Variable of interest
xlab = "BPD",
ylab = "Relative Abundance",
main = "Box Plot") 

We can plot other taxa information besides the relative abundance (such as the raw counts) by specifying the y argument. This function also allows for multiple variables of stratification, and will give you the interaction of all categorical variables given. There is an apparent trend here, but be aware that raw counts are rarely the information of interest for analysis. Relative abundance or the centered log ratio is almost always more meaningful.

taxa_boxplot(micro.set,        ## Our dataset
taxa = staph,     ## Taxa we are interested in
y = clr,          ## Making Boxplot of CLR
bpd1, gender,     ## Variables of interest
ylab = "Staphylococcaceae CLR",
main = "Box plot", subtitle = "Subtitle") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))

## Correlation Plots

Correlations between taxa RA and continuous variables can be useful descriptives during data exploration. The functions cor_heatmap and cor_rocky_mtn will calculate correlation between the selected variable(s) and taxa information. The taxa information (counts, CLR, or RA) can be controlled through the y argument, and the type of correlation can be controlled through the cor_type argument. We recommend using either CLR transformed counts with Spearman’s rank based correlation or taxa RA with Kendall’s rank based correlation. Since the large number of 0 counts often present will create many ties Kendall’s correlation more appropriate for taxa RA.

### Heat Maps

Correlations for each taxa within the selected table are plotted in a heat map for one or several continuous variables. This style of non-symmetric heat map is sometimes called a “lasagna plot.”

micro.set %>% cor_heatmap(table = "Class", weight, gestational_age)

### Rocky Mountain Plots

The rocky mountain plot shows the correlation for each taxa within the selected table displayed along the horizontal axis. They will be color coded by the phylum they belong to (characters before the first “/” in the taxa name), and taxa with a correlation magnitude greater than or equal to a specified cutoff (cor_label) will be labeled. This function uses the same options for cor_type and y as cor_heatmap.

micro.set %>% cor_rocky_mtn(table = "Class",
weight,
cor_label = 0.3)

# Alpha Diversities

alpha_div calculates Good’s Coverage and the following alpha diversities through rarefied bootstrap samples and attaches them as columns to the given micro_set:

• Richness: $$S_{obs}$$ and Chao1
• Evenness: Shannon’s and Simpsons E
• Diversity: Shannon’s H and Simpsons D

It will either calculate the alpha diversity based on the specified table and attach it within each table of your micro_set, or calculate the alpha diversity of each table if no table is specified. Either way the alpha diversities will be attached as if they were a part of your clinical data. If your micro_set contains OTU tables of different taxonomic ranks, we recommend only calculating and analyzing the alpha diversity of the lowest rank.

This step can be computationally intensive. Lowering the number of bootstrap iterations with the iter argument can speed this up. We recommend removing subjects with poor sequencing as well (for all analyses). There are two options for removing subjects with poor sequencing:

1. min_depth: Remove libraries with sequencing depths (Total) below min_depth.
2. min_goods: Remove libraries with Good’s Coverage below min_goods.
micro_alpha <- alpha_div(micro.set,
table = "Family",  ## Table of interest
min_depth = 5000,  ## Requires a Seq Depth of 5000 to be included
min_goods = 80)    ## Requires a Good's coverage of %80 
## Warning in alpha_div(micro.set, table = "Family", min_depth = 5000, min_goods = 80): The minimum Library size is 8851.
## No libraries dropped based on sequencing depth or Good's coverage.

## Regress on Alpha Diversity

Once alpha diversities are calculated, standard regression can be used to analyse alpha diversities. micro_alpha_reg is a simple wrapper function that will run linear regression on each alpha diversity with your specified covariate pattern within the specified rank. Covariates included in the function will be added together in your covariate structure. For instance, if we include Group, Age, Sex, the function will fit $\hat{alpha} \sim Group + Age + Sex.$ You can also include an interaction by typing Group * Age.

The function will output a summary table for every model containing a column for the alpha diversity measure, the model coefficients, the $$\beta$$ estimates, the standard errors of that estimate, the test statistic, the p-value, and a 95% confidence interval for the $$\beta$$ estimates.

micro_alpha %>%
micro_alpha_reg(table = "Family", bpd1, gender) %>%
knitr::kable()
##    Alpha_Div         Coef  Beta std.error  t.stat p.value              CI_95
## 1       Sobs  (Intercept) 17.88      2.83  6.3163  0.0000 (11.9735, 23.7817)
## 2       Sobs bpd1Moderate -2.08      3.24 -0.6404  0.5292    (-8.844, 4.689)
## 3       Sobs   bpd1Severe -0.58      3.40 -0.1699  0.8668  (-7.6796, 6.5226)
## 4       Sobs      genderM -1.15      2.36 -0.4880  0.6308  (-6.0671, 3.7664)
## 5      Chao1  (Intercept) 21.85      3.27  6.6895  0.0000 (15.0392, 28.6684)
## 6      Chao1 bpd1Moderate -1.15      3.74 -0.3067  0.7623  (-8.9582, 6.6618)
## 7      Chao1   bpd1Severe  0.87      3.93  0.2209  0.8274   (-7.3284, 9.064)
## 8      Chao1      genderM -1.78      2.72 -0.6554  0.5197   (-7.4581, 3.892)
## 9   ShannonE  (Intercept)  0.10      0.06  1.6796  0.1086  (-0.0237, 0.2195)
## 10  ShannonE bpd1Moderate  0.01      0.07  0.1395  0.8905    (-0.13, 0.1487)
## 11  ShannonE   bpd1Severe  0.04      0.07  0.6123  0.5472  (-0.1033, 0.1892)
## 12  ShannonE      genderM -0.06      0.05 -1.1641  0.2581  (-0.1578, 0.0448)
## 13  ShannonH  (Intercept)  0.41      0.24  1.7039  0.1039   (-0.0912, 0.905)
## 14  ShannonH bpd1Moderate  0.03      0.27  0.0948  0.9254   (-0.545, 0.5968)
## 15  ShannonH   bpd1Severe  0.18      0.29  0.6117  0.5477  (-0.4235, 0.7748)
## 16  ShannonH      genderM -0.24      0.20 -1.1990  0.2445  (-0.6533, 0.1764)
## 17  SimpsonD  (Intercept)  1.17      0.22  5.3593  0.0000   (0.7139, 1.6239)
## 18  SimpsonD bpd1Moderate  0.13      0.25  0.5038  0.6199  (-0.3955, 0.6474)
## 19  SimpsonD   bpd1Severe  0.27      0.26  1.0374  0.3119  (-0.2751, 0.8194)
## 20  SimpsonD      genderM -0.22      0.18 -1.2277  0.2338  (-0.6019, 0.1559)

# Beta Diversities

beta_div is a simple wrapper around the vegdist function. It requires you to specify the OTU table and the method of beta diversity you want calculated.

## Beta Diversity
bray <- beta_div(micro.set, table = "Class", method = "bray")

## Heat Map

The beta_heatmap function can be used to create a heat map of your calculated beta diversities ordered by some categorical variable of interest.

bray %>%
beta_heatmap(micro.set, bpd1)

## PERMANOVA

The vegan package contains a useful function called adonis2 for running a PERMANOVA test. PERMANOVA is a permutation based ANOVA using a distance matrix as your response. micro_PERMANOVA is simply a wrapper function to make working within your micro_set easy. The output is analogous to that of a standard ANOVA. Please note that the F statistic given is a “pseudo-F” statistic from a “pseudo-F test” since it is based off of a permutation distribution not a true F distribution.

micro_PERMANOVA(micro.set, ## micro_set to pull covariates from
bray,      ## Beta diversity matrix (or any distance matrix)
method = "bray",   ## method used to calculate the beta diversity
bpd1, mom_ethncty_2)  ## Covariates
## Permutation test for adonis under reduced model
## Terms added sequentially (first to last)
## Permutation: free
## Number of permutations: 999
##
## vegan::adonis2(formula = f, data = micro_set, permutations = nperm, method = method)
##               Df SumOfSqs      R2      F Pr(>F)
## bpd1           2   0.2157 0.04301 0.5041  0.864
## mom_ethncty_2  1   0.5207 0.10381 2.4335  0.052 .
## Residual      20   4.2797 0.85318
## Total         23   5.0162 1.00000
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

# Taxa Models and Plots

## Filtering on a subset

There are several reasons you might want to aggregate rare taxa counts into an “Other” category. Some plots and figures (e.g. corHeatmap, corRockyMtn, ) will look nicer without all of the extra names cluttering the legend, some taxa might not be fully classified, or their models might be unstable and therefore a waste of computation time. We can use the otu_filter function to apply filtering requirements to each table within the micro_set. There are three functions built in to filter out taxa with low counts.

1. prev_cutoff is a prevalence cutoff where $$X%$$ of subjects must have this taxa present or it will be included in the “Other” category.
2. ra_cutoff is a relative abundance (RA) cutoff where at least one subject must have a RA above the cutoff or the taxa will be included in the “Other” category.
3. exclude_taxa can be used to specify any taxa that you would like to make sure are included in the “Other” category. For instance, our OTU tables have taxa that are named “Unclassified” that we’d like to exclude. This can be a character string of any length.
1. Please note that the taxa specified with exclude_taxa will be filtered out of every OTU table given. To avoid this, you can make a subset of the data before filtering.
## Taxa names "Bacteria" are essentially unclassified
exclude_taxa <- c("Unclassified", "Bacteria")

micro.filt <- micro.set %>%
otu_filter(prev_cutoff = 1,               ## Prevalence cutoff
ra_cutoff = 0.1,               ## Relative abundance cutoff
exclude_taxa = exclude_taxa)   ## Uninteresting taxa
## Filter for Class counts
## Found 'Unclassified' category in input data.
## Created new 'Other' category.
## Found 'Bacteria' category in input data.
## Found 34 OTUs.
## Collapsed 2 OTUs into 'Other' in OTU table.
## Converted 61662 counts to 'Other' otu category.
## Remaining OTUs: 33 (Including 'Other').
## Prevalence cutoff: 1% (i.e., at least 1% of libaries must be represented to keep OTU)
## Found 33 OTUs.
## Found 'Other' category in input data.
## Collapsed 12 OTUs into 'Other' in OTU table.
## Converted 0 counts to 'Other' in otu category.
## Remaining OTUs: 21 (Including 'Other').
## Relative abundance cutoff: 0.1% (i.e., at least one library must have RA > 0.1% to keep OTU).
## Found 21 OTUs.
## Found 'Other' category in input data.
## Collapsed 12 OTUs into 'Other' in OTU table.
## Converted 337 counts to 'Other' otu category.
## Remaining OTUs: 9 (Including 'Other').
## Filter for Family counts
## Found 'Unclassified' category in input data.
## Created new 'Other' category.
## Found 'Bacteria' category in input data.
## Found 116 OTUs.
## Collapsed 2 OTUs into 'Other' in OTU table.
## Converted 61662 counts to 'Other' otu category.
## Remaining OTUs: 115 (Including 'Other').
## Prevalence cutoff: 1% (i.e., at least 1% of libaries must be represented to keep OTU)
## Found 115 OTUs.
## Found 'Other' category in input data.
## Collapsed 45 OTUs into 'Other' in OTU table.
## Converted 0 counts to 'Other' in otu category.
## Remaining OTUs: 70 (Including 'Other').
## Relative abundance cutoff: 0.1% (i.e., at least one library must have RA > 0.1% to keep OTU).
## Found 70 OTUs.
## Found 'Other' category in input data.
## Collapsed 50 OTUs into 'Other' in OTU table.
## Converted 751 counts to 'Other' otu category.
## Remaining OTUs: 20 (Including 'Other').
## Filter for Order counts
## Found 'Unclassified' category in input data.
## Created new 'Other' category.
## Found 'Bacteria' category in input data.
## Found 62 OTUs.
## Collapsed 2 OTUs into 'Other' in OTU table.
## Converted 61662 counts to 'Other' otu category.
## Remaining OTUs: 61 (Including 'Other').
## Prevalence cutoff: 1% (i.e., at least 1% of libaries must be represented to keep OTU)
## Found 61 OTUs.
## Found 'Other' category in input data.
## Collapsed 21 OTUs into 'Other' in OTU table.
## Converted 0 counts to 'Other' in otu category.
## Remaining OTUs: 40 (Including 'Other').
## Relative abundance cutoff: 0.1% (i.e., at least one library must have RA > 0.1% to keep OTU).
## Found 40 OTUs.
## Found 'Other' category in input data.
## Collapsed 24 OTUs into 'Other' in OTU table.
## Converted 525 counts to 'Other' otu category.
## Remaining OTUs: 16 (Including 'Other').
## Filter for Phylum counts
## Found 'Unclassified' category in input data.
## Created new 'Other' category.
## Found 'Bacteria' category in input data.
## Found 15 OTUs.
## Collapsed 2 OTUs into 'Other' in OTU table.
## Converted 61662 counts to 'Other' otu category.
## Remaining OTUs: 14 (Including 'Other').
## Prevalence cutoff: 1% (i.e., at least 1% of libaries must be represented to keep OTU)
## Found 14 OTUs.
## Found 'Other' category in input data.
## Collapsed 3 OTUs into 'Other' in OTU table.
## Converted 0 counts to 'Other' in otu category.
## Remaining OTUs: 11 (Including 'Other').
## Relative abundance cutoff: 0.1% (i.e., at least one library must have RA > 0.1% to keep OTU).
## Found 11 OTUs.
## Found 'Other' category in input data.
## Collapsed 4 OTUs into 'Other' in OTU table.
## Converted 50 counts to 'Other' otu category.
## Remaining OTUs: 7 (Including 'Other').

### Important Note on Filtering

Please note that we did not filter before calculating our diversity measures! If we aggregate rare taxa into a single category, this will bias all of our diversity measures. Our Sobs and Choa1 in particular will be greatly biased. If you do not wish to calculate alpha or beta diversities, these filtering options are also available in the initial tidy_micro read in step.

## Named List
otu_tabs <- list(Phylum = phy, Class = cla, Ord = ord, Family = fam)

tidy.filt <- tidy_micro(otu_tabs = otu_tabs,  ## OTU Table
clinical = clin,      ## Clinical Data
prev_cutoff = 1,      ## Prevalence cutoff
ra_cutoff = 1,        ## Relative abundance cutoff
exclude_taxa = exclude_taxa)   ## Uninteresting taxa

## Modeling

### Negative Binomial Models

It is a standard practice to model the relative abundance of each taxa using a negative binomial distribution. The function nb_mods will fit negative binomial models for each taxa within a specified table using the observed counts and the total sequencing counts as an offset. It uses glm.nb from the MASS package to fit these models; the profile likelihood confidence intervals are also calculated through the MASS package using the confint function.

As in micro_alpha_reg and bb_mods, nb_mods will take each variable you specify as a new term to add into the model. For instance, if we include Group, Age, Sex the function will run the model $log(\hat{cts}) \sim \beta_0 + \beta_1 Group + \beta_2 Age + \beta_3 Sex + log(Total).$ The offset of log(Total) will be included automatically and is recommended, however it is possible to remove this term with Offset = FALSE

nb_fam <- micro.filt %>%       ## micro_set
nb_mods(table = "Family",    ## Rank of taxa we want to model
bpd1)                ## The covariate in our model

As in micro_alpha_reg and bb_mods, you can also include interaction terms (such as Age*Sex) as you can with any other model.

## If we wanted the covariates to be bpd1+gender+bpd1*gender we just need input bpd1*gender.

nb_int <- micro.filt %>%
nb_mods(table = "Class", bpd1*gender)

Notice that the function will tell you how many different models converged and how many did not. To explore the convergent models we can access the Convergent_Summary from nb_mods. The convergent summary table gives the taxa name, the model coefficients, the estimated $$\beta s$$, profile likelihood confidence limits, the Z-score, the p-value from a Wald test, an FDR adjusted p-value, and finally an Anova test from a likelihood-ratio test. Below is the summary information from the first two convergent taxa.

nb_fam$Convergent_Summary %>% .[1:6,] %>% knitr::kable() Taxa Coef Beta CI Z P_val FDR_Pval LRT Actinobacteria/Actinobacteria/Actinomycetales/Actinomycetaceae (Intercept) -8.2921218 (-9.5266, -6.4186) -11.1387 0.0000000 0.0000 NA Actinobacteria/Actinobacteria/Actinomycetales/Actinomycetaceae bpd1Moderate -0.4842894 (-2.5126, 1.1272) -0.5550 0.5789104 0.7105 0.5330622 Actinobacteria/Actinobacteria/Actinomycetales/Actinomycetaceae bpd1Severe -0.9958385 (-3.0842, 0.7387) -1.0918 0.2749094 0.4241 0.5330622 Actinobacteria/Actinobacteria/Corynebacteriales/Corynebacteriaceae (Intercept) -5.3792759 (-7.6366, 1.9534) -2.9617 0.0030599 0.0083 NA Actinobacteria/Actinobacteria/Corynebacteriales/Corynebacteriaceae bpd1Moderate -2.2474168 (-9.7445, 1.7989) -1.0590 0.2896069 0.4344 0.0000000 Actinobacteria/Actinobacteria/Corynebacteriales/Corynebacteriaceae bpd1Severe 0.1151231 (-7.4138, 4.5979) 0.0527 0.9579410 0.9599 0.0000000 nb_mods also provides an Estimate_Summary table that contains a table of model summaries for convergent taxa that is more readily exportable into publications. It gives the taxa name, the model coefficients, the exponentiated beta coefficients for the rate ratio, a Wald confidence interval, the Z-score, and FDR_Pvalue. For interaction terms, the rate ratios are calculated taking their main effects into account. That is, rate ratios are the exponentiation sum of interaction and main effect $$\beta$$ coefficients, and confidence intervals are for the exponentiation sum of the $$\beta$$ coefficients. The FDR p-values are for the individual $$\beta$$ coefficients. Below is the summary information from the first two convergent taxa. nb_fam$Estimate_Summary %>% .[1:4,] %>% knitr::kable()
Taxa Coef RR CI_95 Z FDR_Pval
Actinobacteria/Actinobacteria/Actinomycetales/Actinomycetaceae bpd1Moderate 0.6161 (0.1114, 3.4078) -0.5550 0.7105
Actinobacteria/Actinobacteria/Actinomycetales/Actinomycetaceae bpd1Severe 0.3694 (0.0618, 2.2075) -1.0918 0.4241
Actinobacteria/Actinobacteria/Corynebacteriales/Corynebacteriaceae bpd1Moderate 0.1057 (0.0016, 6.7678) -1.0590 0.4344
Actinobacteria/Actinobacteria/Corynebacteriales/Corynebacteriaceae bpd1Severe 1.1220 (0.0156, 80.9383) 0.0527 0.9599

For the non-convergent taxa we can explore their summary information within the RA_Summary from nb_mods. This is a table of summary measures for all taxa (not just the non-convergent taxa) that is stratified by the categorical variables in the models. It includes the counts (n), percent of subjects with counts of 0, the average, median, standard deviation, IQR, percentiles of taxa RA, and finally an indicator for whether or not this taxa’s negative binomial model converged.

### Beta Binomial Models

Another standard practice is to model taxa abundance using a beta binomial distribution. The function bb_mods will fit beta binomial model to each taxa within a specified table using the vglm function from the VGAM package. Confidence intervals are fit using the confintvglm function. The current default for confidence intervals around individual $$\beta$$ parameters are Wald intervals, although this can be change to profile likelihood confidence intervals using CI_method = "profile". Profile likelihoods are much more computationally intensive.

bb_fam <- micro.filt %>%       ## micro_set
bb_mods(table = "Phylum",    ## Table we want to model
bpd1)                ## The covariate in our model

As in micro_alpha_reg and nb_mods, bb_mods will take each variable you specify as a new term to add into the model. For instance, if we include Group, Age, Sex the function will run the model $logit(\hat{RA}) \sim \beta_0 + \beta_1 Group + \beta_2 Age + \beta_3 Sex$

Again, as in micro_alpha_reg and nb_mods, you can also include interaction terms (such as Age*Sex) as you can with any other model.

## If we wanted the covariates to be bpd1+gender+bpd1*gender we just need input bpd1*gender.

bb_int <- micro.filt %>%
bb_mods(table = "Class", bpd1*gender)

## Plotting

The model output from nb_mods and bb_mods can easily cause some information overload. tidyMicro includes several useful functions to visualize the results of all convergent models. They function the same way for both model types, so we will only show examples from our nb_mods output.

### Stacked Bar Charts

We have created functions that will calculate the estimated RA of all taxa based on the convergent models. This gives us the ability to visualize stacked bar charts of taxa RA while controlling for other variables in the model. For instance, if our model is $log(\hat{cts}) \sim \beta_0 + \beta_1 Group + \beta_2 Age + \beta_3 Sex + log(Total),$ we can visualize the estimated differences in RA among different groups while holding Age and Sex constant.

The functions nb_mods and bb_mods will create these stacked bar charts based on the output of from nb_mods and bb_mods, respectively. They requires the name of a covariate in your model and can create plots based on main effects or interactions. If a continuous variable is one of the supplied covariates there are two options for visualization available through the quant_style argument. You can either visualize two points as two separate bars (quant_style = "discrete") or visualize the continuous change (quant_style = "continuous"). By default the functions will use the first and third quartiles of the continuous variable as your endpoints, but this can be changed through the range option. nb_bars and bb_bars also give you the ability to aggregate estimated RA into and “Other” category just like the ra_bars function.

nb_fam %>% nb_bars(bpd1,                 ## Covariate of interest
top_taxa = 5,         ## How many named taxa we want
xlab = "Group",
xaxis = c("1","2","3")) ## Labels

#### Object Flexibility

Including every taxonomic level in your legend often looks cluttered. The nb_bars and bb_bars functions use the Model_Coef data frame from nb_mods and bb_mods output to create their bar charts. You can manipulate the taxa names here to avoid confusion further up in the pipeline.

nb_fam$Model_Coef$Taxa %<>%
stringr::str_split("/") %>%  ## Splitting by "/" in taxa names
lapply(function(x) x[length(x)]) %>% ## selecting piece after the last "/"
unlist ## Unlisting to put back into data frame

## Reordering to put "Other" at the bottom of the legend
## our functions usually do this automatically, but we'll need to do
## this externally since we are messing with the output
non.other <- nb_fam$Model_Coef %>% filter(Taxa != "Other") %>% arrange(Taxa) %>% distinct(Taxa) %>% pull(Taxa) nb_fam$Model_Coef$Taxa <- factor(nb_fam$Model_Coef\$Taxa,
levels = c(non.other, "Other"))

nb_fam %>% nb_bars(bpd1,                 ## Covariate of interest
top_taxa = 5,         ## How many named taxa we want
xlab = "BPD Severity", main = "Cleaner Legend") ## Labels

### Rocky Mountain Plots

The function micro_rocky_mtn displays magnitude of the log FDR adjusted p-values for each of the taxa in nb_mods as vertical bars next to each other along the x-axis. The direction of the bars will be determined by the direction of the estimated relationship. The taxa will be color coded by the phylum they belong to, and taxa that have FRD adjusted p-values for the specified covariate below your desired significance cutoff will be labeled. This significance cutoff is 0.05 by default and can be changed through the alpha argument. You can also turn off the labels with sig_text = FALSE.

## Order level models
nb_ord <- micro.filt %>%       ## micro_set
nb_mods(table = "Order",     ## Rank of taxa we want to model
bpd1)                ## The covariate in our model

nb_ord %>%
micro_rocky_mtn(bpd1, ## Covariate of interest
xlab = "Taxa", main = "Rocky Mountain Plot",
subtitle = "Direction of bar indicates direction of relationship", facet_labels = c("Moderate", "Severe"))

### Forest Plots

micro_forest will create forest plots for a specified covariate from each taxa’s nb_mod or bb_mod. Forest plots display the estimated $$\beta$$ coefficients with their 95% confidence intervals.

## Class level models
nb_cla <- micro.filt %>%       ## micro_set
nb_mods(table = "Class",     ## Rank of taxa we want to model
bpd1)                ## The covariate in our model

nb_cla %>%
micro_forest(bpd1, ## Covariate of interest
main = "Forest Plot for BPD Severity")