# Normalization

:BEGIN norm-control
We use control barcodes that should not change in abundance between samples.
Either they are non-targeting controls or they target genes that are not involved in survival/proliferation.

```{r}
use.for.norm <- logical(nrow(y))
norm.features <- character(0)
```

:BEGIN barcodes
We add a pre-defined set of control barcodes:

```{r}
norm.barcodes <- <%= NORM_BARCODES %>
use.for.norm.barcodes <- rownames(y) %in% unlist(norm.barcodes)
summary(use.for.norm.barcodes)
use.for.norm[use.for.norm.barcodes] <- TRUE
norm.features <- c(norm.features, names(norm.barcodes))
```
:END

:BEGIN genes
We add a pre-defined set of control genes:

```{r}
norm.genes <- <%= NORM_GENES %>
use.for.norm.genes <- SummarizedExperiment::rowData(se)[y$genes$origin, <%= NORM_GENE_FIELD %>] %in% unlist(norm.genes)
summary(use.for.norm.genes)
use.for.norm[use.for.norm.genes] <- TRUE 
norm.features <- c(norm.features, names(norm.genes))
```
:END

:BEGIN types
We add control barcodes based on the row annotation in the `SummarizedExperiment`:

```{r}
norm.types <- <%= NORM_TYPES %>
use.for.norm.types <- SummarizedExperiment::rowData(se)[y$genes$origin, <%= NORM_TYPE_FIELD %>] %in% norm.types
summary(use.for.norm.types)
use.for.norm[use.for.norm.types] <- TRUE
norm.features <- c(norm.features, norm.types)
```
:END

We look at the total number of control barcodes:

```{r}
stopifnot(any(use.for.norm)) # make sure we have at least some controls.
norm.features <- paste(norm.features, collapse=', ')
summary(use.for.norm)
```

:BEGIN tmm
We define a normalization factor for each sample with the TMM method across all control barcodes.
This avoids composition biases due to genuine changes in abundance for barcodes with inadvertent biological activity. 

```{r, fig.wide=TRUE}
ysub <- edgeR::calcNormFactors(y[use.for.norm,,keep.lib.sizes=TRUE])
y$samples$norm.factors <- ysub$samples$norm.factors
norm.meta <- paste0('TMM on controls (', norm.features, ')')
head(y$samples)

par(mfrow=c(1,2))
hist(y$samples$lib.size)
hist(y$samples$norm.factors)
```
:END

:BEGIN lib
We define a normalization factor for each sample based on the sequencing depth (i.e., total counts) across all control barcodes.
Normalization factors are scaled to have a geometric mean of 1 to preserve the scale of the library sizes. 

```{r, fig.wide=TRUE}
ratio <- colSums(y$counts[use.for.norm,,drop=FALSE])/y$samples$lib.size
y$samples$norm.factors <- ratio/exp(mean(log(ratio)))
norm.meta <- paste0('library size on controls (', norm.features, ')')
head(y$samples)

par(mfrow=c(1,2))
hist(y$samples$lib.size)
hist(y$samples$norm.factors)
```
:END 

We inspect the normalization results by creating MD (mean-difference) plots.
Ideally, the log-fold changes for most control barcodes should be centered around zero if the normalization was successful.

```{r plot-md-norm, fig.wide=TRUE, fig.asp=1/3, fig.cap="Mean-difference plots for each sample against the average reference of all other samples. Each plot corresponds to a sample while each point represents a barcode, with control barcodes highlighted in red."}
y.cpm <- edgeR::cpm(y, log=TRUE, prior.count=3)

# Avoid creating too many plots, otherwise the compiled report is too big.
par(mfrow=c(1, 3), mar=c(5.1, 4.1, 4.1, 0.1))
for (i in seq_len(min(9, ncol(y.cpm)))) {
    limma::plotMD(y.cpm, column=i, status=as.integer(use.for.norm))
    abline(h=0, col='red', lty=2)
}
```
:END

:BEGIN norm-all
:BEGIN tmm
We define a normalization factor for each sample with the TMM method across all barcodes.
This avoids composition biases due to genuine changes in abundance for barcodes associated with relevant biological processes.

```{r, fig.wide=TRUE}
ysub <- edgeR::calcNormFactors(y)
y$samples$norm.factors <- ysub$samples$norm.factors
norm.meta <- "TMM on all barcodes"
head(y$samples)

par(mfrow=c(1,2))
hist(y$samples$lib.size)
hist(y$samples$norm.factors)
```
:END

:BEGIN lib
We perform library size normalization across all barcodes.

```{r, fig.wide=TRUE}
y$samples$norm.factors <- rep(1, nrow(y$samples))
norm.meta <- "library size on all barcodes"
head(y$samples)

par(mfrow=c(1,2))
hist(y$samples$lib.size)
hist(y$samples$norm.factors)
```
:END

We inspect the normalization results by creating MD (mean-difference) plots.
Ideally, the log-fold changes for most barcodes should be centered around zero if the normalization was successful.

```{r plot-md-norm, fig.wide=TRUE, fig.asp=1/3, fig.cap="Mean-difference plots for each sample against the average reference of all other samples. Each plot corresponds to a sample while each point represents a barcode."}
y.cpm <- edgeR::cpm(y, log=TRUE, prior.count=3)

# Avoid creating too many plots, otherwise the compiled report is too big.
par(mfrow=c(1, 3), mar=c(5.1, 4.1, 4.1, 0.1))
for (i in seq_len(min(9, ncol(y.cpm)))) {
    limma::plotMD(y.cpm, column=i)
    abline(h=0, col='red', lty=2)
}
```
:END
