# Understanding GEO data formats

This article explains what the NCBI Gene Expression Omnibus (GEO) actually stores, why it is shaped the way it is, and how GEOquery maps each form onto a Bioconductor object. Understanding the formats is the fastest way to predict what `getGEO()` will return and to debug the occasional surprising record.

    library(GEOquery)

## The four GEO entity types

GEO is organized around four accession types, and every GEOquery workflow starts by recognizing which one you have:

Prefix | Entity | What it is
--- | --- | ---
`GPL` | Platform | The array or sequencing platform: the list of probes/features and their annotation.
`GSM` | Sample | One hybridization / sequencing run: the measurements for a single biological sample, plus its metadata.
`GSE` | Series | A study: a set of related `GSM` samples, optionally spanning several platforms.
`GDS` | DataSet | A curated, normalized collection assembled by GEO staff from a `GSE`. Far fewer exist; new submissions do not get one.

A `GSE` is the unit you almost always want. The subtlety is that a single `GSE` can contain samples from more than one platform, which is why `getGEO("GSE...")` returns a list — one element per platform.

## Two file formats, two code paths

GEO exposes the same underlying data in two very different file formats, and GEOquery has a distinct parser for each. Knowing which one you are using explains the object you get back.

### SOFT — the complete, verbose record

SOFT (“Simple Omnibus Format in Text”) is the canonical, loss-less GEO representation. It is a line-oriented text format where metadata lines begin with `!`, `^`, or `#`, and data tables are delimited by `!..._table_begin` / `!..._table_end` markers. SOFT contains everything GEO knows about an entity.

GEOquery parses SOFT into its own S4 classes — `GSE`, `GSM`, `GPL`, `GDS` — that mirror the file structure. You reach the pieces with accessors:

    gsm <- getGEO("GSM11805")
    names(head(Meta(gsm), 8))   # Meta(): the metadata as a named list
    #> [1] "channel_count"      "contact_address"    "contact_city"
    #> [4] "contact_country"    "contact_department" "contact_email"
    #> [7] "contact_fax"        "contact_institute"

    Columns(gsm)                # descriptions of the data-table columns
    #>     Column
    #> 1
    #> 2    VALUE
    #> 3 ABS_CALL
    #>                                                                  Description
    #> 1                                                                   ID_REF =
    #> 2                         MAS 5.0 Statistical Algorithm (mean scaled to 500)
    #> 3 MAS 5.0 Absent, Marginal, Present call  with Alpha1 = 0.05, Alpha2 = 0.065

    head(Table(gsm))            # Table(): the data table as a data.frame
    #>            ID_REF  VALUE ABS_CALL
    #> 1  AFFX-BioB-5_at  953.9        P
    #> 2  AFFX-BioB-M_at 2982.8        P
    #> 3  AFFX-BioB-3_at 1657.9        P
    #> 4  AFFX-BioC-5_at 2652.7        P
    #> 5  AFFX-BioC-3_at 2019.5        P
    #> 6 AFFX-BioDn-5_at 3531.5        P

SOFT is the right choice when you need fields that only exist in the full record. The cost is size and speed: a large `GSE` family SOFT file can be enormous, and parsing it is correspondingly slow.

### Series Matrix — the fast, analysis-ready path

For a `GSE`, GEO also publishes a Series Matrix file: a compact, tab-delimited table with the sample metadata as a header block (`!Sample_...` lines) followed by a single expression matrix (rows = features, columns = samples). This is what most analyses actually need, and it parses orders of magnitude faster than SOFT.

This is the GEOquery default (`GSEMatrix = TRUE`). The result is a Bioconductor `SummarizedExperiment`:

    gse <- getGEO("GSE2553")   # a list, one SummarizedExperiment per platform
    se <- gse[[1]]
    se
    #> class: RangedSummarizedExperiment
    #> dim: 12600 181
    #> metadata(3): experimentData annotation protocolData
    #> assays(1): exprs
    #> rownames(12600): 1 2 ... 12599 12600
    #> rowData names(13): ID PenAt ... LLID Chimeric_Cluster_IDs
    #> colnames(181): GSM48681 GSM48682 ... GSM48860 GSM48861
    #> colData names(30): title geo_accession ... supplementary_file
    #>   data_row_count

    assay(se)[1:5, 1:3]        # the expression matrix
    #>     GSM48681   GSM48682   GSM48683
    #> 1  0.2701103  0.3925373  0.4186763
    #> 2  6.3459203  2.1304703  2.0750533
    #> 3 -0.0918793 -0.2411003 -0.2499943
    #> 4  1.4679053  0.6252703  0.4673843
    #> 5 -0.2817373  0.6492023 -1.4973643

If you ever need a field that the Series Matrix omits, drop to SOFT with `getGEO("GSE...", GSEMatrix = FALSE)`.

### Why `getGEO()` returns different classes

This is the most common source of confusion, and it follows directly from the formats:

- `GSE` + Series Matrix (default) → a list of `SummarizedExperiment` (or `ExpressionSet` with `returnType = "ExpressionSet"`)
- `GSE` + SOFT (`GSEMatrix = FALSE`) → a single `GSE` S4 object
- `GSM` / `GPL` / `GDS` → the corresponding S4 object

When in doubt, check `class()` and remember that the GSE-matrix path always returns a list because a study may span platforms.

## ExpressionSet vs. SummarizedExperiment

`SummarizedExperiment` is the modern Bioconductor standard and the substrate that single-cell (`SingleCellExperiment`) and spatial (`SpatialExperiment`) classes build on; it is now the GEOquery default. `ExpressionSet` (from Biobase) is the historical container, still available on request:

    # default: SummarizedExperiment
    se_list <- getGEO("GSE2553")
    # opt back into the legacy ExpressionSet:
    eset_list <- getGEO("GSE2553", returnType = "ExpressionSet")
    # or convert an existing ExpressionSet without re-downloading:
    se <- as_SummarizedExperiment(eset_list[[1]])

The accessor vocabulary differs — `assay()`/`colData()`/`rowData()` for `SummarizedExperiment` versus `exprs()`/`pData()`/`fData()` for `ExpressionSet` — so pick one and stay consistent.

## Supplementary files: where everything else lives

Series Matrix and SOFT cover processed expression tables, but a great deal of GEO data — raw reads, processed counts, single-cell matrices, peak calls, images — is attached as supplementary files that GEO does not parse and whose format it does not constrain. GEOquery lists and downloads them but, by design, does not try to interpret arbitrary content:

    # see what is attached, without downloading
    getGEOSuppFiles("GSE63137", fetch_files = FALSE)
    #>                                                               fname
    #> 1                   GSE63137_ATAC-seq_PV_neurons_HOMER_peaks.bed.gz
    #> 2                  GSE63137_ATAC-seq_VIP_neurons_HOMER_peaks.bed.gz
    #> 3           GSE63137_ATAC-seq_excitatory_neurons_HOMER_peaks.bed.gz
    #> 4   GSE63137_ChIP-seq_H3K27ac_excitatory_neurons_SICER_peaks.bed.gz
    #> 5  GSE63137_ChIP-seq_H3K27me3_excitatory_neurons_SICER_peaks.bed.gz
    #> 6   GSE63137_ChIP-seq_H3K4me1_excitatory_neurons_SICER_peaks.bed.gz
    #> 7   GSE63137_ChIP-seq_H3K4me3_excitatory_neurons_SICER_peaks.bed.gz
    #> 8                         GSE63137_MethylC-seq_DMRs_methylpy.txt.gz
    #> 9                  GSE63137_MethylC-seq_PV_neurons_UMRs_LMRs.txt.gz
    #> 10                GSE63137_MethylC-seq_VIP_neurons_UMRs_LMRs.txt.gz
    #> 11         GSE63137_MethylC-seq_excitatory_neurons_UMRs_LMRs.txt.gz
    #> 12                                                 GSE63137_RAW.tar
    #>                                                                                                                                 url
    #> 1                   https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ATAC-seq_PV_neurons_HOMER_peaks.bed.gz
    #> 2                  https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ATAC-seq_VIP_neurons_HOMER_peaks.bed.gz
    #> 3           https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ATAC-seq_excitatory_neurons_HOMER_peaks.bed.gz
    #> 4   https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ChIP-seq_H3K27ac_excitatory_neurons_SICER_peaks.bed.gz
    #> 5  https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ChIP-seq_H3K27me3_excitatory_neurons_SICER_peaks.bed.gz
    #> 6   https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ChIP-seq_H3K4me1_excitatory_neurons_SICER_peaks.bed.gz
    #> 7   https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_ChIP-seq_H3K4me3_excitatory_neurons_SICER_peaks.bed.gz
    #> 8                         https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_MethylC-seq_DMRs_methylpy.txt.gz
    #> 9                  https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_MethylC-seq_PV_neurons_UMRs_LMRs.txt.gz
    #> 10                https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_MethylC-seq_VIP_neurons_UMRs_LMRs.txt.gz
    #> 11         https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_MethylC-seq_excitatory_neurons_UMRs_LMRs.txt.gz
    #> 12                                                 https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63137/suppl/GSE63137_RAW.tar

This matters enormously for sequencing-era data. RNA-seq counts and all single-cell data live here, not in the Series Matrix — which is why those have dedicated entry points (see the RNA-seq and single-cell articles).

## Where to go next

- Finding and downloading data — search GEO, control `getGEO()`, and cache downloads.
- Single-cell data from GEO — why single-cell lives in supplementary files and how to load it into a `SingleCellExperiment`.
- From GEO to downstream analysis — taking a GEOquery object into differential expression and beyond.