Spectra data representation backends

spectra-data-representation-backends

Summary

Design and implement MsBackend classes that provide mass spectrometry data to Spectra objects while avoiding circular dependencies between spectraData() and spectraVariables() methods. This skill bridges MS data storage (in-memory, on-disk, or database) with the Spectra analysis framework.

When to use

You are creating a new backend to expose MS data (m/z, intensity, retention time, and other spectral properties) from a specific storage format or data source (e.g., mzML files, SQL databases, in-memory matrices, or spectral libraries) to Spectra objects. The backend must handle multiple spectra with internally sorted m/z values and provide both accessor and optional replacement methods for spectra variables.

When NOT to use

• You are not creating a new data source type — use an existing backend (MsBackendMemory, MsBackendMzR, MsBackendSql, etc.) if your MS data already fits a supported format.
• Your spectra have missing or unsorted m/z values — backends require m/z values to be sorted increasingly within each spectrum and do not support NA values for m/z.
• You need analysis-level functionality (filtering, processing, visualization) rather than data storage/provision — use the Spectra class directly instead of implementing a new backend.

Inputs

• MS data source (mzML/mzXML/CDF files, SQL database, HDF5 file, in-memory matrix list, or spectral library file)
• Spectra variable metadata (retention time, scan index, precursor m/z, collision energy, etc.)
• Storage backend specification (read-only or read-write mode)

Outputs

• MsBackend subclass instance with implemented spectraData(), spectraVariables(), peaksData(), mz(), and intensity() methods
• DataFrame object containing all spectra variables with m/z and intensity as NumericList columns
• Character vector of spectra variable names (union of core and backend-specific variables)

How to apply

Extend the virtual MsBackend class and implement core methods in this order: (1) backendInitialize() to load/prepare MS data and populate dataStorage and dataOrigin variables; (2) spectraVariables() as a character vector returning the union of core variable names ('mz', 'intensity', 'rtime', 'scanIndex', etc.) and backend-specific column names — this method must NOT call spectraData() to prevent circularity; (3) spectraData() to call fillCoreSpectraVariables() to populate missing core spectra variables into a DataFrame, extract backend-specific columns from internal storage, and combine them using S4Vectors::DataFrame(), with m/z and intensity returned as NumericList columns when requested; (4) peaksData() to extract m/z and intensity as a list of numerical matrices with m/z values sorted increasingly; (5) intensity() and mz() as NumericList accessors; (6) extractByIndex() and backendMerge() for subsetting and combining backends. Ensure spectraData() and spectraVariables() do not call each other to prevent infinite recursion. For read-only backends, implement only accessor methods; for partially read-only backends (e.g., MsBackendMzR), implement replacement methods for spectra variables but not peaks data.

Related tools

• Spectra (R package defining the virtual MsBackend class and providing fillCoreSpectraVariables(), Spectra class for analysis, and utility backends (MsBackendMemory, MsBackendMzR, MsBackendDataFrame)) — https://github.com/RforMassSpectrometry/Spectra
• S4Vectors (Bioconductor package providing DataFrame object for storing and returning combined spectra data with support for NumericList columns)
• MsBackendMzR (Example backend retrieving m/z and intensity on-the-fly from mzML/mzXML/CDF files using mzR package while keeping general spectra variables in memory)
• MsBackendSql (Example backend storing all MS data in SQL database with minimal memory footprint for large datasets) — https://github.com/rformassspectrometry/MsBackendSql
• MsBackendMemory (Example default backend keeping all data in-memory, optimized for fast processing of smaller datasets)

Examples

setClass('MsBackendExample', contains = 'MsBackend', slots = c(data = 'data.frame', peaks = 'list')); setMethod('spectraVariables', 'MsBackendExample', function(object) union(c('mz', 'intensity', 'rtime', 'scanIndex'), names(object@data))); setMethod('spectraData', 'MsBackendExample', function(object, columns = spectraVariables(object)) { df <- fillCoreSpectraVariables(object@data[, intersect(names(object@data), columns)]); df$mz <- IRanges::NumericList(object@peaks[, 1]); df$intensity <- IRanges::NumericList(object@peaks[, 2]); df })

Evaluation signals

• spectraData() returns a DataFrame with all core variables (mz, intensity, rtime, scanIndex) and backend-specific columns; m/z and intensity columns are NumericList objects; no NA values present in m/z columns
• spectraVariables() returns a character vector containing the union of core variable names and backend column names with no duplicates
• No circular function calls: spectraVariables() does not call spectraData() and spectraData() does not call spectraVariables()
• peaksData() returns a list of numerical matrices with m/z values sorted increasingly within each matrix; matrix row count matches spectrum peak count
• extractByIndex() correctly subsets backend to selected spectra; backendMerge() combines multiple backend instances without data loss or reordering

Limitations

• m/z values within each spectrum must be sorted increasingly and cannot contain NA values — unsorted or incomplete peak data will violate backend contract
• Backends can be read-only (e.g., spectral libraries, database queries) or partially read-only (e.g., MsBackendMzR allows spectra variable modification but not peak data modification) — implement only the data accessor or replacement methods appropriate to your data source
• Circular dependencies between spectraData() and spectraVariables() cause infinite recursion — fillCoreSpectraVariables() must be called inside spectraData(), not spectraVariables()
• For parallel processing, backends may need to implement backendParallelFactor() to suggest a preferred splitting factor for load balancing

Evidence

• [intro] core_api_definition: "The MsBackend virtual class defines the API that new backend classes need to implement in order to be used with the Spectra object."
• [intro] spectra_data_representation: "One Spectra object is supposed to contain MS (spectral) data of multiple MS spectra. m/z values within each spectrum are expected to be sorted increasingly."
• [intro] core_variables_requirement: "Properties of a spectrum are called spectra variables. While backends can define their own properties, a minimum required set of spectra variables must be provided by each backend"
• [intro] mz_constraints: "m/z values within each spectrum are expected to be sorted increasingly. Missing values (NA) for m/z values are not supported."
• [intro] spectraData_method_design: "The spectraData() method should return the full spectra data within a backend as a DataFrame object"
• [intro] spectraVariables_method_design: "The spectraVariables() method should return a character vector with the names of all available spectra variables of the backend"
• [intro] peaksData_method_design: "The peaksData() method extracts the MS peaks data from a backend, which includes the m/z and intensity values of each MS peak of a spectrum"
• [intro] backend_read_only_flexibility: "MsBackend implementations can also represent purely read-only data resources. In this case only data accessor methods need to be implemented but not data replacement methods."
• [intro] partially_read_only_example: "Backends can also be partially read-only, such as the MsBackendMzR. This backend allows for example to change spectra variables, but not the peaks data"
• [other] fillCoreSpectraVariables_usage: "spectraData() uses fillCoreSpectraVariables() to populate missing core spectra variables into a DataFrame, while spectraVariables() returns the union of core variable names and backend column names"
• [other] circularity_prevention: "Verify that spectraVariables() does not call spectraData() and that spectraData() does not call spectraVariables() to prevent circular dependencies."