Metabolic network database output and serialization

metabolic-network-database-output-and-serialization

Summary

Serialize and persist the output of a metabolic network expansion (compounds and reactions generated by Pickaxe) to either a MongoDB instance or local TSV files, preserving the full reaction network topology and compound metadata for downstream retrieval and analysis.

When to use

After completing a Pickaxe reaction network expansion across N generations, when you need to store the predicted compounds and reactions in a format suitable for querying, sharing, or integration with other bioinformatic pipelines. Choose this skill when your output must be either persisted to a local database for programmatic access or written to flat files for portability.

When NOT to use

• If the reaction network is still being expanded across generations — serialize only after the final generation is complete.
• If you require real-time streaming output during the expansion; TSV/MongoDB writes are applied after generation completion.
• If you need the output in a format other than TSV or MongoDB (e.g., JSON, Neo4j, or RDF); this skill covers only the two built-in serialization formats.

Inputs

• Pickaxe reaction network object (in-memory compound set and reaction edges)
• MongoDB connection URI string (local or remote, optional mongo_uri.csv file)
• Output directory path (for TSV serialization, optional)

Outputs

• MongoDB collections: compounds collection and reactions collection
• Local TSV files: compounds table and reactions table in specified output directory

How to apply

Configure the output destination by either specifying a MongoDB URI (local or remote) or designating a local output directory for TSV files. If using MongoDB, pass the mongo_uri parameter to the Pickaxe instance; if using local files, specify the output directory path. Call the Pickaxe write method after the final generation completes. The system will automatically serialize the reaction network (edges representing transformations) and compound table (containing SMILES, structure metadata, and generation assignment) in the chosen format. Verify that all generations of compounds are present in the output and that reaction rules are correctly attributed to each transformation.

Related tools

• MongoDB (Persistent document store for writing reaction network and compound collections; allows querying and retrieval of predicted metabolites and reactions)
• Python (Runtime environment for Pickaxe write operations; handles file I/O and database connection logic)
• RDKit (Validates chemical structures (SMILES) before serialization and ensures integrity of compound representations) — https://rdkit.org/

Examples

from minedatabase.pickaxe import Pickaxe; pk = Pickaxe(compounds, rules, coreactants, mongo_uri='mongodb://localhost:27017/mine_db'); pk.expand(generations=3); # or: pk.write_output(output_dir='./results/')

Evaluation signals

• All compounds from all N generations are present in the output collection/files, with correct generation assignment (0 to N).
• Reaction table contains entries for every transformation applied; each row maps reactants → products with the applied rule ID.
• SMILES strings in the output are valid and can be parsed by RDKit without error.
• MongoDB collections are queryable and contain all expected fields (e.g., compound ID, SMILES, generation, reaction rule); local TSV files have correct header rows and consistent field counts.
• Row count in output matches expected total: sum of compounds across all generations + starting compounds; reactions count ≤ compounds × rules × generations (upper bound).

Limitations

• Scalability: expanding 10,000 compounds from KEGG with 272 rules yields 5 million compounds, which may exceed practical MongoDB or local storage capacity depending on infrastructure.
• MongoDB URI must be correctly specified (local or remote); connection failures will cause the write operation to fail silently or raise an exception depending on error handling configuration.
• TSV serialization is human-readable but lacks query indexing; MongoDB queries are more efficient for large networks but require database setup and maintenance.
• No changelog or versioning mechanism is documented for database outputs, making reproducibility and schema evolution tracking difficult across runs.

Evidence

• [intro] There are two ways to output data: 1. Writing to a mongo database that is specified by a mongo uri, either local or in mongo_uri.csv 2. Local .tsv files: "There are two ways to output data: 1. Writing to a mongo database that is specified by a mongo uri, either local or in mongo_uri.csv 2. Local .tsv files"
• [other] write final reaction network and compound data to TSV files in the specified output directory or to a MongoDB instance via the provided mongo URI: "write final reaction network and compound data to TSV files in the specified output directory or to a MongoDB instance via the provided mongo URI"
• [intro] Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded: "Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded"
• [readme] An example file, pickaxe_run_template.py, provides a framework for running pickaxe through a python file. Feel free to download it and change it to your needs. The starting compounds, rules and cofactors, optional database information, and Pickaxe run options are specified. After running the results are stored in a specified database or written to .tsv files.: "The starting compounds, rules and cofactors, optional database information, and Pickaxe run options are specified. After running the results are stored in a specified database or written to .tsv"
• [intro] For example, expanding 10,000 compounds from KEGG with 272 rules from metacyc yields 5 million compounds: "expanding 10,000 compounds from KEGG with 272 rules from metacyc yields 5 million compounds"