ecoli.experiments.ecoli_master_sim

Interface for configuring and running single-cell E. coli simulations.

Note

Simulations can be configured to divide through this interface, but full colony-scale simulations are best run using the ecoli_engine_process module for efficient multiprocessing.

class ecoli.experiments.ecoli_master_sim.EcoliSim(config)

Bases: object

Main interface for running single-cell E. coli simulations. Typically instantiated using one of two methods:

1. from_file()

2. from_cli()

Config options can be modified after the creation of an EcoliSim object in one of two ways.

1. sim.total_time = 100

2. sim.config['total_time'] = 100

Parameters:

config (dict[str, Any]) – Automatically generated from SimConfig when EcoliSim is instantiated using from_file() or from_cli()

_retrieve_process_configs(process_configs, processes)

Sets up process configs to be interpreted by generate_processes_and_steps().

Parameters:

• process_configs (dict[str, dict[str, Any]]) – Mapping of process names to user-specified process configuration dictionaries.

• processes (list[str]) – List of process names to set up process config for.

Returns:

Mapping of process names to process configs.

Return type:

dict[str, Any]

_retrieve_processes(processes, add_processes, exclude_processes, swap_processes)

Retrieve process classes from process_registry (processes are registered in ecoli/processes/__init__.py).

Parameters:

• processes (dict[str, str]) – Base list of process names to retrieve classes for

• add_processes (list[str]) – Additional process names to retrieve classes for

• exclude_processes (list[str]) – Process names to not retrieve classes for

• swap_processes (dict[str, str]) – Mapping of process names to the names of the processes they should be swapped for. It is assumed that the swapped processes share the same topologies.

Returns:

Mapping of process names to process classes.

Return type:

dict[str, Process]

_retrieve_topology(topology, processes, swap_processes, log_updates)

Retrieves topologies for processes from topology_registry.

Parameters:

• topology (dict[str, dict[str, tuple[str]]]) – Mapping of process names to user-specified topologies. Will be merged with topology from topology_registry, if exists.

• processes (list[str]) – List of process names for which to retrive topologies.

• swap_processes (dict[str, str]) – Mapping of process names to the names of processes to swap them for. By default, the new processes are assumed to have the same topology as the processes they replaced. When this is not the case, users can add/modify the original process topology with custom values in topology under either the new or the old process name.

• log_updates (bool) – Whether to emit process updates. Adds topology for log_update port.

Returns:

Mapping of process names to process topologies.

Return type:

dict[str, dict[str, tuple[str]]]

build_ecoli()

Creates the E. coli composite. MUST be called before calling run().

For all processes in config['processes']:

1. Retrieves process class from process_registry, which is populated in ecoli/processes/__init__.py.

2. Retrieves process topology from topology_registry and merge with user-specified topology from config['topology'], if applicable

3. Retrieves process configs from config['process_configs'] if present, else indicate that process config should be loaded from pickled simulation data using get_config_by_name()

Adds spatial environment if config['spatial_environment'] is True. Spatial environment config options are loaded from config['spatial_environment_config`]. See ecoli/composites/ecoli_configs/spatial.json for an example.

Note

When loading from a saved state with a file name of the format vivecoli_t{save time}, the simulation seed is automatically set to config['seed'] + {save_time} to prevent create_unique_indexes() from generating clashing indices.

ecoli

Contains the fully instantiated processes, steps, topologies, and flow necessary to run simulation. Generated by build_ecoli() and cleared when run() is called to potentially free up memory after division.

Type:

vivarium.core.composer.Composite

ecoli_experiment

Engine that runs the simulation. Instantiated by run().

Type:

vivarium.core.engine.Engine

export_json(filename='/home/runner/work/vEcoli/vEcoli/ecoli/composites/ecoli_configs/export.json')

Saves current simulation settings along with git hash and final list of process names as a JSON that can be reloaded using from_file().

Parameters:

filename (str) – Filepath and name for saved JSON (include .json).

static from_cli()

Used to instantiate EcoliSim with a config loaded from the command-line arguments parsed by SimConfig.

Return type:

EcoliSim

static from_file(filepath='/home/runner/work/vEcoli/vEcoli/ecoli/composites/ecoli_configs/default.json')

Used to instantiate EcoliSim with a config loaded from the JSON at filepath by SimConfig.

Parameters:

filepath – String filepath of JSON file with config options to apply on top of the options laid out in the default JSON located at the default value for filepath.

Return type:

EcoliSim

generated_initial_state

Fully populated initial state for simulation. Generated by build_ecoli() and cleared when run() is called to potentially free up memory after division.

Type:

dict

get_metadata()

Compiles all simulation settings, git hash, and process list into a single dictionary.

Return type:

dict[str, Any]

get_output_metadata()

Filters all ports schemas to include only output metadata located at the path ('_properties', 'metadata') for each schema by invoking extract_metadata(). See listener_schema() for usage details.

This dictionary of output metadata is flattened (see flatten_dict()) into columns with prefix METADATA_PREFIX and emitted as part of the simulation config by the Parquet emitter. It can be retrieved later using get_field_metadata().

Return type:

dict[str, Any]

merge(other)

Combine settings from this EcoliSim with another, overriding current settings with those from the other EcoliSim.

Parameters:

other (EcoliSim) – Simulation with settings to override current simulation.

query(query=None)

Query data that was emitted to RAMEmitter (config['emitter'] == 'timeseries'). For the Parquet emitter, query sim output with an analysis script run using runscripts.analysis or with ad-hoc DuckDB SQL queries built using get_dataset_sql() as a base.

Parameters:

query (list[tuple[str]] | None) – List of tuple-style paths in the simulation state to retrieve emitted values for. Returns all emitted data if None.

Returns:

Dictionary of emitted data in one of two forms.

• Raw data (if self.raw_output): Data is keyed by time (e.g. {0: {'data': ...}, 1: {'data': ...}, ...})

• Timeseries: Data is reorganized to match the structure of the simulation state. Leaf values in the returned dictionary are lists of the simulation state value over time (e.g. {'data': [..., ..., ...]}).

run()

Create and run an EcoliSim experiment.

Warning

Run build_ecoli() before calling run()!

save_states(daughter_outdir='')

Runs the simulation while saving the states of specific timesteps to files named data/vivecoli_t{time}.json. Invoked by run() if config['save'] == True. State is saved as a JSON that can be reloaded into a simulation as described in initial_state().

Parameters:

daughter_outdir (str) – Location to write JSON files for daughter cell(s). Only used if config contains generations key specifying number of generations to simulate. Nextflow chains simulations together by passing saved daughter states to new processes.

ecoli.experiments.ecoli_master_sim.LIST_KEYS_TO_MERGE = ('save_times', 'add_processes', 'exclude_processes', 'processes', 'engine_process_reports', 'initial_state_overrides')

Special configuration keys that are list values which are concatenated together when they are found in multiple sources (e.g. default JSON and user-specified JSON) instead of being directly overriden.

class ecoli.experiments.ecoli_master_sim.SimConfig(config=None, parser=None)

Bases: object

Stores configuration options for a simulation. Has dictionary-like interface (e.g. bracket indexing, get, keys).

config

Current configuration.

parser

Argument parser for the command-line interface.

Parameters:

• config (Dict[str, Any] | None) – Configuration options. If not provided, the default configuration is loaded from the file path default_config_path.

• parser (ArgumentParser | None) – Useful for scripts that leverage the inheritance features of the JSON config files but want to have their own CLI args for clarity.

default_config_path = '/home/runner/work/vEcoli/vEcoli/ecoli/composites/ecoli_configs/default.json'

Path to default JSON configuration file.

get(key, default=None)

keys()

static merge_config_dicts(d1, d2)

Helper function to safely merge two config dictionaries. Config options whose values are lists (e.g. save_times, add_processes, etc.) are handled separately so that the lists from each config are concatenated in the merged output.

Parameters:

• d1 (dict[str, Any]) – Config to mutate by merging in d2.

• d2 (dict[str, Any]) – Config to merge into d1.

Return type:

None

to_dict()

update_from_cli()

Parses command-line options defined in __init__ and updates config.

update_from_dict(dict_config)

Updates loaded config with user-specified dictionary.

Parameters:

dict_config (dict[str, Any])

update_from_json(path)

Loads config dictionary from file path path and merges it into the currently loaded config.

Parameters:

path (str) – The file path of the JSON config to merge in.

Return type:

None

exception ecoli.experiments.ecoli_master_sim.TimeLimitError

Bases: RuntimeError

Error raised when fail_at_total_time is True and simulation reaches total_time.

ecoli.experiments.ecoli_master_sim.extract_metadata(ports_schema, properties=False)

Filters ports schema to contain only a mapping of ports to user-supplied metadata (pulled from path (‘_properties’, ‘metadata’) for each schema). See listener_schema() for usage details.

Parameters:

• ports_schema (dict[str, Any]) – Ports schema to filter and compile metadata for

• properties (bool) – Flag used internally during recursive filtering

Returns:

Dictionary with same structure as ports schema but with only metadata as leaf nodes instead of complete schema

ecoli.experiments.ecoli_master_sim.get_git_revision_hash()

Returns current Git hash for model repository to include in metadata that is emitted when starting a simulation.

Return type:

str

ecoli.experiments.ecoli_master_sim.get_git_status()

Returns Git status of model repository to include in metadata that is emitted when starting a simulation.

Return type:

str

ecoli.experiments.ecoli_master_sim.main()

Runs a simulation with CLI options.

ecoli.experiments.ecoli_master_sim.parse_key_value_args(args_list)

Parses key-value pairs specified as strings of the form key=value via CLI. See emitter_arg option in SimConfig.

Parameters:

• argument_string – Key-value pair as a string of the form key=value

• args_list (list[str])

Returns:

[key, value]

Return type:

dict[str, str]

ecoli.experiments.ecoli_master_sim.prepare_save_state(state)

Prepares simulation state to be saved to a JSON file by pruning unsaveable values and adding necessary metadata. Mutates in-place.

Parameters:

state (dict[str, Any])

Return type:

None

ecoli.experiments.ecoli_master_sim.report_profiling(stats)

Prints out a summary of profiling statistics when profile option is True in the config given to EcoliSim

Parameters:

stats (Stats) – Profiling statistics.

Return type:

None

ecoli.experiments.ecoli_master_sim.tuplify_topology(topology)

JSON files allow lists but do not allow tuples. This function transforms the list paths in topologies loaded from JSON into standard tuple paths.

Parameters:

topology (dict[str, Any]) – Topology to recursively iterate over, converting all paths to tuples

Returns:

Topology with tuple paths (e.g. ['bulk'] turns into ('bulk',))

Return type:

dict[str, Any]