Tracks¶
Functions for creating, importing, modifying, and managing genomic tracks, including dense, sparse, 2D, and indexed track types, as well as track attributes and variables.
pymisha.gtrack_ls ¶
gtrack_ls(*patterns: str, ignore_case: bool = False, db: str | None = None, **attr_filters: str) -> list[str] | None
Return a list of track names in the Genomic Database.
Returns track names that match all supplied patterns. Name patterns are applied as regex searches against track names. Attribute patterns are matched against the corresponding track attribute values. Multiple patterns are applied conjunctively (all must match).
| PARAMETER | DESCRIPTION |
|---|---|
*patterns
|
Regex patterns to filter track names. Each pattern is applied sequentially; only tracks matching all patterns are returned.
TYPE:
|
ignore_case
|
If True, pattern matching is case-insensitive.
TYPE:
|
db
|
Only the tracks of this database or dataset (a path, normalized), as
:func:
TYPE:
|
**attr_filters
|
Keyword arguments of the form
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
list of str or None
|
Sorted list of matching track names, or None if no tracks match. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If a regex pattern is invalid. |
See Also
gtrack_exists : Test whether a single track exists. gtrack_info : Get metadata for a track. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_info ¶
Return metadata about a track.
Returns a dictionary containing track properties such as type, dimensions, bin size, total size in bytes, and any user-defined attributes. The fields vary depending on the track type (Dense, Sparse, Rectangles, Points).
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict
|
Dictionary of track properties. Common keys include |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist. |
See Also
gtrack_exists : Test whether a track exists. gtrack_ls : List available tracks.
Examples:
pymisha.gtrack_exists ¶
Test for track existence in the Genomic Database.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name to check.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if the track exists, False otherwise. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If track is None. |
Examples:
>>> import pymisha as pm
>>> _ = pm.gdb_init_examples()
>>> pm.gtrack_exists("dense_track")
True
>>> pm.gtrack_exists("nonexistent_track")
False
See Also
gtrack_ls : List available tracks. gtrack_info : Get metadata for a track. gtrack_rm : Delete a track.
pymisha.gtrack_dataset ¶
Return the database root path that contains a track.
When multiple databases are connected, this identifies which database a track belongs to by returning the filesystem path of that database root.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name, or a list of names.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
(str, None or list)
|
Absolute filesystem path of the database root containing the track, None for a track that does not exist; a list of them for a list of names. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If track is None or the track does not exist. |
See Also
gtrack_info : Get full metadata for a track. gtrack_ls : List available tracks.
Examples:
pymisha.gtrack_create ¶
gtrack_create(track: str, description: str, expr: str, iterator: Any = None, band: tuple[int, int] | None = None) -> None
Create a track from a track expression.
Creates a new track whose values are determined by evaluating expr over the entire genome. The type of the new track (Dense, Sparse, or Rectangles) is determined by the iterator policy. The description is stored as a track attribute.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track. Must start with a letter and contain only alphanumeric characters, underscores, and dots.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
expr
|
Numeric track expression to evaluate.
TYPE:
|
iterator
|
Track expression iterator: a bin size, a track or intervals set
name, intervals, or for a 2D track a
TYPE:
|
band
|
Diagonal band
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, expr is None, or the dimension of the iterator does not match the tracks of expr. |
See Also
gtrack_create_sparse : Create a Sparse track from intervals/values. gtrack_create_dense : Create a Dense track from intervals/values. gtrack_2d_create : Create a 2D track. gtrack_smooth : Create a smoothed track. gtrack_modify : Modify an existing Dense track. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_create_dense ¶
gtrack_create_dense(track: str, description: str, intervals: Intervals, values: Any, binsize: int, defval: float = np.nan, func: str = 'weighted.mean') -> None
Create a Dense (fixed-bin) track from intervals and values.
Creates a new Dense track whose genome is tiled into fixed-size bins. Each bin stores a single numeric value reduced from the intervals overlapping that bin, plus an optional synthetic uncovered contribution at value defval. The description is stored as a track attribute.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track. Must start with a letter and contain only alphanumeric characters, underscores, and dots.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
intervals
|
One-dimensional intervals with columns
TYPE:
|
values
|
Numeric values, one per interval. Length must match the number of rows in intervals.
TYPE:
|
binsize
|
Bin size in base pairs. Must be a positive integer.
TYPE:
|
defval
|
Default value for bins not covered by any interval. Acts as a
synthetic contribution with value=defval and
overlap=uncovered_bases for every
TYPE:
|
func
|
Per-bin reduction over the intervals overlapping each bin. One of:
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, binsize is not positive, values
length does not match intervals, |
See Also
gtrack_create_sparse : Create a Sparse track. gtrack_create : Create a track from a track expression. gtrack_modify : Modify values of an existing Dense track. gtrack_import : Import a track from a file. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_create_sparse ¶
gtrack_create_sparse(track: str, description: str, intervals: Intervals, values: Any = None) -> None
Create a Sparse track from intervals and values.
Creates a new Sparse track where each interval carries an associated
numeric value. Intervals must be non-overlapping within each
chromosome. Chromosome names are matched through the alias chain
(chr1 <-> 1, M <-> MT); a chromosome the database does
not have is an error. The description is stored as a track attribute.
values is matched to intervals row by row, in the order the intervals
are passed; intervals need not be sorted. Note however that
:func:gintervals returns its result sorted, so building intervals with
gintervals() while keeping values in the original order will bind
values to the wrong intervals. Keep the values in a value column of
intervals and omit values to make such a mismatch impossible.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track. Must start with a letter and contain only alphanumeric characters, underscores, and dots.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
intervals
|
One-dimensional intervals with columns
TYPE:
|
values
|
Numeric values, one per interval, in the same order as the rows
of intervals. Length must match the number of rows in
intervals. If
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, intervals overlap, values length does not match intervals, or an interval has a missing coordinate, a negative start, start >= end, an unknown chromosome or an end past the chromosome end. |
See Also
gtrack_create_dense : Create a Dense (fixed-bin) track. gtrack_create : Create a track from a track expression. gtrack_import : Import a track from a file. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_create_empty_indexed ¶
Create empty indexed files for an existing track directory.
Writes an empty track.idx and track.dat pair in the track
directory. Useful when the track has no data yet but indexed format
is required by the database.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name of an existing track whose directory should receive the indexed files.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
See Also
gtrack_convert_to_indexed : Convert per-chromosome files to indexed. gdb_convert_to_indexed : Convert an entire database.
Examples:
pymisha.gtrack_import ¶
gtrack_import(track: str, description: str, file: str, binsize: int | None = None, defval: float = np.nan, attrs: dict[str, str] | None = None, func: str = 'weighted.mean') -> None
Create a track from a WIG, BigWig, BedGraph, BED, or tab-delimited file.
Parses the input file and creates either a Sparse or Dense track
depending on binsize. The format is found as R misha finds it.
Compressed files (.gz, or a .zip holding one file) are unzipped
to a temporary file first, and the format is detected on it. A Parquet
file is recognized by its first bytes, a BED file by its name (.bed,
.bed.gz, .bed.zip), a bigWig by its first bytes or its name
(.bw, .bigwig). Any other file is read as WIG / bedGraph,
whatever its name, and, if it is not one, as a tab-delimited file, which
must have a header with columns chrom, start, end, and
exactly one value column; a file that is neither is an error giving
both readers' reasons.
A WIG / bedGraph / tab-delimited file with no records at all makes an
empty track; one whose records are all on chromosomes the database
lacks is an error. Records on a chromosome the database lacks are
skipped and reported: a log message for a scaffold or patch, a warning
when the name looks like a primary chromosome.
Empty BED records (zero length) are dropped; a BED file with no record left is an error.
In every format, records that run past the end of their chromosome are clipped to it, and records that start at or past the end are dropped, with one warning giving the counts and the first such record in the file.
BigWig files are read directly at full resolution and full float
precision (zoom levels are not used). A truncated or corrupt bigWig is
an error. A bigWig imports bin for bin like the WIG / bedGraph text it
was made from, and both follow the same rules: the records of a
chromosome must be together (a block that resumes after another
chromosome, known to the database or not, is an error), sorted and
must not overlap. An empty bedGraph
record is an error in text and skipped in a bigWig. Two bigWig
chromosomes that name the same database chromosome (chr1 and 1)
are an error. A WIG file may mix fixedStep / variableStep sections and
bedGraph lines; a bedGraph line ends the section before it.
Dense bins are computed as R misha's gtrack.import computes them, which
rounds a little differently from :func:gtrack_create_dense for the
default func.
A Parquet file (recognized by its first bytes, whatever its name) is
imported when it has exactly one value column besides chrom,
start and end; it always makes a Dense track, so binsize must
be > 0, and the column's field metadata becomes attributes of the track
as in :func:gtrack_import_parquet (attrs wins over it). Use
:func:gtrack_import_parquet for several columns.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
file
|
Path to the input file: WIG, bedGraph, BED, bigWig, Parquet or
tab-delimited (see above). May include
TYPE:
|
binsize
|
Bin size for a Dense track. If None or 0, a Sparse track is created. If positive, a Dense track with the given bin size is created.
TYPE:
|
defval
|
Default value for Dense track bins not covered by any interval. Ignored when creating Sparse tracks.
TYPE:
|
attrs
|
Additional attributes to set on the track after import, as a dict mapping attribute names to string values.
TYPE:
|
func
|
Per-bin reduction applied when creating a Dense track (i.e. when
binsize is positive). Forwarded to :func:
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, file is None, the file contains no valid intervals, or WIG / bedGraph records break the rules above. |
error
|
If a bigWig is truncated, corrupt or breaks the rules above. |
See Also
gtrack_import_set : Batch-import multiple files into tracks. gtrack_import_parquet : Import the value columns of Parquet files. gtrack_create_sparse : Create a Sparse track programmatically. gtrack_create_dense : Create a Dense track programmatically. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_import_mappedseq ¶
gtrack_import_mappedseq(track: str, description: str, file: str, pileup: int = 0, binsize: int = -1, cols_order: tuple[int, int, int, int] | None = _DEFAULT_MAPPEDSEQ_COLS, remove_dups: bool = True, paired: bool = False, min_mapq: int = 0, max_fraglen: float = 2000, one_based: bool = False) -> dict[str, Any]
Import mapped sequences from SAM, BAM or tab-delimited text into a track.
Reads aligned sequence data and creates either a Sparse (per-read) or a
Dense (pileup or paired-end fragment coverage) track. The file can be in
SAM format, in a tab-delimited format where each line describes a single
read, in gzipped variants of either (.sam.gz, .tsv.gz), or in BAM
format (detected from its content; requires samtools on PATH).
Without paired, any bgzipped file is read as SAM. file may also be a
named pipe (FIFO) carrying SAM or text, plain or gzipped, but not BAM. A
pipe is not inspected, so the BAM and bgzip rules above do not apply to
it: pass cols_order=None for SAM. An import that brings in no read at
all gives a warning.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
file
|
Path to a SAM, BAM, tab-delimited or fragment file, or a named pipe.
TYPE:
|
pileup
|
If 0, create a Sparse track with one interval per mapped read. If positive, create a Dense pileup track where each bin stores the mean number of reads covering its bases. Each read is extended to this length from its 5' end, downstream.
TYPE:
|
binsize
|
Bin size for Dense tracks. Required when pileup > 0 or paired. Must be -1 when pileup is 0 and paired is False.
TYPE:
|
cols_order
|
Column indices (1-based) for sequence, chromosome, coordinate, and strand in a tab-delimited file. Set to None for SAM. For BAM input the default is treated as SAM; any other value is an error.
TYPE:
|
remove_dups
|
If True, duplicated coordinates (fragments with the same start and end, with paired) are counted once.
TYPE:
|
paired
|
Import paired-end fragments (see Notes).
TYPE:
|
min_mapq
|
Minimal MAPQ of an imported SAM / BAM record.
TYPE:
|
max_fraglen
|
Maximal fragment length when paired is True (
TYPE:
|
one_based
|
If True the coordinates of a tab-delimited file are 1-based (see Notes). Not used for SAM / BAM, which are always 1-based.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict
|
Dictionary with keys |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, file is None, column indices are invalid, or the arguments do not fit together. |
See Also
gtrack_import : Import from WIG/BedGraph/BED/BigWig files. gtrack_create_sparse : Create a Sparse track from intervals. gtrack_create_dense : Create a Dense track from intervals. gtrack_rm : Delete a track.
Notes
Coordinates. SAM / BAM POS is 1-based and is converted to the 0-based
leftmost aligned base. The 5' end of a reverse-strand read is its
rightmost aligned base, from the CIGAR. A mapped SAM record without a
CIGAR (*), or with a malformed one, is counted as unmapped, as htslib
does. In a tab-delimited file the coordinate is the leftmost base of the
read, and the 5' end of a reverse read is found from the length of the
sequence. With one_based=True the coordinate is read as 1-based and
reads are placed as for SAM; Illumina export files (the default
cols_order layout) are 1-based. With the default one_based=False a
tab-delimited file keeps the placement of earlier versions: the
coordinate is used as is, which places dense coverage correctly for a
0-based file, but in a sparse track a reverse read is recorded one base
past its 5' end. Valid strand values are + or F for the forward
strand and - or R for the reverse strand.
For SAM and BAM input, unmapped, secondary, QC-failed and supplementary records (FLAG 0x4, 0x100, 0x200, 0x800) are never imported, and min_mapq drops records with a lower MAPQ.
If paired is True each fragment covers its own span instead of being
expanded by pileup (which must be 0), and the track is dense with bins
of binsize. Each bin holds the mean number of fragments covering its
bases (the last, partial bin of a chromosome is also divided by
binsize). For SAM / BAM input (cols_order=None, or any BAM) the
fragment is read from each proper pair (FLAG 0x1 and 0x2 set, 0x8 unset)
by its first mate, as [min(POS, PNEXT), min(POS, PNEXT) + abs(TLEN)), the
way MACS3 does with -f BAMPE: records of second mates (FLAG 0x80) are
skipped. The input may be sorted in any order. min_mapq is tested on
the first mate only. With remove_dups fragments with the same start and
end are counted once. Any other input (not a BAM, and cols_order not
None) is read as a fragment file: tab-delimited, 0-based half-open
chromosome, start and end in columns 1-3 (BED, or a 10x
fragments.tsv.gz), lines starting with # skipped. Fragment files
are taken as already deduplicated, so remove_dups does not apply to
them, and cols_order must not be set. Fragments longer than
max_fraglen are skipped in both cases. With paired the records of
second mates are not counted in the returned statistics.
The statistics report total records, total.mapped (imported,
duplicates included), total.unmapped (unmapped, unparsable, with an
unusable CIGAR, or with its 5' end outside the chromosomes of the
database), total.dups and total.filtered (left out by the flag,
MAPQ, proper-pair or fragment length filters).
R's missing(cols.order) is read here as cols_order equal to its
default: passing (9, 11, 13, 14) explicitly counts as not passing it.
Examples:
pymisha.gtrack_import_set ¶
gtrack_import_set(description: str, path: str, binsize: int, track_prefix: str | None = None, defval: float = np.nan) -> dict[str, list[str]]
Create one or more tracks from multiple WIG/BedGraph/BigWig/tab files.
Similar to gtrack_import but operates on multiple files at once.
Files can be specified by a local glob pattern or an FTP URL with
wildcards. Each file produces one track named
{track_prefix}{filestem}. Existing tracks are skipped. The
function continues importing even if individual files fail.
| PARAMETER | DESCRIPTION |
|---|---|
description
|
Human-readable description stored as a track attribute on every imported track.
TYPE:
|
path
|
Local file glob pattern (e.g.,
TYPE:
|
binsize
|
Bin size for Dense tracks. If 0, Sparse tracks are created.
TYPE:
|
track_prefix
|
Prefix prepended to each track name derived from the filename stem. If None, no prefix is used.
TYPE:
|
defval
|
Default value for Dense track bins not covered by any interval.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict
|
Dictionary with keys |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If description, path, or binsize is None, or no files match the pattern. |
See Also
gtrack_import : Import a single file into a track. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_import_parquet ¶
gtrack_import_parquet(description: str, path: str, binsize: int, track_prefix: str | None = None, columns: str | list[str] | None = None, defval: float = np.nan) -> list[str]
Create Dense tracks from the columns of Parquet files.
Creates one Dense track per value column of one or more Parquet files.
The files must have the interval columns chrom (string), start
and end (0-based, end exclusive; integers, or floating point numbers
that are whole) and one or more value columns (integer or floating
point), such as the files :func:gextract writes with
file="x.parquet" or pandas' to_parquet. path may contain
wildcards, so that per-chromosome files import in one call; all the
files must have the same value columns. Columns are matched by name,
not by position.
The track of column col is named {track_prefix}col. All the
track names are checked before anything is written: a name must start
with a letter and hold only letters, digits, _ and . (columns
are never renamed; leave such a column out with columns). Existing
tracks are not overwritten and no directories are created.
Values are binned as :func:gtrack_import bins the same rows read from
a tab-delimited file: each bin holds the mean of the values that overlap
it, weighted by the overlap. Bases no row covers, and rows whose value
is null or NaN, count as defval (unless it is NaN). The rows of a
chromosome may be spread over several files and row groups in any
order, but may not overlap. Chromosome names are matched through the
chromosome alias chain as in :func:gtrack_import, with the same
reporting of names the database does not have.
Rows that run past the end of their chromosome (such as a last 32 bp
bin written with end = start + 32) are clipped to it, and rows that
start at or past the end are dropped, with one warning that gives the
counts and the first such row. Messages number the rows of a file from
0, as pyarrow and pandas do.
The field metadata of a column in the first file (Arrow field metadata,
as :func:gextract writes the attributes of a track column, or as
pyarrow writes pa.field(name, type, metadata=...)) becomes
attributes of its track, except created.by, created.date and
created.user, which the import sets, type and binsize,
which describe how the source track was stored, and description
unless the description argument is an empty string. A file without
field metadata imports with only those attributes. See
:func:gtrack_attr_set for the attributes of model predictions.
The import is all or nothing: if it fails or is interrupted, none of the tracks is created.
Supported: flat schemas; PLAIN, dictionary and BYTE_STREAM_SPLIT encodings; data pages v1 and v2; UNCOMPRESSED, SNAPPY and GZIP compression. Other compression codecs (e.g. ZSTD) and encodings (e.g. DELTA_BINARY_PACKED) are an error that names them. Unsigned integer columns are read as unsigned; decimal, date, time and timestamp columns are not numbers and are refused.
| PARAMETER | DESCRIPTION |
|---|---|
description
|
Description stored as a track attribute of every created track.
TYPE:
|
path
|
Parquet file path (may contain wildcards).
TYPE:
|
binsize
|
Bin size of the new Dense tracks (> 0).
TYPE:
|
track_prefix
|
Prefix of the track names.
TYPE:
|
columns
|
Names of the value columns to import, in the order of the created
tracks. With None every column other than
TYPE:
|
defval
|
Default value of the bins no row covers.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
list of str
|
Names of the created tracks. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If an argument is invalid, no file matches path, the files do not have the columns, or a track name is invalid or exists. |
See Also
gtrack_import : Import a single file into a track. gtrack_import_set : Batch-import multiple files into tracks. gextract : Write track expressions to a Parquet file.
Examples:
>>> import os, tempfile
>>> import pymisha as pm
>>> _ = pm.gdb_init_examples()
>>> f = os.path.join(tempfile.mkdtemp(), "x.parquet")
>>> pm.gextract(["dense_track", "dense_track * 2"], pm.gintervals("1", 0, 10000),
... iterator=50, file=f, progress=False)
>>> pm.gtrack_import_parquet("from parquet", f, 50, track_prefix="pq_", columns="dense_track")
['pq_dense_track']
>>> pm.gtrack_rm("pq_dense_track", force=True)
pymisha.gtrack_rm ¶
Remove a track from disk.
Permanently deletes the track directory and all associated files (per-chromosome data, attributes, variables). Empty parent directories are cleaned up automatically.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name of the track to remove.
TYPE:
|
force
|
If True, suppress errors when the track does not exist and
allow deletion without confirmation. If False, raises
TYPE:
|
db
|
Explicit database root path. If None, the track is located in the currently initialized databases.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist (when force is False) or if force is False (safety guard). |
See Also
gtrack_ls : List available tracks. gtrack_exists : Test whether a track exists. gtrack_mv : Rename or move a track. gtrack_copy : Copy a track.
Examples:
pymisha.gtrack_mv ¶
Rename or move a track within the same database.
Renames a track or moves it to a different namespace (directory)
within its source database. The track cannot be moved across
databases; use gtrack_copy followed by gtrack_rm for that.
| PARAMETER | DESCRIPTION |
|---|---|
src
|
Current track name.
TYPE:
|
dest
|
New track name.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If source and destination are identical, the source track does not exist, or the destination track already exists. |
See Also
gtrack_copy : Copy a track (possibly across databases). gtrack_rm : Delete a track. gtrack_exists : Test whether a track exists. gtrack_ls : List available tracks.
Examples:
pymisha.gtrack_copy ¶
gtrack_copy(src: str | Iterable[str], dest: str | None = None, db: str | None = None, overwrite: bool = False) -> list[str]
Copy one or more tracks, optionally to a different database.
Transparently handles format mismatches (per-chromosome vs indexed) and chromosome-order differences between source and destination databases. Chromosomes that exist in the source database but not in the destination are dropped with a warning.
For 2D tracks (rectangles, points), cross-database copy requires identical chromosome order in source and destination.
| PARAMETER | DESCRIPTION |
|---|---|
src
|
Source track name(s). Either a single name or an iterable of names.
TYPE:
|
dest
|
Destination name. For a single source, this is the destination
track name (defaults to src). For a sequence of sources, this
is interpreted as a namespace prefix (e.g.
TYPE:
|
db
|
Destination database root. Must be the active GROOT, a member of
the loaded datasets, or a path that looks like a valid misha
database (contains
TYPE:
|
overwrite
|
If True, replace an existing destination track. Every check runs first, and the existing track is kept until the copy has succeeded.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
list[str]
|
The created destination track names. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
Source track does not exist; destination already exists with
|
See Also
gtrack_mv : Rename / move a track within the same database. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_modify ¶
Modify a Dense track's values in-place by evaluating an expression.
Overwrites the values of an existing Dense track with the result of evaluating expr. The iterator policy is automatically set to the track's bin size. Only Dense (fixed-bin) tracks are supported.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name of the dense track to modify.
TYPE:
|
expr
|
Track expression to evaluate (may reference the track itself).
TYPE:
|
intervals
|
Genomic scope for modification. If None, the entire genome (ALLGENOME) is used.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist, is not a Dense track, or expr is None. |
See Also
gtrack_create : Create a new track from a track expression. gtrack_smooth : Create a smoothed copy of a track. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_smooth ¶
gtrack_smooth(track: str, description: str, expr: str, winsize: float, weight_thr: float = 0, smooth_nans: bool = False, alg: str = 'LINEAR_RAMP', iterator: int | None = None) -> None
Create a new Dense track with smoothed values from a track expression.
Each output bin at coordinate C is computed by smoothing the non-NaN values of expr within a window of size winsize (in coordinate units) around C. The smoothing algorithm and handling of NaN / edge-of-chromosome gaps are controlled by the remaining parameters.
The window is a whole number of iterator bins: the central bin and
k bins on each side, 2 * k + 1 bins in all, where
k = winsize / (2 * binsize) rounded to the nearest integer, a half
rounded up (binsize being the bin size of the iterator). The window
can therefore be wider than winsize: winsize=100 with a 20 bp
iterator gives k = 3 (2.5 rounded up), 7 bins or 140 bp;
winsize=288 with a 32 bp iterator gives k = 5 (4.5 rounded up),
11 bins or 352 bp, and winsize=256 gives 9 bins. A winsize below
the bin size (k = 0) is an error.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name of the new track to create.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
expr
|
Track expression whose values are smoothed.
TYPE:
|
winsize
|
Smoothing window size in coordinate units. Defines the total region considered on both sides of the central point.
TYPE:
|
weight_thr
|
Weight sum threshold below which the smoothed value is NaN instead of a partial-window estimate.
TYPE:
|
smooth_nans
|
If False, output NaN whenever the central window value is NaN, regardless of weight_thr. If True, NaN center values are filled from surrounding non-NaN values.
TYPE:
|
alg
|
Smoothing algorithm.
TYPE:
|
iterator
|
Fixed-bin iterator bin size for the new track. If None, the bin size is inferred from the track expression.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, expr is None, winsize is not positive, or alg is not one of the supported algorithms. |
See Also
gtrack_create : Create a track from a track expression. gtrack_modify : Modify an existing Dense track in-place. gtrack_create_sparse : Create a Sparse track. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_convert_to_indexed ¶
Convert a per-chromosome track to indexed format.
Reads the per-chromosome binary files and writes a unified
track.idx / track.dat pair. Optionally removes the original
per-chromosome files after conversion.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name of the track to convert.
TYPE:
|
remove_old
|
If True, remove the original per-chromosome files after successful conversion.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
See Also
gtrack_create_empty_indexed : Create empty indexed files. gdb_convert_to_indexed : Convert an entire database.
Examples:
pymisha.gtrack_2d_convert_to_indexed ¶
Convert a 2D track to indexed format (track.dat + track.idx).
Consolidates per-chromosome-pair files into a single indexed format, reducing file descriptor usage from O(N^2) to O(1).
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
remove_old
|
If True, remove old per-pair files after conversion.
TYPE:
|
force
|
If True, re-convert even if already in indexed format.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist, is not a 2D track, or conversion fails. |
See Also
gtrack_convert_to_indexed : Convert a 1D track to indexed format. gdb_convert_to_indexed : Convert an entire database.
Examples:
pymisha.gtrack_2d_create ¶
Create a 2D track from intervals and values.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name (dot-separated namespace).
TYPE:
|
description
|
Track description.
TYPE:
|
intervals
|
2D intervals with columns: chrom1, start1, end1, chrom2, start2, end2.
TYPE:
|
values
|
Numeric values, one per interval.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, values length does not match intervals, no valid intervals remain after normalization, or overlapping intervals are detected within the same chromosome pair. |
See Also
gtrack_2d_import : Create a 2D track from a file. gtrack_2d_import_contacts : Import HiC contact data as a 2D track. gtrack_create_sparse : Create a 1D Sparse track. gtrack_rm : Delete a track.
Notes
Creates a RECTS track, even when every interval is 1 x 1, as R misha
does; :func:gtrack_2d_import and :func:gtrack_2d_import_contacts
create POINTS tracks. The files are the bytes R writes for the same
intervals and values. Overlapping intervals within the same chromosome
pair raise an error.
Examples:
>>> import pymisha as pm
>>> import pandas as pd
>>> _ = pm.gdb_init_examples()
>>> ivs = pd.DataFrame({
... "chrom1": ["1"], "start1": [0], "end1": [100],
... "chrom2": ["1"], "start2": [200], "end2": [300],
... })
>>> pm.gtrack_2d_create("test_2d", "Test", ivs, [1.0])
>>> pm.gtrack_rm("test_2d", force=True)
pymisha.gtrack_2d_import ¶
Import a 2D track from one or more tab-delimited files.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
description
|
Track description.
TYPE:
|
file
|
Path(s) to tab-delimited file(s) with header:
chrom1, start1, end1, chrom2, start2, end2,
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, any file is not found, the file list is empty, a file is malformed (the message names the file and line), or no interval is left to import. |
See Also
gtrack_2d_create : Create a 2D track from a DataFrame. gtrack_2d_import_contacts : Import HiC contacts as a 2D track. gtrack_rm : Delete a track.
Notes
The first 6 columns must be named chrom1, start1, end1, chrom2, start2
and end2; the value is the 7th column. A POINTS track if every interval
is 1 x 1, a RECTS track otherwise, as in R misha. Intervals on a
chromosome the database does not have are skipped and the names
reported, as :func:gtrack_import reports them; a file that names
chromosomes and matches none is an error. Each chromosome pair
is written as R writes it - its records in input order, a pair of at
least 4 x CONFIG['max_data_size'] records in parts (R's
gmax.data.size) - so the files are the bytes R writes.
If CONFIG['multitasking'] is on (the default), the input files are read
by several processes in parallel, each process reading a contiguous subset
of the files, and the chromosome pairs are then written in parallel, one
pair per process at a time. A single input file is read by one process, so
split a large input into several files to read it in parallel. The number
of processes is bounded by CONFIG['max_processes']. The resulting track
is identical to the one created with multitasking off. A pair is started
only when its estimated memory, added to that of the pairs being written,
stays within CONFIG['max_mem_usage'] (KB). Larger pairs are taken
first, but a smaller pair that fits may start while a larger one waits, and
a pair estimated above CONFIG['max_mem_usage'] is written alone.
Examples:
pymisha.gtrack_2d_import_contacts ¶
gtrack_2d_import_contacts(track: str, description: str, contacts: str | list[str], fends: str | None = None, allow_duplicates: bool = True) -> None
Create a 2D Points track from inter-genomic contacts.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name (dot-separated namespace).
TYPE:
|
description
|
Track description.
TYPE:
|
contacts
|
Path(s) to contact files. If
TYPE:
|
fends
|
Path to a fragment-ends file with columns: fend, chr, coord.
TYPE:
|
allow_duplicates
|
If True, duplicate contacts (same midpoint pair) are summed.
If False, duplicates raise
TYPE:
|
Notes
- Intervals are converted to midpoints: X = (start1+end1)//2, Y = (start2+end2)//2.
- Contacts are canonically ordered: if chrom2 < chrom1 (or same chrom and coord2 < coord1) the two sides are swapped.
- Cis contacts (same chromosome) are mirrored: both (X,Y) and (Y,X) are stored unless X == Y.
- Trans contacts (different chromosomes) are written in both directions: a chrA-chrB file and a chrB-chrA file (with swapped coordinates) are created so that queries work regardless of chromosome pair order.
- Contacts on a chromosome the database does not have, or on a fend the
fends file does not define, are skipped; the unknown chromosome names
are reported, as :func:
gtrack_importreports them. A file that names chromosomes and matches none is an error, as is an import that leaves no contact. - Fend ids must be in 0 - 2147483647 and listed once.
- Each chromosome pair is written as R misha writes it - the contacts
summed and inserted in R's order, a pair of at least
4 x
CONFIG['max_data_size']records in parts (R's gmax.data.size; 2 x for a chromosome with itself) - so the files are the bytes R writes for the same input.
If CONFIG['multitasking'] is on (the default), the input files are read
by several processes in parallel, each process reading a contiguous subset
of the files, and the chromosome pairs are then written in parallel, one
pair per process at a time. A single input file is read by one process, so
split a large input into several files to read it in parallel. The number
of processes is bounded by CONFIG['max_processes']. The resulting track
is identical to the one created with multitasking off. A pair is started
only when its estimated memory, added to that of the pairs being written,
stays within CONFIG['max_mem_usage'] (KB). Larger pairs are taken
first, but a smaller pair that fits may start while a larger one waits, and
a pair estimated above CONFIG['max_mem_usage'] is written alone.
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, no contact files are provided, a file is malformed, no contact is left to import, or duplicates are found when allow_duplicates is False. |
See Also
gtrack_2d_create : Create a 2D track from a DataFrame. gtrack_2d_import : Import a 2D track from a tab-delimited file. gtrack_rm : Delete a track.
Examples:
pymisha.gtrack_create_pwm_energy ¶
gtrack_create_pwm_energy(track: str, description: str, pssmset: str, pssmid: int, prior: float, iterator: int) -> None
Create a track from a PSSM energy function.
Creates a new Dense track with values of a PSSM energy function
(log-sum-exp scoring). PSSM parameters are read from
{pssmset}.key and {pssmset}.data files in GROOT/pssms/.
Internally creates a temporary PWM virtual track, extracts values at
the given iterator resolution, and writes them to a new Dense track.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track.
TYPE:
|
description
|
Human-readable description stored as a track attribute.
TYPE:
|
pssmset
|
Name of PSSM set. Files
TYPE:
|
pssmid
|
PSSM id within the set.
TYPE:
|
prior
|
Dirichlet prior for the PSSM.
TYPE:
|
iterator
|
Fixed-bin iterator bin size for the new track. Must be a positive integer.
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track already exists, any required argument is None, iterator is not positive, or the PSSM set/id is not found. |
FileNotFoundError
|
If the PSSM key or data file does not exist. |
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
See Also
gtrack_create : Create a track from a general track expression. gtrack_create_dense : Create a Dense track from intervals/values. gtrack_smooth : Create a smoothed track. gtrack_rm : Delete a track.
Examples:
Array Tracks¶
Multi-column per-position tracks. The PyMisha implementation reads and writes
the on-disk format byte-compatibly with R misha. Array tracks dispatch through
the C++ track-expression scanner, so they work directly in expressions
(gextract("my.array", ...), gextract("2 * my.array", ...)), as an iterator,
and as the source of value / slice virtual tracks. Use gtrack_array_extract
to dump the full multi-column matrix.
pymisha.gtrack_array_create ¶
gtrack_array_create(track: str, description: str, intervals: DataFrame, values: Any, colnames: str | list[str]) -> None
Create an array track from intervals + a per-position matrix.
Lower-level analogue of R's gtrack.array.import for the
in-memory case (R's import shells out to a C parser for files; here
you supply intervals + values directly).
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Name for the new track.
TYPE:
|
description
|
Human-readable description (stored as a track attribute).
TYPE:
|
intervals
|
TYPE:
|
values
|
Float values.
TYPE:
|
colnames
|
Column names;
TYPE:
|
Examples:
pymisha.gtrack_array_extract ¶
gtrack_array_extract(track: str, slice: str | int | list[str] | list[int] | None = None, intervals: DataFrame | str | None = None, file: str | None = None) -> pd.DataFrame | None
Extract per-position array values from an array track.
R parity for gtrack.array.extract. Returns a DataFrame with
chrom, start, end, <selected colnames>, intervalID. One output
row per overlapping track interval (clipped to the query); columns
where the track has no value are NaN.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Array track name.
TYPE:
|
slice
|
Column subset to return. Strings are matched against the track
colnames; integers are 0-based column indices.
TYPE:
|
intervals
|
Query intervals (
TYPE:
|
file
|
If given, write the result as a tab-separated table to this path
and return
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
DataFrame or None
|
The extracted DataFrame, or |
pymisha.gtrack_array_get_colnames ¶
Return the column names of an array track.
R parity for gtrack.array.get_colnames. The names live in
<track_dir>/.colnames as an R-serialized named integer vector.
pymisha.gtrack_array_set_colnames ¶
Set the column names of an array track.
R parity for gtrack.array.set_colnames. Writes
<track_dir>/.colnames in a format both R misha and pymisha read.
pymisha.gtrack_attr_get ¶
Get a single track attribute value.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
attr
|
Attribute name.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
str
|
Attribute value, or empty string if attribute doesn't exist. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If track does not exist. |
Examples:
>>> import pymisha as pm
>>> _ = pm.gdb_init_examples()
>>> pm.gtrack_attr_get("sparse_track", "created.by")
'...'
See Also
gtrack_attr_set : Set a track attribute. gtrack_attr_export : Export attributes for multiple tracks. gtrack_attr_import : Batch-import attributes from a table.
pymisha.gtrack_attr_set ¶
Set a track attribute value.
Tracks of model predictions conventionally carry these attributes, which
pymisha only stores and carries along: value.scale, the scale of the
values (e.g. "log2(x+1)" or "linear"); model, the model that
made them; checkpoint, the model checkpoint; and heldout.chroms,
the chromosomes held out of training. Like any attribute, they are
written with the track's column by gextract(file="x.parquet") and
restored by :func:gtrack_import_parquet.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
attr
|
Attribute name.
TYPE:
|
value
|
Attribute value. Set to empty string "" to remove the attribute.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If track does not exist or the attribute is read-only. |
See Also
gtrack_attr_get : Read a single track attribute. gtrack_attr_export : Export attributes for multiple tracks. gtrack_attr_import : Batch-import attributes from a table.
Examples:
pymisha.gtrack_attr_export ¶
gtrack_attr_export(tracks: str | list[str] | None = None, attrs: str | list[str] | None = None) -> pd.DataFrame
Export track attributes as a DataFrame.
| PARAMETER | DESCRIPTION |
|---|---|
tracks
|
Track name(s). If None, all tracks.
TYPE:
|
attrs
|
Attribute name(s) to include, in this order. If None, all attributes, the most common first.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
DataFrame
|
DataFrame with tracks as rows and attributes as columns. A name given twice appears once. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If any specified track does not exist. |
Examples:
>>> import pymisha as pm
>>> _ = pm.gdb_init_examples()
>>> pm.gtrack_attr_export()
>>> pm.gtrack_attr_export(tracks=["sparse_track", "dense_track"])
>>> pm.gtrack_attr_export(attrs=["created.by"])
See Also
gtrack_attr_import : Batch-import attributes from a DataFrame. gtrack_attr_get : Read a single attribute. gtrack_attr_set : Set a single attribute.
pymisha.gtrack_attr_import ¶
Bulk import track attributes from a DataFrame.
| PARAMETER | DESCRIPTION |
|---|---|
table
|
DataFrame with track names as index and attribute names as columns. Values are converted to strings. Empty string values are skipped (attribute not set for that track).
TYPE:
|
remove_others
|
If True, remove all non-readonly attributes not present in the table for tracks listed in the table.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If table is empty, any track in the index does not exist, or any attribute is read-only. |
See Also
gtrack_attr_export : Export attributes to a DataFrame. gtrack_attr_get : Read a single attribute. gtrack_attr_set : Set a single attribute.
Examples:
pymisha.gtrack_var_ls ¶
List track variables.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
pattern
|
Regex pattern to filter variable names. Default
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
list of str
|
Sorted list of variable names matching the pattern. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If track does not exist. |
Examples:
See Also
gtrack_var_get : Read a variable's value. gtrack_var_set : Store a variable. gtrack_var_rm : Delete a variable.
pymisha.gtrack_var_get ¶
Get the value of a track variable.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
var
|
Variable name.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
object
|
The stored Python object. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track or variable does not exist. |
Examples:
>>> import pymisha as pm
>>> _ = pm.gdb_init_examples()
>>> # pm.gtrack_var_get("dense_track", "my_var")
See Also
gtrack_var_set : Store a variable. gtrack_var_ls : List variables for a track. gtrack_var_rm : Delete a variable.
pymisha.gtrack_var_set ¶
Set the value of a track variable.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
var
|
Variable name.
TYPE:
|
value
|
Value to store. Can be any pickle-able Python object.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist. |
See Also
gtrack_var_get : Read a variable's value. gtrack_var_ls : List variables for a track. gtrack_var_rm : Delete a variable.
Examples:
pymisha.gtrack_var_rm ¶
Remove a track variable.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name.
TYPE:
|
var
|
Variable name to remove.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist. |
See Also
gtrack_var_set : Store a variable. gtrack_var_get : Read a variable's value. gtrack_var_ls : List variables for a track.
Examples:
Track Export¶
Functions to export tracks to standard genomic file formats.
pymisha.gtrack_export_bedgraph ¶
gtrack_export_bedgraph(track: str, file: str, intervals: DataFrame | str | None = None, iterator: int | None = None, name: str | None = None, header: bool = True) -> None
Export a track or track expression to bedGraph format.
Evaluates a track expression over the specified genomic intervals and
writes the result in standard bedGraph format (4-column, tab-separated:
chrom, start, end, value). NaN values are omitted from the output. The
lines are those R misha writes: values as R's as.character() writes
them (15 significant digits), coordinates as integers.
If the output file path ends in .gz, the output is gzip-compressed.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name or track expression (e.g.
TYPE:
|
file
|
Output file path. If it ends in
TYPE:
|
intervals
|
Genomic intervals to export, or the name of an interval set. If
TYPE:
|
iterator
|
Iterator bin size. If
TYPE:
|
name
|
Track name for the bedGraph header line. If
TYPE:
|
header
|
Write the
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
Called for its side effect of writing a file. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the track does not exist or is a 2D track. |
FileNotFoundError
|
If the output directory does not exist. |
Examples:
pymisha.gtrack_export_bigwig ¶
gtrack_export_bigwig(track: str, file: str, intervals: DataFrame | None = None, iterator: int | None = None) -> None
Export a track or track expression to BigWig format.
Creates a temporary bedGraph file via :func:gtrack_export_bedgraph and
then converts it to BigWig using the UCSC bedGraphToBigWig utility.
| PARAMETER | DESCRIPTION |
|---|---|
track
|
Track name or track expression.
TYPE:
|
file
|
Output file path (typically ending in
TYPE:
|
intervals
|
Genomic intervals to export. If
TYPE:
|
iterator
|
Iterator bin size. If
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
None
|
Called for its side effect of writing a file. |
| RAISES | DESCRIPTION |
|---|---|
RuntimeError
|
If |
ValueError
|
If the track is a 2D track. |
Examples: