`pc`: Point cloud data utilities (CLI)¶

Commands of moraine.cli.pc (source); the same functions are called in Python as mc.<name>(...) with import moraine.cli as mc. The help below is generated from the docstrings, like moraine COMMAND --help.

command	summary
`gix2bool`	Convert pc grid index to bool 2d array
`bool2gix`	Convert bool 2d array to grid index
`ras2pc`	Convert raster data to point cloud data
`pc-concat`	concatenate (and sort) point cloud dataset.
`ras2pc-ras-chunk`	Convert raster data to point cloud data that sorted by ras chunk
`pc2ras`	Convert point cloud data to raster data, filled with nan
`pc-hix`	Compute the hillbert index from grid index for point cloud data.
`pc-gix`	Compute the grid index from hillbert index for point cloud data.
`pc-sort`	Sort point cloud data according to the indices that sort `idx_in`.
`pc-union`	Get the union of two point cloud datasets
`pc-intersect`	Get the intersection of two point cloud datasets.
`pc-diff`	Get the points of the first point cloud dataset that are not in the second one.
`pc-logic-ras`	generate point cloud index based on logical operation of one raster image.
`pc-logic-pc`	generate point cloud index and data based on logical operation one point cloud data.
`pc-select-data`	generate point cloud data based on its index and one point cloud data

gix2bool¶

Convert pc grid index to bool 2d array

moraine gix2bool --gix STR --is_pc STR --shape INT INT [--chunks INT INT]

argument	type	default	description
`--gix`	str	required	input grid index (azimuth, range) of the point cloud, shape (n_points, 2), int
`--is_pc`	str	required	output bool raster, True at the points, shape (nlines, width)
`--shape`	tuple[int, int]	required	raster shape (nlines, width)
`--chunks`	tuple[int, int], 1000)	`(1000, 1000)`	output chunk size

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (not used by the example pipelines; required arguments only):

[[step]]
name = "gix2bool"
run = "gix2bool"
gix = "..."
is_pc = "..."
shape = "..."

In Python:

import moraine.cli as mc
mc.gix2bool(gix, is_pc, shape, chunks=(1000, 1000))

bool2gix¶

Convert bool 2d array to grid index

moraine bool2gix --is_pc STR --gix STR [--chunks VALUE]

argument	type	default	description
`--is_pc`	str	required	input bool array
`--gix`	str	required	output, point cloud grid index
`--chunks`	int	`100000`	output point chunk size

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (not used by the example pipelines; required arguments only):

[[step]]
name = "bool2gix"
run = "bool2gix"
is_pc = "..."
gix = "..."

In Python:

import moraine.cli as mc
mc.bool2gix(is_pc, gix, chunks=100000)

ras2pc¶

Convert raster data to point cloud data

moraine ras2pc --idx STR --ras PATH [PATH ...] --pc PATH [PATH ...] [--chunks VALUE] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx`	str	required	input grid index or hillbert index of the point cloud
`--ras`	str \| list	required	input path or list of paths of raster data, shape (nlines, width, ...)
`--pc`	str \| list	required	output, path (in string) or list of path for point cloud data
`--chunks`	int	`None`	point chunk size of the output data, same as `idx` by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/02_ps.toml, step ps_can_data):

[[step]]
name = "ps_can_data"
run = "ras2pc"
idx = "ps/ps_can_hix.zarr"
ras = ["raw/rslc.zarr", "raw/e.zarr", "raw/n.zarr", "raw/lon.zarr", "raw/lat.zarr", "ps/ras_adi.zarr"]
pc = ["ps/ps_can_rslc.zarr", "ps/ps_can_e.zarr", "ps/ps_can_n.zarr", "ps/ps_can_lon.zarr", "ps/ps_can_lat.zarr", "ps/ps_can_adi.zarr"]

Also in examples/03_ds.toml (step ds_en), examples/04_refine.toml (step lon_lat), examples/04_refine.toml (step geometry).

In Python:

import moraine.cli as mc
mc.ras2pc(idx, ras, pc, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

The array function mr.ras2pc does the same on numpy / cupy arrays in memory.

pc-concat¶

concatenate (and sort) point cloud dataset.

moraine pc-concat --pcs PATH [PATH ...] --pc PATH [PATH ...] [--key PATH [PATH ...]] [--chunks VALUE] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--pcs`	list \| str	required	list of path to pc or directory that hold one pc, or a list of that
`--pc`	list \| str	required	output, path of output or a list of that
`--key`	list \| str	`None`	input key(s) that sort the concatenated data (e.g. from `ras2pc_ras_chunk`); no sorting by default
`--chunks`	int	`None`	pc chunk size in output data, optional, same as first pc in pcs by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/03_ds.toml, step ds_can_concat):

[[step]]
name = "ds_can_concat"            # chunks -> one array each, in hilbert order
run = "pc-concat"
pcs = ["ds/chunks/ds_can_ph", "ds/chunks/ds_can_t_coh", "ds/chunks/ds_can_t_coh_w", "ds/chunks/ds_can_n_components"]
pc = ["ds/ds_can_ph.zarr", "ds/ds_can_t_coh.zarr", "ds/ds_can_t_coh_w.zarr", "ds/ds_can_n_components.zarr"]
key = ["ds/chunks/ds_can_key.zarr", "ds/chunks/gix2hix_key.zarr"]
chunks = 200000

In Python:

import moraine.cli as mc
mc.pc_concat(pcs, pc, key=None, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

ras2pc-ras-chunk¶

Convert raster data to point cloud data that sorted by ras chunk

moraine ras2pc-ras-chunk --gix STR --ras PATH [PATH ...] --pc PATH [PATH ...] --key STR [--chunks INT INT] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--gix`	str	required	input grid index (azimuth, range) of the point cloud, shape (n_points, 2), int
`--ras`	str \| list	required	input path or list of paths of raster data, shape (nlines, width, ...)
`--pc`	str \| list	required	output, path (directory) or list of path for point cloud data
`--key`	str	required	output, path for the key to sort generated pc in the directory back to gix order
`--chunks`	tuple[int, int]	`None`	(azimuth, range) raster chunk size used to split the points, same as the first `ras` by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/03_ds.toml, step ds_can_is_shp):

[[step]]
name = "ds_can_is_shp"            # SHPs of the candidates, one zarr per raster chunk
run = "ras2pc-ras-chunk"
gix = "ds/ds_can_gix_ras_order.zarr"
ras = "ds/ras_is_shp.zarr"
pc = "ds/chunks/ds_can_is_shp"
key = "ds/chunks/ds_can_key.zarr"
chunks = [1000, 1000]

In Python:

import moraine.cli as mc
mc.ras2pc_ras_chunk(gix, ras, pc, key, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

pc2ras¶

Convert point cloud data to raster data, filled with nan

moraine pc2ras --idx STR --pc PATH [PATH ...] --ras PATH [PATH ...] --shape INT INT [--chunks INT INT] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx`	str	required	input grid index or hillbert index of the point cloud
`--pc`	str \| list	required	input path or list of paths of point cloud data, shape (n_points, ...)
`--ras`	str \| list	required	output, path (in string) or list of path for raster data
`--shape`	tuple[int, int]	required	shape of one image (nlines,width)
`--chunks`	tuple[int, int], 1000)	`(1000, 1000)`	output chunk size
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (not used by the example pipelines; required arguments only):

[[step]]
name = "pc2ras"
run = "pc2ras"
idx = "..."
pc = "..."
ras = "..."
shape = "..."

In Python:

import moraine.cli as mc
mc.pc2ras(idx, pc, ras, shape, chunks=(1000, 1000), processes=False, n_workers=1, threads_per_worker=4)

The array function mr.pc2ras does the same on numpy / cupy arrays in memory.

pc-hix¶

Compute the hillbert index from grid index for point cloud data.

moraine pc-hix --gix STR --hix STR --shape INT INT

argument	type	default	description
`--gix`	str	required	input grid index (azimuth, range) of the point cloud, shape (n_points, 2), int
`--hix`	str	required	output hillbert index of the point cloud, shape (n_points,), int64
`--shape`	tuple[int, int]	required	raster shape (nlines, width)

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/02_ps.toml, step ps_can_hix):

[[step]]
name = "ps_can_hix"
run = "pc-hix"
gix = "ps/ps_can_gix.zarr"
hix = "ps/ps_can_hix_unsorted.zarr"
shape = "${shape}"

Also in examples/03_ds.toml (step ds_can_hix).

In Python:

import moraine.cli as mc
mc.pc_hix(gix, hix, shape)

The array function mr.pc_hix does the same on numpy / cupy arrays in memory.

pc-gix¶

Compute the grid index from hillbert index for point cloud data.

moraine pc-gix --hix STR --gix STR --shape INT INT

argument	type	default	description
`--hix`	str	required	input hillbert index of the point cloud, shape (n_points,), int64
`--gix`	str	required	output grid index (azimuth, range) of the point cloud, shape (n_points, 2), int
`--shape`	tuple[int, int]	required	raster shape (nlines, width)

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/05_unwrap.toml, step gix):

[[step]]
name = "gix"                      # mcf-pc needs the grid index of the points
run = "pc-gix"
hix = "pc/pc_hix.zarr"
gix = "unw/pc_gix.zarr"
shape = "${shape}"

In Python:

import moraine.cli as mc
mc.pc_gix(hix, gix, shape)

The array function mr.pc_gix does the same on numpy / cupy arrays in memory.

pc-sort¶

Sort point cloud data according to the indices that sort idx_in.

moraine pc-sort --idx_in STR --idx STR [--pc_in PATH [PATH ...]] [--pc PATH [PATH ...]] [--shape INT INT] [--chunks VALUE] [--key STR] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx_in`	str	required	input unsorted grid index or hillbert index of the input data
`--idx`	str	required	output, the sorted grid index or hillbert index
`--pc_in`	str \| list	`None`	input path or list of paths of the input point cloud data
`--pc`	str \| list	`None`	output, path (in string) or list of path for the output point cloud data
`--shape`	tuple[int, int]	`None`	(nline, width), faster if provided for grid index input
`--chunks`	int	`None`	chunk size in output data, same as `idx_in` by default
`--key`	str	`None`	output, path (in string) for the key of sorting
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/02_ps.toml, step ps_can_sort):

[[step]]
name = "ps_can_sort"              # hilbert order
run = "pc-sort"
idx_in = "ps/ps_can_hix_unsorted.zarr"
idx = "ps/ps_can_hix.zarr"

Also in examples/03_ds.toml (step ds_can_sort).

In Python:

import moraine.cli as mc
mc.pc_sort(idx_in, idx, pc_in=None, pc=None, shape=None, chunks=None, key=None, processes=False, n_workers=1, threads_per_worker=4)

The array function mr.pc_sort does the same on numpy / cupy arrays in memory.

pc-union¶

Get the union of two point cloud datasets. Points in both keep the data of the first point cloud.

moraine pc-union --idx1 STR --idx2 STR --idx STR [--pc1 PATH [PATH ...]] [--pc2 PATH [PATH ...]] [--pc PATH [PATH ...]] [--shape INT INT] [--chunks VALUE] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx1`	str	required	input grid index or hillbert index of the first point cloud, sorted
`--idx2`	str	required	input grid index or hillbert index of the second point cloud, sorted
`--idx`	str	required	output grid index or hillbert index of the union
`--pc1`	str \| list	`None`	input path or list of paths of the first point cloud data
`--pc2`	str \| list	`None`	input path or list of paths of the second point cloud data
`--pc`	str \| list	`None`	output path or list of paths of the point cloud data of the union
`--shape`	tuple[int, int]	`None`	image shape, faster if provided for grid index input
`--chunks`	int	`None`	point chunk size of the output data, same as `idx1` by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/02_ps.toml, step ps_can):

[[step]]
name = "ps_can"                   # union of both candidate sets
run = "pc-union"
idx1 = "ps/ps_can_1_gix.zarr"
idx2 = "ps/ps_can_2_gix.zarr"
idx = "ps/ps_can_gix.zarr"
shape = "${shape}"

Also in examples/04_refine.toml (step merge).

In Python:

import moraine.cli as mc
mc.pc_union(idx1, idx2, idx, pc1=None, pc2=None, pc=None, shape=None, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

The array function mr.pc_union does the same on numpy / cupy arrays in memory.

pc-intersect¶

Get the intersection of two point cloud datasets.

moraine pc-intersect --idx1 STR --idx2 STR --idx STR [--pc1 PATH [PATH ...]] [--pc2 PATH [PATH ...]] [--pc PATH [PATH ...]] [--shape INT INT] [--chunks VALUE] [--prefer_1 | --no-prefer_1] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx1`	str	required	input grid index or hillbert index of the first point cloud, sorted
`--idx2`	str	required	input grid index or hillbert index of the second point cloud, sorted
`--idx`	str	required	output grid index or hillbert index of the intersection
`--pc1`	str \| list	`None`	input path or list of paths of the first point cloud data
`--pc2`	str \| list	`None`	input path or list of paths of the second point cloud data
`--pc`	str \| list	`None`	output path or list of paths of the point cloud data of the intersection, taken from `pc1` or `pc2` (see `prefer_1`)
`--shape`	tuple[int, int]	`None`	image shape, faster if provided for grid index input
`--chunks`	int	`None`	point chunk size of the output data, same as `idx1` by default
`--prefer_1`	bool	`True`	take the output data from `pc1` (True) or from `pc2` (False)
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (not used by the example pipelines; required arguments only):

[[step]]
name = "pc_intersect"
run = "pc-intersect"
idx1 = "..."
idx2 = "..."
idx = "..."

In Python:

import moraine.cli as mc
mc.pc_intersect(idx1, idx2, idx, pc1=None, pc2=None, pc=None, shape=None, chunks=None, prefer_1=True, processes=False, n_workers=1, threads_per_worker=4)

The array function mr.pc_intersect does the same on numpy / cupy arrays in memory.

pc-diff¶

Get the points of the first point cloud dataset that are not in the second one.

moraine pc-diff --idx1 STR --idx2 STR --idx STR [--pc1 PATH [PATH ...]] [--pc PATH [PATH ...]] [--shape INT INT] [--chunks VALUE] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx1`	str	required	input grid index or hillbert index of the first point cloud, sorted
`--idx2`	str	required	input grid index or hillbert index of the second point cloud, sorted
`--idx`	str	required	output grid index or hillbert index of the points in `idx1` but not in `idx2`
`--pc1`	str \| list	`None`	input path or list of paths of the first point cloud data
`--pc`	str \| list	`None`	output path or list of paths of the point cloud data of these points, taken from `pc1`
`--shape`	tuple[int, int]	`None`	image shape, faster if provided for grid index input
`--chunks`	int	`None`	point chunk size of the output data, same as `idx1` by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (not used by the example pipelines; required arguments only):

[[step]]
name = "pc_diff"
run = "pc-diff"
idx1 = "..."
idx2 = "..."
idx = "..."

In Python:

import moraine.cli as mc
mc.pc_diff(idx1, idx2, idx, pc1=None, pc=None, shape=None, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

The array function mr.pc_diff does the same on numpy / cupy arrays in memory.

pc-logic-ras¶

generate point cloud index based on logical operation of one raster image.

moraine pc-logic-ras --ras VALUE --gix VALUE --operation STR [--chunks VALUE]

argument	type	default	description
`--ras`		required	input raster used in `operation`, shape (nlines, width)
`--gix`		required	output grid index (azimuth, range) of the point cloud, shape (n_points, 2), int, of the pixels where `operation` is True
`--operation`	str	required	numexpr expression on the raster, which is named `ras`, e.g. '(ras>=0)&(ras<=0.3)'
`--chunks`	int	`100000`	chunk size in output data, optional

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/02_ps.toml, step ps_can_adi):

[[step]]
name = "ps_can_adi"               # candidates 1: ADI below 0.4
run = "pc-logic-ras"
ras = "ps/ras_adi.zarr"
gix = "ps/ps_can_1_gix.zarr"
operation = "ras<0.4"
chunks = 200000

Also in examples/02_ps.toml (step ps_can_n2f), examples/03_ds.toml (step ds_can).

In Python:

import moraine.cli as mc
mc.pc_logic_ras(ras, gix, operation, chunks=100000)

pc-logic-pc¶

generate point cloud index and data based on logical operation one point cloud data.

moraine pc-logic-pc --idx_in STR --pc_in STR --idx STR --operation STR [--chunks VALUE]

argument	type	default	description
`--idx_in`	str	required	input grid index or hillbert index of the input point cloud
`--pc_in`	str	required	input point cloud data used in `operation`, shape (n_points,)
`--idx`	str	required	output grid index or hillbert index of the points where `operation` is True
`--operation`	str	required	numexpr expression on the point cloud data, which is named `pc_in`, e.g. '(pc_in>=0.1)&(pc_in<=0.5)'
`--chunks`	int	`None`	point chunk size of the output data, same as `idx_in` by default

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/03_ds.toml, step ds_connected):

[[step]]
name = "ds_connected"             # one connected component
run = "pc-logic-pc"
idx_in = "ds/ds_can_hix.zarr"
pc_in = "ds/ds_can_n_components.zarr"
idx = "ds/ds_connected_hix.zarr"
operation = "pc_in==1"

Also in examples/03_ds.toml (step ds), examples/04_refine.toml (step select).

In Python:

import moraine.cli as mc
mc.pc_logic_pc(idx_in, pc_in, idx, operation, chunks=None)

pc-select-data¶

generate point cloud data based on its index and one point cloud data. The index of generated point cloud data must in the index of the old one.

moraine pc-select-data --idx_in STR --idx STR --pc_in PATH [PATH ...] --pc PATH [PATH ...] [--shape INT INT] [--chunks VALUE] [--processes | --no-processes] [--n_workers VALUE] [--threads_per_worker VALUE]

argument	type	default	description
`--idx_in`	str	required	input grid index or hillbert index of the input point cloud, sorted
`--idx`	str	required	input grid index or hillbert index of the points to select, a subset of `idx_in`, sorted
`--pc_in`	str \| list	required	input path or list of paths of the input point cloud data
`--pc`	str \| list	required	output, path (in string) or list of path for the output point cloud data
`--shape`	tuple[int, int]	`None`	shape of the raster data the point cloud from, must be provided if `idx` is hix
`--chunks`	int	`None`	point chunk size of the output data, same as `idx` by default
`--processes`		`False`	use processes for the workers instead of threads
`--n_workers`		`1`	number of workers
`--threads_per_worker`		`4`	tasks a worker runs at the same time

Global options: --json, --traceback, --log FILE, -q / --quiet (conventions).

In a pipeline (examples/03_ds.toml, step ds_connected_t_coh_w):

[[step]]
name = "ds_connected_t_coh_w"     # the weighted temporal coherence of those points
run = "pc-select-data"
idx_in = "ds/ds_can_hix.zarr"
idx = "ds/ds_connected_hix.zarr"
pc_in = ["ds/ds_can_t_coh_w.zarr"]
pc = ["ds/ds_connected_t_coh_w.zarr"]

Also in examples/03_ds.toml (step ds_data), examples/04_refine.toml (step select_data).

In Python:

import moraine.cli as mc
mc.pc_select_data(idx_in, idx, pc_in, pc, shape=None, chunks=None, processes=False, n_workers=1, threads_per_worker=4)

pc: Point cloud data utilities (CLI)¶

gix2bool¶

bool2gix¶

ras2pc¶

pc-concat¶

ras2pc-ras-chunk¶

pc2ras¶

pc-hix¶

pc-gix¶

pc-sort¶

pc-union¶

pc-intersect¶

pc-diff¶

pc-logic-ras¶

pc-logic-pc¶

pc-select-data¶

`pc`: Point cloud data utilities (CLI)¶