跳转至

dl: Deep learning operators (CLI)

moraine.cli.dl 的命令(源码);同样的函数在 Python 里以 mc.<name>(...) 调用(import moraine.cli as mc)。下面的帮助由 docstring 生成,与 moraine COMMAND --help 相同(docstring 为英文)。

命令 摘要
n2f Noise2Fringe (n2f) filtering of raster interferograms
n2ft Noise2Fringe Transformer (n2ft) filtering of point cloud interferograms

n2f

Noise2Fringe (n2f) filtering of raster interferograms. With cuda the network runs in half precision. A task filters one output chunk with all image pairs: it holds the rslc of one processing chunk with its overlap for all images (8 bytes per pixel and image), the filtered interferograms of the output chunk (8 bytes per pixel and image pair) and the model's activations of one interferogram (about 1 kB per pixel of the processing chunk on the GPU, 2 kB on the CPU); with cuda all of it on the GPU. A worker runs threads_per_worker tasks at a time.

moraine n2f --rslc STR --intf STR --image_pairs FILE [--chunks INT INT] [--out_chunks INT INT] [--depths INT INT] [--model STR] [--cuda | --no-cuda] [--processes VALUE] [--n_workers VALUE] [--threads_per_worker VALUE]
参数 类型 默认值 说明
--rslc str 必需 input rslc stack, shape (nlines, width, nimages)
--intf str 必需 output filtered interferograms, complex64 with unit amplitude (nan where the input is nan), shape (nlines, width, n_image_pairs)
--image_pairs np.ndarray 必需 input image pairs (reference, secondary), shape (n_image_pairs, 2); make a file with moraine image-pairs
--chunks tuple[int, int] None (azimuth, range) processing chunk size, same as rslc by default; the processing chunks are cut at the output chunks
--out_chunks tuple[int, int] None (azimuth, range) chunk size of the output, same as rslc by default; it sets the memory of a worker
--depths tuple[int, int], 0) (0, 0) (azimuth, range) overlap in pixels between processing chunks, reduces chunk border effects
--model str None path to the model weights (.pth), use the model comes with this package by default
--cuda bool False if use cuda for processing, false by default
--processes None use processes (True) or threads (False) for the workers, only for cpu processing. Default: True
--n_workers None number of workers. Default: 1 for cpu, one per GPU for cuda
--threads_per_worker None output chunks a worker filters at the same time. Default: 1

全局选项:--json、--traceback、--log FILE、-q / --quiet(约定)。

在 pipeline 文件中(examples/02_ps.toml,步骤 n2f):

[[step]]
name = "n2f"                      # Noise2Fringe filtering of the sequential interferograms
run = "n2f"
rslc = "raw/rslc.zarr"
intf = "ps/ras_n2f_intf.zarr"
image_pairs = "ps/pairs_seq.txt"

在 Python 中:

import moraine.cli as mc
mc.n2f(rslc, intf, image_pairs, chunks=None, out_chunks=None, depths=(0, 0), model=None, cuda=False, processes=None, n_workers=None, threads_per_worker=None)

数组函数 mr.n2f 对内存中的 numpy / cupy 数组做同样的事。

n2ft

Noise2Fringe Transformer (n2ft) filtering of point cloud interferograms. Every worker filters one output chunk of points with all image pairs at a time: it holds the rslc of the points of the chunk and of their halos for the images of the pairs (8 bytes per point and image), the filtered interferograms of the chunk (8 bytes per point and image pair) and about 400 bytes per point of the chunk more; with cuda a model call holds about a fifth of the GPU memory. Before, the main process finds the halos of all processing chunks with about 70 bytes per point and 32 threads of 16 x k x chunks bytes (41 MB each by default).

moraine n2ft --x STR --y STR --rslc STR --intf STR --image_pairs FILE [--chunks VALUE] [--out_chunks VALUE] [--k VALUE] [--model STR] [--cuda | --no-cuda] [--compile | --no-compile] [--processes VALUE] [--n_workers VALUE] [--threads_per_worker VALUE]
参数 类型 默认值 说明
--x str 必需 input x coordinate of the points (e.g. longitude or easting), shape (n_points,)
--y str 必需 input y coordinate of the points (e.g. latitude or northing), shape (n_points,)
--rslc str 必需 input rslc of the points, shape (n_points, nimages)
--intf str 必需 output filtered interferograms, complex64 with unit amplitude, shape (n_points, n_image_pairs)
--image_pairs np.ndarray 必需 input image pairs (reference, secondary), shape (n_image_pairs, 2); make a file with moraine image-pairs
--chunks int None number of points per processing chunk, same as rslc by default; the processing chunks do not cross the output chunks
--out_chunks int None point chunk size of the output, same as rslc by default; it sets the memory of a worker
--k int 128 number of nearest neighbours of every point of a processing chunk that are filtered with it (halo)
--model str None path to the model weights (.pth), use the model comes with this package by default
--cuda bool False if use cuda for processing, false by default
--compile bool None compile the model with torch.compile in every worker: the model then runs about 3 times faster on a GPU, but the compilation takes 15-40 s per worker (less when torch has cached it on disk). Default: when points times image pairs is at least 1e8
--processes None use processes (True) or threads (False) for the workers, only for cpu processing. Default: True
--n_workers None number of workers. Default: 1 for cpu, one per GPU for cuda
--threads_per_worker None tasks a worker runs at the same time. Default: 1

全局选项:--json、--traceback、--log FILE、-q / --quiet(约定)。

在 pipeline 文件中(examples/04_refine.toml,步骤 n2ft):

[[step]]
name = "n2ft"                     # Noise2Fringe Transformer filtering of the sequential interferograms
run = "n2ft"
x = "pc/pc_can_lon.zarr"
y = "pc/pc_can_lat.zarr"
rslc = "pc/pc_can_ph.zarr"
intf = "pc/pc_can_n2ft_intf.zarr"
image_pairs = "pc/pairs_seq.txt"
chunks = 20000

在 Python 中:

import moraine.cli as mc
mc.n2ft(x, y, rslc, intf, image_pairs, chunks=None, out_chunks=None, k=128, model=None, cuda=False, compile=None, processes=None, n_workers=None, threads_per_worker=None)

数组函数 mr.n2ft 对内存中的 numpy / cupy 数组做同样的事。