SurfaceTopography.IO package

Submodules

SurfaceTopography.IO.AL3D module

class SurfaceTopography.IO.AL3D.AL3DReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.BCR module

class SurfaceTopography.IO.BCR.BCRReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.DATX module

class SurfaceTopography.IO.DATX.DATXReader(fobj)

Bases: ReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(fobj)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.DI module

class SurfaceTopography.IO.DI.DIReader(file_path)

Bases: ReaderBase

__init__(file_path)

Load Digital Instrument’s Nanoscope files.

Parameters:: file_path (filename or file object) – File or data stream to open.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, channel_id=None, height_channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.DZI module

SurfaceTopography.IO.DZI.write_dzi(data: ndarray, name: str, physical_sizes: Tuple[float, float], unit: str, root_directory: str = '.', tile_size: int = 256, overlap: int = 1, format: str = 'jpg', meta_format: str = 'xml', colorbar_title=None, cmap=None, **kwargs)

Write generic numpy array to a Deep Zoom Image file. This can for example be used to create a zoomable topography with OpenSeadragon (https://openseadragon.github.io/).

Additional keyword parameters are passed to Pillow’s save function.

Parameters:

data (np.ndarray) – Two-dimensional array containing the data.
name (str) – Name of the exported file. This is used as a prefix. Output filter create the file name.xml that contains the metadata and a directory `name`_files that contains the rendered image files at different levels.
physical_sizes (tuple of floats) – Linear physical sizes of the two-dimensional array.
unit (str) – Length units of physical sizes.
root_directory (str, optional) – Root directory where to place name.xml and `name`_files. (Default: ‘.’)
tile_size (int, optional) – Size of individual tiles. (Default: 256)
overlap (int, optional) – Overlap of tiles. (Default: 1)
format (str, optional) – Data output format. Note that PNG files have seams at the boundary between tiles. Use ‘npy’ to output raw data in the native numpy format. Use ‘nc’ to output raw data as NetCDF files. (Default: ‘jpg’)
meta_format (str, optional) – Format for metadata information (the DZI file), can be ‘xml’ or ‘json’. (Default: ‘xml’)
colorbar_title (str, optional) – Additional title for the color bar that is dumped into the DZI file. Ignored if format is ‘npy’ or ‘nc’. (Default: None)
cmap (str or colormap, optional) – Color map for rendering the topography. Ignored if format is ‘npy’ or ‘nc’. (Default: None)

Returns:

manifest – List with names of files created during write operation

Return type:

list of str

SurfaceTopography.IO.DZI.write_topography_dzi(self: UniformTopographyInterface, name: str, root_directory: str = '.', tile_size: int = 256, overlap: int = 1, format: str = 'jpg', meta_format: str = 'xml', cmap=None, **kwargs)

Write topography to a Deep Zoom Image file. This can for example be used to create a zoomable topography with OpenSeadragon (https://openseadragon.github.io/).

Additional keyword parameters are passed to Pillow’s save function.

Parameters:

self (Topography) – Topogaphy to export
name (str) – Name of the exported file. This is used as a prefix. Output filter create the file name.xml that contains the metadata and a directory `name`_files that contains the rendered image files at different levels.
root_directory (str) – Root directory where to place name.xml and `name`_files.
tile_size (int, optional) – Size of individual tiles. (Default: 256)
overlap (int, optional) – Overlap of tiles. (Default: 1)
format (str, optional) – Image format. Note that PNG files have seems at the boundary between tiles. (Default: jpg)
meta_format (str, optional) – Format for metadata information (the DZI file), can be ‘xml’ or ‘json’. (Default: ‘xml’)
cmap (str or colormap, optional) – Color map for rendering the topography. (Default: None)

Returns:

filenames – List with names of files created during write operation

Return type:

list of str

SurfaceTopography.IO.DZI.generate_manifest(name: str, nb_grid_pts: Tuple[int, int], root_directory: str = '.', tile_size: int = 256, format: str = 'jpg', meta_format: str = 'xml')

Generate the file manifest, i.e. the list of files, that would be generated when writing the DZI.

Parameters:

name (str) – Name of the exported file. This is used as a prefix. Output filter create the file name.xml that contains the metadata and a directory `name`_files that contains the rendered image files at different levels.
nb_grid_pts (tuple of ints) – Number of grid points.
root_directory (str, optional) – Root directory where to place name.xml and `name`_files. (Default: ‘.’)
tile_size (int, optional) – Size of individual tiles. (Default: 256)
format (str, optional) – Data output format. Note that PNG files have seams at the boundary between tiles. Use ‘npy’ to output raw data in the native numpy format. Use ‘nc’ to output raw data as NetCDF files. (Default: ‘jpg’)
meta_format (str, optional) – Format for metadata information (the DZI file), can be ‘xml’ or ‘json’. (Default: ‘xml’)

Returns:

manifest – List with names of files created during write operation

Return type:

list of str

SurfaceTopography.IO.EZD module

class SurfaceTopography.IO.EZD.EZDReader(file_path)

Bases: ReaderBase

__init__(file_path)

Load NanoSurf easyScan data files.

Parameters:: file_path (filename or file object) – File or data stream to open.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.FRT module

class SurfaceTopography.IO.FRT.FRTReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.FromFile module

SurfaceTopography profile from file input

SurfaceTopography.IO.FromFile.binary(func)

SurfaceTopography.IO.FromFile.make_wrapped_reader(reader_func, class_name='WrappedReader', format=None, mime_types=None, file_extensions=None, name=None, description=None)

SurfaceTopography.IO.FromFile.read_hgt(fobj, *args, **kwargs)

class SurfaceTopography.IO.FromFile.HGTReader(fobj)

Bases: ReaderBase

emulates the new implementation of the readers

__init__(fobj)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.GWY module

class SurfaceTopography.IO.GWY.GWYReader(file_path)

Bases: ReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(file_path)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.GWY.write_gwy(self, fobj, name='Topography')

Write topography to a Gwyddion (GWY) file.

GWY is the native file format of the open-source SPM visualization and analysis software Gwyddion.

Parameters:

self (Topography) – The topography to write.
fobj (str or file-like object) – File path or file-like object to write to.
name (str, optional) – Name/title for the channel. (Default: ‘Topography’)

SurfaceTopography.IO.H5 module

class SurfaceTopography.IO.H5.H5Reader(fobj)

Bases: ReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(fobj)

close()

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.IBW module

class SurfaceTopography.IO.IBW.IBWReader(file_path)

Bases: ReaderBase

__init__(file_path)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, channel_id=None, height_channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.JPK module

class SurfaceTopography.IO.JPK.JPKReader(file_path)

Bases: DeclarativeReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

SurfaceTopography.IO.LEXT module

class SurfaceTopography.IO.LEXT.LEXTReader(file_path)

Bases: DeclarativeReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

SurfaceTopography.IO.MDT module

class SurfaceTopography.IO.MDT.MDTReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.MI module

class SurfaceTopography.IO.MI.MIReader(file_path)

Bases: DeclarativeReaderBase

property info: dict: Return all the available file-wide metadata as a dict.

SurfaceTopography.IO.MNT module

Reader for Digital Surf Mountains MNT files.

The MNT format is a Microsoft Compound Document File (OLE) containing: - ImagePreview: JPEG preview image - ScopedContents: Binary data with TLV-encoded metadata and zlib-compressed

ScopedResults: Parameter data
XmlHeader: UTF-16 encoded XML metadata

The ScopedContents stream uses a hierarchical TLV (Tag-Length-Value) structure: - First 8 bytes: uint64 LE size field (= stream size - 8) - Remaining bytes: TLV entries with tag (uint16 LE) + size (uint64 LE) + data

Height data container (tag 0x02BD) structure: - compressed_blocks contains multiple sections separated by uint64 size fields:

Image parameters (in Section 3, innermost tag 0x0002 container): - 0x0007: Width in pixels (uint32) - 0x0008: Height in pixels (uint32) - 0x0009: Physical size X in mm (double) - 0x000a: Physical size Y in mm (double)

Height data format: - Stored as int32 values - Data type tag (tag 0x0001 inside height_data container) indicates masking:

Height scale factor stored in pixel_scales container (typically 10 nm/count)

Note: 0xFFFF tags are used as section markers/delimiters and must be skipped when parsing nested containers.

class SurfaceTopography.IO.MNT.MNTReader(fobj)

Bases: ReaderBase

__init__(fobj)

Load Digital Surf Mountains data files.

Parameters:: fobj (filename or file object) – File or data stream to open.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info=None, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.Matlab module

class SurfaceTopography.IO.Matlab.MatReader(fobj)

Bases: ReaderBase

__init__(fobj)

Reads a surface profile from a Matlab file and presents in in a SurfaceTopography-conformant manner.

All two-dimensional arrays present in the matlab data file are returned.

Parameters:: fobj (filename or file object) – File to read.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.MetroPro module

class SurfaceTopography.IO.MetroPro.MetroProReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.Mitutoyo module

class SurfaceTopography.IO.Mitutoyo.MitutoyoReader(fobj)

Bases: ReaderBase

Mitutoyo SurfTest surface roughness testers produce specifically formatted Excel spreadsheets.

__init__(fobj)

Open Excel spreadsheet produced by Mitutoyo SurfTest surface roughness testers.

The reader expects a line scan by positions and heights in columns 5 and 6 (E, F) and tries to extract standard roughness metrics from column 1 (A) on the sheet ‘DATA’. The reader expects the acquisition date in column 5 (E) row 2 on sheet ‘Certificate’.

Parameters:: fobj (filename or file object) – File to read.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.NC module

class SurfaceTopography.IO.NC.NCReader(fobj, communicator=None)

Bases: ReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(fobj, communicator=None)

close()

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

property communicator

SurfaceTopography.IO.NC.write_nc_uniform(topography, fobj, format='NETCDF3_64BIT_OFFSET')

Write topography into a NetCDF file.

Parameters:

topography (SurfaceTopography) – The topography to write to disk.
fobj (str or stream) – Name of the NetCDF file or file stream
format (str) – NetCDF file format. Default is ‘NETCDF3_64BIT_OFFSET’.

SurfaceTopography.IO.NC.write_nc_nonuniform(line_scan, fobj, format='NETCDF3_64BIT_OFFSET')

Write nonuniform line scan into a NetCDF file.

Parameters:

line_scan (SurfaceTopography) – The topography to write to disk.
fobj (str or stream) – Name of the NetCDF file or file stream
format (str) – NetCDF file format. Default is ‘NETCDF3_64BIT_OFFSET’.

SurfaceTopography.IO.NMM module

class SurfaceTopography.IO.NMM.NMMReader(fobj, rtol=1e-06)

Bases: ReaderBase

__init__(fobj, rtol=1e-06)

Initialize the NMMReader object.

Parameters:

fobj (file-like object or callable) – The file object or a callable that returns a file object to the ZIP file containing the NMM data.
rtol (float, optional) – Relative tolerance for detecting uniform grids. (Default: 1e-6).

Raises:

FileFormatMismatch – If the ZIP file does not contain exactly two files or if the files are not a DSC and a DAT file.
CorruptFile – If the number of columns in the DAT file does not match the number of entries in the DSC file or if the number of data points in the DSC file does not match the number of rows in the DAT file.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.NMM.read_nmm(dsc_file, dat_file, rtol=1e-06)

Convenience function for reading a Nanomeasuring Machine (NMM) profile.

Parameters:

dsc_file (str or file-like object) – Path to the DSC file or file-like object.
dat_file (str or file-like object) – Path to the DAT file or file-like object.
rtol (float, optional) – Relative tolerance for detecting uniform grids. (Default: 1e-6).

SurfaceTopography.IO.NMS module

Reader for Nanofocus NMS files.

class SurfaceTopography.IO.NMS.NMSReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.NPY module

In MPI Parallelized programs:

we have to use MPI.File.Open instead of open to allow several processors to access the same file simultaneously
make the file reading in 3 steps:
- read the nb_grid_pts only (Reader.__init__)
- make the domain decomposition according to the nb_grid_pts
- load the relevant subdomain on each processor in Reader.topography()

class SurfaceTopography.IO.NPY.NPYReader(fobj, communicator=<NuMPI.MPIStub.Intracomm object>)

Bases: ReaderBase

NPY is a file format made specially for numpy arrays. They contain no extra metadata so we use directly the implementation from numpy and NuMPI.

For a description of the file format, see here: https://docs.scipy.org/doc/numpy/reference/generated/numpy.lib.format.html

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(fobj, communicator=<NuMPI.MPIStub.Intracomm object>)

Open file in the NPY format.

Parameters:

fobj (str) – Name of the file
communicator (mpi4py MPI communicator or NuMPI stub communicator) – MPI communicator object for parallel loads.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.NPY.save_npy(fn, topography)

SurfaceTopography.IO.OIR module

class SurfaceTopography.IO.OIR.OirChunkType(*values)

Bases: IntEnum

METADATA_LIST = 0

METADATA = 1

IMAGE = 2

DATA_BLOCK = 3

SPACER = 5

class SurfaceTopography.IO.OIR.OirMetadataBlockType(*values)

Bases: IntEnum

VERSION = 1

PROPERTIES = 2

ANNOTATIONS = 3

OVERLAYS = 4

LOOKUP_TABLES = 5

TOPOGRAPHY_PREFIX = 6

DATASETS = 7

TOPOGRAPHY_UUIDS = 8

TOPOGRAPHY_PREFIX_AGAIN = 9

CAMERA = 10

LOOKUP_TABLES2 = 11

CAMERA_PREFIX = 12

CAMERA_UUIDS = 13

EVENTS = 14

class SurfaceTopography.IO.OIR.OIRReader(file_path)

Bases: DeclarativeReaderBase

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

class SurfaceTopography.IO.OIR.POIRReader(fobj)

Bases: ReaderBase

__init__(fobj)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.OPD module

class SurfaceTopography.IO.OPD.OPDReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.OPDx module

class SurfaceTopography.IO.OPDx.OPDxReader(file_path)

Bases: ReaderBase

__init__(file_path)

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

info_from_manifest(prefix)

read_linescan_channel_infos(channel_index): Read line scan (profile) information

read_topography_channel_infos(channel_index): Read topography (2D map) information

class SurfaceTopography.IO.OPDx.DektakUnit(name, symbol, value, extra)

Bases: tuple

extra: Alias for field number 3

name: Alias for field number 0

symbol: Alias for field number 1

value: Alias for field number 2

SurfaceTopography.IO.OPDx.read_dimension2d_content(stream)

Reads in information about a 2d dimension.

Parameters:

stream (bytes) – The input buffer

Returns:

unit (DektakUnit) – The unit
divisor (float) – Divisor

SurfaceTopography.IO.OS3D module

Reader for Digital Metrology OmniSurf3D OS3D files.

Format documentation: https://digitalmetrology.com/omnisurf3d-file-format/

class SurfaceTopography.IO.OS3D.OS3DReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.PLU module

class SurfaceTopography.IO.PLU.PLUReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.PLUX module

class SurfaceTopography.IO.PLUX.PLUXReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.PS module

class SurfaceTopography.IO.PS.PSReader(file_path)

Bases: DeclarativeReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

SurfaceTopography.IO.QEP module

class SurfaceTopography.IO.QEP.QEPReader(fobj)

Bases: ReaderBase

Reader for contour/profile scans of Mahr instruments driven by the MarWin software (e.g. the MarSurf and MarForm series).

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(fobj)

Open a QEP profile archive.

Parameters:: fobj (str or file object) – File name or stream.

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, channel_id=None, height_channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.Reader module

class SurfaceTopography.IO.Reader.MagicMatch(*values)

Bases: Enum

Result of magic-based file format check.

YES = 'yes'

NO = 'no'

MAYBE = 'maybe'

class SurfaceTopography.IO.Reader.DataKind(*values)

Bases: Enum

Kind of data stored in a channel.

HEIGHT = 'height'

VOLTAGE = 'voltage'

CURRENT = 'current'

PHASE = 'phase'

AMPLITUDE = 'amplitude'

ERROR = 'error'

DEFLECTION = 'deflection'

FRICTION = 'friction'

OTHER = 'other'

class SurfaceTopography.IO.Reader.ChannelInfo(reader, index, name=None, dim=None, nb_grid_pts=None, physical_sizes=None, height_scale_factor=None, periodic=None, uniform=None, undefined_data=None, unit=None, data_kind=None, data_unit=None, info={}, tags={})

Bases: object

Information on topography channels contained within a file.

__init__(reader, index, name=None, dim=None, nb_grid_pts=None, physical_sizes=None, height_scale_factor=None, periodic=None, uniform=None, undefined_data=None, unit=None, data_kind=None, data_unit=None, info={}, tags={})

Initialize the channel. Use as much information from the file as possible by passing it in the keyword arguments. Keyword arguments can be None if the information cannot be determined. (This is the default for all keyword arguments.)

Parameters:

reader (ReaderBase) – Reader instance this channel is coming from.
index (int) – Index of channel in the file, where zero is the first channel.
name (str, optional) – Name of the channel. If no name is given, “channel <index>” will be used, where “<index>” is replaced with the index.
dim (int, optional) – Number of dimensions.
nb_grid_pts (tuple of ints, optional) – Number grid points in each dimension.
physical_sizes (tuple of floats, optional) – Physical dimensions.
height_scale_factor (float, optional) – Number by which all heights have been multiplied.
periodic (bool, optional) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
uniform (bool, optional) – Data is uniform.
has_undefined_data (bool, optional) – Underlying data has missing/undefined points.
unit (str, optional) – Length unit of measurement (lateral dimensions).
data_kind (DataKind, optional) – Kind of data stored in this channel (HEIGHT, VOLTAGE, etc.). Defaults to HEIGHT if not specified.
data_unit (str, optional) – Unit for data values (z-axis). For height channels, this is typically the same as unit. For non-height channels, this could be ‘V’, ‘A’, ‘deg’, etc.
info (dict, optional) – Meta data found in the file. (Default: {})
tags (dict, optional) – Additional meta data required internally by the reader

topography(physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. The method allows to override data found in the data file.

Parameters:

physical_sizes (tuple of floats, optional) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float, optional) – Factor by which the heights should be multiplied. This parameter is only available for file formats that do not provide metadata information about units and associated length scales.
unit (str, optional) – Length unit of measurement.
info (dict, optional) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool, optional) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing)
subdomain_locations (tuple of ints, optional) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints, optional) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

property index: Unique integer channel index.

property name: Name of the channel. Can be used in a UI for identifying a channel.

property dim: 1 for line scans and 2 for topography maps.

property nb_grid_pts: Number of grid points in each direction, either 1 or 2 elements depending on the dimension of the topography.

property physical_sizes

If the physical size can be determined from the file, a tuple is returned. The tuple has 1 or 2 elements (size_x, size_y), depending on the dimension of the topography.

If the physical size can not be determined from the file, then None is returned.

Note that the physical sizes obtained from the file can be overwritten by passing a physical_sizes argument to the topography method that returns the topography object.

property height_scale_factor

If a height scale factor can be determined from the file, a float is returned.

If no height scale factor can be determined from the file, then None is returned.

Note that the height scale factor obtained from the file can be overwritten by passing a height_scale_factor argument to the topography method that returns the topography object.

property is_periodic: Return whether the topography is periodically repeated at the boundaries.

property is_uniform: Return whether the topography is uniform.

property has_undefined_data: Return whether the topography has undefined data.

property pixel_size

The pixel size is returned as the phyiscal size divided by the number of grid points. If the physical size can be determined from the file, a tuple is returned. The tuple has 1 or 2 elements (size_x, size_y), depending on the dimension of the topography.

If the physical size can not be determined from the file, then None is returned.

Note that the physical sizes obtained from the file can be overwritten by passing a physical_sizes argument to the topography method that returns the topography object. Overriding the physical size will also affect the pixel size.

property area_per_pt: The area per point is returned as the product over the pixel size tuple. If pixel_size returns None, than also area_per_pt returns None.

property unit

Length unit for lateral dimensions (x, y).

This is the unit for physical_sizes. For height channels, this is typically also the unit of the height data. For non-height channels, see data_unit for the unit of the data values.

property lateral_unit

Length unit for lateral dimensions (x, y).

Alias for unit property.

property data_kind

Kind of data stored in this channel.

Returns:: The kind of data (HEIGHT, VOLTAGE, CURRENT, PHASE, etc.)
Return type:: DataKind

property data_unit

Unit for data values (z-axis).

For height channels, this returns the lateral unit if no specific data unit was provided (since height is typically in the same unit as lateral dimensions). For non-height channels, this could be ‘V’, ‘A’, ‘deg’, etc.

Returns:: Unit string for the data values.
Return type:: str or None

property is_height_channel

Return whether this channel contains height data.

Returns:: True if this is a height channel, False otherwise.
Return type:: bool

property channel_id

Stable unique string identifier for this channel.

This ID is stable across different reads of the same file and does not depend on channel ordering. Format is ‘{name}’ if the name is unique, or ‘{name}#{n}’ if there are multiple channels with the same name.

Returns:: Stable channel identifier.
Return type:: str

property height_index

Index among height channels only.

This provides backwards compatibility for code that expects channels to be indexed only among height data. Non-height channels return None.

Returns:: Index among height channels, or None for non-height channels.
Return type:: int or None

property info: dict: A dictionary containing additional information (metadata) not used by SurfaceTopography itself, but required by third-party application.

property tags: A dictionary containing additional information (metadata) used internally by the reader.

property reader

class SurfaceTopography.IO.Reader.ReaderBase

Bases: object

Base class for topography readers. These are object that allow to open a file (from filename or stream object), inspect its metadata and then request to load a topography from it. Metadata is typically extracted without reading the full file.

Readers should adhere to the following design rules: 1. Opening a file should be fast and therefore not read the whole data.

Data corruption must be detected when opening the file. The topography method must not fail because of file corruption issues.

These rules are important to allow smooth operation of the readers in the web application TopoBank.

classmethod format(): Short string identifier for this file format. Identifier must be unique and is typically equal to the file extension of this format.

classmethod mime_types(): MIME types supported by this reader.

classmethod file_extensions(): A list of typical file extensions for this reader. Can be None if there are no typical file extensions.

classmethod name(): Short name of this file format.

classmethod description(): Long description of this file format. Should be formatted as markdown.

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

close()

height_index_to_channel_id(height_index)

Convert a height channel index to a stable channel ID.

This utility function is useful for migrating databases that stored channel references using the old height-only index system to the new stable channel_id system.

Parameters:: height_index (int) – Index among height channels (0-based).
Returns:: The stable channel_id for the height channel at that index.
Return type:: str
Raises:: ValueError – If no height channel exists at the given index.

Examples

>>> reader = open_topography("scan.ibw")
>>> # Migrate old database entry
>>> old_index = 0  # stored in database
>>> new_id = reader.height_index_to_channel_id(old_index)
>>> # Store new_id in database instead

channel_id_to_height_index(channel_id)

Convert a stable channel ID to a height channel index.

This utility function can be used to convert channel IDs back to height indices for backwards compatibility with systems that require numeric indices.

Parameters:: channel_id (str) – The stable channel identifier.
Returns:: The height index for the channel with that ID.
Return type:: int
Raises:: ValueError – If no channel with the given ID is found, or if the channel is not a height channel.

Examples

>>> reader = open_topography("scan.ibw")
>>> height_idx = reader.channel_id_to_height_index("Height")

abstract property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

property height_channels

Returns a list of :obj:`ChannelInfo`s for height channels only.

This provides backwards compatibility for code that expects only height data channels.

property default_channel: Return the default channel. This is often the first channel with height information.

abstractmethod topography(channel_index=None, channel_id=None, height_channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

class SurfaceTopography.IO.Reader.CompoundLayout(structures, name=None, context_mapper=<function _identity>)

Bases: LayoutWithNameBase

Declare a file layout

__init__(structures, name=None, context_mapper=<function _identity>)

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.If(*args, context_mapper=None)

Bases: object

Switch structure type dependent on data

__init__(*args, context_mapper=None)

to_dict()

name(context)

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.Switch(key, cases, default=None)

Bases: object

Select a structure from a mapping, keyed by a context value.

This is the layout-level analog of a switch/case statement, for tag-driven formats where a tag read earlier determines the layout of the following block (see the TLVContainer alternative for formats where tag and block are adjacent).

Parameters:

key (callable or expression) – Evaluated against the context; the result selects the case.
cases (dict) – Maps key values to layout class instances.
default (layout, optional) – Layout used when no case matches. If None, a non-matching key raises CorruptFile. (Default: None)

__init__(key, cases, default=None)

name(context)

to_dict()

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.Check(condition, exception=<class 'SurfaceTopography.Exceptions.CorruptFile'>, comment=None)

Bases: object

Validates a condition against the context without reading from the stream. Use this for consistency checks between previously parsed values, e.g. cross-structure validation that a Validate field hook cannot express.

Parameters:

condition (bool, callable or expression) – Evaluated against the context; a false result raises.
exception (Exception, optional) – Exception class to raise. (Default: CorruptFile)
comment (str, optional) – Message of the raised exception (and documentation of the check).

__init__(condition, exception=<class 'SurfaceTopography.Exceptions.CorruptFile'>, comment=None)

to_dict()

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.Skip(size=None, comment=None)

Bases: object

Skips over bytes in a binary stream without storing them.

Use this for TLV entries or file sections that should be ignored.

Parameters:

size (int, callable, or None) – Size of the data block in bytes. Can be: - An integer for fixed size - A callable that takes context and returns size - None to read size from context[‘_block_size’] (for TLV parsing)
comment (str, optional) – Description of what is being skipped (for documentation).

__init__(size=None, comment=None)

to_dict()

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.Seek(offset, comment=None)

Bases: object

Moves the stream to an absolute position. Use this for formats whose header carries absolute offsets to data regions.

Parameters:

offset (int, callable or expression) – Absolute stream position to seek to.
comment (str, optional) – Description of the target region (for documentation).

__init__(offset, comment=None)

to_dict()

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.SizedChunk(size, structure, mode='read-once', name=None, context_mapper=None, debug=False)

Bases: LayoutWithNameBase

__init__(size, structure, mode='read-once', name=None, context_mapper=None, debug=False)

Declare a file portion of a specific size. This allows bounds checking, i.e. raising exception if too much or too little data is read.

Parameters:

size (function or int) – Function that returns the size of the chunk and takes as input the current context.
structure (structure definition) – Definition of the structure within this chunk.
mode (str) – Reading mode, can be ‘read-once’, ‘skip-missing’ or ‘loop’. ‘read-once’ and ‘skip-missing’ read the containing block once, but ‘skip-missing’ does not complain if this is not the complete chunk. ‘loop’ repeats the containing block until the full chunk has been read. (Default: ‘read-once’)
name (str) – Context name of this block. Required if mode is ‘loop’.
context_mapper (callable) – Function that takes a context and returns a new context.
debug (Bool) – Print boundary offsets of chunk. (Default: False)

from_stream(stream_obj, context)

Decode stream into dictionary.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

decoded_data – Dictionary with decoded data entries.

Return type:

dict

class SurfaceTopography.IO.Reader.For(range, structure, name=None)

Bases: LayoutWithNameBase

Repeat structure

__init__(range, structure, name=None)

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.ForEach(items, structure, name=None)

Bases: LayoutWithNameBase

Repeat a structure for each element of a previously parsed list, e.g. for the payloads of a directory of blocks. The structure’s context contains the current element as item and its index as item_index (mirroring the foreach of channel bindings); the per-element results are collected into a list.

Parameters:

items (callable or expression) – Evaluated against the context; must yield a list.
structure (structure definition) – Structure parsed once per list element.
name (str) – Context name of the list of results.

__init__(items, structure, name=None)

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.Let(values)

Bases: object

Stores computed values into the context without reading from the stream. Use this to make values from an enclosing scope (e.g. the item of a ForEach) part of a structure’s result, or to name intermediate expressions.

Parameters:: values (dict) – Maps names to literals or expressions evaluated against the context.

__init__(values)

to_dict()

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.While(*args, name=None)

Bases: LayoutWithNameBase

Repeat as long as condition is met

__init__(*args, name=None)

from_stream(stream_obj, context)

class SurfaceTopography.IO.Reader.DeclarativeReaderBase(file_path)

Bases: ReaderBase

Base class for automatic readers.

Subclasses declare the file structure via the _file_layout class member and either override the channels property or declare the mapping from parsed metadata to channels via the _channel_bindings class member (see the format description contract, docs/format_description_contract.rst).

_channel_bindings is a list of dictionaries with the entries

name, dim, unit, periodic, uniform: literals or expressions
nb_grid_pts, physical_sizes: expressions evaluating to tuples
height_scale_factor: literal, expression or absent (unscaled)
info: (nested) dictionary of literals or expressions; entries that evaluate to None are omitted from the channel information (except within raw_metadata, which is reported verbatim and may legitimately contain nulls, e.g. NaN-sanitized header fields)
data: expression resolving to a lazy array reader within the parsed metadata
mask: optional dictionary {“source”: <expression resolving to a lazy array reader>, “rule”: <expression over the source array (`V) and the parsed metadata (C), true marks undefined pixels>}`
foreach: optional expression evaluating to a list; one channel is emitted per element, with the element available as C.item and its index as C.item_index in all other expressions of the binding
where: optional expression; if it evaluates falsy for a foreach element (or for the whole binding without foreach), no channel is emitted
checks: optional list of dictionaries {“condition”: <expression>, “error”: <error taxonomy name>, “message”: <str>}, evaluated per emitted channel; a falsy condition raises the named error

Expressions are evaluated against the parsed metadata context.

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

__init__(file_path)

property metadata

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, channel_id=None, height_channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.SDF module

Reader for ISO 25178-71 SDF (Surface Data File) format.

This format is defined in ISO 25178-71 and supports both ASCII and binary variants.

class SurfaceTopography.IO.SDF.SDFReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.SUR module

class SurfaceTopography.IO.SUR.SURReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.TMD module

Reader for TrueMap TMD files.

class SurfaceTopography.IO.TMD.TMDReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.Text module

SurfaceTopography.IO.Text.to_str(x)

class SurfaceTopography.IO.Text.AscReader(file_path)

Bases: ReaderBase

classmethod to_float(s)

parse_data(line)

parse_metadata(line)

__init__(file_path)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=False, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.Text.write_matrix(self, fname): Saves the topography using np.savetxt. Warning: This only saves the heights; the physical_sizes is not contained in the file

SurfaceTopography.IO.VK module

class SurfaceTopography.IO.VK.VKReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.WSXM module

class SurfaceTopography.IO.WSXM.WSXMReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.X3P module

class SurfaceTopography.IO.X3P.X3PReader(file_path)

Bases: DeclarativeReaderBase

classmethod can_read(buffer: bytes) → MagicMatch

Check if this reader can handle a file based on magic bytes.

This method performs a fast check using the first N bytes of a file to determine if this reader can handle the format. It is used to quickly reject incompatible formats during auto-detection.

Parameters:: buffer (bytes) – First N bytes of the file (typically 512 bytes).
Returns:: YES if magic matches and this reader should work. NO if magic does NOT match and this reader should be skipped. MAYBE if cannot determine from magic alone (default).
Return type:: MagicMatch

SurfaceTopography.IO.X3P.write_x3p(self, fobj, dtype='D', manufacturer='SurfaceTopography', model='Python Library', version=None)

Write topography to an X3P file (ISO 5436-2 / ISO 25178-72 format).

X3P is a container format conforming to the ISO 5436-2 standard for surface texture data exchange.

Parameters:

self (Topography) – The topography to write.
fobj (str or file-like object) – File path or file-like object to write to.
dtype (str, optional) – Data type for height values. Options are: - ‘D’ : 64-bit float (default) - ‘F’ : 32-bit float - ‘L’ : 32-bit unsigned integer (requires height_scale_factor) - ‘I’ : 16-bit unsigned integer (requires height_scale_factor)
manufacturer (str, optional) – Manufacturer name to include in metadata. (Default: ‘SurfaceTopography’)
model (str, optional) – Model/software name. (Default: ‘Python Library’)
version (str, optional) – Version string. If None, uses the SurfaceTopography version.

SurfaceTopography.IO.XYZ module

SurfaceTopography.IO.XYZ.read_text_header_hfm(fobj, unit, height_scale_factor)

Read header HFM files.

Parameters:

fobj (file object) – File object to read from.
unit (str) – Length unit of the data. Raises an error if not `None`because HFM files always give a unit.
height_scale_factor (float) – Conversion factor for the height data. Raises an error if not None because HFM files always define a height scale.

Returns:

sep (str) – Separator between columns.
usecols (list of int) – List of column indices to read.
skiprows (int) – Number of rows to skip before reading data.
unit (str) – Length unit of the data.
height_scale_factor (float) – Conversion factor for the height data.
info (dict) – Additional information.

SurfaceTopography.IO.XYZ.read_text_header_dektak(fobj, unit, height_scale_factor)

Read header of Dektak CSV files.

Parameters:

fobj (file object) – File object to read from.
unit (str) – Length unit of the data, only if not present in the file.
height_scale_factor (float) – Conversion factor for the height data, only if not present in the file.

Returns:

sep (str) – Separator between columns.
usecols (list of int) – List of column indices to read.
skiprows (int) – Number of rows to skip before reading data.
unit (str) – Length unit of the data.
height_scale_factor (float) – Conversion factor for the height data.
info (dict) – Additional information.

SurfaceTopography.IO.XYZ.read_csv(fobj, sep=None, usecols=None, skiprows=0)

Simple reader for tabular data in C(omma) S(eparated) V(alue) format. The comma should not be taken literal as the reader tries to be fuzzy (and flexible).

Parameters:

fobj (file object) – File object to read from.
sep (str, optional) – Separator between columns. (Default: White space, comma or semicolon)
usecols (list of int, optional) – List of column indices to read. (Default: All columns)
skiprows (int, optional) – Number of rows to skip before reading data. (Default: 0)

Returns:

data – List of arrays, one for each column in the file.

Return type:

list of np.ndarray

class SurfaceTopography.IO.XYZ.XYZReader(file_path)

Bases: ReaderBase

__init__(file_path)

property channels: Returns a list of :obj:`ChannelInfo`s describing the available data channels.

topography(channel_index=None, physical_sizes=None, height_scale_factor=None, unit=None, info={}, periodic=None, subdomain_locations=None, nb_subdomain_grid_pts=None)

Returns an instance of a subclass of HeightContainer that contains the topography data. Which specific type of height container (1D, 2D, uniform, nonuniform) is returned may depend on the file content and is determined dynamically.

The returned object needs to have physical_sizes and periodicity information. If this information is not provided by the data file, then the user must specify it when calling this method. Conversely, metadata that is present in the file cannot be overridden but needs to lead to a MetadataAlreadyFixedByFile exception. The user can check if the metadata is present by inspecting the ChannelInfo of the respective data channel.

There are three ways to specify which channel to load: 1. channel_index: Index into the full channels list (all data types) 2. channel_id: Stable string identifier (e.g., “Height”, “Phase#2”) 3. height_channel_index: Index among height channels only (backwards

Only one of these parameters should be specified. If none are given, the default channel is loaded.

Parameters:

channel_index (int) – Index of the channel to load. See also channels method. (Default: None, which loads the default channel)
channel_id (str) – Stable channel identifier string. This is the recommended way to identify channels for storage in databases as it remains stable even if channel ordering changes.
height_channel_index (int) – Index among height channels only. This provides backwards compatibility for databases that stored channel indices before non-height channels were supported.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size, then this parameter will override the physical size found in the data file.
height_scale_factor (float) – Override height scale factor found in the data file.
unit (str) – Length unit.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Whether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing).
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyFixedByFile – Raised if physical_sizes, unit or `height_scale_factor have already been defined in the file, because they should not be overridden by the user.
ValueError – Raised if more than one channel selection parameter is provided, or if the specified channel cannot be found.

SurfaceTopography.IO.ZMG module

Reader for KLA Zeta ZMG files.

class SurfaceTopography.IO.ZMG.ZMGReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.ZON module

Reader for Keyence ZON files.

class SurfaceTopography.IO.ZON.ZONReader(file_path): Bases: DeclarativeReaderBase

SurfaceTopography.IO.binary module

exception SurfaceTopography.IO.binary.ValidationError

Bases: CorruptFile

Raised when a Validate hook without an explicit exception class fails. A plain validation failure means the file is corrupt, hence this is a CorruptFile (and thus ReadFileError); the format description contract maps it to the corrupt_file error taxonomy name.

class SurfaceTopography.IO.binary.AttrDict(*args, **kwargs)

Bases: dict

__init__(*args, **kwargs)

SurfaceTopography.IO.binary.decode(stream_obj, structure_format, byte_order='@', return_size=False, context={})

Decode a binary stream given the sequence of binary entries. Strings are stripped of zeros and white spaces.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
structure_format (list of tuples) –
List of tuples describing the sequence of entries in the binary stream. Each tuple consists of two entries

(name, format)

that give the name of the entry and the format. We support the format defined in the struct module, plus ‘u’ for UTF-8, ‘U’ for UTF-16, ‘t’ for a Pascal string with 16-bit length and ‘T’ for a Pascal string with 32-bit length. Decoder also supports per-entry endianness.
byte_order (str, optional) – Byte order (see struct.unpack). (Default: ‘@’)
return_size (bool, optional) – Return the total size the structure in addition to the decoded data. (Default: False)
context (dict, optional) – Context dictionary passed to validation and converter functions. (Default: {})

Returns:

data (dict) – Dictionary with decoded data entries.
size (int) – Size of the structure in the native binary form. (Only returned if return_size is True.)

class SurfaceTopography.IO.binary.Convert(fun, exception=None)

Bases: object

__init__(fun, exception=None)

class SurfaceTopography.IO.binary.Validate(value, exception=<class 'SurfaceTopography.IO.binary.ValidationError'>)

Bases: object

__init__(value, exception=<class 'SurfaceTopography.IO.binary.ValidationError'>)

class SurfaceTopography.IO.binary.DebugOutput(prefix='', context=False)

Bases: object

__init__(prefix='', context=False)

class SurfaceTopography.IO.binary.LayoutWithNameBase

Bases: object

Base class for file layout classes

name(context)

to_dict(): Serialize this layout to the JSON representation defined by the format description contract (see SurfaceTopography.IO.description).

class SurfaceTopography.IO.binary.BinaryStructure(structure_format, byte_order='@', name=None)

Bases: LayoutWithNameBase

__init__(structure_format, byte_order='@', name=None)

Define a binary stream given the sequence of binary entries.

Parameters:

structure_format (list of tuples) –
List of tuples describing the sequence of entries in the binary stream. Each tuple consists of two entries

(name, format)

that give the name of the entry and the format. We support the format defined in the struct module, plus ‘u’ for UTF-8, ‘U’ for UTF-16, ‘t’ for a Pascal string with 16-bit length and ‘T’ for a Pascal string with 32-bit length. Decoder also supports per-entry endianness.
byte_order (str, optional) – Byte order (see struct.unpack). (Default: ‘@’)
name (str, optional) – Name of this structure. (Default: None)

from_stream(stream_obj, context)

Decode stream into dictionary.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
data (dict) – Dictionary with data that has been decoded at this point.

Returns:

decoded_data – Dictionary with decoded data entries.

Return type:

dict

class SurfaceTopography.IO.binary.BinaryArray(name, shape, dtype, conversion_fun=<function _identity>, mask_fun=None)

Bases: object

__init__(name, shape, dtype, conversion_fun=<function _identity>, mask_fun=None)

Defines flat binary data to be read into a numpy array.

Parameters:

name (str) – Name of the array.
shape (function or tuple) – Function that returns the shape and takes a input the current data dictionary.
dtype (function or dtype) – Function that returns the dtype and takes a input the current data dictionary.
conversion_fun (function) – Function that converts the array after reading. This can be useful for example to change the data format or transpose the array. The function receives the array as its first argument and can optionally accept the context dictionary as a second argument.
mask_fun (function) – Function that returns a mask with undefined data points.

name(context)

from_stream(stream_obj, context)

Skip over data block and return reader for block.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

context – Context dictionary with file reader.

Return type:

dict

read(stream_obj, context)

Read data block into numpy array.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

data – Nunpy array containing the data from the file.

Return type:

numpy.ndarray

class SurfaceTopography.IO.binary.RawBuffer(name, size=None, lazy=True)

Bases: object

__init__(name, size=None, lazy=True)

Defines a raw binary data block.

Parameters:

name (str) – Name of the data block.
size (int, callable, or None) – Size of the data block in bytes. Can be: - An integer for fixed size - A callable that takes context and returns size - None to read size from context[‘_block_size’] (for TLV parsing)
lazy (bool, optional) – If True, return a ReaderProxy for deferred reading. If False, read data immediately. Default: True.

name(context)

from_stream(stream_obj, context)

Read or skip over data block.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

Dictionary with the data or a reader proxy.

Return type:

dict

read(stream_obj, context)

Read data block.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

buffer – Buffer containing the raw data.

Return type:

bytes

class SurfaceTopography.IO.binary.TextBuffer(name, size=None, encoding='ascii')

Bases: LayoutWithNameBase

Reads a text block from a binary stream.

Parameters:

name (str) – Name of the text entry.
size (int, callable, or None) – Size of the text block in bytes. Can be: - An integer for fixed size - A callable that takes context and returns size - None to read size from context[‘_block_size’] (for TLV parsing)
encoding (str, optional) – Text encoding. Default: ‘ascii’.

__init__(name, size=None, encoding='ascii')

name(context)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.TextLine(name, encoding='latin-1')

Bases: LayoutWithNameBase

Reads a single text line from a binary stream and stores its stripped content.

Parameters:

name (str) – Name of the entry in the result dictionary.
encoding (str, optional) – Text encoding of the line. Default: ‘latin-1’.

__init__(name, encoding='latin-1')

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.TextHeader(name=None, encoding='latin-1', separator='=', key_width=None, comment_prefixes=(), sections=None, section_key=None, sections_name='sections', terminator=None, stop_key=None, size=None, converters=None, strict=False)

Bases: LayoutWithNameBase

Parses a line-oriented text header of key <separator> value entries from a binary stream.

The header region is delimited either by an exact byte count (size), a terminator line (terminator), a terminating key (stop_key), or the end of the stream. In line-delimited mode, the stream is left positioned directly after the last consumed line, so binary data following the header can be read by subsequent layout nodes. (Line mode requires an ASCII-compatible encoding of the newline character; use size for encodings like UTF-16.)

Parameters:

name (str, optional) – Name of this structure in the result dictionary. If None, the parsed entries are merged into the enclosing context. (Default: None)
encoding (str, optional) – Text encoding. Undecodable bytes are replaced. (Default: ‘latin-1’)
separator (str, optional) – Separator between key and value. Ignored if key_width is set. (Default: ‘=’)
key_width (int, optional) – If set, the key is a fixed-width column of this many characters and the value is the remainder of the line. (Default: None)
comment_prefixes (tuple of str, optional) – Lines starting with one of these prefixes are skipped. The check runs on the unstripped line, so ‘ ‘ skips indented lines. (Default: ())
sections (str, optional) – If ‘ini’, lines of the form [Name] start a named section and subsequent entries are stored in a nested dictionary under that name. (Default: None)
section_key (str, optional) – If set, each occurrence of this key starts a new record; subsequent entries (including this key) are stored in the record, and the records form a list stored under sections_name. (Default: None)
sections_name (str, optional) – Name of the record list for section_key. (Default: ‘sections’)
terminator (str, optional) – Exact (stripped) line that ends the header. The line is consumed but not stored. (Default: None)
stop_key (str, optional) – Key whose entry ends the header. The entry is stored at the top level and consumed. (Default: None)
size (int, callable or expression, optional) – Exact size of the header region in bytes. The whole region is consumed and parsed; unparseable content is skipped. (Default: None)
converters (dict, optional) – Maps keys to callables (or expressions) that convert the value, e.g. {‘headersize’: F.int(V)}. (Default: None)
strict (bool, optional) – If True, a non-empty line that is neither a comment, a section marker nor a key-value pair raises CorruptFile; if False, such lines are skipped. (Default: False)

__init__(name=None, encoding='latin-1', separator='=', key_width=None, comment_prefixes=(), sections=None, section_key=None, sections_name='sections', terminator=None, stop_key=None, size=None, converters=None, strict=False)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.TextMatrix(name, shape, encoding='ascii', bad_markers=('BAD', ), conversion_fun=<function _identity>)

Bases: LayoutWithNameBase

Reads a matrix of whitespace-separated numbers from a text region of a binary stream.

Reads lines until the number of values given by shape has been consumed; the values are parsed into a float64 array. Unlike BinaryArray, the values are parsed when the layout is executed (a text region cannot be skipped without scanning it); the result dictionary nevertheless holds a lazy-style reader proxy for consistency.

Parameters:

name (str) – Name of the array.
shape (tuple, callable or expression) – Shape of the resulting array.
encoding (str, optional) – Text encoding. (Default: ‘ascii’)
bad_markers (tuple of str, optional) – Tokens marking invalid values, parsed as NaN. (Default: (‘BAD’,))
conversion_fun (function or expression, optional) – Conversion applied to the parsed array, e.g. a transposition. (Default: identity)

__init__(name, shape, encoding='ascii', bad_markers=('BAD', ), conversion_fun=<function _identity>)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.TLVContainer(tag_map, name=None, tag_format='<H', size_format='<Q', count=None, container_size=None, store_by_name=True, entry_prefix_format=None, default=None, hex_tag_keys=False)

Bases: LayoutWithNameBase

Reads TLV (Tag-Length-Value) encoded blocks from a binary stream.

TLV format: - Tag: identifies the block type (uint16 by default) - Length: size of the data section - Value: the actual data

Parameters:

tag_map (dict) – Maps tag IDs to layout classes. Each layout class must have a from_stream(stream_obj, context) method. Special values: - None or missing: store raw bytes (but see default) - ‘text’: treat as ASCII text - Layout class instance: use that layout to parse The context passed to each entry’s layout contains the previously parsed named entries of this container, so later entries can reference earlier ones (like within a CompoundLayout), e.g. for data-dependent array shapes.
default (layout, optional) – Layout used for tags that are missing from tag_map, e.g. Skip() to skip unknown blocks without reading them. If None, unmapped tags store their raw bytes. Default: None.
hex_tag_keys (bool, optional) – If True, entries are stored under hexadecimal string keys (e.g. ‘0x66’) instead of integer tags. Use this when the parsed entries end up in reported metadata, which must survive a JSON round trip (JSON has no integer object keys). Default: False.
name (str, optional) – Name for this container in the result dict. Default: None.
tag_format (str, optional) – Format for tag field. Default: ‘<H’ (uint16 LE).
size_format (str, optional) – Format for size field. Default: ‘<Q’ (uint64 LE). Use ‘<I’ for uint32.
count (int or callable, optional) – Number of TLV entries to read. If None, reads until end of container (requires knowing the container size). Default: None.
container_size (int or callable, optional) – Total size of the container in bytes. Used when count is None to determine when to stop reading. Default: None.
store_by_name (bool, optional) – If True, also store entries by their layout’s name (if available) in addition to by tag ID. Default: True.
entry_prefix_format (str, optional) – Format for a prefix field before each TLV entry. If set, this many bytes are read and skipped before each entry. Used for structures where each TLV entry is preceded by a size hint. Default: None (no prefix).

__init__(tag_map, name=None, tag_format='<H', size_format='<Q', count=None, container_size=None, store_by_name=True, entry_prefix_format=None, default=None, hex_tag_keys=False)

name(context)

from_stream(stream_obj, context)

Parse TLV entries from stream.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

Dictionary with parsed entries keyed by tag ID.

Return type:

dict

class SurfaceTopography.IO.binary.LayoutWithTrailingData(name, fields)

Bases: LayoutWithNameBase

Layout that parses structured fields then stores remaining bytes as raw data.

This is useful for TLV blocks that have a known header structure followed by variable-length data (e.g., compressed data blocks).

Parameters:

name (str) – Name for this block in the result dict.
fields (list) – Field definitions for the header. Can include: - Tuples: (name, format) following struct module conventions - Layout classes: Objects with from_stream() method (like For)

__init__(name, fields)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.XMLStructure(name=None, converters=None)

Bases: LayoutWithNameBase

Parses an XML document from a stream into a nested dictionary.

Elements with children become attribute-accessible dictionaries keyed by tag name; leaf elements are reported as their text content.

Parameters:

name (str, optional) – Name of this structure in the result dictionary. If None, the parsed dictionary is merged into the enclosing context. (Default: None)
converters (dict, optional) – Maps leaf tag names to callables that convert the element’s text content, e.g. {‘MeterPerPixel’: float}. (Default: None)

__init__(name=None, converters=None)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.ZipMemberLoop(items, member_name, structure, name)

Bases: object

Entry of a ZipContainer that parses one archive member per element of a list, e.g. per-layer data files whose names come from previously parsed metadata. The member-name expression and the layout see the current element as item and its index as item_index in their context; the per-element results are collected into a list.

Parameters:

items (callable or expression) – Evaluated against the context (which contains the previously parsed members); must yield a list.
member_name (callable or expression) – Name of the archive member to parse for the current element.
structure (layout) – Layout used to parse each member.
name (str) – Context name of the list of results.

__init__(items, member_name, structure, name)

class SurfaceTopography.IO.binary.ZipContainer(members, name=None, stream_filter=None, mismatch_error=<class 'SurfaceTopography.Exceptions.CorruptFile'>)

Bases: LayoutWithNameBase

Interprets the stream as a ZIP archive and parses selected members of the archive, each with its own layout.

Only members that are named in the layout are parsed; all other members of the archive are ignored. Lazy readers (e.g. BinaryArray proxies) returned by member layouts are wrapped such that calling them with the outer file stream transparently reopens the archive member. Note that the position of the stream is undefined after this container has been parsed.

Parameters:

members (list) –
List describing the archive members to parse, processed in order. Each entry is one of
- a tuple of two or three entries
  (member_name, layout[, optional])
  
  where member_name is the name of the file within the archive (a string, or an expression evaluated against the previously parsed members), layout is a layout class instance used to parse it, and optional indicates whether the member may be missing from the archive (or the name expression may evaluate to None). (Default for optional: False)
- a ZipMemberLoop, parsing one member per element of a list
- a layout node that does not read from the stream (e.g. Check or Let), executed against the previously parsed members
name (str, optional) – Name of this structure in the result dictionary. If None, the parsed members are merged into the enclosing context. (Default: None)
stream_filter (callable, optional) – Callable with signature stream_filter(stream_obj, context) that wraps the raw member stream, e.g. for decompression. Note that the returned stream only needs to support reading and forward seeking. (Default: None)
mismatch_error (Exception, optional) – Exception raised when the stream is not a ZIP archive or a required member is missing. Use FileFormatMismatch when the member is what identifies the format (e.g. main.xml of an X3P). (Default: CorruptFile)

__init__(members, name=None, stream_filter=None, mismatch_error=<class 'SurfaceTopography.Exceptions.CorruptFile'>)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.TIFFContainer(name=None, tag_names=None, tag_groups=None, tag_layouts=None, image_conversion=<function _identity>)

Bases: LayoutWithNameBase

Interprets the stream as a TIFF file and reports its pages.

The result contains a list pages; each page is a record with

index: the page index,
shape: the shape of the page’s raster image,
tags: the page’s tag values, keyed by the standard tag name, by the name from tag_names, or by the hexadecimal tag code (e.g. ‘0x8050’) for unknown private tags. Values follow the contract’s value model (enums become [name, value] pairs; byte blobs and arrays are omitted),
image: a lazy reader materializing the page’s raster image,
one list per tag_groups entry (see below), and
the results of tag_layouts (see below).

Parameters:

name (str, optional) – Name of this structure in the result dictionary. If None, the pages list is merged into the enclosing context. (Default: None)
tag_names (dict, optional) – Maps numeric tag codes of private tags to names, e.g. {0x8000: ‘FileFormatVersion’}. (Default: None)
tag_groups (dict, optional) – Repeating private-tag blocks, mapping a group name to {‘first’: <code>, ‘stride’: <code stride>, ‘names’: {<offset>: <name>}}. Tags with code first + i * stride + offset are collected into the i-th record of the group’s list. (Default: None)
tag_layouts (dict, optional) – Maps numeric tag codes to layouts that parse the tag’s byte value, in declaration order; the results are merged into the page record, and lazy readers within them re-read the tag on demand. Use this for vendor headers and rasters stored in private tags. (Default: None)
image_conversion (function or expression, optional) – Conversion applied to page raster images, e.g. a transposition. (Default: identity)

__init__(name=None, tag_names=None, tag_groups=None, tag_layouts=None, image_conversion=<function _identity>)

from_stream(stream_obj, context)

class SurfaceTopography.IO.binary.ZlibBlockChain(name, prefix_format='<QII', min_decompressed_size=1000)

Bases: LayoutWithNameBase

Reads a chain of sequential zlib-compressed blocks from a binary stream.

This class is designed for file formats like MNT that store height data as a series of zlib-compressed blocks with a fixed prefix structure. The blocks are stored sequentially in memory and can be chained together using the compressed_size field in each prefix.

Block structure: ` [prefix][zlib data][prefix][zlib data]... `

Default prefix format (16 bytes): - Bytes 0-7: uint64 LE - element_offset (for logical ordering) - Bytes 8-11: uint32 LE - elements_per_block - Bytes 12-15: uint32 LE - compressed_size

The class scans for the first zlib block by looking for the zlib magic byte (0x78), then chains through all subsequent blocks using the compressed_size field.

Parameters:

name (str) – Name for the parsed result in the context dictionary.
prefix_format (str, optional) – Struct format for the block prefix. Default: ‘<QII’ (uint64 + 2*uint32). The last field must be the compressed_size.
min_decompressed_size (int, optional) – Minimum decompressed size to consider a valid block. This filters out false positive zlib matches. Default: 1000.

Examples

>>> layout = ZlibBlockChain('height_blocks')
>>> result = layout.from_stream(stream, {})
>>> blocks = result['height_blocks']
>>> for block in blocks:
...     elem_offset = block['elem_offset']
...     data = block['data']  # decompressed bytes

ZLIB_COMPRESSION_LEVELS = [1, 94, 156, 218]

__init__(name, prefix_format='<QII', min_decompressed_size=1000)

from_stream(stream_obj, context)

Parse zlib-compressed blocks from stream.

Parameters:

stream_obj (stream-like object) – Binary stream to decode.
context (dict) – Dictionary with data that has been decoded at this point.

Returns:

Dictionary with {name: list of block dicts}, where each block dict contains ‘elem_offset’, ‘elem_per_block’, and ‘data’ keys.

Return type:

dict

SurfaceTopography.IO.common module

SurfaceTopography.IO.common.is_binary_stream(fobj): Check whether fobj is a binary stream

SurfaceTopography.IO.common.text(encoding='utf-8')

class SurfaceTopography.IO.common.OpenFromAny(fobj, mode='r', encoding=None)

Bases: object

Context manager for turning file names, callables that open streams or already open streams into a single stream format (binary or text with specific encoding) for subsequent reading. The file is left in an identical state, including its cursor position, when the context manager returns.

__init__(fobj, mode='r', encoding=None)

Open file

Parameters:

fobj (str or stream) – The file to be opened, specified either as a file name or a stream object.
mode (str) – Open as text (‘r’) or binary (‘rb’). (Default: None)
encoding (str) – Character encoding when opening text files. (Default: None)

SurfaceTopography.IO.description module

Serialization of declarative file layouts to and from the JSON documents defined by the format description contract (docs/format_description_contract.rst).

layout_to_dict turns a layout tree (built from the classes in binary.py and Reader.py) into a JSON-compatible dictionary; layout_from_dict rehydrates it. Hooks that are Python callables rather than expressions (see SurfaceTopography.IO.expr) serialize to opaque markers — a document containing opaque markers is incomplete and cannot be rehydrated or executed by a foreign-language engine. count_opaque reports them.

SurfaceTopography.IO.description.encode_value(value): Encode a hook or parameter value: expressions to their AST, literals to plain JSON, remaining callables to opaque markers.

SurfaceTopography.IO.description.decode_value(value): Inverse of encode_value.

SurfaceTopography.IO.description.layout_to_dict(layout)

Serialize a layout tree to the contract’s JSON representation.

Hooks that are plain Python callables serialize to opaque markers; use count_opaque to check whether the result is complete.

SurfaceTopography.IO.description.layout_from_dict(d): Rehydrate a layout tree from its JSON representation.

SurfaceTopography.IO.description.count_opaque(document): Count opaque (non-serializable) hooks in an encoded document.

SurfaceTopography.IO.description.capabilities(document): Compute the sorted capability list of an encoded document, from its layout node types and registry function calls.

SurfaceTopography.IO.description.channel_bindings_to_dict(bindings): Serialize the _channel_bindings of a declarative reader.

SurfaceTopography.IO.description.channel_bindings_from_dict(document): Rehydrate channel bindings from their JSON representation.

SurfaceTopography.IO.description.reader_description(reader_class): Build the complete format description document for a declarative reader class, per the format description contract.

SurfaceTopography.IO.description.reader_from_dict(document): Create a reader class from a format description document. This is the generic, data-driven counterpart of the hand-authored declarative reader classes; it is also how the round-trip CI gate proves that a document contains all format knowledge.

SurfaceTopography.IO.export module

Exporter for format description documents and conformance goldens, per the format description contract (docs/format_description_contract.rst).

SurfaceTopography is the reference implementation; the artifacts written here are vendored into consumers (e.g. SDSAlgorithms/libsdsio) and pinned by their CI.

Run as a script:

python -m SurfaceTopography.IO.export --descriptions DIR
python -m SurfaceTopography.IO.export --goldens DIR --corpus CORPUS_DIR

SurfaceTopography.IO.export.exported_reader_classes(): The reader classes exported to description documents, by format id.

SurfaceTopography.IO.export.export_descriptions(output_dir): Write one description document per exported reader into output_dir. Refuses to export incomplete (opaque-hook-containing) documents.

SurfaceTopography.IO.export.golden_document(reader_class, file_path, fixture_name): Build the conformance golden document for one corpus fixture.

SurfaceTopography.IO.export.export_goldens(output_dir, corpus_dir): Write one conformance golden document per corpus fixture of every exported reader into output_dir.

SurfaceTopography.IO.export.main(argv=None)

SurfaceTopography.IO.expr module

Serializable expression mini-language for the declarative file layouts.

PROTOTYPE. The declarative readers currently express data-dependent quantities (array shapes, validation conditions, conversions) as Python lambdas. Lambdas cannot be serialized, which ties the format descriptions to Python. This module provides drop-in replacements: expression objects that are callable with the same signatures as the lambdas, but that can also be serialized to a JSON-compatible AST and rehydrated, e.g. by an expression interpreter written in another language.

The normative definition of the AST, its operator semantics and the function registry is docs/format_description_contract.rst; the design rationale is docs/declarative_readers_design.rst. Where this module and the contract disagree, the contract wins and this module needs fixing.

Authoring vocabulary

C: the parser context, e.g. C.header.nb_grid_pts_x
V: the value currently being processed (in validators, converters and array conversion functions)
F: registered named functions, e.g. F.dtype(“<i2”); the function registry is the fixed set of primitives that a foreign-language interpreter must provide
Tup(…): a tuple of expressions, e.g. an array shape
Cond(condition, then, otherwise): conditional expression

Standard Python operators (+, -, *, /, //, %, &, |, ^, <<, >>, comparisons, unary -), indexing and slicing are overloaded on expression objects. Note that and/or/not cannot be overloaded in Python; use &/| on boolean subexpressions (parenthesize, &/| bind tighter than comparisons) or Cond.

Calling convention

Expression objects are callable so that they can stand in for the lambdas the layout classes accept. The layout classes call these hooks either with (context) or with (value, context); a single dict-like argument is interpreted as the context, a single non-dict argument as the value.

Examples

>>> from SurfaceTopography.IO.binary import AttrDict
>>> shape = Tup(C.header.nb_grid_pts_y, C.header.row_bytes // 4)
>>> shape(AttrDict({'header': AttrDict({'nb_grid_pts_y': 3, 'row_bytes': 16})}))
(3, 4)
>>> is_valid = (V & 0x03) == 0
>>> is_valid(0x80, {})
True
>>> from_dict(is_valid.to_dict())(0x02, {})
False

SurfaceTopography.IO.expr.register_function(name, fun): Register a named function usable as F.<name>(…) in expressions.

class SurfaceTopography.IO.expr.Expr

Bases: object

Base class of all expression nodes.

evaluate(context, value=None)

to_dict()

isin(*values): Membership test, e.g. V.isin(‘KPK0’, ‘KPK1’).

SurfaceTopography.IO.expr.ensure_expr(value): Coerce a value into an expression node.

class SurfaceTopography.IO.expr.Lit(value)

Bases: Expr

Literal (JSON-representable) value.

__init__(value)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.BytesLit(value)

Bases: Expr

Bytes literal, serialized as base64.

__init__(value)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.DictExpr(items)

Bases: Expr

Mapping with static string keys, e.g. for context restructuring.

__init__(items)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.Val

Bases: Expr

The value currently being processed (validated or converted).

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.CtxRef(path=())

Bases: Expr

Reference to an entry of the parser context, e.g. C.header.magic.

__init__(path=())

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.BinaryOp(op, left, right)

Bases: Expr

__init__(op, left, right)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.UnaryOp(op, arg)

Bases: Expr

__init__(op, arg)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.Call(name, args)

Bases: Expr

Call to a function from the named function registry.

__init__(name, args)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.TupleExpr(*items)

Bases: Expr

Tuple of expressions, e.g. an array shape.

__init__(*items)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.Cond(condition, then, otherwise)

Bases: Expr

Conditional expression with short-circuit evaluation.

__init__(condition, then, otherwise)

evaluate(context, value=None)

to_dict()

class SurfaceTopography.IO.expr.GetItem(base, index)

Bases: Expr

Indexing and slicing, e.g. V[:, :C.header.nb_grid_pts_x].

__init__(base, index)

evaluate(context, value=None)

to_dict()

SurfaceTopography.IO.expr.from_dict(d): Rehydrate an expression from its JSON-compatible AST.

SurfaceTopography.IO.expr.Tup: alias of TupleExpr

Module contents

SurfaceTopography.IO.detect_format(fobj, comm=None)

Detect file format based on its content.

Parameters:

fobj (filename or file object)
comm (mpi communicator, optional)

SurfaceTopography.IO.open_topography(fobj, format=None, communicator=None)

Returns a reader object for the file fobj. The reader interface mirrors the topography interface and can be used to extract meta data (number of grid points, physical sizes, etc.) without reading the full topography in memory.

Parameters:

fobj (str or filelike object) – path of the file or filelike object
format (str, optional) – specify in which format the file should be interpreted
communicator (mpi4py or NuMPI communicator object) – MPI communicator handling inter-process communication

Returns:

Instance of a SurfaceTopography.IO.Reader.ReaderBase subclass
according to the format.

Examples

Simplest read workflow:

>>> reader = open_topography("filename")
>>> top = reader.topography()

The first topography in file is returned, independently of whether the file has multiple channels or not.

You can always check the channels and their metadata with reader.channels(). This returns a list of dicts containing attributes name, physical_sizes, nb_grid_pts, ``unit` and `height_scale_factor:

>>> reader.channels
[{'name': 'ZSensor',
  'nb_grid_pts': (256, 256),
  'physical_sizes': (9999.999999999998, 9999.999999999998),
  'unit': 'nm',
  'height_scale_factor': 0.29638271279074097},
 {'name': 'AmplitudeError',
  'nb_grid_pts': (256, 256),
  'physical_sizes': (10.0, 10.0),
  'unit': ('µm', None),
  'height_scale_factor': 0.04577566528320313}]

Here the channel ‘ZSensor’ offers a topography with sizes of 10000 nm in each dimension.

You can choose it by giving the index 0 to channel (you would use channel=1 for the second):

>>> top = reader.topography(channel=0)

The returned topography has the physical sizes found in the file.

You can also prescribe some attributes when reading the topography:

>>> top = reader.topography(channel=0, physical_sizes=(10.,10.), info={"unit":"µm"})

In order to plot the topography with matplotlib, you can use

>>> plt.pcolormesh(*top.positions_and_heights())

with origin in the lower left and correct tick labels at x and y axes, or

>>> plt.imshow(top.heights().T)

with origin in the upper left (inverted y axis).

SurfaceTopography.IO.read_topography(fn, format=None, communicator=None, **kwargs)

Returns a topography object representing the topograpgy in the file fobj. If there are multiple data channels within this file, the default channel is returned. The default channel depends on the file format; see documentation of the respective reader on this.

Parameters:

fobj (str or filelike object) – path of the file or filelike object
format (str, optional) – specify in which format the file should be interpreted
communicator (mpi4py or NuMPI communicator object) – MPI communicator handling inter-process communication
channel (int) – Number of the channel to load. See also channels method.
physical_sizes (tuple of floats) – Physical size of the topography. It is necessary to specify this if no physical size is found in the data file. If there is a physical size in the file, then specifying this parameter will raise an exception.
height_scale_factor (float) – Can be used to set height scale factor if not found in the data file.
info (dict) – This dictionary will be appended to the info dictionary returned by the reader.
periodic (bool) – Wether the SurfaceTopography should be interpreted as one period of a periodic surface. This will affect the PSD and autocorrelation calculations (windowing)
subdomain_locations (tuple of ints) – Origin (location) of the subdomain handled by the present MPI process.
nb_subdomain_grid_pts (tuple of ints) – Number of grid points within the subdomain handled by the present MPI process.

Returns:

topography – The object containing the actual topography data.

Return type:

subclass of HeightContainer

Raises:

MetadataAlreadyDefined – Raised if given arguments for physical_sizes or height_scale_factor although it’s already given in the file.