cta-observatory/ctapipe: v0.16.0

Summary This next major release of ctapipe includes a large number of improvements and new features, as well as significant changes to the API and data model that fix outstanding problems with both. Due to the nature of these required changes and to keep the code reasonably maintainable (in this pre-1.0 release), files written by earlier versions of ctapipe are not guaranteed to be readable with this one. Contributors @aknierim, @kosack, @maxnoe, @nbiederbeck, @satoshifukami0115 and @vuillaut What has changed since v0.15.0 Pull-requests containing changes of multiple nature are repeated. New features ObsConfig data model (#1973) @kosack Add abstract InvalidPixelHandler and NeighborAverage implementation, apply in CameraCalibrator and TwoPassWindowSum. Pass invalid pixels to extractors. (#2023) @maxnoe Get reference_location and optics/camera name from simtel metadata if available (#2018) @maxnoe Add methods to read run config and shower distribution to TableLoader (#2010) @maxnoe added a better template for release-drafter (#1880) @kosack Add another cleaning method in cleaning.py (#1672) @satoshifukami0115 Round floats for hashing in instrument module, use __slots__ (#1986) @maxnoe Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe recursively validate containers (#1976) @kosack Improve units code in hillas_parameters (#1988) @maxnoe Add TableLoader.read_telescope_events_by_type_chunked (#1984) @maxnoe Maintain event order of input file in TableLoader, fixes #1966 (#1968) @maxnoe Chunked table loader (#1935) @maxnoe Read dl2 in hdf5eventsource (#1940) @maxnoe add joined option to SubarrayDescription.to_table (#1939) @maxnoe compute impact distance to shower axis (#1899) @kosack Improved configuration of reference metadata (#1889) @kosack Allow passing already opened h5file to table loader (#1933) @maxnoe Add overwrite option to write_table (#1932) @maxnoe Support reading files created with extract_calibevent (#1929) @maxnoe improve array display (#1911) @kosack API Changes Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe Remove cutflow (#1766) @maxnoe Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe Remove support for by_type split files (#2032) @maxnoe Rework h5 metadata (#2003) @maxnoe Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe Replace deepcopying defaults with default_factory (#1991) @maxnoe Use module of Tool to setup logging (#1964) @maxnoe Add QualityQuery.get_table_mask, compile python code (#1943) @maxnoe Load all dl2 available groups in TableLoader (#1934) @maxnoe Use container classes for TableReader.read method, do not reuse same container instance (#1921) @maxnoe Data Model Changes ObsConfig data model (#1973) @kosack Rename ReconstructedContainer.tel_ids -> telescopes, fixes 1953 (#2044) @maxnoe Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe Rework h5 metadata (#2003) @maxnoe Fix datalevels (#2004) @maxnoe Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe Rework container prefix handling (#1949) @maxnoe Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe compute impact distance to shower axis (#1899) @kosack Bug Fixes Fix missing f-string (#2043) @nbiederbeck Fix columns order when string column present, fixes #2041 (#2042) @maxnoe Fix writing and merging of dl2 statistics, fixes #1884 (#1885) @maxnoe Fix pixel rotation for very large offsets in NominalFrame (#2037) @maxnoe Fix wrong pixel area and rotation in nominal frame, fixes #2028 (#2029) @maxnoe Fix error message for missing effective focal length (#2026) @maxnoe Remove restriction on existing file in EventSource.input_url, fixes #1821 (#1998) @maxnoe Fix TableLoader reading empty table (#2013) @nbiederbeck Fix datalevels (#2004) @maxnoe Include number of mirror tiles in guessing keys to be able to distinguish between MAGIC 1 and MAGIC 2 (#1977) @maxnoe Use correct traitlet for event type enum in eventtypefilter (#2002) @maxnoe Add dl2 datalevel in hdf5 source, add datalevel tests (#2001) @maxnoe Ignore unknown column metadata in read_table (#1992) @maxnoe Fix comparison of timestamps in calibration code (#1987) @maxnoe Fix missing start/stop for reading impact table, use correct type in simteleventsource (#1982) @maxnoe Hash rounded versions of floats for CameraGeometry (#1956) @maxnoe Fix impact being written for non-triggered telescopes, fixes #1961 (#1967) @maxnoe fixed bug that prevented printing of QualityQuery (#1958) @kosack Fix ArrayDisplay creating legend in wrong axes with multiple axes active (#1946) @maxnoe Store true_image_sum also when true image is not written (#1893) @maxnoe Better error message in case unknown telescopes are selected, fixes #1924 (#1925) @maxnoe Minor visual bugfix: better html repr for Tool and Component (#1905) @kosack Refactoring and Optimization Use tel_id consistently (replacing telid -> tel_id), fixes #1457 (#2045) @maxnoe Remove cutflow (#1766) @maxnoe Add 'raises' to run_tool, which raises exceptions instead of logging them. (#2038) @nbiederbeck ExpressionEngine for QualityQuery and FeatureGenerator (#2014) @nbiederbeck Return tuples from SubArray properties, add suport for single value in get_tel_ids (#2027) @maxnoe Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe Ignore containers and maps by default in HDF5Writer (#1945) @maxnoe Hdf5EventSource performance improvements and skip empty events, fixes #1916 (#1917) @maxnoe Directly return peak position from neighbor average waveform (#1914) @maxnoe Maintenance updated incorrect comments in base_config.yaml (#2040) @kosack Remove support for by_type split files (#2032) @maxnoe Fixed Typo in docstring (#2009) @aknierim added a better template for release-drafter (#1880) @kosack Add flake8 pre-commit hook. Change black hook to also format notebooks. (#1993) @nbiederbeck Avoid warning in case of reading DL1a only files with HDF5EventSource (#1994) @maxnoe Automatically include fields in docstrings for containers (#1990) @maxnoe Create a fresh ArrayEventContainer for each event in HDF5EventSource (#1970) @maxnoe Create a fresh ArrayEventContainer for each event in SimTelEventSource (#1952) @maxnoe Update CI actions versions (#1951) @maxnoe Simplify units code in subarray (#1950) @maxnoe Use a simple kwarg instead of a traitlet for TableLoader.h5file (#1937) @maxnoe Improve TableLoader docs, fixes #1930 (#1931) @maxnoe Print version info in pytest header (#1919) @maxno