Awkward Array

These are differences from 1.8.0 to 1.9.0. Most only affect Awkward Array version 2, in the awkward._v2 submodule of 1.9.0. The exceptions are called out with "v1" in the subject line. Features feat: C++ refactoring: ak.to_pandas by @ioanaif in https://github.com/scikit-hep/awkward/pull/1369 feat: C++ refactoring: ak.nan_to_num by @ioanaif in https://github.com/scikit-hep/awkward/pull/1352 feat: C++ refactoring: ak.run_lengths by @ioanaif in https://github.com/scikit-hep/awkward/pull/1347 feat: add is_tuple describe operation by @agoose77 in https://github.com/scikit-hep/awkward/pull/1351 feat: C++ refactoring: ak.unzip by @ioanaif in https://github.com/scikit-hep/awkward/pull/1354 feat: pure Cling demo and improvements to C++ JIT infrastructure by @jpivarski in https://github.com/scikit-hep/awkward/pull/1359 feat: C++ refactoring: ak.broadcast_arrays by @ioanaif in https://github.com/scikit-hep/awkward/pull/1368 feat: irst version of ak._v2.from_parquet by @jpivarski in https://github.com/scikit-hep/awkward/pull/1338 feat: C++ refactoring: ak.copy by @ioanaif in https://github.com/scikit-hep/awkward/pull/1367 feat: C++ refactoring: ak.unflatten by @ioanaif in https://github.com/scikit-hep/awkward/pull/1360 feat: add depth_limit to ak.broadcast_arrays by @agoose77 in https://github.com/scikit-hep/awkward/pull/1373 feat: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1376 feat: This PR sets up the architecture to call CuPy Raw Kernels from Awkward. by @swishdiff in https://github.com/scikit-hep/awkward/pull/1355 feat: Reducers with axis=None and typetracers. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1380 feat: This PR adds the generated kernels and simplifies the template specialization generation process. by @swishdiff in https://github.com/scikit-hep/awkward/pull/1381 feat: High-level ak._v2.Array clean-ups. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1392 feat: This PR adds all the remaining kernels in the studies directory by @swishdiff in https://github.com/scikit-hep/awkward/pull/1390 feat: This PR adds JAX as a new nplike by @swishdiff in https://github.com/scikit-hep/awkward/pull/1399 feat: Passing behaviour in ak._v2 functions by @ioanaif in https://github.com/scikit-hep/awkward/pull/1415 feat: Enable broadcasting of string equality. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1427 feat: Enabled string/categorical behavior by @ioanaif in https://github.com/scikit-hep/awkward/pull/1421 feat: Implements ak.nan_to_none and all of the ak.nan* functions to override NumPy's. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1428 feat: Pretty-printing types by @jpivarski in https://github.com/scikit-hep/awkward/pull/1430 feat: Register both v1 and v2 Arrays in Numba entry_points. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1432 feat: Add Array and Record.delitem. And fix show(type=True). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1433 feat: Enable categorical behavior - testing by @ioanaif in https://github.com/scikit-hep/awkward/pull/1434 feat: Enable mixins behavior by @ioanaif in https://github.com/scikit-hep/awkward/pull/1437 feat: Implementing ak._v2.to_parquet. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1440 feat: awkward to rdataframe by @ianna in https://github.com/scikit-hep/awkward/pull/1374 feat: Enable ak.firsts by @ioanaif in https://github.com/scikit-hep/awkward/pull/1443 feat: Enable ak.singletons by @ioanaif in https://github.com/scikit-hep/awkward/pull/1444 feat: Revamping the to_json/from_json interface. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1449 feat: This PR attempts to add autodifferentiation support for Awkward Arrays using JAX pytrees. by @swishdiff in https://github.com/scikit-hep/awkward/pull/1447 feat: Adding a Forth Based Avro Reader by @aryan26roy in https://github.com/scikit-hep/awkward/pull/1491 feat: Adding repr overriden behavior by @ioanaif in https://github.com/scikit-hep/awkward/pull/1487 feat: rdataframe to awkward by @ianna in https://github.com/scikit-hep/awkward/pull/1474 feat: add missing _like methods to TypeTracer by @agoose77 in https://github.com/scikit-hep/awkward/pull/1505 feat: from rdataframe for awkward arrays by @ianna in https://github.com/scikit-hep/awkward/pull/1508 feat: Add typeparser to v2. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1513 feat: add C++ headers-only distribution configuration by @ianna in https://github.com/scikit-hep/awkward/pull/1523 feat: Growable Buffer header by @ManasviGoyal in https://github.com/scikit-hep/awkward/pull/1535 feat: parquet redux by @martindurant in https://github.com/scikit-hep/awkward/pull/1476 feat: Templated LayoutBuilder by @ManasviGoyal in https://github.com/scikit-hep/awkward/pull/1494 feat: raise AttributeError for public Array attributes by @agoose77 in https://github.com/scikit-hep/awkward/pull/1573 feat: Public interface for layout.recursively_apply and broadcast_and_apply. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1610 feat: prevent reducers like ak.sum on records (v2) by @ioanaif in https://github.com/scikit-hep/awkward/pull/1607 feat: Drop ak.behavior['.', 'Name'] = cls, which isn't working/isn't tested. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1651 feat: retrieve multiple columns from RDataFrame in a single event loop by @ianna in https://github.com/scikit-hep/awkward/pull/1625 Bug-fixes fix: Fixed ak.num with axis=0 in typetracer. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1329 fix: ak.flatten and ak.ravel should test for nplike.ndarray, not np.ndarray. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1340 fix: Straighten out error handling via a thread-local (but otherwise global) context by @jpivarski in https://github.com/scikit-hep/awkward/pull/1327 fix: Fix PR #788: avoid materializing VirtualArrays in ak.with_name. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1346 fix: fix docstring line in is_tuple (v1) by @agoose77 in https://github.com/scikit-hep/awkward/pull/1356 fix: fixes #1363 by ensuring that arguments documented as 'iterable of X' aren't used in 'len(X)'. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1364 fix: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1372 fix: Fixes nonlocal reducers in which the first list is empty. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1378 fix: Fix _prettyprint after 'for i in range' changed to 'for i, val in enumerate'. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1384 fix: bump black to 22.3.0 due to click 8.1 release by @henryiii in https://github.com/scikit-hep/awkward/pull/1385 fix: No zero-length shortcuts for ak.argsort (v1 & v2). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1387 fix: ErrorContexts should only contain strings. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1393 fix: ROOT doesn't recognize for-each iterators without operator== by @jpivarski in https://github.com/scikit-hep/awkward/pull/1398 fix: Implement recursively_apply for Record by @agoose77 in https://github.com/scikit-hep/awkward/pull/1401 fix: from_numpy references ListArray64 by @agoose77 in https://github.com/scikit-hep/awkward/pull/1404 fix: Windows builds stopped working; be looser about directory name. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1407 fix: pypy 3.9 by @henryiii in https://github.com/scikit-hep/awkward/pull/1412 fix: replace llvmlite.ir instead of llvmlite.llvmpy.core by @Ahmad-AlSubaie in https://github.com/scikit-hep/awkward/pull/1413 fix: Fix performance issue in v2 tolist. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1418 fix: Fix iteration over NumpyArray type. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1419 fix: Removed bytemask() in favour of mask_as_bool() by @ioanaif in https://github.com/scikit-hep/awkward/pull/1410 fix: Fix ak._v2.to_arrow for sliced ListOffsetArray. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1425 fix: ListOffsetArray._reduce_next is not implemented for 32-bit. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1426 fix: Fixed miscellaneous optiontype-in-Parquet bugs by @jpivarski in https://github.com/scikit-hep/awkward/pull/1431 fix: Remove duplicated import of to/from-parquet by @douglasdavis in https://github.com/scikit-hep/awkward/pull/1435 fix: is_unique() for IndexedArray by @ioanaif in https://github.com/scikit-hep/awkward/pull/1429 fix: Fixes to_layout with allow_records=False and allows single-record writing to Arrow and Parquet by @jpivarski in https://github.com/scikit-hep/awkward/pull/1456 fix: Fix RDataFrame GetColumnNames order in test. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1457 fix: Streamline recursively_apply for small slices of big arrays. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1458 fix: getitem* functions must consistently set the slicer in handle_error. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1469 fix: to_rdataframe extensive tests and bug fixes by @ianna in https://github.com/scikit-hep/awkward/pull/1478 fix: Fix selecting columns from Parquet. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1486 fix: Fix categorical equality handling (bad copy-paste from v1). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1492 fix: _to_numpy method should return a numpy array by @swishdiff in https://github.com/scikit-hep/awkward/pull/1496 fix: Fix/Fixed slicing shape for array of booleans by @ioanaif in https://github.com/scikit-hep/awkward/pull/1497 fix: Fixed typo in unmaskedarray.py. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1498 fix: Fix slicing for UnmaskedArrays (which come from Arrow). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1499 fix: Fix for issue 1406 by @ioanaif in https://github.com/scikit-hep/awkward/pull/1502 fix: Update type-parser for v2 by @jpivarski in https://github.com/scikit-hep/awkward/pull/1514 fix: numba pre-commit issues by @ioanaif in https://github.com/scikit-hep/awkward/pull/1533 fix: Initialize values behind the mask in ak.to_numpy by @ioanaif in https://github.com/scikit-hep/awkward/pull/1531 fix: Fixed RecordArray.repr (last vestige of 'override' misunderstanding). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1537 fix: Error when using ak.copy in v2 by @ioanaif in https://github.com/scikit-hep/awkward/pull/1532 fix: ak._v2.is_none check for axis value by @ioanaif in https://github.com/scikit-hep/awkward/pull/1539 fix: Solving the endian bug on MacOS by @aryan26roy in https://github.com/scikit-hep/awkward/pull/1538 fix: fix ak2 convert class name msg by @Moelf in https://github.com/scikit-hep/awkward/pull/1544 fix: GrowableBuffer.h's missing import broke localbuild.py; fixing it. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1555 fix: Remove std::cout from ArrayBuilder code. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1561 fix: Missing axis_wrap_if_negative Record method in both v1 and v2 by @ioanaif in https://github.com/scikit-hep/awkward/pull/1565 fix: ufuncs on records should not be allowed unless overridden by @ioanaif in https://github.com/scikit-hep/awkward/pull/1559 fix: add int64_t definition for Windows by @ianna in https://github.com/scikit-hep/awkward/pull/1572 fix: allow empty RecordArrays in ak.to_layout by @agoose77 in https://github.com/scikit-hep/awkward/pull/1581 fix: tolist() bug from Uproot (longlong type for Index) by @ioanaif in https://github.com/scikit-hep/awkward/pull/1567 fix: Lengths of empty regular slices by @ioanaif in https://github.com/scikit-hep/awkward/pull/1568 fix: RegularArray: maybe_toNumpy() by @ioanaif in https://github.com/scikit-hep/awkward/pull/1589 fix: Raises an exception on mixed Awkward/NumPy slices. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1598 fix: ak.concatenate (mergemany) should preserve regular-type. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1604 fix: ak.concatenate of identical Forms preserves the Form by @jpivarski in https://github.com/scikit-hep/awkward/pull/1605 fix: ak.concatenate should preserve regular-type for axis>0, too. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1609 fix: improve ak.from_iter performance for long (axis=0) arrays by @agoose77 in https://github.com/scikit-hep/awkward/pull/1614 fix: generator tolayout in cling by @ianna in https://github.com/scikit-hep/awkward/pull/1613 fix: Use dtype=np.int64, not int, for platform independence (Windows). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1636 fix: support root empty field in Parquet file by @agoose77 in https://github.com/scikit-hep/awkward/pull/1619 fix: empty slice lists of record arrays (#1593) by @agoose77 in https://github.com/scikit-hep/awkward/pull/1597 fix: ak.from_iter should interpret top-level tuples as ak.Array (v1 and v2). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1642 fix: Ensure that ak._v2.to_json raises errors when appropriate. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1649 fix: pass a copy of RecordArray's internal fields in HL API by @Saransh-cpp in https://github.com/scikit-hep/awkward/pull/1650 fix: include self._length in RegularArray.mergemany by @agoose77 in https://github.com/scikit-hep/awkward/pull/1645 fix: jagged slicing for ListArray by @agoose77 in https://github.com/scikit-hep/awkward/pull/1408 fix: to_list must follow getitem implementations, even in Records by @jpivarski in https://github.com/scikit-hep/awkward/pull/1652 Other perf: ak.concatenate should flatten for first axis, single-array by @agoose77 in https://github.com/scikit-hep/awkward/pull/1641 refactor: cleanup reducer by @agoose77 in https://github.com/scikit-hep/awkward/pull/1365 refactor: split up functions by @martindurant in https://github.com/scikit-hep/awkward/pull/1397 refactor: Flatten directory structure under src/awkward/_v2/operations. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1467 refactor: Rename low-level methods to match high-level function names. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1468 refactor: Rename fillna -> fill_none. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1470 refactor: Refactoring to include index_nplike and reducers by @swishdiff in https://github.com/scikit-hep/awkward/pull/1490 refactor: LayoutBuilder migration to v2 by @ianna in https://github.com/scikit-hep/awkward/pull/1484 refactor: restructure cpp headers by @ianna in https://github.com/scikit-hep/awkward/pull/1524 refactor: migrate ArrayBuilder to new GrowableBuffer by @ianna in https://github.com/scikit-hep/awkward/pull/1542 refactor: builder options and layout builder updates by @ianna in https://github.com/scikit-hep/awkward/pull/1560 refactor: Renamed ak.to_pandas -> ak.to_dataframe. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1603 refactor: replace 3 ak.from_json* functions with one by @jpivarski in https://github.com/scikit-hep/awkward/pull/1617 refactor: from_rdataframe to use LayoutBuilder by @ianna in https://github.com/scikit-hep/awkward/pull/1620 docs: add Ahmad-AlSubaie as a contributor for code by @allcontributors in https://github.com/scikit-hep/awkward/pull/1416 docs: Update AwkwardForth documentation and move it from the wiki to the standard docs. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1422 docs: update README: from awkward-1.0 to awkward by @ianna in https://github.com/scikit-hep/awkward/pull/1475 docs: Add the beginning of an example of templated LayoutBuilder. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1485 docs: add ManasviGoyal as a contributor for code by @allcontributors in https://github.com/scikit-hep/awkward/pull/1540 docs: add aryan26roy as a contributor for code by @allcontributors in https://github.com/scikit-hep/awkward/pull/1541 docs: Doxygen Documentation of GrowableBuffer and LayoutBuilder. by @ManasviGoyal in https://github.com/scikit-hep/awkward/pull/1579 docs: Documented ak.from_json and completed it. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1621 docs: add Saransh-cpp as a contributor for code by @allcontributors in https://github.com/scikit-hep/awkward/pull/1653 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1370 ci: Protect test 1300 from ROOT without C++17 (or, at least, without std::optional). by @jpivarski in https://github.com/scikit-hep/awkward/pull/1383 ci: Build(deps): bump pypa/cibuildwheel from 2.3.1 to 2.4.0 by @dependabot in https://github.com/scikit-hep/awkward/pull/1394 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1395 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1409 ci: Pass on skipped v2 tests by @ioanaif in https://github.com/scikit-hep/awkward/pull/1445 ci: Build(deps): bump pypa/cibuildwheel from 2.4.0 to 2.5.0 by @dependabot in https://github.com/scikit-hep/awkward/pull/1455 ci: awkward rdataframe source tests by @ianna in https://github.com/scikit-hep/awkward/pull/1446 ci: Ignore a NumPy 1.22 warning in Numba and fix the flake8-print T001 --> T201 change. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1464 ci: Build(deps): bump docker/setup-qemu-action from 1.2.0 to 2.0.0 by @dependabot in https://github.com/scikit-hep/awkward/pull/1462 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1465 ci: Build(deps): bump pypa/cibuildwheel from 2.5.0 to 2.6.1 by @dependabot in https://github.com/scikit-hep/awkward/pull/1493 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1503 ci: Build(deps): bump pypa/cibuildwheel from 2.6.1 to 2.7.0 by @dependabot in https://github.com/scikit-hep/awkward/pull/1510 ci: Swap 'merged dtype same as NumPy' test of v1 for test of v2. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1515 ci: Build(deps): bump actions/upload-artifact from 2 to 3 by @dependabot in https://github.com/scikit-hep/awkward/pull/1519 ci: Build(deps): bump actions/checkout from 2 to 3 by @dependabot in https://github.com/scikit-hep/awkward/pull/1520 ci: Build(deps): bump actions/download-artifact from 2 to 3 by @dependabot in https://github.com/scikit-hep/awkward/pull/1521 ci: Build(deps): bump pypa/cibuildwheel from 2.7.0 to 2.8.0 by @dependabot in https://github.com/scikit-hep/awkward/pull/1536 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1527 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1543 ci: Build(deps): bump pypa/cibuildwheel from 2.8.0 to 2.8.1 by @dependabot in https://github.com/scikit-hep/awkward/pull/1556 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1566 ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1576 ci: Build(deps): bump pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1 by @dependabot in https://github.com/scikit-hep/awkward/pull/1564 ci: use GHA instead of Azure :hammer: by @agoose77 in https://github.com/scikit-hep/awkward/pull/1550 ci: Upgrade Linux 18.04 -> 20.04 and DOS line endings. by @jpivarski in https://github.com/scikit-hep/awkward/pull/1616 ci: lint PR titles according to conventional commits by @agoose77 in https://github.com/scikit-hep/awkward/pull/1615 ci: aarch64 should also use cibuildwheel v2.9.0 by @jpivarski in https://github.com/scikit-hep/awkward/pull/1627 ci: run semantic-pr-title in concurrency group by @agoose77 in https://github.com/scikit-hep/awkward/pull/1632 ci: add Linux ROOT build to run ROOT-based tests by @ianna in https://github.com/scikit-hep/awkward/pull/1629 package: fix MANIFEST to include cpp-headers by @agoose77 in https://github.com/scikit-hep/awkward/pull/1516 chore: remove unneeded lines by @henryiii in https://github.com/scikit-hep/awkward/pull/1518 chore: include Python 3.11 by @henryiii in https://github.com/scikit-hep/awkward/pull/1602 chore: update pre-commit hooks by @pre-commit-ci in https://github.com/scikit-hep/awkward/pull/1618 New Contributors @Ahmad-AlSubaie made their first contribution in https://github.com/scikit-hep/awkward/pull/1413 @martindurant made their first contribution in https://github.com/scikit-hep/awkward/pull/1397 @aryan26roy made their first contribution in https://github.com/scikit-hep/awkward/pull/1491 @ManasviGoyal made their first contribution in https://github.com/scikit-hep/awkward/pull/1535 @Moelf made their first contribution in https://github.com/scikit-hep/awkward/pull/1544 Full Changelog: https://github.com/scikit-hep/awkward/compare/1.8.0...v1.9.0If you use this software, please cite it as below