GilesStrong/lumin: V0.3 Tears in Rain

Important changes norm_in default value for get_pre_proc_pipes is now True rather than False layer width in dense=True FullyConnected now no longer scales with input size to prevent parameter count from exploding Biases in FullyConnected linear layers are now initialised to zero, rather that default PyTorch init Bias in ClassRegMulti linear layer is now intitialised to 0.5 if sigmoid output, zero if linear output, and 1/n_out if softmax, unless a bias_init value is specified Breaking Changed order of arugments in AMS and MultiAMS and removed some default values Removed default for return_mean in RegAsProxyPull and RegPull Changedsettings to plot_settings in rf_rank_features Removed some default parameters for NN blocks in ModelBuilder ModelBuilder model_args should now be a dictionary of dictionaries of keyword arguments, one for head, body, and tail blocks, previously was a single dictionary of keyword arguments Embedder.from_fy now no longer works: change to CatEmbedder.from_fy CatEmbHead now no longer has a n_cont_in argument, instead one should pass a list of feature names to cont_feats Additions Added n_estimators parameter to rf_rank_features and get_opt_rf_params to adjust the number of trees Added n_rfs parameter to rf_rank_features to average feature importance over several random forests Added automatic computation of 3-momenta magnitude to add_mass if it's missing n_components to get_pre_proc_pipes to be passed to PCA Pipeline configuration parameters to fit_input_pipe Ability to pass an instantiated Pipeline to fit_input_pipe Callbacks now receive model_num and savepath in on_train_begin Random Forest like ensembling: BootstrapResample callback for resampling training and validation data Feature subsambling: FeatureSubsample callback for training on random selection of features Model now has an input_mask to automatically mask inputs at inference time (train-time inputs should be masked at BatchYielder level) plot_roc now returns aucs as dictionary growth_rate scaling coefficient to FullyConnected to adjust layer width by depth n_in parameter to FullyConnected so it works on arbitray size inputs freeze_tail to ModelBuilder and ClassRegMulti Abstract blocks for head, body, and tail cont_feats argument to ModelBuilder to allow passing of list of named features, eventually allowing more advanced methods based on named outputs of head blocks. CatEmbHead now computes a mapping from named input features to their outputs body blocks now expect to be passed a dictionary mapping from named input features to the model to the outputs of the head block Model and AbsBlock classes now have a method to compute total number of (trainable) parameters MultiBlock body, providing possibility for multiple, parallel body blocks taking subsets of input features Explicit initialisation paramater for bias in ClassRegMulti plot_1d_partial_dependence now takes pdp_isolate_kargs and pdp_plot_kargs to pass to pdp_isolate and pdp_plot, respectively plot_2d_partial_dependence now takes pdp_interact_kargs and pdp_interact_plot_kargs to pass to pdp_interact and pdp_interact_plot, respectively ForwardHook class plot_multibody_weighted_outputs an interpration plot for MultiBlock models Better documentation for methods and classes Removals Some default values of arugments in AMS and MultiAMS Default for return_mean in RegAsProxyPull and RegPull Fixes Missing bbox_inches in plot_embedding Typing for cont_feats and savename in fit_input_pipe Typing for targ_feats and savename in fit_output_pipe Moved predictions to after callback on_eval_begin Updated from_model_builder class method of ModelBuilderto use and CatEmbedder Hard coded savename in Model during save to hopefull solve occaisional permission error during save Typing for val_fold in SWA 'lr' to 'momentum' in Model.set_mom Model.get_mom now actually returns momentum (beta_1) rather than lr Added catch for infinite uncertainties being passed to uncert_round Added catch for plot_roc with bootstraping when resamples data only contains one class Error when attempting to plot categorical feature in plot_1d_partial_dependence layer width in dense=True FullyConnected scaling with input size Fixed lookup_act for linear function plot_1d_partial_dependence not using n_points parameter Errors in plot_rocs when passing non-lists and when requesting plot_params and bootsrapping Missing to_device call when exporting to ONNX on a CUDA device Changes to_pt_eta_phi now infers presence of z momentum from dataframe norm_in default value for get_pre_proc_pipes is now True rather than False fold_train_ensemble now always trains n_models, and validation fold IDs are cycled through according to fy.n_folds % model_num FoldYielder.set_ignore changed to FoldYielder.add_ignore Changed HEPAugFoldYielder.rotate and HEPAugFoldYielder.reflect to private methods compute method of RegPull now private Renamed data to fy in RegPull.evaluate and RegAsProxyPull.evaluate Made get_layer in FullyConnected private Made get_dense and load_embeds in CatEmbHead private Made build_layers in 'ClassRegMulti` private Made parse methods and build_opt in ModelBuilder private Made get_folds private Changed settings to plot_settings in rf_rank_features Dense layer from CatEmbHead removed and placed in FullyConnected Swapped order of continuous and categorical embedding concatination in CatEmbHead in order to match input data arr in plot_kdes_from_bs changed to x weighted partial dependencies in plot_1d_partial_dependence are now computed by passing the name of the weight coulmn in the dataframe and normalisation is done automatically data argument for plot_binary_class_pred renamed to df plot_1d_partial_dependence and plot_2d_partial_dependence both now expect to be passed a list on training features, rather than expecteing the DataFrame to only contain the trainign features rfpimp package nolonger requires manual installation Depreciations Passing eta argument to to_pt_eta_phi. Will be removed in v0.4 binary_class_cut renamed to binary_class_cut_by_ams. Code added to call binary_class_cut_by_ams. Will be removed in v0.4 plot_dendrogram renamed to plot_rank_order_dendrogram. Code added to call plot_rank_order_dendrogram. Will be removed in v0.4 Embedder renamed to CatEmbedder. Code added to call CatEmbedder. Will be removed in v0.4 n_cont_in (number of continuous input features) argument of ModelBuilder depreciated in favour of cont_feats (list of named continuous input features). Code added to create this by encoding numbers as string. Will be removed in v0.4. Comments Online documentation to be created soo