Greetings,

I can not reproduce notebooks given in the ./notebooks. For example, run code given in the Real World MERF Examples notebook thrower out two errors.

When I tried to import the modules, the first error popped out

No module named 'merf.evaluator'

When I tried to call mrf.fit(X_train, Z_train, clusters_train, y_train), the error popped out

ValueError: operands could not be broadcast together with shapes (10,) (162,)

I also attached the full information below FYI. BTW, I installed the merf using pip install merf. It should be the latest version.

Any suggestion is appreciated.

ValueError Traceback (most recent call last) in () 13 clusters_train = train['Subject'] 14 y_train = train['Reaction'] ---> 15 mrf.fit(X_train, Z_train, clusters_train, y_train) 16 17 # Mixed Effects Random Forest Test

~/anaconda/lib/python3.6/site-packages/merf/merf.py in fit(self, X, Z, clusters, y) 142 143 # Compute y_star for this cluster and put back in right place --> 144 y_star_i = y_i - Z_i.dot(b_hat_i) 145 y_star[indices_i] = y_star_i 146

~/anaconda/lib/python3.6/site-packages/pandas/core/ops.py in wrapper(left, right, name, na_op) 713 lvalues = lvalues.values 714 --> 715 result = wrap_results(safe_na_op(lvalues, rvalues)) 716 return construct_result( 717 left,

~/anaconda/lib/python3.6/site-packages/pandas/core/ops.py in safe_na_op(lvalues, rvalues) 674 try: 675 with np.errstate(all='ignore'): --> 676 return na_op(lvalues, rvalues) 677 except Exception: 678 if isinstance(rvalues, ABCSeries):

~/anaconda/lib/python3.6/site-packages/pandas/core/ops.py in na_op(x, y) 650 try: 651 result = expressions.evaluate(op, str_rep, x, y, --> 652 raise_on_error=True, **eval_kwargs) 653 except TypeError: 654 if isinstance(y, (np.ndarray, ABCSeries, pd.Index)):

~/anaconda/lib/python3.6/site-packages/pandas/computation/expressions.py in evaluate(op, op_str, a, b, raise_on_error, use_numexpr, **eval_kwargs) 208 if use_numexpr: 209 return _evaluate(op, op_str, a, b, raise_on_error=raise_on_error, --> 210 **eval_kwargs) 211 return _evaluate_standard(op, op_str, a, b, raise_on_error=raise_on_error) 212

~/anaconda/lib/python3.6/site-packages/pandas/computation/expressions.py in _evaluate_numexpr(op, op_str, a, b, raise_on_error, truediv, reversed, **eval_kwargs) 119 120 if result is None: --> 121 result = _evaluate_standard(op, op_str, a, b, raise_on_error) 122 123 return result

~/anaconda/lib/python3.6/site-packages/pandas/computation/expressions.py in _evaluate_standard(op, op_str, a, b, raise_on_error, **eval_kwargs) 61 _store_test_result(False) 62 with np.errstate(all='ignore'): ---> 63 return op(a, b) 64 65

ValueError: operands could not be broadcast together with shapes (10,) (162,)

When trying to set arguments like n_estimators for MERF, MERF.fit returns the above initialisation error. It works fine with default setting. This seems to suggest that MERF to sklearn random forest API is broken. Is this a version issue? I'm using: Python 3.9.5 scikit-learn==0.24.2

It would be nice if there was a function for creating partial dependency plot data or plots. This would help with translating information to consumers and clients

The current implementation of MERF only allows the user to set the number of trees in the random forest. This pull request allows the user to set any parameter they want but prevents them from changing important params like having the out of bag score on.

I hope you accept this it has been useful for me.

Also, would using the ExtraTreesRegressor also in this framework? It seems simple to just add a base_estimator parameter, allowing the user to use whatever he/she wants?

I can submit a pull request if it makes sense to add these features.

Thank you for this wonderful package and your efforts!

For future users, would it be possible to add type-checks and more verbose error statements to the fit func of the MERF class when the input type deviates from the expected input type? This led to a bit of reverse engineering to figure out why the underlying linear alg was failing when, for example, inputting a NUMPY array in place of an expected pandas series.

(Also happy to contribute...when I have the time).

Thanks!

I am using Python 3.6.5 and Pandas version 1.0.1. I receive the above when attempting to plot a specific cluster by its respective id.

I noticed here in the example notebook that Panel is deprecated. I welcome feedback on the best way to address this.

Add ability to specify additional parameters to RandomForestRegressor, with some minor error checking and two additional tests.

Options are passed in as dictionary argument rf_opts. n_estimators is ignored in favor of the argument to the constructor; oob_score is ignored and always set to True; warm_start is ignored (assumed to be False).

Two reasons this is helpful for me, at least: the random forest fit is often faster (particularly on my laptop) with n_jobs set to 1. I'm also hoping to investigate the shap interpretability package, but the underlying algorithms perform extremely poorly on deep trees, so the ability to specify max_depth would make that a bit easier.

This is a repeat of another pull request, closed by me, attempting to fix formatting issues that cause continuous integration to fail.

First, thank you for creating this package! It is kind of exactly what I am looking for. However, I have some questions that stem from my prior experience with mixed models on other platforms (notably the lme4 package for R), and the documentation and examples aren't helping me translate my knowledge of how random effects are specified/named in MERF.

For example, in lme4, random effects are designated as having slopes and intercepts (which I think correspond to "clusters" and "covariates" in MERF? With 1s in the covariates matrix indicating the intercepts?).

So if "subject" (in an experiment) or "county") like in the radon example are grouping variables over which there are multiple observations, one could specify a random intercept for subject (or county). Then, one could additionally specify a random slope for some variable by the grouping variable (such as to specify a random slope for experimental condition by subject, to allow the model to estimate the variance of how much subjects differ in their response to the experimental manipulation, or that counties could have a random slope for floor, allowing counties to differ in how much each floor impacts random levels in the model). I'm not quite sure if I'm translating these to the MERF nomenclature correctly.

Moreover, lme4 allows the researcher to specify multiple, crossed random effects (e.g., random intercepts for both experimental subjects and experimental items, as well as random slopes for variables by both subject and item).

Getting to the point:

1. My looking through the examples leads me to think that the clusters argument is the column containing the IDs for which random intercepts are generated: Is this the case?
2. I get the inclination that the Z matrix includes 1s for the random intercepts, but can include a second column for random slopes (i.e., the covariates): Is this the case? Can there be more columns?

More generally, some comments in the notebooks or documentation about what these variables are (concretely, and what form they must/can/cannot take, and possibly relationships to how random effects are specified in other mixed modeling packages) would be very helpful.

3. Can MERF handle crossed random effects structures?
4. Finally, does MERF provide variable importance measures from the fit forest (analogous to those produced in sklearn, and from randomForest and party::cforest in R)? I couldn't find mention of that in the readme or the notebooks.

Thanks!

Thanks for an excellent package! I am very keen to try out your latest updates from the last few days (including using lighgbm, SHAP values). However, I cannot seem to pip install directly from github or clone because of this error:

Clone failed
				Invalid path 'data/Rossmann Store Sales | Kaggle eval.pdf'
				unable to checkout working tree
				warning: Clone succeeded, but checkout failed.
				You can inspect what was checked out with 'git status'
				and retry with 'git restore --source=HEAD :/'

It seems like this error is because of the file name 'Rossmann Store Sales | Kaggle eval.pdf' in the data directory. Is it possible to rename the two pdf files in that directory?

Hi, it took me a long time to train the MERF on my dataset. After training, I tried saving it as a pickle object and reload it to make prediction on test data. However, I got an error saying the pkl is a dictionary that doesn't have a predict method. Any ideas on how to get around this? I don't want to train the model every time I pass new test data. Thanks.

AttributeError                            Traceback (most recent call last)

<ipython-input-55-0bd2bd5ab064> in <module>()
----> 1 y_hat_known_merf = loaded_model.predict(X_known, Z_known, clusters_known)
      2 y_hat_known_merf

AttributeError: 'dict' object has no attribute 'predict'

Bumps numpy from 1.18.4 to 1.22.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hello, Thanks for your great work on merf! When I debug merf, I found that there is one line that does not work in any case:

63   def predict(self, X: np.ndarray, Z: np.ndarray, clusters: pd.Series):
           ...
          for cluster_id in self.cluster_counts.index:
                indices_i = clusters == cluster_id

               # If cluster doesn't exist in test data that's ok. Just move on.
96           if len(indices_i) == 0:  < ------------------ might revise to: if sum(indices_i) == 0
                    continue

               # If cluster does exist, apply the correction.
                ...

I marked Line # 96 and suggested changing "len()" to "sum()", otw, this if will never run in any case because indices_i is a pd.Series has the same shape with the input cluster.

And if possible, I would like to ask another question about the random effect matrix Z.

In the given example notebook, I noticed that when considering one variance as a random effect, the Z is not simply composed of one column but also has another column of ones. Why are the ones necessary?

Thanks in advance! meng

MERF does not implement get_params. This means that scikit-learn cannot clone a MERF predictor as needed for e.g., RandomSearchCV to optimize the hyper parameter of the fixed effects model.

Hi Dey, Thank you so much for providing this useful python pachage. I watched your video that is very enlighting, and is curretly using the merf package to handle my data. I'm just wondering if it is possible to specify the random_state for merf for repetition. I tried to specify random_state for RandomForestRegressor, but it still returns different results every time I run the code. Below are my codes, mrf = MERF(fixed_effects_model=RandomForestRegressor(n_estimators=100, n_jobs=-1,random_state = 2),max_iterations=20)

Do you have any idea? Thanks, Zixiao

mrf.fit(X_train, Z_train, clusters_train, y_train, X_val, Z_val, clusters_val, y_val) y_hat_known = mrf.predict(X_known, Z_known, clusters_known)

is the mrf model the best performance model based on X_val, Z_val, clusters_val, y_val?

BW, Yang