Hi, I ran into the issue:

phylotoast 1.4.0rc2 has requirement scikit-bio<=0.4.2, but you'll have scikit-bio 0.5.1 which is incompatible. gneiss 0.4.2 has requirement scikit-bio==0.5.1, but you'll have scikit-bio 0.4.2 which is incompatible.

Either way I cant install both versions, because pip overwrites the other. Even having an environment would not solve that I guess. How can you deal with that?

Cheers, Robert

Hey @Zsailer, great to meet you at SciPy 2018!

I think pandas_flavor is what I'd like to switch over to in pyjanitor, where I simply register functions as a pandas accessor rather than subclass the entire dataframe outright.

There is something a bit magical about how pandas_flavor works though. With subclassing, everything is quite transparent - I subclass pandas DataFrames, then have the users wrap their existing dataframe inside a Janitor dataframe, following which, all of the data cleaning methods are available:

import pandas as pd
import janitor as jn

df = pd.DataFrame(...)
df = jn.DataFrame(df).clean_names()...

Say I decorated the Janitor functions as pandas accessors. How would things look like for an end-user? Would it be like the following?

import pandas as pd

df = pd.DataFrame(...).clean_names().remove_empty()...

I guess I'm just wondering, where and when does a decorated function get exposed up to pandas?

Thanks again for putting this out!

Hey @Zsailer, please let me know if you'd like help doing a release to PyPI -- I've got tests failing on the latest pyjanitor PR (here) but only because I'm relying on the latest code in PR #4!

Support xarray in the same way done in pyjanitor PR

Example of a small run for validating Regression + New stuff:

import pandas as pd

import pandas_flavor as pf

@pf.register_dataframe_method @pf.register_xarray_dataarray_method @pf.register_xarray_dataset_method def print_df(df): print(df) return df

df = pd.DataFrame(data={"x": [10, 20, 25], "y": [0, 2, 5]}) df.print_df() df.to_xarray().print_df()

Support xarray in the same way done in pyjanitor PR

Example of a small run for validating Regression + New stuff:

import pandas as pd

import pandas_flavor as pf

@pf.register_dataframe_method @pf.register_xarray_dataarray_method @pf.register_xarray_dataset_method def print_df(df): print(df) return df

df = pd.DataFrame(data={"x": [10, 20, 25], "y": [0, 2, 5]}) df.print_df() df.to_xarray().print_df()

@Zsailer let me know if you'd like more changes, happy to make them.

I've also allowed edits from maintainers so you can make quickie changes without needing to wait for me.

Closes #3.

Major changes here:

1. Remove code that supported older versions of pandas.
2. Add dev container for easy development.
3. Use lazy loading to improve import times in downstream packages that depend on pandas-flavor (e.g. pyjanitor).
4. Add minimal test suite to check for breaking changes.
5. Add code quality checks via code linters.
6. Add docstrings to satisfy code linters.
7. Add a GitHub action to automatically run tests on pull requests.

Reflecting back on this #3, it's not clear from the README how to make a pandas "flavor" that's importable.

The README shows how to register functions to DataFrames, but I think it needs documentation on how to write these functions in a module/package that you can import.

Something like:

1. Add "flavoring" functions in a module or package file my_flavor.py

   import pandas_flavor as pf
   
   @pf.register_series_method
   @pf.register_dataframe_method
   def cool_func(df, arg):
       print(arg)
       return df 
   

2. Import Pandas and that "flavor" module/package in a Python session.

   import pandas as pd
   import my_flavor
   
   df = pd.DataFrame(...).cool_func()
   

The new lazy submodule loader breaks the dependency analysis of PyInstaller. Normally, we'd manually patch this up on PyInstaller's side but, given that pandas_flavours only has two submodules (minus the one which holds the package's version), one imports only pandas and functools, the other imports the first and functools and neither of them do any processing at all on initialisation, what actually is the point in using lazy loading here anyway? Asides from breaking PyInstaller's analysis (and just looking ugly), it screws with IDE completion and static analysis tools.

I saw there have a lot of duplicate codes. And they both have the same structure. Maybe we could extract the same codes via the factory design pattern?

Duplicate codes:

• https://github.com/Zsailer/pandas_flavor/blob/f9308140d559ef1cb80587dedf6a7e32ca1f0b67/pandas_flavor/register.py#L19-L26
• https://github.com/Zsailer/pandas_flavor/blob/f9308140d559ef1cb80587dedf6a7e32ca1f0b67/pandas_flavor/register.py#L38-L47
• https://github.com/Zsailer/pandas_flavor/blob/f9308140d559ef1cb80587dedf6a7e32ca1f0b67/pandas_flavor/xarray.py#L13-L21

The prototype of this idea. It could work for pandas-like object, such as pandas.DataFrame, pandas.Series, pandas.Index, geopandas.GeoDataFrame, and geopandas.GeoSeries.

def register_method_factory(register_accessor):
    @wraps(register_accessor)
    def decorator(method):
        def method_accessor(pd_obj):
            @wraps(method)
            def wrapper(*args, **kwargs):
                return method(pd_obj, *args, **kwargs)

            return wrapper

        # Register method as pandas object inner method.
        register_accessor(method.__name__)(method_accessor)

        # Must return method itself, otherwise would get None.
        return method

    return decorator


# or register_dataframe_method = register_method_factory(register_dataframe_accessor)
@register_method_factory
def register_dataframe_method(method):
    """Docstring"""

    return register_dataframe_accessor(method)


@register_method_factory
def register_dataarray_method(method):
    """Docstring"""

    return register_dataarray_accessor(method)

When I run pip install pandas-flavor==0.3.0, I get the following error

ERROR: Could not find a version that satisfies the requirement lazy-loader==0.1rc2 (from pandas-flavor)

Problem description: It's difficult to determine how/when to use inplace or create a copy and the behavior is inconsistent. Does pandas_flavor always require a copy/modifying the original dataframe? Is there a way to avoid this in order to save memory?

import pandas_flavor

# Does not replace original dataframe
@pandas_flavor.register_dataframe_method
def drop_empty_rows(dataframe):
    return dataframe.dropna(axis=0, how='all')
 
# Should replace original dataframe
@pandas_flavor.register_dataframe_method
def drop_empty_rows(dataframe):
    dataframe = dataframe.dropna(axis=0, how='all')
    return dataframe

# Should replace original dataframe
@pandas_flavor.register_dataframe_method
def drop_empty_rows(dataframe):
    dataframe_processed = dataframe.copy()
    dataframe_processed = dataframe.dropna(axis=0, how='all')
    return dataframe_processed

If I call the first function on a dataframe, it returns the dataframe with dropped rows but does not change the original dataframe.

dict_rows = {}
dict_rows['A']  = [20,numpy.nan,40,10,50]
dict_rows['B'] = [50,numpy.nan,10,40,50]
dict_rows['C'] = [30,numpy.nan,50,40,50]

dataframe = pandas.DataFrame(dict_rows)

This function returns the reduced dataframe, but doesn't affect the original dataframe.

>>> dataframe.drop_empty_rows()
      A     B     C
0  20.0  50.0  30.0
2  40.0  10.0  50.0
3  10.0  40.0  40.0
4  50.0  50.0  50.0

First of all thank you for making pandas_flavor. It is very useful.

One issue: Is it possible to make it so that registering the method also enables tab/autocomplete for that method in some common IDEs (spyder, pycharm, vs)?

Relevant link: https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000665110-auto-completion-for-dynamic-module-attributes-in-python

Registering the method in the dir may also work?

Again, thank you for pandas_flavour.