I am able to generate the same report on Titanic data as in the Medium articles. However, when I try to test the Boston housing data, I get the errors as below:

ValueError Traceback (most recent call last) ~\AppData\Local\Continuum\anaconda3\envs\envSDS\lib\site-packages\pandas\core\indexes\base.py in get_slice_bound(self, label, side, kind) 5166 try: -> 5167 return self._searchsorted_monotonic(label, side) 5168 except ValueError:

~\AppData\Local\Continuum\anaconda3\envs\envSDS\lib\site-packages\pandas\core\indexes\base.py in _searchsorted_monotonic(self, label, side) 5127 -> 5128 raise ValueError("index must be monotonic increasing or decreasing") 5129

ValueError: index must be monotonic increasing or decreasing

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last) in ----> 1 my_report = sv.analyze(dfx)

Any ideas on the error?

Thanks.

I ran into a "FloatingPointError: divide by zero encountered in true_divide" in the pairwise feature portion of the code. Apparently there was a divide by zero issue in the cov part of the underlying code.

The trace of the error is as follows: file: sv_public.py, line 13, in analyze, pairwise_analysis, feat_cfg) file: dataframe_report.py, line 243, in init, self.process_associations(features_to_process, source_target_series, compare_target series file: dataframe_report.py, line 423, in process_associations, feature.source.corr(other.source, method='pearson') file: series.py line 2254, in corr, this.values, other.values, method=method, min_periods=min_periods file: nanops.py, line 69, in _f, return f(*args,*kwargs) file: nanops.py, line 1240, in nancorr, return f(a,b) file: nanops.py, line 1256, in _pearson, return np.corrcoef(a,b)[0,1] file: <array_function internals>, line 6, in corrcoef file: function_base.py,line 2526 in corrcoef, c=cov(x,y,rowvar) file: <array_function internals>, line 6, in cov file: function_base.py, line 2455, in cov, c=np.true_divide(1,fact)

My dataframe had some empty strings where nulls should have been, but there were other columns that had similar features, but they never threw this error.

Hey guys, I'm getting an error when handling integer columns but the error message is not very clear for me to understand what is going on. So far it looks like a bug to me. Here it goes.

We start by importing basic stuff and generate a pandas dataframe with 4 columns containing random real numbers, plus an integer column named 'target' with values 1 and 2.

import sweetviz as sv
import pandas as pd
import numpy as np

np.random.seed(42)
np_data = np.random.randn(10, 4)
df = pd.DataFrame(np_data, columns=['col1', 'col2', 'col3', 'col4'])
df['target'] = 1.0
df['target'].iloc[5:] = 2.
df['target'] = df['target'].astype(int)

Taking a look at the original types of the dataframe (df.dtypes), we have as a result: col1 float64 col2 float64 col3 float64 col4 float64 target int32 dtype: object

Error: TypeError

compareReport = sv.compare_intra(df, df['target'] == 1, ["Complete", "Incomplete"])
compareReport.show_html()

gives this message:

TypeError                                 Traceback (most recent call last)
<ipython-input-54-8e3e89553904> in <module>
      1 #feature_config = sv.FeatureConfig(force_num=['col1', 'col2', 'col3', 'col4'], force_cat='target')
----> 2 compareReport = sv.compare_intra(df, df['target'] == 1, ["Complete", "Incomplete"])#, feat_cfg=feature_config, target_feat='target')
      3 compareReport.show_html() # Default arguments will generate to "SWEETVIZ_REPORT.html"

~\AppData\Local\Continuum\anaconda3\envs\sweetbug\lib\site-packages\sweetviz\sv_public.py in compare_intra(source_df, condition_series, names, target_feat, feat_cfg, pairwise_analysis)
     42     report = sweetviz.DataframeReport([data_true, names[0]], target_feat,
     43                                       [data_false, names[1]],
---> 44                                       pairwise_analysis, feat_cfg)
     45     return report
     46 

~\AppData\Local\Continuum\anaconda3\envs\sweetbug\lib\site-packages\sweetviz\dataframe_report.py in __init__(self, source, target_feature_name, compare, pairwise_analysis, fc)
    215             # start = time.perf_counter()
    216             self.progress_bar.set_description(':' + f.source.name + '')
--> 217             self._features[f.source.name] = sa.analyze_feature_to_dictionary(f)
    218             self.progress_bar.update(1)
    219             # print(f"DONE FEATURE------> {f.source.name}"

~\AppData\Local\Continuum\anaconda3\envs\sweetbug\lib\site-packages\sweetviz\series_analyzer.py in analyze_feature_to_dictionary(to_process)
     92         compare_type = determine_feature_type(to_process.compare,
     93                                               to_process.compare_counts,
---> 94                                               returned_feature_dict["type"], "COMPARED")
     95         if compare_type != FeatureType.TYPE_ALL_NAN and \
     96             source_type != FeatureType.TYPE_ALL_NAN:

~\AppData\Local\Continuum\anaconda3\envs\sweetbug\lib\site-packages\sweetviz\type_detection.py in determine_feature_type(series, counts, must_be_this_type, which_dataframe)
     73             var_type = FeatureType.TYPE_TEXT
     74         else:
---> 75             raise TypeError(f"Cannot force series '{series.name}' in {which_dataframe} to be from its type {var_type} to\n"
     76                             f"DESIRED type {must_be_this_type}. Check documentation for the possible coercion possibilities.\n"
     77                             f"This can be solved by changing the source data or is sometimes caused by\n"

TypeError: Cannot force series 'target' in COMPARED to be from its type FeatureType.TYPE_CAT to
DESIRED type FeatureType.TYPE_BOOL. Check documentation for the possible coercion possibilities.
This can be solved by changing the source data or is sometimes caused by
a feature type mismatch between source and compare dataframes.

If I explicitly supply the feat_cfg argument the result is the same.

feature_config = sv.FeatureConfig(force_num=['col1', 'col2', 'col3', 'col4'], force_cat='target')
compareReport = sv.compare_intra(df, df['target'] == 1, ["Complete", "Incomplete"], feat_cfg=feature_config)
compareReport.show_html() # Default arguments will generate to "SWEETVIZ_REPORT.html"

However, if I add 10 to the 'target' column (it will now have 11 and 12 as values), the report is generated without errors. Am I missing something or it is indeed a bug?

In some cases the key is a boolean value not a string. A keyerror is produced when a boolean value appears in key. Reference #42

I was able to recreate the issue as user described and was able to fix by casting key as string. It seems like the key should always be a string.

I've a dataset which has date_time column of the format: 2020-07-12 11:37:25

I get the following error:

:date_time:                        |███                  | [ 14%]   00:00  -> (00:03 left)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-77-cbd387f7f43e> in <module>()
      1 #analyzing the dataset
----> 2 techglares_report = sv.analyze(df)

6 frames
/usr/local/lib/python3.6/dist-packages/sweetviz/sv_public.py in analyze(source, target_feat, feat_cfg, pairwise_analysis)
     11             pairwise_analysis: str = 'auto'):
     12     report = sweetviz.DataframeReport(source, target_feat, None,
---> 13                                       pairwise_analysis, feat_cfg)
     14     return report
     15 

/usr/local/lib/python3.6/dist-packages/sweetviz/dataframe_report.py in __init__(self, source, target_feature_name, compare, pairwise_analysis, fc)
    214             # start = time.perf_counter()
    215             self.progress_bar.set_description(':' + f.source.name + '')
--> 216             self._features[f.source.name] = sa.analyze_feature_to_dictionary(f)
    217             self.progress_bar.update(1)
    218             # print(f"DONE FEATURE------> {f.source.name}"

/usr/local/lib/python3.6/dist-packages/sweetviz/series_analyzer.py in analyze_feature_to_dictionary(to_process)
     90 
     91     # Establish base stats
---> 92     add_series_base_stats_to_dict(to_process.source, to_process.source_counts, returned_feature_dict)
     93     if to_process.compare is not None:
     94         add_series_base_stats_to_dict(to_process.compare, to_process.compare_counts, compare_dict)

/usr/local/lib/python3.6/dist-packages/sweetviz/series_analyzer.py in add_series_base_stats_to_dict(series, counts, updated_dict)
     42     base_stats = updated_dict["base_stats"]
     43     num_total = counts["num_rows_total"]
---> 44     num_zeros = series[series == 0].sum()
     45     non_nan = counts["num_rows_with_data"]
     46     base_stats["total_rows"] = num_total

/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py in stat_func(self, axis, skipna, level, numeric_only, min_count, **kwargs)
  11180             skipna=skipna,
  11181             numeric_only=numeric_only,
> 11182             min_count=min_count,
  11183         )
  11184 

/usr/local/lib/python3.6/dist-packages/pandas/core/series.py in _reduce(self, op, name, axis, skipna, numeric_only, filter_type, **kwds)
   3901             numeric_only=numeric_only,
   3902             filter_type=filter_type,
-> 3903             **kwds,
   3904         )
   3905 

/usr/local/lib/python3.6/dist-packages/pandas/core/base.py in _reduce(self, op, name, axis, skipna, numeric_only, filter_type, **kwds)
   1058         if func is None:
   1059             raise TypeError(
-> 1060                 f"{type(self).__name__} cannot perform the operation {name}"
   1061             )
   1062         return func(skipna=skipna, **kwds)

TypeError: DatetimeIndex cannot perform the operation sum

I'm running sweetviz on Google Colab.

Is there any way to solve this error?

First of all it’s awesome! Many thanks for your effort on data visualization! There is a small issue maybe, the html report lacks a meta tag showing the charset as “utf-8”; by adding it, the report can correctly show the MBCS characters and will catch eyes of more global analysts. Thanks again! Hope this project goes better!

Hi there,

I try to use sweetviz in local:

• Ubuntu 20.04

And in anaconda enterprise:

• K8s with centOS

Both lead to the same issue. The display of the output in jupyter lab and notebook isn't visible.

Local:

• Jupyter_lab=2.0 AE:
• jupyter=1.0.0
• jupyter_client=5.3.3
• jupyter_console=6.0.0
• jupyter_core=4.5.0
• Jupyter_lab=1.1.3
• ipython=7.8.0

The report has been generated but not display.

How to fix it?

Best

Hey there, this is a great package and it is pretty handy. However, I run into a strange layout issue when generating the plot.

sweet = stz.analyze(data) sweet.show_html()

Above is the code I used, and I attached the result's layout as a png file below.

1: Would you kindly inform me of the option for layout in show_html()? 2: How I should solve this issue?

Thank you so much!

hi,

sweetviz does not work for a special table

i get the following error AAfindfont: Font family ['STIXGeneral'] not found. Falling back to DejaVu Sans. findfont: Font family ['DejaVu Sans'] not found. Falling back to DejaVu Sans. findfont: Font family ['DejaVu Sans'] not found. Falling back to DejaVu Sans. findfont: Font family ['DejaVu Sans'] not found. Falling back to DejaVu Sans. findfont: Font family ['DejaVu Sans'] not found. Falling back to DejaVu Sans. ... RecursionError: maximum recursion depth exceeded in comparison

how can i resolve this ?

thanks for any hint

Hi, attempting to install sweetviz using pip install sweetviz, but kept encountering following error message (reproduced below) Am using pandas version 1.0.1. Kindly advise, thanks.

Installing collected packages: importlib-resources, pandas, tqdm, sweetviz Attempting uninstall: pandas Found existing installation: pandas 1.0.1 Uninstalling pandas-1.0.1: ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\users\\65943\\anaconda3\\lib\\site-packages\\~andas\\_libs\\algos.cp37-win_amd64.pyd' Consider using the--useroption or check the permissions.

Error thrown up during analyze(dataframe), right after :PAIRWISE DONE: and Creating Associations graph... Traceback (most recent call last):

File "", line 1, in myreport = sv.analyze(df)

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\sv_public.py", line 13, in analyze pairwise_analysis, feat_cfg)

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\dataframe_report.py", line 246, in init self._association_graphs["all"] = GraphAssoc(self, "all", self._associations)

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\graph_associations.py", line 165, in init f = corrplot(graph_data, dataframe_report)

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\graph_associations.py", line 410, in corrplot dataframe_report = dataframe_report

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\graph_associations.py", line 318, in heatmap cur_size[1] / 2, facecolor=value_to_color(color[index]),

File "C:\Users\cnble\anaconda37\lib\site-packages\sweetviz\graph_associations.py", line 210, in value_to_color ind = int(val_position * (n_colors - 1)) # target index in the color palette

ValueError: cannot convert float NaN to integer

I am fun of this library. I have used, I would like to use for all EDA, however it is giving error with a dataframe out of json file. After making a dataframe, I didn't think this would be a problem.

My code: import sweetviz as sv my_report = sv.analyze(df) my_report.show_html() # Default arguments will generate to "SWEETVIZ_REPORT.html"

Warnings:

C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\dataframe_report.py:74: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. all_source_names = [cur_name for cur_name, cur_series in source_df.iteritems()] C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\dataframe_report.py:109: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. filtered_series_names_in_source = [cur_name for cur_name, cur_series in source_df.iteritems()

C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_cat.py:28: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. for item in category_counts.iteritems(): C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_text.py:19: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. for item in to_process.source_counts["value_counts_without_nan"].iteritems(): C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_text.py:19: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. for item in to_process.source_counts["value_counts_without_nan"].iteritems(): C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_cat.py:28: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead. for item in category_counts.iteritems(): C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_numeric.py:25: FutureWarning: The 'mad' method is deprecated and will be removed in a future version. To compute the same result, you may do (df - df.mean()).abs().mean(). stats["mad"] = series.mad() C:\Users\thomp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sweetviz\series_analyzer_numeric.py:25: FutureWarning: The 'mad' method is deprecated and will be removed in a future version. To compute the same result, you may do (df - df.mean()).abs().mean(). stats["mad"] = series.mad()

I am trying to plot the distribution of a variable and target rate in each of its value, sweetviz shows wrong target rate. Below is the reproducible code.

import pandas as pd
import sweetviz as sv

var1 = [0.]*10 + [1.]*10 + [2]*10 + [3]*10
target = [0]*2 + [1]*8 + [0]*4 +[1]*6 + [0]*8 + [1]*2 + [0]*10
df = pd.DataFrame({'var1':var1, 'target':target})

fc = sv.FeatureConfig(force_num=['var1'])
report = sv.analyze([df, 'Train'], target_feat='target', feat_cfg=fc, pairwise_analysis='off')
report.show_html('report.html')
report.show_notebook('report.html')

I know that, if var1 is forcefully set to categorical, it shows the correct output. But it is not useful for me, since categorical variables sweetviz charts are not sorted based axis labels, but on the size of category.

How to make this work, by keep the variable numerical itself?

Is it possible to add an argument to silence the progress bar ?

We want to use SweetViz in an automatique pipeline and store the report in a database. We already have a lot of logs in our process, hence we would love to get rid of the progress bar logs. We can deactivate tqdm before loading SweetViz, but that would also impact others parts of our process.

One solution might be to add an argument in DataframeReport.__init__ and set self.progress_bar to a fake logger.

Hi,

I have tried to insert the line sv.config_parser.read("override.ini") into my code right after the import after I have set "show_logo = 0" under the layout section but I noticed it is still reading from the default .ini file as opposed to the new one (which I have duplicated from existing one before apply my change).

If I proceed to set the "show_logo = 0" in teh default ini file, it is working (logo no longer shows), Any advise?

Thanks.

If we have more than 100 features, no label is clear in current correlation map.

But if we create heatmap by seaborn or just pandas, user can zoom html to see characters clearly.

Further more, use html+js can provide hover infomation on heatmap cells.