mood.model_analysis

Tools for model analysis

class mood.model_analysis.ModelPerformanceAnalyzer(data: DataFrame, relevant_metrics: list[str] = <factory>, model_type_column: str = 'model_name', attribute_column: str = 'semantic_attribute', data_type_column: str = 'data_type')

Analyze model performance metrics across different semantic attributes and model types.

This class provides methods to generate statistical summaries and visualizations for comparing model performance across different metrics and semantic attributes.

>>> import pandas as pd
>>> # Sample data
>>> data = pd.DataFrame({
...     'semantic_attribute': ['irony', 'irony', 'moral', 'moral'],
...     'model_name': ['svm', 'log_reg', 'svm', 'log_reg'],
...     'accuracy': [0.7, 0.6, 0.9, 0.8]
... })
>>> analyzer = ModelPerformanceAnalyzer(data)
>>> analyzer.get_metrics()
['accuracy']

filter_data(attributes: list[str] | None = None, models: list[str] | None = None, data_types: list[str] | None = None) → DataFrame

Filter the dataframe based on specified attributes, models, and data types.

Parameters:

• attributes – List of semantic attributes to include

• models – List of model names to include

• data_types – List of data types to include

Returns:

Filtered dataframe

Return type:

pd.DataFrame

generate_attribute_modelability_table(primary_metric: str, secondary_metrics: list[str] | None = None, models: list[str] | None = None, data_types: list[str] | None = None) → DataFrame

Generate a table showing which semantic attributes are most modelable.

Parameters:

• primary_metric – The main metric to sort by (e.g., accuracy, r2)

• secondary_metrics – Additional metrics to include in the table

• models – List of model names to include (default: all)

• data_types – List of data types to include (default: all)

Returns:

A table with attributes as rows and metrics as columns

Return type:

pd.DataFrame

generate_model_comparison_table(metric: str, attributes: list[str] | None = None, models: list[str] | None = None, data_types: list[str] | None = None) → DataFrame

Generate a comparison table of models across different semantic attributes for a given metric.

Parameters:

• metric – The metric to compare (e.g., accuracy, f1, r2)

• attributes – List of semantic attributes to include (default: all)

• models – List of model names to include (default: all)

• data_types – List of data types to include (default: all)

Returns:

A table with attributes as rows and models as columns

Return type:

pd.DataFrame

get_attributes() → list[str]

Get the list of unique semantic attributes in the dataset.

Returns:

List of semantic attribute names

Return type:

List[str]

get_data_types() → list[str]

Get the list of unique data types in the dataset.

Returns:

List of data types (e.g., binary, numerical, ordinal)

Return type:

List[str]

get_metrics() → list[str]

Get the list of available metrics in the dataset.

Returns:

List of metric names

Return type:

List[str]

get_models() → list[str]

Get the list of unique model names in the dataset.

Returns:

List of model names

Return type:

List[str]

visualize_attribute_modelability(metric: str, subset_idx: int | None = None, models: list[str] | None = None, data_types: list[str] | None = None, figsize: tuple[int, int] = (10, 6), palette: str = 'viridis', tight_xlim: bool = True) → Figure

Create a horizontal bar chart showing which attributes are most/least modelable.

Parameters:

• metric – The metric to visualize

• subset_idx – When positive, show first N attributes; when negative, show last N attributes (None = show all attributes)

• models – List of model names to include (default: all)

• data_types – List of data types to include (default: all)

• figsize – Figure size (width, height)

• palette – Color palette for the plot

• tight_xlim – Whether to use tight x-axis limits (default: True)

Returns:

The matplotlib figure object

Return type:

plt.Figure

visualize_correlation_matrix(metrics: list[str] | None = None, figsize: tuple[int, int] = (10, 8)) → Figure

Create a heatmap showing the correlation between different metrics.

Parameters:

• metrics – List of metrics to include in the correlation matrix (default: all)

• figsize – Figure size (width, height)

Returns:

The matplotlib figure object

Return type:

plt.Figure

visualize_metric_distributions(metrics: list[str], models: list[str] | None = None, data_types: list[str] | None = None, figsize: tuple[int, int] = (12, 8)) → Figure

Create box plots showing the distribution of metrics across semantic attributes.

Parameters:

• metrics – List of metrics to visualize

• models – List of model names to include (default: all)

• data_types – List of data types to include (default: all)

• figsize – Figure size (width, height)

Returns:

The matplotlib figure object

Return type:

plt.Figure

visualize_model_comparison(metric: str, attributes: list[str] | None = None, models: list[str] | None = None, data_types: list[str] | None = None, figsize: tuple[int, int] = (10, 6), palette: str = 'viridis', tight_ylim: bool = True) → Figure

Create a grouped bar chart comparing model performance across semantic attributes.

Parameters:

• metric – The metric to visualize

• attributes – List of semantic attributes to include (default: all)

• models – List of model names to include (default: all)

• data_types – List of data types to include (default: all)

• figsize – Figure size (width, height)

• palette – Color palette for the plot

• tight_ylim – Whether to use tight y-axis limits (default: True)

Returns:

The matplotlib figure object

Return type:

plt.Figure

visualize_performance_radar(metrics: list[str], models: list[str], attribute: str, data_type: str | None = None, figsize: tuple[int, int] = (10, 8)) → Figure

Create a radar chart comparing models across multiple metrics for a specific attribute.

Parameters:

• metrics – List of metrics to include in the radar chart

• models – List of model names to include

• attribute – The semantic attribute to analyze

• data_type – Data type to filter by (default: None)

• figsize – Figure size (width, height)

Returns:

The matplotlib figure object

Return type:

plt.Figure

mood.model_analysis.analyze_all(classifier_stats, regression_stats, attribute='irony_humor')

Run all analyses and generate a complete report.

mood.model_analysis.analyze_by_data_type(classifier_stats, regression_stats)

Analyze performance by data type.

mood.model_analysis.analyze_classifiers(classifier_stats)

Example analysis of classifier models.

mood.model_analysis.analyze_regression_models(regression_stats)

Example analysis of regression models.

mood.model_analysis.analyze_specific_attribute(classifier_stats, regression_stats, attribute='irony_humor')

Focused analysis on a specific semantic attribute.

mood.model_analysis.compare_datasets(classifier_stats, regression_stats)

Compare model performance across classifier and regression datasets.

mood.model_analysis.compare_models_across_datasets(classifier_data: DataFrame, regression_data: DataFrame, common_models: list[str] | None = None, common_attributes: list[str] | None = None) → dict[str, DataFrame]

Compare model performance across classifier and regression datasets.

Parameters:

• classifier_data – Dataframe with classifier metrics

• regression_data – Dataframe with regression metrics

• common_models – List of model names to compare (must exist in both datasets)

• common_attributes – List of attributes to compare (must exist in both datasets)

Returns:

Dictionary with comparison tables

Return type:

Dict[str, pd.DataFrame]