User Guide
==========

This guide provides a step-by-step workflow for using EXPLICAD to evaluate explainability heatmaps.

Installation and Setup
----------------------

Ensure you have the project dependencies installed as per `requirements.txt`. To use the tools, you will need a Vision Transformer (ViT) model and its corresponding processor (e.g., from HuggingFace `transformers`).

Example Imports:

.. code-block:: python

    from transformers import ViTForImageClassification, AutoProcessor
    from xai_methods.gradcam_vit import ViTCAM
    from evaluator.plausibility import Plausibility
    from evaluator.fidelity.insertion import Insertion
    from evaluator.fidelity.deletion import Deletion
    from evaluator.fidelity.faithfulness_corr import FaithfulnessCorrelation
    from evaluator.consistency.RIS import RIS
    from evaluator.consistency.ROS import ROS
    from evaluator.consistency.SSIM import SSIM

Workflow Overview
-----------------

The typical workflow consists of two main phases: generating the explanation (heatmap) and evaluating it.

1. Generating Heatmaps
~~~~~~~~~~~~~~~~~~~~~~

Use the methods in `xai_methods` to generate a heatmap for your image. For example, using `ViTCAM`:

.. code-block:: python

    model = ViTForImageClassification.from_pretrained("google/vit-base-patch16-224")
    processor = AutoProcessor.from_pretrained("google/vit-base-patch16-224")
    
    # Define target layers for Grad-CAM
    target_layer = [model.vit.encoder.layer[6].output.dense]
    
    cam = ViTCAM(
        model=model,
        processor=processor,
        target_layers=target_layer,
        targets=None,
        cam_method="gradcamelementwise",
    )
    
    _, image_arr, grayscale_cam, cam_image, label = cam.generate(image)
    heatmap = grayscale_cam[0]

2. Evaluating Explanations
~~~~~~~~~~~~~~~~~~~~~~~~~~

Once you have the heatmap, you can evaluate it across three dimensions: Plausibility, Fidelity, and Consistency.

Plausibility
------------

Plausibility measures how well the explanation aligns with human-annotated ground truth regions.

.. code-block:: python

    def bbox_from_heatmap(heatmap):
        # Implementation to extract a bounding box from the heatmap
        ...

    plausibility = Plausibility(
        model=model,
        processor=processor,
        target_layers=target_layer,
        targets=None,
        bbox_func=bbox_from_heatmap,
        explainability_method="cam",
        cam_method="gradcamelementwise",
    )

    scores = plausibility.plausibility(image=image, ground_truth=ground_truth_box)
    print(f"IoU: {scores['iou']}")

Fidelity
--------

Fidelity tests whether the explanation faithfully reflects the model's behavior by perturbing important regions.

.. code-block:: python

    # Insertion AUC
    insertion = Insertion(model=model, processor=processor, target_layers=target_layer, targets=None, cam_method="gradcamelementwise")
    ins_auc = insertion([image])

    # Deletion AUC
    deletion = Deletion(model=model, processor=processor, target_layers=target_layer, targets=None, cam_method="gradcamelementwise")
    del_auc = deletion([image])

    # Faithfulness Correlation
    faithfulness = FaithfulnessCorrelation(model=model, processor=processor, target_layers=target_layer, targets=None, cam_method="gradcamelementwise")
    fc_score = faithfulness([image])

Consistency
-----------

Consistency measures the stability of explanations under small input transformations.

.. code-block:: python

    ris = RIS(model=model, processor=processor, target_layers=target_layer, targets=None, cam_method="gradcamelementwise")
    ris_scores = ris(images=[image], transform_type="rotation", p_val=2)