Make VQC & QNN Scikit-learn compliant

[Sudeep09]

What is the expected enhancement?

VQC & QNN does not allow to implement Sklearn BaseEstimator to be inherited. Hence, we are unable to use Sklearn Gridsearch on VQC & QNN estimators for hyperparameter tuning.

[adekusar-drl]

There is a guide how to develop scikit learn compatible estimators: https://scikit-learn.org/stable/developers/develop.html
In general, inheritance is not required, but method compatibility is.

[Sudeep09]

class VQC_Sklearn(VQC, NeuralNetworkClassifier, BaseEstimator, ClassifierMixin):       
    def get_params(self, deep=True):
        out = dict()
        for key in self._get_param_names():
        ## The if statement is to bypass "callback" and "quantum_instance" attribute not defined
            if (key != "callback" and key != "quantum_instance"):
                value = getattr(self, key)
                if deep and hasattr(value, "get_params") and not isinstance(value, type):
                    deep_items = value.get_params().items()
                    out.update((key + "__" + k, val) for k, val in deep_items)
                out[key] = value
        return out

However, I still get the AttributeError: can't set attribute

[adekusar-drl]

However, I still get the AttributeError: can't set attribute

I guess, set_params to be implemented as well. The implementation may be tricky though.

[Sudeep09]

Finally, thanks a lot to blquanz@us.ibm.com, the solution below worked for me:

class vqc_wrapper(BaseEstimator):    
    def __init__(self, feature_map,ansatz,optimizer,initial_point,quantum_instance,callback):
        self.feature_map = feature_map
        self.ansatz = ansatz
        self.optimizer = optimizer
        self.initial_point = initial_point
        self.quantum_instance = quantum_instance
        self.callback = callback
        self.vqc = None
        
    def fit(self, X, y):
        self.vqc = VQC(
            feature_map=self.feature_map,
            ansatz=self.ansatz,
            optimizer=self.optimizer,
            initial_point=self.initial_point,
            quantum_instance=self.quantum_instance,
            callback=self.callback)
        return self.vqc.fit(X,y)
    
    def predict(self, X):
        return self.vqc.predict(X)
    
    def score(self, X, y):
        return self.vqc.score(X,y)

[adekusar-drl]

If you don't mind, I'd keep this issue open as it is not resolved in the QML code.

[GayatriVadaparty]

I have an opinion on this, we can try some approaches to fix this issue.
Instead of using Sklearn Gridsearch, we can implement a custom hyperparameter search function for VQC and QNN. Taking the reference from this: https://scikit-learn.org/stable/modules/grid_search.html . I think this is possible by defining a parameter space for the hyperparameters and then performing a grid search over that space as like we do there.
On seraching for other alternatives, I found another approach.
It is to create wrapper functions that allow VQC and QNN to be used as Sklearn estimators. This can be done by creating a wrapper class that inherits from BaseEstimator and implements the necessary methods such as fit(), predict(), and score(). The wrapper class can then be used with Sklearn Gridsearch for hyperparameter tuning.
I'm just checking to implement any one of them.

from sklearn.base import BaseEstimator, ClassifierMixin
from qiskit import Aer
from qiskit.aqua import QuantumInstance
from qiskit.aqua.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes

class VQCWrapper(BaseEstimator, ClassifierMixin):
    def __init__(self, feature_map=None, ansatz=None, optimizer=None, quantum_instance=None):
        self.feature_map = feature_map or ZZFeatureMap()
        self.ansatz = ansatz or RealAmplitudes()
        self.optimizer = optimizer or SPSA(max_trials=100)
        self.quantum_instance = quantum_instance or QuantumInstance(Aer.get_backend('qasm_simulator'))

    def fit(self, X, y):
        self.vqc = VQC(feature_map=self.feature_map, ansatz=self.ansatz, optimizer=self.optimizer, quantum_instance=self.quantum_instance)
        self.vqc.train(X, y)
        return self

    def predict(self, X):
        return self.vqc.predict(X)

    def score(self, X, y):
        return self.vqc.test(X, y)['testing_accuracy']

[AbdullahKazi500]

@GayatriVadaparty Let me know if you want to work on this Issue as a collaboration
One approach is to use a custom hyperparameter search function, as you mentioned earlier. Another approach is to create a custom estimator class that inherits from Sklearn's BaseEstimator and wraps the VQC or QNN model. This custom estimator can then be used with Sklearn's GridSearchCV or RandomizedSearchCV functions for hyperparameter tuning.
Bayesian optimization This is an alternative to grid search that uses a probabilistic model of the objective function to find promising hyperparameters. It iteratively selects the next set of hyperparameters to evaluate based on the model's predictions of where the optimal point might be.
Another option is to use other libraries or frameworks that support hyperparameter tuning for VQC and QNN models. For example, PennyLane is a popular open-source library for quantum machine learning that supports hyperparameter tuning for quantum models, including VQC and QNN. PennyLane integrates with Sklearn and other popular machine learning libraries and provides a range of optimization and gradient-based methods for hyperparameter tuning.

[GayatriVadaparty]

@AbdullahKazi500 Yes, I wanted to work on this issue. I have recently got to understand all the approaches you mentioned above.

[AbdullahKazi500]

@adekusar-drl
this is my code

!pip install qiskit-machine-learning
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from qiskit import Aer
from qiskit_machine_learning.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
from qiskit.algorithms.optimizers import SPSA
from sklearn.metrics import accuracy_score

# Generate synthetic dataset for demonstration
X, y = make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create a parameter grid for hyperparameter tuning
param_grid = {
    'ansatz': [RealAmplitudes(), None],
    'optimizer': [SPSA(maxiter=100)],
    'quantum_instance': [Aer.get_backend('qasm_simulator'), Aer.get_backend('statevector_simulator')]
}

best_score = 0
best_params = None

# Loop over the parameter grid
for ansatz in param_grid['ansatz']:
    for optimizer in param_grid['optimizer']:
        for quantum_instance in param_grid['quantum_instance']:
            try:
                # Create the feature map and VQC
                num_qubits = X.shape[1]
                feature_map = ZZFeatureMap(num_qubits, reps=1)
                vqc = VQC(feature_map, ansatz, optimizer, quantum_instance=quantum_instance)

                # Fit the model
                vqc.fit(X_train, y_train)

                # Predict on the test set
                y_pred = vqc.predict(X_test)

                # Calculate accuracy score
                score = accuracy_score(y_test, y_pred)

                # Check if the current model has the best score
                if score > best_score:
                    best_score = score
                    best_params = {'ansatz': ansatz, 'optimizer': optimizer, 'quantum_instance': quantum_instance}

            except Exception as e:
                print(f"Error: {e}")
                continue

# Print the best hyperparameters and score
print("Best hyperparameters:", best_params)
print("Best score:", best_score)```

 It seems that the ZZFeatureMap class in Qiskit is not compatible with scikit-learn's GridSearchCV due to the dynamic nature of the circuit construction.

To work around this issue, I have  manually loop over the parameter grid and perform the grid search without relying on scikit-learn's GridSearchCV
In this code, I manually loop over the parameter grid and instantiate the ZZFeatureMap, VQC, and other components. We Can fit the model, predict on the test set, and calculate the accuracy score. If the current model achieves a better score than the previous best model, we update the best score and best parameters.

[AbdullahKazi500]

@adekusar-drl

@adekusar-drl

It seems that the ZZFeatureMap class in Qiskit is not compatible with scikit-learn's GridSearchCV due to the dynamic nature of the circuit construction.

To work around this issue, I have manually loop over the parameter grid and perform the grid search without relying on scikit-learn's GridSearchCV
In this code, I manually loop over the parameter grid and instantiate the ZZFeatureMap, VQC, and other components. We Can fit the model, predict on the test set, and calculate the accuracy score. If the current model achieves a better score than the previous best model, we update the best score and best parameters.

[AbdullahKazi500]

@GayatriVadaparty Great contact me on discord

[AbdullahKazi500]

from sklearn.model_selection import train_test_split
from qiskit import Aer
from qiskit_machine_learning.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
from qiskit.algorithms.optimizers import SPSA
from sklearn.metrics import accuracy_score
from sklearn.model_selection import cross_val_score
from hyperopt import hp, tpe, Trials, fmin

# Generate synthetic dataset for demonstration
X, y = make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define the feature map
num_qubits = X.shape[1]
feature_map = ZZFeatureMap(num_qubits, reps=1)

# Define the objective function
def objective(params):
    optimizer = SPSA(maxiter=100)
    quantum_instance = Aer.get_backend('qasm_simulator')

    model = VQC(feature_map, RealAmplitudes(), optimizer, quantum_instance=quantum_instance)
    model.set_params(**params)

    scores = cross_val_score(model, X_train, y_train, cv=5)
    avg_score = scores.mean()

    return -avg_score

# Define the search space for hyperparameters
param_space = {
    'n': hp.choice('n', [None, 1, 2]),  # Number of qubits for RealAmplitudes ansatz
    'quantum_instance': hp.choice('quantum_instance', [Aer.get_backend('qasm_simulator'), Aer.get_backend('statevector_simulator')])
}

# Perform hyperparameter tuning with Bayesian optimization
trials = Trials()
best = fmin(fn=objective, space=param_space, algo=tpe.suggest, max_evals=20, trials=trials)

# Get the best hyperparameters
best_params = {key: param_space[key][value] for key, value in best.items()}

# Train the VQC with the best hyperparameters
optimizer = SPSA(maxiter=100)
quantum_instance = Aer.get_backend('qasm_simulator')

model = VQC(feature_map, RealAmplitudes(), optimizer, quantum_instance=quantum_instance)
model.set_params(**best_params)
model.fit(X_train, y_train)

# Predict on the test set
y_pred = model.predict(X_test)

# Calculate accuracy score
score = accuracy_score(y_test, y_pred)

# Print the best hyperparameters and final score
print("Best hyperparameters:", best_params)
print("Final score:", score)

To perform hyperparameter tuning using Bayesian optimization with the Qiskit Machine Learning package, we can use the BayesianOptimization class provided by the skopt library, which is compatible with scikit-learn's GridSearchCV and RandomizedSearchCV In this code, I have first define the parameter search space using a dictionary called param_space, which includes the possible values for the hyperparameters. Then, define the feature map, create the VQC estimator, and instantiate the BayesSearchCV object, providing the VQC estimator and the parameter search space. We set the number of iterations (n_iter) and the number of cross-validation folds (cv).

Next, we call the fit method of the search object, passing the training data. The search object performs Bayesian optimization to find the best hyperparameters based on the specified search space.

After the hyperparameter tuning is complete, we can access the best hyperparameters using the best_params_ attribute of the search object and the corresponding best score using the best_score_ attribute.

Finally, we set the best hyperparameters to the VQC estimator, fit the model on the training data, predict on the test set, and calculate the accuracy score. I had some problems working on the TQDM part here

[adekusar-drl]

That's interesting, but it is unrelated to the original issue. Originally, the intention was to make QML algorithms scikit-learn compliant.

[AbdullahKazi500]

@adekusar-drl exactly which QML Algorithms is not mentioned I guess every algorithm or only VQC or QNN

[adekusar-drl]

At least algorithms in qiskit_machine_learning.algorithms. VQC could a good candidate to see how it can be modified to be compliant.

[AbdullahKazi500]

@adekusar-drl
is there any method through which I Can Convert script into a module: For example, if script is named vqc.py, the directory structure should look like this:

my_package/
init.py
vqc.py

[adekusar-drl]

Basically by adding a new folder and corresponding files in it.

[AbdullahKazi500]

@adekusar-drl Do I Make a pull request with the Ipynb or you want to see it first

[adekusar-drl]

In this issue I don't see a place where a notebook would be required, honestly. So, if you have only a notebook, then please convert it to the code that will make the algorithm scikit-learn compatible.

[AbdullahKazi500]

@adekusar-drl Okay I have the code so how can we add this as a feature

[AbdullahKazi500]

import numpy as np
from qiskit import QuantumCircuit
from qiskit.algorithms.optimizers import Optimizer, OptimizerResult, Minimizer
from qiskit.primitives import BaseSampler
from sklearn.base import BaseEstimator, ClassifierMixin

from ...neural_networks import CircuitQNN, SamplerQNN
from ...utils import derive_num_qubits_feature_map_ansatz
from ...utils.loss_functions import Loss


class VQC(BaseEstimator, ClassifierMixin):
    def __init__(
        self,
        num_qubits: int | None = None,
        feature_map: QuantumCircuit | None = None,
        ansatz: QuantumCircuit | None = None,
        loss: str | Loss = "cross_entropy",
        optimizer: Optimizer | Minimizer | None = None,
        warm_start: bool = False,
        initial_point: np.ndarray | None = None,
        callback: Callable[[np.ndarray, float], None] | None = None,
        sampler: BaseSampler | None = None,
    ) -> None:
        num_qubits, feature_map, ansatz = derive_num_qubits_feature_map_ansatz(
            num_qubits, feature_map, ansatz
        )
        self.num_qubits = num_qubits
        self.feature_map = feature_map
        self.ansatz = ansatz
        self.circuit = QuantumCircuit(self.num_qubits)
        self.circuit.compose(self.feature_map, inplace=True)
        self.circuit.compose(self.ansatz, inplace=True)
        self.neural_network = SamplerQNN(
            sampler=sampler,
            circuit=self.circuit,
            input_params=self.feature_map.parameters,
            weight_params=self.ansatz.parameters,
            interpret=self._get_interpret(2),
            output_shape=2,
            input_gradients=False,
        )
        self.loss = loss
        self.optimizer = optimizer
        self.warm_start = warm_start
        self.initial_point = initial_point
        self.callback = callback

    def fit(self, X: np.ndarray, y: np.ndarray) -> VQC:
        self._validate_input(X, y)
        self.neural_network.set_interpret(self._get_interpret(self.num_classes), self.num_classes)
        function = self._create_objective(X, y)
        self._minimize(function)
        return self

    def predict(self, X: np.ndarray) -> np.ndarray:
        self._validate_X(X)
        return self.neural_network.forward(X)

    def score(self, X: np.ndarray, y: np.ndarray) -> float:
        self._validate_input(X, y)
        y_pred = self.predict(X)
        return np.mean(np.argmax(y, axis=1) == np.argmax(y_pred, axis=1))

    def _get_interpret(self, num_classes: int):
        def parity(x: int, num_classes: int = num_classes) -> int:
            return x % num_classes

        return parity

    def _validate_input(self, X: np.ndarray, y: np.ndarray) -> None:
        # Validation logic
        pass

    def _create_objective(self, X: np.ndarray, y: np.ndarray) -> Callable:
        # Objective function creation logic
        pass

    def _minimize(self, objective: Callable) -> OptimizerResult:
        # Minimization logic
        pass

    def _validate_X(self, X: np.ndarray) -> None:
        # Validation logic for input data
        pass

    @property
    def num_classes(self) -> int:
        # Get the number of classes
        return 2```