Natural Gradient implementation

[WanderingMike]

Implementation of the Natural Gradient. First, the optimizer class has to be merged and only then will the tests run

Checklist:

• Reviewers confirm new code works as expected.
• Tests are passing.
• Coverage does not decrease.
• Documentation is updated.

[renatomello]

I remember during one of the past Qibo meetings I made a suggestion to look into calculating the Fubini-Study metric tensor using a modified version of the Circuit.light_cone method.

I haven't tested anything but it seems to me it would be faster + introduce way less complexity to the underlying code, since it would not require a graph representation of the circuit.

Was that tested by anyone? @WanderingMike @MatteoRobbiati .

I'm asking because calculating this matrix is relevant not only here, but it could also be a function in the quantum_info module that calculates the Fisher information matrix in general, as it was requested in previous issues (e.g. #657)

[WanderingMike]

@renatomello I am currently trying to benchmark both methods to see if there is a big gap in performance. The big advantage with the graph method is that you don't need to build a new circuit for every gate that you're considering. You can go layer by layer through the circuit and calculate the variance directly on all qubits with the same circuit. This should in theory become advantageous if you have circuit sizes >> 1.

Sidenote: Also, from a high-level point-of-view, it might make sense to start looking at using graphs as a general method in the qibo backend, though this is a larger discussion to be had. It might be a better representation for large circuits

[renatomello]

When this moves forward, I'd like to ask for the subroutine that calculates the Quantum Fisher Information / geometric tensor to be implemented inside the quantum_info module, maybe in quantum_info.metrics. I ask this because it is important on its own and necessary for other things.

[marekgluza]

@renatomello and @scarrazza do you have some notes/docs on the gradient types that you are implementing? Is this linked to a larger feature request?

@derenliu (phd student of Pinaki Sengupta at NTU SG) will be looking at gradients from the geometrical perspective for the dbi model and @wrightjandrew and @MatteoRobbiati and @Sam-XiaoyueLi wanted to also look at gradients to optimize iterations.
Is there something that we can make use of for DBI?

We could use onboarding Deren as an occasion to add some notes to the docs and examples/tutorial notebooks. @jeongrak-son and @Ashwinie-Ghanesh are also interested in the theory so maybe could help add to notes

[MatteoRobbiati]

Closing this here, but I am going to open an issue in the Qiboml repo.