The Finite Difference Method: 1D steady state heat transfer

These examples are based on code originally written by Krzysztof Fidkowski and adapted by Venkat Viswanathan.

import jax
import jax.numpy as jnp
from jax import grad, jit

from jax.scipy.linalg import solve
from scipy.optimize import minimize

jax.config.update("jax_enable_x64", True)  # Enable 64-bit precision for better numerical stability

import matplotlib.pyplot as plt
import niceplots

plt.style.use(niceplots.get_style())

# Force the jupyter notebook to use vector graphics
import matplotlib_inline

matplotlib_inline.backend_inline.set_matplotlib_formats("pdf", "svg")

This code implements the example case from section 3.1.1 of the course notes.

We will solve the steady state heat transfer equation in 1D:

\[ -\kappa\frac{d^2T}{dx^2} = q(x) \]

where \(\kappa\) is the thermal conductivity, \(T\) is the temperature, \(x\) is the position, and \(q(x)\) is a heat source term, which in this case is \(\sin\left(\pi x /L\right)\).

# Define the symbolic function q(x)
def q(x, L):
    return jnp.sin(jnp.pi * x / L)

The 1D domain spans \(0 \le x \le L\) and is split into \(N\) intervals of length \(\Delta x = L/N\), this gives \(N+1\) nodes in the grid. The temperatures at the nodes are \(T_0, T_1, \ldots, T_N\). Dirichlet boundary conditions are applied at \(x=0\) and \(x=L\), such that \(T_0 = 1\) and \(T_N=4\).

Using the central difference approximation for the second derivative, we can write the equation at each node as:

\[ -\kappa\frac{T_{i-1} - 2T_i + T_{i+1}}{\Delta x^2} = q(x_i) \Rightarrow -T_{i-1} + 2T_i - T_{i+1} = \frac{1}{\kappa} \Delta x^2 q(x_i)\]

We will enforce this equation at all nodes except the boundary nodes, which have known temperatures. This gives \(N-1\) equations for the \(N+1\) unknown temperatures. At nodes \(i=1\) and \(i=N-1\), we have to rearrange the equation to account for the known temperature values that appear in the finite difference stencil:

\[\text{at } i=1, \quad 2T_1 - T_2 = \frac{1}{\kappa} \Delta x^2 q(x_i) + T_0\]

\[\text{at } i=N-1, \quad -T_{N-2} + 2T_{N-1} = \frac{1}{\kappa} \Delta x^2 q(x_i) + T_N\]

Writing this all up in matrix-form we get:

\[\begin{split}\begin{bmatrix} 2 & -1 & 0 & 0 & \cdots & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & \cdots & 0 & 0 & -1 & 2 \end{bmatrix} \begin{bmatrix} T_1 \\ T_2 \\ T_3 \\ \vdots \\ T_{N-1} \end{bmatrix} = \begin{bmatrix} q(x_1) \Delta x^2 / \kappa + T_0 \\ q(x_2) \Delta x^2 / \kappa \\ q(x_3) \Delta x^2 / \kappa \\ \vdots \\ q(x_{N-1}) \Delta x^2 / \kappa + T_N \end{bmatrix} \end{split}\]

Alternatively, we can keep the boundary nodes in the matrix equation, and enforce the boundary conditions using the first and last rows of the matrix:

\[\begin{split}\begin{bmatrix} 1 & 0 & 0 & 0 & \cdots & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & \cdots & 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} T_0 \\ T_1 \\ T_2 \\ \vdots \\ T_{N-1} \\ T_{N} \end{bmatrix} = \begin{bmatrix} T_0 \\ q(x_1)\Delta x^2 / \kappa \\ q(x_2)\Delta x^2 / \kappa \\ \vdots \\ q(x_{N-1})\Delta x^2 / \kappa \\ T_N) \end{bmatrix} \end{split}\]

The code below implements the second approach, and solves the matrix equation using a direct sparse linear solver.

def heat_conduction(T_left=1.0, T_right=4.0, L=2.0, kappa=0.5, Nx=10):
    """Setup and solve the heat conduction problem using the finite difference method

    Parameters
    ----------
    T_left : float, optional
        Left boundary temperature, by default 1.0
    T_right : float, optional
        Right boundary temperature, by default 4.0
    L : float, optional
        Length of domain, by default 2.0
    kappa : float, optional
        Thermal conductivity, by default 0.5
    Nx : int, optional
        Number of intervals, by default 10

    Returns
    -------
    array
        Nodal temperature values
    array
        Nodal x coordinates
    """
    # Define the parameters
    dx = L / Nx  # Grid spacing

    # Create the matrix A (tridiagonal with 2 on the diagonal and -1 on the off-diagonals)
    diagonal = 2.0 * jnp.ones(Nx - 1)
    off_diagonal = -jnp.ones(Nx - 2)
    A = jnp.diag(diagonal) + jnp.diag(off_diagonal, k=1) + jnp.diag(off_diagonal, k=-1)

    # Add the Boundary conditions
    A = jnp.vstack((A, jnp.zeros(Nx - 1)))
    A = jnp.vstack((jnp.zeros(Nx - 1), A))
    A = jnp.column_stack([jnp.zeros(A.shape[0]), A])
    A = jnp.column_stack([A, jnp.zeros(A.shape[0])])
    A = A.at[(0, 0)].set(1.0)
    A = A.at[(-1, -1)].set(1.0)  # Set the bottom-right diagonal element to 1
    A = A.at[(1, 0)].set(-1.0)
    A = A.at[(-2, -1)].set(-1.0)  # Set the bottom-right diagonal element to 1

    # Create the vector representing the heat source (modify q(x) as needed)
    x_values = jnp.linspace(0, L, Nx + 1)
    b = jnp.zeros(Nx + 1)
    b = b.at[1:Nx].set(q(x_values[1:Nx], L) / kappa * dx**2)

    # Define boundary conditions (e.g., fixed temperature at both ends)
    T_left = 1.0
    T_right = 4.0
    b = b.at[0].set(T_left)
    b = b.at[-1].set(T_right)

    T = solve(A, b)
    return T, x_values

Now let’s solve the system for \(N=3\) and plot the results compared to the analytical solution:

\[T_\text{exact} (x) = \frac{L^2}{\kappa \pi^2}\sin\left(\frac{\pi x}{L}\right) + T_0 + \frac{x}{L}\left(T_N - T_0\right)\]

# Define the true solution
def true_solution(x, L, kappa, T_left, T_right):
    return L**2 / (jnp.pi**2 * kappa) * jnp.sin(jnp.pi * x / L) + T_left + (T_right - T_left) * x / L


# Define the parameters
L = 2.0  # Length of domain
kappa = 0.5  # Thermal conductivity
Nx = 3  # Number of intervals
T0 = 1.0  # Left boundary condition
TN = 4.0  # Right boundary condition

# Solve the finite difference problem
T_soln, x_vals = heat_conduction(T0, TN, L, kappa, Nx)

# Plot the results against the true solution
fig, ax = plt.subplots()
xTrue = jnp.linspace(0, 2.0, 100)
ax.plot(xTrue, true_solution(xTrue, L, kappa, T0, TN), "-", clip_on=False, label="True solution")
ax.plot(x_vals, T_soln, "-o", clip_on=False, label="FD solution")
ax.set_xlabel("$x$")
ax.set_xticks([0, L])
ax.set_ylabel("$T$")
ax.set_yticks([T0, TN])
ax.legend(labelcolor="linecolor", loc="lower right")
niceplots.adjust_spines(ax)
plt.show()

Even with only 3 intervals, the solution is already quite close to the exact solution.

Convergence study

Since we are using a second-order accurate approximation of \(\frac{d^2T}{dx^2}\), we expect the error to be proportional to \(\Delta x^2\). Let’s verify this by solving the system for a range of values of \(N\) and plotting the error as a function of \(\Delta x\).

The error is computed using the \(L_2\) norm as given in chapter 1 of the course notes:

\[\text{Error} = \sqrt{\frac{1}{N+1}\sum_0^{N}\left(T_i - T_\text{exact}(x_i)\right)^2}\]

This is not equivalent to the typical \(L_2\) computed by numpy/JAX, which do not normalize by the number of elements, and would therefore give the wrong order of convergence.

Nsweep = 2 ** jnp.arange(1, 8)
errors = []

for Nx in Nsweep:
    T_soln, x_vals = heat_conduction(T0, TN, L, kappa, Nx)
    error = jnp.sqrt(1 / (Nx + 1) * jnp.sum((T_soln - true_solution(x_vals, L, kappa, T0, TN)) ** 2))
    errors.append(error)

fig, ax = plt.subplots()
ax.set_xlabel("$\Delta x$")
ax.set_ylabel(r"$||T_{FD} - T_{true}||_2$")
ax.set_xscale("log")
ax.set_yscale("log")

ax.plot(L / Nsweep, errors, "-o", clip_on=False)

# Compute the convergence rate by fitting a line to the log-log plot
rate = jnp.polyfit(jnp.log(L / Nsweep), jnp.log(jnp.array(errors)), 1)[0]

ax.annotate(f"Convergence rate: {float(rate):.2f}", xy=(L / Nsweep[4], errors[5]), ha="left", va="top")

niceplots.adjust_spines(ax)

<>:10: SyntaxWarning: invalid escape sequence '\D'
<>:10: SyntaxWarning: invalid escape sequence '\D'
/tmp/ipykernel_2338/2780833931.py:10: SyntaxWarning: invalid escape sequence '\D'
  ax.set_xlabel("$\Delta x$")

Learning thermal conductivity from measurement data

Now we will leverage the power of JAX to solve an inverse problem: given some measured temperatures, we will try to find the value of the thermal conductivity that best matches the data.

In this case, we will generate our “experimental data” by running the FD code with a given value of \(\kappa\), this way we can tell if our learning process has been successful.

Nx = 10
measured_temps, measurement_locations = heat_conduction(T0, TN, L, 0.5, Nx)

Next we define our “loss” or “objective” function, this is the function that we will try to minimize. In this case, we will use the \(L_2\) norm of the difference between the measured temperatures and the temperatures computed by the FD code. We will also use JAX to create a function that computes \(\frac{df}{d\kappa}\).

def obj_function(kappa):
    predicted_temps = heat_conduction(T0, TN, L, kappa, Nx)[0]
    error = jnp.linalg.norm(predicted_temps - measured_temps)
    # Print the current kappa and error, the try/except block is so that the printing is skipped when JAX AD's this function
    try:
        print(f"kappa = {float(kappa[0]): .7e}, error = {float(error): .7e}")
    except TypeError:
        pass
    return error


obj_grad = grad(obj_function)

Now we will use an algorithm called an optimizer to find the value of \(\kappa\) that minimizes the objective function. For problems where we expect the objective function to be smooth, gradient-based optimizers are the most efficient methods. Here we use an optimizer provided by JAX which will use the gradient information computed by JAX’s AD to find the minimum.

sol = minimize(obj_function, 2.349822365, jac=obj_grad, method="SLSQP", tol=1e-6, options={"disp": True})

if sol.success:
    print(f"\nSUCCESS: My best guess at kappa is {sol.x[0]:.7e}")
else:
    print("\nFAILURE: Optimization did not converge")

kappa =  2.3498224e+00, error =  1.4386167e+00

kappa =  2.1843406e+00, error =  1.4091580e+00
kappa =  1.2268172e+00, error =  1.0826684e+00
kappa = -1.3950688e+01, error =  1.8929669e+00
kappa = -5.7485267e+00, error =  1.9864205e+00
kappa = -1.6474461e+00, error =  2.3821065e+00
kappa =  4.0309418e-01, error =  4.3933260e-01
kappa =  1.1465550e+00, error =  1.0305301e+00
kappa =  7.2877184e-01, error =  5.7366863e-01
kappa =  5.5480509e-01, error =  1.8052217e-01
kappa =  5.0238946e-01, error =  8.6917907e-03
kappa =  1.8277191e-01, error =  3.1718475e+00
kappa =  4.5958774e-01, error =  1.6069220e-01
kappa =  4.9158610e-01, error =  3.1278644e-02
kappa =  4.9896527e-01, error =  3.7897058e-03
kappa =  5.0068908e-01, error =  2.5150622e-03

kappa =  4.9983015e-01, error =  6.2101403e-04
kappa =  5.0026035e-01, error =  9.5106246e-04
kappa =  5.0000795e-01, error =  2.9059218e-05
kappa =  4.9991908e-01, error =  2.9580400e-04
kappa =  4.9998355e-01, error =  6.0116422e-05
kappa =  4.9999890e-01, error =  4.0097427e-06
kappa =  5.0000343e-01, error =  1.2525187e-05
kappa =  5.0000040e-01, error =  1.4473893e-06
kappa =  4.9999965e-01, error =  1.2811645e-06
kappa =  5.0000000e-01, error =  5.3908343e-09
Optimization terminated successfully    (Exit mode 0)
            Current function value: 5.390834251021253e-09
            Iterations: 12
            Function evaluations: 26
            Gradient evaluations: 12

SUCCESS: My best guess at kappa is 5.0000000e-01

Another optimization algorithm we could use is gradient-descent. There are a few different versions of gradient descent, but here we will use the simplest one. At each iteration we compute the gradient of the objective function and then simply take a step in the downhill direction, scaled by a value \(\lambda\) which people in the machine learning community like to call the “learning rate”:

\[x_{i+1} = x_i - \lambda \frac{df}{dx}\]

def gradient_descent(f, grad_f, x0, step_size=0.01, max_iter=500, tol=1e-6):
    x = x0
    converged = False
    for ii in range(max_iter):
        func_val = f(x)
        dfdx = grad_f(x)
        if jnp.linalg.norm(dfdx) < tol:
            converged = True
            break
        x = x - step_size * dfdx
    if converged:
        print(f"Converged after {ii} iterations")
    else:
        print(f"Did not converge after {max_iter} iterations")
    return x, func_val


kappa_grad_descent, error_grad_descent = gradient_descent(obj_function, obj_grad, jnp.array([2.349822365]))

kappa =  2.3498224e+00, error =  1.4386167e+00

kappa =  2.3481675e+00, error =  1.4383427e+00
kappa =  2.3465104e+00, error =  1.4380679e+00
kappa =  2.3448509e+00, error =  1.4377923e+00
kappa =  2.3431891e+00, error =  1.4375159e+00
kappa =  2.3415249e+00, error =  1.4372388e+00
kappa =  2.3398583e+00, error =  1.4369608e+00
kappa =  2.3381894e+00, error =  1.4366821e+00

kappa =  2.3365180e+00, error =  1.4364026e+00
kappa =  2.3348443e+00, error =  1.4361222e+00

kappa =  2.3331682e+00, error =  1.4358411e+00
kappa =  2.3314897e+00, error =  1.4355592e+00
kappa =  2.3298087e+00, error =  1.4352764e+00
kappa =  2.3281254e+00, error =  1.4349928e+00
kappa =  2.3264396e+00, error =  1.4347084e+00
kappa =  2.3247513e+00, error =  1.4344232e+00
kappa =  2.3230606e+00, error =  1.4341371e+00

kappa =  2.3213674e+00, error =  1.4338502e+00
kappa =  2.3196718e+00, error =  1.4335625e+00

kappa =  2.3179737e+00, error =  1.4332739e+00
kappa =  2.3162731e+00, error =  1.4329845e+00
kappa =  2.3145700e+00, error =  1.4326943e+00
kappa =  2.3128644e+00, error =  1.4324031e+00
kappa =  2.3111562e+00, error =  1.4321112e+00
kappa =  2.3094456e+00, error =  1.4318183e+00
kappa =  2.3077324e+00, error =  1.4315246e+00

kappa =  2.3060167e+00, error =  1.4312300e+00
kappa =  2.3042984e+00, error =  1.4309345e+00

kappa =  2.3025775e+00, error =  1.4306382e+00
kappa =  2.3008541e+00, error =  1.4303409e+00
kappa =  2.2991281e+00, error =  1.4300428e+00
kappa =  2.2973995e+00, error =  1.4297438e+00
kappa =  2.2956683e+00, error =  1.4294438e+00
kappa =  2.2939345e+00, error =  1.4291430e+00
kappa =  2.2921981e+00, error =  1.4288412e+00

kappa =  2.2904590e+00, error =  1.4285386e+00
kappa =  2.2887173e+00, error =  1.4282350e+00

kappa =  2.2869729e+00, error =  1.4279305e+00
kappa =  2.2852259e+00, error =  1.4276250e+00
kappa =  2.2834762e+00, error =  1.4273187e+00
kappa =  2.2817239e+00, error =  1.4270113e+00
kappa =  2.2799688e+00, error =  1.4267031e+00
kappa =  2.2782110e+00, error =  1.4263939e+00
kappa =  2.2764505e+00, error =  1.4260837e+00

kappa =  2.2746873e+00, error =  1.4257726e+00
kappa =  2.2729214e+00, error =  1.4254605e+00

kappa =  2.2711527e+00, error =  1.4251474e+00
kappa =  2.2693813e+00, error =  1.4248334e+00
kappa =  2.2676070e+00, error =  1.4245183e+00
kappa =  2.2658301e+00, error =  1.4242023e+00
kappa =  2.2640503e+00, error =  1.4238853e+00
kappa =  2.2622677e+00, error =  1.4235673e+00
kappa =  2.2604823e+00, error =  1.4232483e+00

kappa =  2.2586941e+00, error =  1.4229283e+00
kappa =  2.2569031e+00, error =  1.4226072e+00

kappa =  2.2551092e+00, error =  1.4222852e+00
kappa =  2.2533124e+00, error =  1.4219621e+00
kappa =  2.2515128e+00, error =  1.4216380e+00
kappa =  2.2497104e+00, error =  1.4213128e+00
kappa =  2.2479050e+00, error =  1.4209866e+00
kappa =  2.2460967e+00, error =  1.4206594e+00
kappa =  2.2442855e+00, error =  1.4203311e+00

kappa =  2.2424714e+00, error =  1.4200017e+00
kappa =  2.2406544e+00, error =  1.4196713e+00

kappa =  2.2388344e+00, error =  1.4193397e+00
kappa =  2.2370114e+00, error =  1.4190072e+00
kappa =  2.2351855e+00, error =  1.4186735e+00
kappa =  2.2333566e+00, error =  1.4183387e+00
kappa =  2.2315247e+00, error =  1.4180029e+00
kappa =  2.2296897e+00, error =  1.4176659e+00
kappa =  2.2278518e+00, error =  1.4173278e+00

kappa =  2.2260108e+00, error =  1.4169886e+00
kappa =  2.2241668e+00, error =  1.4166483e+00

kappa =  2.2223197e+00, error =  1.4163068e+00
kappa =  2.2204696e+00, error =  1.4159642e+00
kappa =  2.2186163e+00, error =  1.4156205e+00
kappa =  2.2167600e+00, error =  1.4152756e+00
kappa =  2.2149006e+00, error =  1.4149296e+00
kappa =  2.2130380e+00, error =  1.4145824e+00
kappa =  2.2111723e+00, error =  1.4142340e+00

kappa =  2.2093035e+00, error =  1.4138844e+00
kappa =  2.2074314e+00, error =  1.4135337e+00

kappa =  2.2055562e+00, error =  1.4131818e+00
kappa =  2.2036779e+00, error =  1.4128286e+00
kappa =  2.2017963e+00, error =  1.4124743e+00
kappa =  2.1999115e+00, error =  1.4121187e+00
kappa =  2.1980234e+00, error =  1.4117620e+00
kappa =  2.1961322e+00, error =  1.4114040e+00
kappa =  2.1942376e+00, error =  1.4110447e+00

kappa =  2.1923398e+00, error =  1.4106842e+00
kappa =  2.1904387e+00, error =  1.4103225e+00

kappa =  2.1885343e+00, error =  1.4099595e+00
kappa =  2.1866266e+00, error =  1.4095953e+00
kappa =  2.1847156e+00, error =  1.4092297e+00
kappa =  2.1828012e+00, error =  1.4088629e+00
kappa =  2.1808834e+00, error =  1.4084948e+00
kappa =  2.1789623e+00, error =  1.4081254e+00
kappa =  2.1770378e+00, error =  1.4077547e+00

kappa =  2.1751099e+00, error =  1.4073827e+00
kappa =  2.1731785e+00, error =  1.4070094e+00

kappa =  2.1712438e+00, error =  1.4066347e+00
kappa =  2.1693055e+00, error =  1.4062587e+00
kappa =  2.1673639e+00, error =  1.4058814e+00
kappa =  2.1654187e+00, error =  1.4055026e+00
kappa =  2.1634700e+00, error =  1.4051226e+00
kappa =  2.1615179e+00, error =  1.4047411e+00
kappa =  2.1595622e+00, error =  1.4043583e+00

kappa =  2.1576029e+00, error =  1.4039741e+00
kappa =  2.1556401e+00, error =  1.4035885e+00

kappa =  2.1536737e+00, error =  1.4032015e+00
kappa =  2.1517038e+00, error =  1.4028130e+00
kappa =  2.1497302e+00, error =  1.4024232e+00
kappa =  2.1477530e+00, error =  1.4020319e+00
kappa =  2.1457721e+00, error =  1.4016391e+00
kappa =  2.1437876e+00, error =  1.4012450e+00
kappa =  2.1417994e+00, error =  1.4008493e+00

kappa =  2.1398075e+00, error =  1.4004522e+00
kappa =  2.1378120e+00, error =  1.4000536e+00

kappa =  2.1358126e+00, error =  1.3996535e+00
kappa =  2.1338096e+00, error =  1.3992519e+00
kappa =  2.1318028e+00, error =  1.3988487e+00
kappa =  2.1297922e+00, error =  1.3984441e+00
kappa =  2.1277778e+00, error =  1.3980380e+00
kappa =  2.1257595e+00, error =  1.3976302e+00
kappa =  2.1237375e+00, error =  1.3972210e+00

kappa =  2.1217116e+00, error =  1.3968102e+00
kappa =  2.1196818e+00, error =  1.3963978e+00

kappa =  2.1176482e+00, error =  1.3959838e+00
kappa =  2.1156106e+00, error =  1.3955682e+00
kappa =  2.1135691e+00, error =  1.3951511e+00
kappa =  2.1115237e+00, error =  1.3947323e+00
kappa =  2.1094742e+00, error =  1.3943119e+00
kappa =  2.1074209e+00, error =  1.3938898e+00
kappa =  2.1053635e+00, error =  1.3934661e+00

kappa =  2.1033020e+00, error =  1.3930407e+00
kappa =  2.1012366e+00, error =  1.3926137e+00

kappa =  2.0991671e+00, error =  1.3921850e+00
kappa =  2.0970935e+00, error =  1.3917546e+00
kappa =  2.0950157e+00, error =  1.3913225e+00
kappa =  2.0929339e+00, error =  1.3908886e+00
kappa =  2.0908479e+00, error =  1.3904531e+00
kappa =  2.0887578e+00, error =  1.3900158e+00
kappa =  2.0866635e+00, error =  1.3895767e+00

kappa =  2.0845649e+00, error =  1.3891359e+00
kappa =  2.0824622e+00, error =  1.3886933e+00

kappa =  2.0803552e+00, error =  1.3882489e+00
kappa =  2.0782439e+00, error =  1.3878027e+00
kappa =  2.0761283e+00, error =  1.3873547e+00
kappa =  2.0740085e+00, error =  1.3869048e+00
kappa =  2.0718842e+00, error =  1.3864531e+00
kappa =  2.0697557e+00, error =  1.3859996e+00
kappa =  2.0676227e+00, error =  1.3855441e+00

kappa =  2.0654853e+00, error =  1.3850868e+00
kappa =  2.0633436e+00, error =  1.3846276e+00

kappa =  2.0611973e+00, error =  1.3841665e+00
kappa =  2.0590466e+00, error =  1.3837035e+00
kappa =  2.0568914e+00, error =  1.3832385e+00
kappa =  2.0547317e+00, error =  1.3827716e+00
kappa =  2.0525674e+00, error =  1.3823027e+00
kappa =  2.0503986e+00, error =  1.3818318e+00
kappa =  2.0482252e+00, error =  1.3813589e+00

kappa =  2.0460472e+00, error =  1.3808840e+00
kappa =  2.0438645e+00, error =  1.3804071e+00

kappa =  2.0416771e+00, error =  1.3799282e+00
kappa =  2.0394851e+00, error =  1.3794472e+00
kappa =  2.0372884e+00, error =  1.3789641e+00
kappa =  2.0350869e+00, error =  1.3784789e+00
kappa =  2.0328806e+00, error =  1.3779916e+00
kappa =  2.0306696e+00, error =  1.3775022e+00
kappa =  2.0284537e+00, error =  1.3770107e+00

kappa =  2.0262330e+00, error =  1.3765170e+00
kappa =  2.0240075e+00, error =  1.3760211e+00

kappa =  2.0217770e+00, error =  1.3755231e+00
kappa =  2.0195416e+00, error =  1.3750228e+00
kappa =  2.0173013e+00, error =  1.3745204e+00
kappa =  2.0150560e+00, error =  1.3740156e+00
kappa =  2.0128056e+00, error =  1.3735087e+00
kappa =  2.0105503e+00, error =  1.3729994e+00
kappa =  2.0082898e+00, error =  1.3724879e+00

kappa =  2.0060243e+00, error =  1.3719741e+00
kappa =  2.0037537e+00, error =  1.3714579e+00

kappa =  2.0014779e+00, error =  1.3709394e+00
kappa =  1.9991969e+00, error =  1.3704185e+00
kappa =  1.9969108e+00, error =  1.3698953e+00
kappa =  1.9946194e+00, error =  1.3693696e+00
kappa =  1.9923227e+00, error =  1.3688415e+00
kappa =  1.9900207e+00, error =  1.3683110e+00
kappa =  1.9877134e+00, error =  1.3677780e+00

kappa =  1.9854007e+00, error =  1.3672426e+00
kappa =  1.9830827e+00, error =  1.3667046e+00

kappa =  1.9807592e+00, error =  1.3661641e+00
kappa =  1.9784303e+00, error =  1.3656211e+00
kappa =  1.9760958e+00, error =  1.3650755e+00
kappa =  1.9737559e+00, error =  1.3645273e+00
kappa =  1.9714104e+00, error =  1.3639765e+00
kappa =  1.9690593e+00, error =  1.3634231e+00
kappa =  1.9667027e+00, error =  1.3628670e+00

kappa =  1.9643403e+00, error =  1.3623083e+00
kappa =  1.9619723e+00, error =  1.3617469e+00

kappa =  1.9595985e+00, error =  1.3611827e+00
kappa =  1.9572190e+00, error =  1.3606158e+00
kappa =  1.9548338e+00, error =  1.3600462e+00
kappa =  1.9524426e+00, error =  1.3594737e+00
kappa =  1.9500457e+00, error =  1.3588985e+00
kappa =  1.9476428e+00, error =  1.3583204e+00

kappa =  1.9452340e+00, error =  1.3577394e+00
kappa =  1.9428192e+00, error =  1.3571556e+00

kappa =  1.9403984e+00, error =  1.3565688e+00
kappa =  1.9379716e+00, error =  1.3559792e+00
kappa =  1.9355387e+00, error =  1.3553865e+00
kappa =  1.9330997e+00, error =  1.3547909e+00
kappa =  1.9306545e+00, error =  1.3541922e+00
kappa =  1.9282031e+00, error =  1.3535905e+00
kappa =  1.9257455e+00, error =  1.3529858e+00

kappa =  1.9232816e+00, error =  1.3523779e+00
kappa =  1.9208114e+00, error =  1.3517669e+00

kappa =  1.9183348e+00, error =  1.3511528e+00
kappa =  1.9158518e+00, error =  1.3505355e+00
kappa =  1.9133624e+00, error =  1.3499150e+00
kappa =  1.9108665e+00, error =  1.3492912e+00
kappa =  1.9083641e+00, error =  1.3486642e+00
kappa =  1.9058551e+00, error =  1.3480339e+00
kappa =  1.9033396e+00, error =  1.3474002e+00

kappa =  1.9008173e+00, error =  1.3467632e+00
kappa =  1.8982884e+00, error =  1.3461228e+00

kappa =  1.8957527e+00, error =  1.3454789e+00
kappa =  1.8932102e+00, error =  1.3448317e+00
kappa =  1.8906609e+00, error =  1.3441809e+00
kappa =  1.8881047e+00, error =  1.3435266e+00
kappa =  1.8855416e+00, error =  1.3428687e+00
kappa =  1.8829715e+00, error =  1.3422073e+00
kappa =  1.8803944e+00, error =  1.3415422e+00

kappa =  1.8778102e+00, error =  1.3408735e+00
kappa =  1.8752189e+00, error =  1.3402011e+00

kappa =  1.8726205e+00, error =  1.3395250e+00
kappa =  1.8700148e+00, error =  1.3388451e+00
kappa =  1.8674018e+00, error =  1.3381614e+00
kappa =  1.8647816e+00, error =  1.3374738e+00
kappa =  1.8621539e+00, error =  1.3367824e+00
kappa =  1.8595189e+00, error =  1.3360871e+00
kappa =  1.8568764e+00, error =  1.3353878e+00

kappa =  1.8542263e+00, error =  1.3346845e+00
kappa =  1.8515687e+00, error =  1.3339772e+00

kappa =  1.8489034e+00, error =  1.3332658e+00
kappa =  1.8462305e+00, error =  1.3325503e+00
kappa =  1.8435498e+00, error =  1.3318307e+00
kappa =  1.8408613e+00, error =  1.3311068e+00
kappa =  1.8381649e+00, error =  1.3303787e+00
kappa =  1.8354606e+00, error =  1.3296463e+00
kappa =  1.8327484e+00, error =  1.3289096e+00

kappa =  1.8300281e+00, error =  1.3281685e+00
kappa =  1.8272997e+00, error =  1.3274230e+00

kappa =  1.8245632e+00, error =  1.3266730e+00
kappa =  1.8218184e+00, error =  1.3259185e+00
kappa =  1.8190654e+00, error =  1.3251594e+00
kappa =  1.8163040e+00, error =  1.3243958e+00
kappa =  1.8135343e+00, error =  1.3236274e+00
kappa =  1.8107561e+00, error =  1.3228544e+00
kappa =  1.8079693e+00, error =  1.3220766e+00

kappa =  1.8051739e+00, error =  1.3212940e+00
kappa =  1.8023699e+00, error =  1.3205065e+00

kappa =  1.7995571e+00, error =  1.3197141e+00
kappa =  1.7967356e+00, error =  1.3189167e+00
kappa =  1.7939052e+00, error =  1.3181143e+00
kappa =  1.7910658e+00, error =  1.3173069e+00
kappa =  1.7882174e+00, error =  1.3164942e+00
kappa =  1.7853600e+00, error =  1.3156764e+00
kappa =  1.7824934e+00, error =  1.3148534e+00

kappa =  1.7796175e+00, error =  1.3140250e+00
kappa =  1.7767324e+00, error =  1.3131912e+00

kappa =  1.7738379e+00, error =  1.3123520e+00
kappa =  1.7709339e+00, error =  1.3115073e+00
kappa =  1.7680204e+00, error =  1.3106571e+00
kappa =  1.7650973e+00, error =  1.3098012e+00
kappa =  1.7621645e+00, error =  1.3089397e+00
kappa =  1.7592219e+00, error =  1.3080723e+00
kappa =  1.7562695e+00, error =  1.3071992e+00

kappa =  1.7533071e+00, error =  1.3063201e+00
kappa =  1.7503347e+00, error =  1.3054351e+00

kappa =  1.7473523e+00, error =  1.3045441e+00
kappa =  1.7443596e+00, error =  1.3036470e+00
kappa =  1.7413566e+00, error =  1.3027436e+00
kappa =  1.7383433e+00, error =  1.3018341e+00
kappa =  1.7353195e+00, error =  1.3009181e+00
kappa =  1.7322852e+00, error =  1.2999958e+00
kappa =  1.7292403e+00, error =  1.2990670e+00

kappa =  1.7261846e+00, error =  1.2981316e+00
kappa =  1.7231181e+00, error =  1.2971896e+00

kappa =  1.7200406e+00, error =  1.2962408e+00
kappa =  1.7169521e+00, error =  1.2952853e+00
kappa =  1.7138526e+00, error =  1.2943228e+00
kappa =  1.7107418e+00, error =  1.2933533e+00
kappa =  1.7076196e+00, error =  1.2923768e+00
kappa =  1.7044861e+00, error =  1.2913930e+00
kappa =  1.7013410e+00, error =  1.2904020e+00

kappa =  1.6981842e+00, error =  1.2894037e+00
kappa =  1.6950158e+00, error =  1.2883979e+00

kappa =  1.6918354e+00, error =  1.2873845e+00
kappa =  1.6886431e+00, error =  1.2863635e+00
kappa =  1.6854387e+00, error =  1.2853348e+00
kappa =  1.6822222e+00, error =  1.2842982e+00
kappa =  1.6789933e+00, error =  1.2832536e+00
kappa =  1.6757520e+00, error =  1.2822009e+00
kappa =  1.6724981e+00, error =  1.2811401e+00

kappa =  1.6692315e+00, error =  1.2800710e+00
kappa =  1.6659522e+00, error =  1.2789935e+00

kappa =  1.6626599e+00, error =  1.2779074e+00
kappa =  1.6593546e+00, error =  1.2768127e+00
kappa =  1.6560361e+00, error =  1.2757093e+00
kappa =  1.6527043e+00, error =  1.2745969e+00
kappa =  1.6493590e+00, error =  1.2734756e+00
kappa =  1.6460002e+00, error =  1.2723451e+00
kappa =  1.6426276e+00, error =  1.2712054e+00

kappa =  1.6392412e+00, error =  1.2700562e+00
kappa =  1.6358408e+00, error =  1.2688975e+00

kappa =  1.6324262e+00, error =  1.2677291e+00
kappa =  1.6289973e+00, error =  1.2665509e+00
kappa =  1.6255540e+00, error =  1.2653627e+00
kappa =  1.6220960e+00, error =  1.2641645e+00
kappa =  1.6186233e+00, error =  1.2629559e+00
kappa =  1.6151357e+00, error =  1.2617369e+00
kappa =  1.6116330e+00, error =  1.2605074e+00

kappa =  1.6081151e+00, error =  1.2592671e+00
kappa =  1.6045817e+00, error =  1.2580159e+00

kappa =  1.6010328e+00, error =  1.2567536e+00
kappa =  1.5974681e+00, error =  1.2554801e+00
kappa =  1.5938875e+00, error =  1.2541951e+00
kappa =  1.5902908e+00, error =  1.2528986e+00
kappa =  1.5866778e+00, error =  1.2515902e+00
kappa =  1.5830484e+00, error =  1.2502699e+00
kappa =  1.5794023e+00, error =  1.2489374e+00

kappa =  1.5757393e+00, error =  1.2475926e+00
kappa =  1.5720592e+00, error =  1.2462351e+00

kappa =  1.5683620e+00, error =  1.2448649e+00
kappa =  1.5646472e+00, error =  1.2434817e+00
kappa =  1.5609148e+00, error =  1.2420853e+00
kappa =  1.5571646e+00, error =  1.2406755e+00
kappa =  1.5533962e+00, error =  1.2392520e+00
kappa =  1.5496096e+00, error =  1.2378146e+00
kappa =  1.5458044e+00, error =  1.2363631e+00

kappa =  1.5419804e+00, error =  1.2348972e+00
kappa =  1.5381375e+00, error =  1.2334167e+00

kappa =  1.5342754e+00, error =  1.2319214e+00
kappa =  1.5303937e+00, error =  1.2304108e+00
kappa =  1.5264924e+00, error =  1.2288849e+00
kappa =  1.5225711e+00, error =  1.2273433e+00
kappa =  1.5186296e+00, error =  1.2257857e+00
kappa =  1.5146675e+00, error =  1.2242118e+00
kappa =  1.5106848e+00, error =  1.2226214e+00

kappa =  1.5066810e+00, error =  1.2210141e+00
kappa =  1.5026559e+00, error =  1.2193896e+00

kappa =  1.4986092e+00, error =  1.2177476e+00
kappa =  1.4945406e+00, error =  1.2160877e+00
kappa =  1.4904498e+00, error =  1.2144097e+00
kappa =  1.4863366e+00, error =  1.2127131e+00
kappa =  1.4822005e+00, error =  1.2109977e+00
kappa =  1.4780413e+00, error =  1.2092629e+00
kappa =  1.4738587e+00, error =  1.2075086e+00

kappa =  1.4696524e+00, error =  1.2057341e+00
kappa =  1.4654219e+00, error =  1.2039393e+00

kappa =  1.4611669e+00, error =  1.2021235e+00
kappa =  1.4568871e+00, error =  1.2002865e+00
kappa =  1.4525822e+00, error =  1.1984278e+00
kappa =  1.4482517e+00, error =  1.1965468e+00
kappa =  1.4438952e+00, error =  1.1946432e+00
kappa =  1.4395125e+00, error =  1.1927165e+00
kappa =  1.4351030e+00, error =  1.1907662e+00

kappa =  1.4306663e+00, error =  1.1887917e+00
kappa =  1.4262021e+00, error =  1.1867925e+00

kappa =  1.4217099e+00, error =  1.1847682e+00
kappa =  1.4171893e+00, error =  1.1827181e+00
kappa =  1.4126398e+00, error =  1.1806416e+00
kappa =  1.4080610e+00, error =  1.1785382e+00
kappa =  1.4034523e+00, error =  1.1764072e+00
kappa =  1.3988133e+00, error =  1.1742480e+00
kappa =  1.3941434e+00, error =  1.1720600e+00

kappa =  1.3894423e+00, error =  1.1698424e+00
kappa =  1.3847093e+00, error =  1.1675946e+00

kappa =  1.3799438e+00, error =  1.1653159e+00
kappa =  1.3751454e+00, error =  1.1630053e+00
kappa =  1.3703135e+00, error =  1.1606623e+00
kappa =  1.3654474e+00, error =  1.1582860e+00
kappa =  1.3605465e+00, error =  1.1558755e+00
kappa =  1.3556103e+00, error =  1.1534301e+00
kappa =  1.3506381e+00, error =  1.1509487e+00

kappa =  1.3456292e+00, error =  1.1484304e+00
kappa =  1.3405829e+00, error =  1.1458744e+00

kappa =  1.3354986e+00, error =  1.1432795e+00
kappa =  1.3303755e+00, error =  1.1406448e+00
kappa =  1.3252129e+00, error =  1.1379691e+00
kappa =  1.3200099e+00, error =  1.1352514e+00
kappa =  1.3147659e+00, error =  1.1324904e+00
kappa =  1.3094800e+00, error =  1.1296850e+00
kappa =  1.3041512e+00, error =  1.1268339e+00

kappa =  1.2987789e+00, error =  1.1239357e+00
kappa =  1.2933620e+00, error =  1.1209892e+00

kappa =  1.2878996e+00, error =  1.1179928e+00
kappa =  1.2823909e+00, error =  1.1149451e+00
kappa =  1.2768346e+00, error =  1.1118445e+00
kappa =  1.2712300e+00, error =  1.1086894e+00
kappa =  1.2655758e+00, error =  1.1054781e+00
kappa =  1.2598709e+00, error =  1.1022088e+00
kappa =  1.2541143e+00, error =  1.0988797e+00

kappa =  1.2483047e+00, error =  1.0954889e+00
kappa =  1.2424409e+00, error =  1.0920343e+00

kappa =  1.2365216e+00, error =  1.0885137e+00
kappa =  1.2305455e+00, error =  1.0849250e+00
kappa =  1.2245113e+00, error =  1.0812658e+00
kappa =  1.2184174e+00, error =  1.0775337e+00
kappa =  1.2122624e+00, error =  1.0737261e+00
kappa =  1.2060447e+00, error =  1.0698402e+00
kappa =  1.1997628e+00, error =  1.0658733e+00

kappa =  1.1934149e+00, error =  1.0618223e+00
kappa =  1.1869993e+00, error =  1.0576840e+00

kappa =  1.1805142e+00, error =  1.0534552e+00
kappa =  1.1739576e+00, error =  1.0491323e+00
kappa =  1.1673275e+00, error =  1.0447116e+00
kappa =  1.1606220e+00, error =  1.0401892e+00
kappa =  1.1538387e+00, error =  1.0355609e+00
kappa =  1.1469755e+00, error =  1.0308223e+00
kappa =  1.1400298e+00, error =  1.0259687e+00

kappa =  1.1329993e+00, error =  1.0209952e+00
kappa =  1.1258813e+00, error =  1.0158965e+00

kappa =  1.1186729e+00, error =  1.0106670e+00
kappa =  1.1113714e+00, error =  1.0053008e+00
kappa =  1.1039736e+00, error =  9.9979139e-01
kappa =  1.0964764e+00, error =  9.9413206e-01
kappa =  1.0888762e+00, error =  9.8831553e-01
kappa =  1.0811696e+00, error =  9.8233402e-01
kappa =  1.0733528e+00, error =  9.7617919e-01

kappa =  1.0654216e+00, error =  9.6984207e-01
kappa =  1.0573720e+00, error =  9.6331306e-01

kappa =  1.0491993e+00, error =  9.5658176e-01
kappa =  1.0408988e+00, error =  9.4963699e-01
kappa =  1.0324654e+00, error =  9.4246666e-01
kappa =  1.0238936e+00, error =  9.3505767e-01
kappa =  1.0151778e+00, error =  9.2739582e-01
kappa =  1.0063116e+00, error =  9.1946567e-01
kappa =  9.9728852e-01, error =  9.1125040e-01

kappa =  9.8810142e-01, error =  9.0273164e-01
kappa =  9.7874269e-01, error =  8.9388930e-01

kappa =  9.6920412e-01, error =  8.8470133e-01
kappa =  9.5947688e-01, error =  8.7514349e-01
kappa =  9.4955141e-01, error =  8.6518902e-01
kappa =  9.3941736e-01, error =  8.5480833e-01
kappa =  9.2906349e-01, error =  8.4396858e-01
kappa =  9.1847755e-01, error =  8.3263322e-01
kappa =  9.0764619e-01, error =  8.2076138e-01

kappa =  8.9655477e-01, error =  8.0830724e-01
kappa =  8.8518723e-01, error =  7.9521920e-01

kappa =  8.7352586e-01, error =  7.8143889e-01
kappa =  8.6155105e-01, error =  7.6689998e-01
kappa =  8.4924105e-01, error =  7.5152671e-01
kappa =  8.3657158e-01, error =  7.3523209e-01
kappa =  8.2351547e-01, error =  7.1791563e-01
kappa =  8.1004209e-01, error =  6.9946049e-01
kappa =  7.9611678e-01, error =  6.7972987e-01

kappa =  7.8170005e-01, error =  6.5856236e-01
kappa =  7.6674666e-01, error =  6.3576587e-01

kappa =  7.5120432e-01, error =  6.1110965e-01
kappa =  7.3501219e-01, error =  5.8431356e-01
kappa =  7.1809879e-01, error =  5.5503347e-01
kappa =  7.0037928e-01, error =  5.2284099e-01
kappa =  6.8175182e-01, error =  4.8719473e-01
kappa =  6.6209254e-01, error =  4.4739843e-01
kappa =  6.4124847e-01, error =  4.0253858e-01

kappa =  6.1902727e-01, error =  3.5138790e-01
kappa =  5.9518209e-01, error =  2.9225066e-01

kappa =  5.6938799e-01, error =  2.2270303e-01
kappa =  5.4120393e-01, error =  1.3913226e-01
kappa =  5.1000798e-01, error =  3.5860769e-02
kappa =  4.7487895e-01, error =  9.6672956e-02
kappa =  5.1539754e-01, error =  5.4595769e-02
kappa =  4.8099936e-01, error =  7.2189481e-02
kappa =  5.2049336e-01, error =  7.1952856e-02

kappa =  4.8676542e-01, error =  4.9686732e-02
kappa =  5.2532930e-01, error =  8.8113342e-02

kappa =  4.9221948e-01, error =  2.8886850e-02
kappa =  5.2993347e-01, error =  1.0322522e-01
kappa =  4.9739648e-01, error =  9.5655268e-03
kappa =  5.3432949e-01, error =  1.1741087e-01
kappa =  5.0232567e-01, error =  8.4608126e-03
kappa =  4.6611393e-01, error =  1.3285542e-01
kappa =  5.0817070e-01, error =  2.9383250e-02

kappa =  4.7278719e-01, error =  1.0518595e-01
kappa =  5.1366511e-01, error =  4.8616432e-02

kappa =  4.7903451e-01, error =  7.9981281e-02
kappa =  5.1885316e-01, error =  6.6403314e-02
kappa =  4.8491165e-01, error =  5.6862950e-02
kappa =  5.2377094e-01, error =  8.2938282e-02
kappa =  4.9046380e-01, error =  3.5531909e-02
kappa =  5.2844828e-01, error =  9.8379283e-02
kappa =  4.9572814e-01, error =  1.5747926e-02

kappa =  5.3291016e-01, error =  1.1285638e-01
kappa =  5.0073564e-01, error =  2.6847622e-03

kappa =  4.6429356e-01, error =  1.4054131e-01
kappa =  5.0668077e-01, error =  2.4095842e-02
kappa =  4.7108886e-01, error =  1.1215342e-01
kappa =  5.1226204e-01, error =  4.3744231e-02
kappa =  4.7744148e-01, error =  8.6345662e-02
Did not converge after 500 iterations

As you can see, gradient descent is not a very good algorithm, even with 500 function evaluations, it fails to get as close to the true answer as SLSQP did in only 26 evaluations. This performance can be improved by tuning the learning rate, but this process is very tedious and requires a lot of trial and error.