Solve automatic numerical differentiation problems in one or more variables.

Last update: Dec 16, 2022

Related tags

Overview

numdifftools

The numdifftools library is a suite of tools written in _Python to solve automatic numerical differentiation problems in one or more variables. Finite differences are used in an adaptive manner, coupled with a Richardson extrapolation methodology to provide a maximally accurate result. The user can configure many options like; changing the order of the method or the extrapolation, even allowing the user to specify whether complex-step, central, forward or backward differences are used.

The methods provided are:

Derivative: Compute the derivatives of order 1 through 10 on any scalar function.
directionaldiff: Compute directional derivative of a function of n variables
Gradient: Compute the gradient vector of a scalar function of one or more variables.
Jacobian: Compute the Jacobian matrix of a vector valued function of one or more variables.
Hessian: Compute the Hessian matrix of all 2nd partial derivatives of a scalar function of one or more variables.
Hessdiag: Compute only the diagonal elements of the Hessian matrix

All of these methods also produce error estimates on the result.

Numdifftools also provide an easy to use interface to derivatives calculated with in _AlgoPy. Algopy stands for Algorithmic Differentiation in Python. The purpose of AlgoPy is the evaluation of higher-order derivatives in the forward and reverse mode of Algorithmic Differentiation (AD) of functions that are implemented as Python programs.

Getting Started

Visualize high order derivatives of the tanh function

>>> import numpy as np
>>> import numdifftools as nd
>>> import matplotlib.pyplot as plt
>>> x = np.linspace(-2, 2, 100)
>>> for i in range(10):
...    df = nd.Derivative(np.tanh, n=i)
...    y = df(x)
...    h = plt.plot(x, y/np.abs(y).max())

plt.show()

https://raw.githubusercontent.com/pbrod/numdifftools/master/examples/fun.png

Compute 1'st and 2'nd derivative of exp(x), at x == 1:

>>> fd = nd.Derivative(np.exp)        # 1'st derivative
>>> fdd = nd.Derivative(np.exp, n=2)  # 2'nd derivative
>>> np.allclose(fd(1), 2.7182818284590424)
True
>>> np.allclose(fdd(1), 2.7182818284590424)
True

Nonlinear least squares:

>>> xdata = np.reshape(np.arange(0,1,0.1),(-1,1))
>>> ydata = 1+2*np.exp(0.75*xdata)
>>> fun = lambda c: (c[0]+c[1]*np.exp(c[2]*xdata) - ydata)**2
>>> Jfun = nd.Jacobian(fun)
>>> np.allclose(np.abs(Jfun([1,2,0.75])), 0) # should be numerically zero
True

Compute gradient of sum(x**2):

>>> fun = lambda x: np.sum(x**2)
>>> dfun = nd.Gradient(fun)
>>> np.allclose(dfun([1,2,3]), [ 2.,  4.,  6.])
True

Compute the same with the easy to use interface to AlgoPy:

>>> import numdifftools.nd_algopy as nda
>>> import numpy as np
>>> fd = nda.Derivative(np.exp)        # 1'st derivative
>>> fdd = nda.Derivative(np.exp, n=2)  # 2'nd derivative
>>> np.allclose(fd(1), 2.7182818284590424)
True
>>> np.allclose(fdd(1), 2.7182818284590424)
True

Nonlinear least squares:

>>> xdata = np.reshape(np.arange(0,1,0.1),(-1,1))
>>> ydata = 1+2*np.exp(0.75*xdata)
>>> fun = lambda c: (c[0]+c[1]*np.exp(c[2]*xdata) - ydata)**2
>>> Jfun = nda.Jacobian(fun, method='reverse')
>>> np.allclose(np.abs(Jfun([1,2,0.75])), 0) # should be numerically zero
True

Compute gradient of sum(x**2):

>>> fun = lambda x: np.sum(x**2)
>>> dfun = nda.Gradient(fun)
>>> np.allclose(dfun([1,2,3]), [ 2.,  4.,  6.])
True

Documentation and code

Numdifftools works on Python 2.7+ and Python 3.0+.

Official releases available at: http://pypi.python.org/pypi/numdifftools

Official documentation available at: http://numdifftools.readthedocs.io/en/latest/

Bleeding edge: https://github.com/pbrod/numdifftools.

Installation

If you have pip installed, then simply type:

$ pip install numdifftools

to get the lastest stable version. Using pip also has the advantage that all requirements are automatically installed.

Unit tests

To test if the toolbox is working paste the following in an interactive python session:

import numdifftools as nd
nd.test('--doctest-modules', '--disable-warnings')

Acknowledgement

The numdifftools package for Python was written by Per A. Brodtkorb based on the adaptive numerical differentiation toolbox written in Matlab by John D'Errico [DErrico06].

Later the package was extended with some of the functionality found in the statsmodels.tools.numdiff module written by Josef Perktold [JPerktold14] which is based on [Rid09]. The implementation of bicomplex numbers is based on the matlab implementation described in the project report of [Ver14] which is based on [GLD12]. For completeness the [For98] method for computing the weights and points in general finite difference formulas as well as the [For81] method for cumputing the taylor coefficients of complex analytic function using FFT, was added.

References

[JPerktold14]

Perktold, J (2014), numdiff package http://statsmodels.sourceforge.net/0.6.0/_modules/statsmodels/tools/numdiff.html

[Ver14]

Adriaen Verheyleweghen, (2014) "Computation of higher-order derivatives using the multi-complex step method", Project report, NTNU

[GLD12]

Gregory Lantoine, R.P. Russell, and T. Dargent (2012) "Using multicomplex variables for automatic computation of high-order derivatives", ACM Transactions on Mathematical Software, Vol. 38, No. 3, Article 16, April 2012, 21 pages, http://doi.acm.org/10.1145/2168773.2168774

[MELEV12]

M.E. Luna-Elizarraras, M. Shapiro, D.C. Struppa1, A. Vajiac (2012), "Bicomplex Numbers and Their Elementary Functions", CUBO A Mathematical Journal, Vol. 14, No 2, (61-80). June 2012.

[Lan10]

Gregory Lantoine (2010), "A methodology for robust optimization of low-thrust trajectories in multi-body environments", Phd thesis, Georgia Institute of Technology

[Rid09]

Ridout, M.S. (2009) "Statistical applications of the complex-step method of numerical differentiation", The American Statistician, 63, 66-74

[DErrico06]

D'Errico, J. R. (2006), "Adaptive Robust Numerical Differentiation", http://www.mathworks.com/matlabcentral/fileexchange/13490-adaptive-robust-numerical-differentiation

[KLLK05]

K.-L. Lai, J.L. Crassidis, Y. Cheng, J. Kim (2005), "New complex step derivative approximations with application to second-order kalman filtering", AIAA Guidance, Navigation and Control Conference, San Francisco, California, August 2005, AIAA-2005-5944.

[For98]

B. Fornberg (1998) "Calculation of weights_and_points in finite difference formulas", SIAM Review 40, pp. 685-691.

[For81]

Fornberg, B. (1981). "Numerical Differentiation of Analytic Functions", ACM Transactions on Mathematical Software (TOMS), 7(4), 512-526. http://doi.org/10.1145/355972.355979

[JML69]

Lyness, J. M., Moler, C. B. (1969). "Generalized Romberg Methods for Integrals of Derivatives", Numerische Mathematik.

[JML66]

Lyness, J. M., Moler, C. B. (1966). "Vandermonde Systems and Numerical Differentiation", Numerische Mathematik.

[NAG]

NAG Library. NAG Fortran Library Document: D04AAF

Comments

Error on evaluating gradient

Hi! I'm facing this problem, I have a function like this:

def function( v, r, z , observation): return numpy.linalg.norm( observation - model(v,r,z) )

and model is a function which takes the (vector parameters) v, r, z and returns a vector of the same size as observation, thus far, everything is ok, but when I try to compute the gradient of function, I'm facing the next error

Traceback (most recent call last): File "Main.py", line 62, in z = my_grad( v, r, z, observation ) File "/Users/Angel/Documents/Projects/Ursula/26Feb/Optimization/Solvers.py", line 33, in grad return dfun( v ) File "/Users/Angel/anaconda/lib/python2.7/site-packages/numdifftools/nd_algopy.py", line 147, in call df = fun(x0, *args, **kwds) File "/Users/Angel/anaconda/lib/python2.7/site-packages/numdifftools/nd_algopy.py", line 346, in _forward return np.atleast_2d(super(Jacobian, self)._forward(x, *args, **kwds)) File "/Users/Angel/anaconda/lib/python2.7/site-packages/numdifftools/nd_algopy.py", line 266, in _forward y = self.fun(tmp, *args, **kwds) File "/Users/Angel/Documents/Projects/Solvers.py", line 29, in fun = lambda x: function( v, r, z , observation) ValueError: setting an array element with a sequence.

My function to compute the gradient is as follows:

def my_grad( v, r, z , observation): fun = lambda x: function( v, r, z , observation) dfun = nda.Gradient(fun) return dfun( v )

I believe that I'm not making any mistake, but if you have any suggestion, please let me know

opened by nkeito 8
ReadTheDocs permission denied

Accessing http://numdifftools.readthedocs.io/ directs to http://numdifftools.readthedocs.io/en/v0.9.14/ and I see the message:

Permission Denied

You don't have the proper permissions to view this page. Please contact the owner of this project to request permission.

opened by andrewfowlie 8
recipe for conda-forge

I'm using numdifftools as a dependency for a package of mine. Because the package uses an extension I make it available on conda-forge (which takes care of all the different OS/numpy/python combinations).

However, numdifftools is not currently available on conda-forge. Would you mind if numdifftools was added as a recipe on conda-forge?

I don't mind doing the groundwork for this.

opened by andyfaff 7
Python 3.6 compatibility

I've opened this PR in case any of the changes I've made here are useful.

The main change I've made is to make algopy an optional dependency, and to ignore it on Python 3.6. Given that algopy development has all but stopped I wouldn't want to have a hard dependency on it IMHO.

I explicitly disabled the doctests as they don't work for me.

I also removed the use of Tester in the __init__.py as that usage seems to be deprecated, broke pytest testing (for me) and IMHO doesn't add much value.

I also updated the meta.yaml so the conda build worked for me.

opened by dhirschfeld 7

Sphinx build error

With Python 3.4 and Sphinx 1.3 I get the following error when I try to build the numdifftools docs:

$ python setup.py build_sphinx
/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/setuptools/dist.py:291: UserWarning: The version specified ('unknown') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
  "details." % self.metadata.version
running build_sphinx
Running Sphinx v1.3
Creating file /Users/deil/code/numdifftools/docs/../docs/_rst/numdifftools.rst.
Creating file /Users/deil/code/numdifftools/docs/../docs/_rst/numdifftools.tests.rst.
Creating file /Users/deil/code/numdifftools/docs/../docs/_rst/modules.rst.
Traceback (most recent call last):
  File "setup.py", line 229, in <module>
    setup_package()
  File "setup.py", line 226, in setup_package
    entry_points={'console_scripts': CONSOLE_SCRIPTS})
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/sphinx/setup_command.py", line 161, in run
    freshenv=self.fresh_env)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/sphinx/application.py", line 143, in __init__
    self.setup_extension(extension)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/sphinx/application.py", line 440, in setup_extension
    ext_meta = mod.setup(self)
  File "/Users/deil/Library/Python/3.4/lib/python/site-packages/numpydoc/numpydoc.py", line 114, in setup
    app.connect('autodoc-process-docstring', mangle_docstrings)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/sphinx/application.py", line 471, in connect
    self._validate_event(event)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/sphinx/application.py", line 468, in _validate_event
    raise ExtensionError('Unknown event name: %s' % event)
sphinx.errors.ExtensionError: Unknown event name: autodoc-process-docstring

cc @maniteja123

opened by cdeil 6

New release?

Seems like the last release is some time ago and there are some deprecations, like:

/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/numdifftools/core.py:330: DeprecationWarning: `factorial` is deprecated!
Importing `factorial` from scipy.misc is deprecated in scipy 1.0.0. Use `scipy.special.factorial` instead.
  misc.factorial(np.arange(offset, step * nterms + offset, step))

That seem fixed on master but not in that latest release. This would make PyMC3's life easier.

opened by twiecki 4

Unable to install on Python 2.7
I can't get numdifftools to install with python 2.7. I resolved the error

RuntimeError: Due to a bug in setuptools, PyScaffold currently needs at least Python 3.4! Install PyScaffold 2.5 for Python 2.7 support.

by changing in setup.py

setup(setup_requires=['pyscaffold>=3.0a0,<3.1a0'] + sphinx,

to

setup(setup_requires=['pyscaffold==2.5.10'] + sphinx,

(2.5.10 is needed to work with setuptools version >= 39 due to blue-yonder/pyscaffold#148). But I then get the error

error: 'egg_base' must be a directory name (got `src`)

which is the setuptools bug that the PyScaffold docs say you need to use PyScaffold 2.5 to avoid. It might be that the project needs to be set up with PyScaffold 2.5 in the first place as well as installed with it. When I go back to the commit https://github.com/pbrod/numdifftools/commit/406a79877e0dd45aefe210b08e73cdd58ff4cb15 (just before numdifftools was updated with PyScaffold 3) then I can get it to install on python 2.7 if I downgrade setuptools to < 39.
opened by rparini 4
Jacobian broken since 0.9.15
I wrote some code back in September that uses Jacobian to calculate all the partial derivatives for a matrix valued function of a vector argument. I started working with this code again this morning and discovered that it no longer works. I traced it down to a change in the behavior of Jacobian. This simple program demonstrates what I mean.

from numpy import array from numdifftools import Jacobian x = array([1,2]) G = lambda x: array([[x[0], x[1]], [x[0], x[1]]]) dGdx = Jacobian(lambda x: G(x)) D = dGdx(x) print G(x).shape print D.shape

This is the output with numdifftools 0.9.14 installed.

(2, 2) (2, 2, 2)

This is the output with numdifftools 0.9.15 or later installed.

(2, 2) (2, 2, 4)

So numdifftools 0.9.15 and later returns the wrong size tensor.
bug
opened by baroobob 4
Support for multiple dimensions?

Hello, is there a way to calculate directional derivatives of a function using numdifftools, as with directionaldiff or gradest from Matlab's derivest?

Kind regards, Joe

opened by solarjoe 4
TypeError: unsupported operand type(s) for /: 'float' and 'Bicomplex'
The following gives an error:

import numpy as np import numdifftools as nd g = lambda x: 1.0/(np.exp(x[0]) + np.cos(x[1]) + 10) print(nd.Gradient(g, method="multicomplex")([1.0, 2.0]))

If I change the method to "complex", it works fine. Is there a way to use the multicomplex method in this situation and other similar ones?
opened by bacalfa 3
Allow fd_derivative to take complex valued functions

At the moment the array which stores the derivatives, du, is always real so if the array of function evaluations, fx, is complex then the imaginary part gets discarded when the elements of du are assigned.

/usr/local/lib/python3.5/site-packages/numdifftools/fornberg.py:173: ComplexWarning: Casting complex values to real discards the imaginary part du[i] = np.dot(fd_weights(x[:size], x0=x[i], n=n), fx[:size])

The proposed change fixes this by matching the data type (as well as the shape) of du to fx.

PS thank you for working on this very useful module!

opened by rparini 3

Computing the Jacobian fails for functions of shape (1,)

The call to Jacobian with a function of shape (1,) fails. Although this can be computed using Gradient, this case is helpful when checking the derivates of constraints in optimization problems.

Example

import numpy as np
import numdifftools as nd
fun = lambda x: np.array([np.sum(x**2) - 1,])
J1 = nd.Jacobian(fun)(x)

This fails with message

Traceback (most recent call last):
  File "/Users/jeffreyh/SVN/ExtOpt/tests/test_numdifftools.py", line 20, in <module>
    J1 = nd.Jacobian(fun)(x)
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/core.py", line 431, in __call__
    return super(Jacobian, self).__call__(np.atleast_1d(x), *args, **kwds)
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/core.py", line 288, in __call__
    results, f_xi = self._derivative(x_i, args, kwds)
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/core.py", line 428, in _derivative_nonzero_order
    return self.fd_rule.apply(results, steps2, step_ratio), fxi
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/finite_difference.py", line 583, in apply
    f_del, h, original_shape = self._vstack(sequence, steps)
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/finite_difference.py", line 684, in _vstack
    h = np.vstack([np.atleast_1d(r).transpose(axes).ravel() for r in steps])
  File "/opt/homebrew/lib/python3.10/site-packages/numdifftools/finite_difference.py", line 684, in <listcomp>
    h = np.vstack([np.atleast_1d(r).transpose(axes).ravel() for r in steps])
ValueError: axes don't match array

In comparison, if we have output of shape (2,), this works

import numpy as np
import numdifftools as nd
fun = lambda x: np.array([np.sum(x**2) - 1,0])
J1 = nd.Jacobian(fun)(x)

opened by jeffrey-hokanson 0

Error when attempting to calculate the Hessian of a vector-valued function

I'm not on the bleeding edge version but I'm getting this error when trying to calculate the Hessian of a function that returns a 1D array:

TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dobos/project/ga_pfsspec_all/python/test/pfs/ga/pfsspec/stellar/rvfit/test_modelgridrvfit.py", line 155, in test_fit_rv
    rv, rv_err, params, params_err = rvfit.fit_rv({'mr': spec}, rv_0=100.0, rv_bounds=(0, 200), params_0=params_0, params_fixed=params_fixed, params_bounds=params_bounds)
  File "/home/dobos/project/ga_pfsspec_all/python/pfs/ga/pfsspec/stellar/rvfit/modelgridrvfit.py", line 155, in fit_rv
    self.calculate_F(spectra, rv, rv_bounds, None, params, params_bounds, None, params_free, params_fixed)
  File "/home/dobos/project/ga_pfsspec_all/python/pfs/ga/pfsspec/stellar/rvfit/modelgridrvfit.py", line 58, in calculate_F
    ddpc = ddphichi(self.params_to_array(rv_0, params_free, **params_0))
  File "/datascope/slurm/miniconda3/envs/astro-dnn/lib/python3.7/site-packages/numdifftools/core.py", line 849, in __call__
    return super(Hessdiag, self).__call__(np.atleast_1d(x), *args, **kwds)
  File "/datascope/slurm/miniconda3/envs/astro-dnn/lib/python3.7/site-packages/numdifftools/core.py", line 376, in __call__
    results = self._derivative(xi, args, kwds)
  File "/datascope/slurm/miniconda3/envs/astro-dnn/lib/python3.7/site-packages/numdifftools/core.py", line 260, in _derivative_nonzero_order
    results = [diff(f, fxi, xi, h) for h in steps]
  File "/datascope/slurm/miniconda3/envs/astro-dnn/lib/python3.7/site-packages/numdifftools/core.py", line 260, in <listcomp>
    results = [diff(f, fxi, xi, h) for h in steps]
  File "/datascope/slurm/miniconda3/envs/astro-dnn/lib/python3.7/site-packages/numdifftools/core.py", line 894, in _central_even
    hess[i, i] = (f(x + 2 * ee[i, :]) - 2 * fx + f(x - 2 * ee[i, :])) / (4. * hess[i, i])
ValueError: setting an array element with a sequence.

It works with the Gradient and the Jacobian. It would also be great if these accepted Nd arrays instead of just 1D arrays and the resulting new dimension would be the last (right now it's the second regardless of the number of input dimensions).

numdifftools 0.9.39 py_0 conda-forge

opened by dobos 0

Parallelizing function evaluations for the Derivative class

Similar to issue #20 , I would be willing to implement parallelizing the function evaluations for the derivative with the multiprocessing module. I have it working in my own implementation, and can add it if you are open to it.

I am unsure if I am breaking anything. Since I am only using it to calculate the jacobian, I have not run any tests on it other than working with that function.

opened by jmeziere 0

discrepancies in values of the jacobian

I tried to compare the values obtained from computing the jacobian of a complex valued function using numdifftools and found that it differed from that computed using autograd only for the parameter having 1j. i would like to know what went wrong.

from numdifftools import Jacobian, Hessian

import autograd.numpy as np
from autograd import jacobian, hessian

frequencies = np.array([1.00000000e+04, 7.94328235e+03, 6.30957344e+03, 5.01187234e+03,3.98107171e+03, 3.16227766e+03, 2.51188643e+03, 1.99526231e+03,
       1.58489319e+03, 1.25892541e+03, 1.00000000e+03, 7.94328235e+02,6.30957344e+02, 5.01187234e+02, 3.98107171e+02, 3.16227766e+02,
       2.51188643e+02, 1.99526231e+02, 1.58489319e+02, 1.25892541e+02,1.00000000e+02, 7.94328235e+01, 6.30957344e+01, 5.01187234e+01,
       3.98107171e+01, 3.16227766e+01, 2.51188643e+01, 1.99526231e+01,1.58489319e+01, 1.25892541e+01, 1.00000000e+01, 7.94328235e+00,
       6.30957344e+00, 5.01187234e+00, 3.98107171e+00, 3.16227766e+00,2.51188643e+00, 1.99526231e+00, 1.58489319e+00, 1.25892541e+00,
       1.00000000e+00])

Z = np.array([ 28.31206108+3.85713361e+00j,  30.65961255-6.15028952e-01j,34.24015216-1.53009927e+00j,  31.28832221+2.00862719e-01j,
        29.16894207-1.90759028e+00j,  31.35415498+8.12935902e+00j,33.29418445-8.44736332e-01j,  31.44975238-4.33909650e+00j,
        28.69757696-4.57807440e-01j,  32.60369585-7.49365253e+00j,38.67554243+1.94558309e+00j,  32.04682449+5.96513060e-02j,
        33.27666601+6.96674118e-02j,  34.03106231-1.63078686e+00j,31.61457921-3.37817364e+00j,  29.10184267-6.59263829e+00j,
        32.50730455-7.49033830e+00j,  36.43172672-3.28250914e+00j,38.06409634-6.87182171e+00j,  28.0217396 -7.79409862e+00j,
        32.56764449-1.28634629e+01j,  35.51511763-2.12395710e+01j,31.9317554 -2.85721873e+01j,  38.87220134-2.99072634e+01j,
        35.5291417 -3.74944224e+01j,  42.9529093 -4.04380053e+01j,40.94710166-5.09880048e+01j,  47.58460761-6.50587929e+01j,
        56.61773977-6.93842057e+01j,  70.84595799-8.97293571e+01j,91.28235232-9.63476214e+01j, 114.19032377-1.06793820e+02j,
       139.78246542-1.00266685e+02j, 161.07272489-1.00733766e+02j,181.230854  -9.02441714e+01j, 205.54084395-8.99491269e+01j,
       215.24421556-7.62099459e+01j, 223.62924894-6.40376670e+01j,229.93028184-4.76770260e+01j, 234.56144072-4.38847706e+01j,
       240.57421683-3.52116682e+01j])

params_init =np.array([10, 1e-5, 20, 50])

w = 2 * np.pi*frequencies # angular frequencies
s = 1j*w
def z_fun(p):
        return p[0] + 1/(s*p[1]+1/(p[2]+(p[3]/np.sqrt(s))))

# define the objective function
def obj_fun_1(p, z): 
    wtRe = 1/(z.real**2 + z.imag**2)
    return (np.sum(np.absolute((wtRe * (z_fun(p)- z)**2))))

# computing the jacobian using numdifftools

def num_jac_1(x, z):
    return Jacobian(lambda x: obj_fun_1(x, z), method='central')(x).real.squeeze()

# check the values
print(num_jac_1(params_init, Z))
# [-6.42302368e-01  1.45242757e-05 -1.07751884e-01 -2.99510421e-02]

# computing the jacobian using autograd
auto_jac_1 = jacobian(obj_fun_1)
# check the values
print(auto_jac_1(params_init, Z))
# [-6.42302368e-01  1.52096259e+05 -1.07751884e-01 -2.99510421e-02]

Now the problem is that when i supply the jacobian from numdifftools (the difference is in the second parameter (p[1]) the optimization fails but when I supply that obtained using autograd the optimization succeeds. I would like to know what the problem is.

opened by richinex 2

Part of docs are almost verbatim copy of original docs by John d'Errico

The original docs of the matlab version can be found here: https://convexoptimization.com/TOOLS/DERIVEST.pdf

The docs give credit to d'Errico, but without explicit permission from the original author, this is still a copyright violation, unless the original docs were released under a copyleft license.

opened by HDembinski 0

Releases(v0.9.1)

v0.9.1(Aug 20, 2015)
This is a major facelift of Numdifftools. Highlights for this release is:

Complex step method for derivative-order 1 through 14 is now available

Updated documentation

Source code(tar.gz)
Source code(zip)

Owner

Per A. Brodtkorb

GitHub Repository

PLUR is a collection of source code datasets suitable for graph-based machine learning.

PLUR (Programming-Language Understanding and Repair) is a collection of source code datasets suitable for graph-based machine learning. We provide scripts for downloading, processing, and loading the

76 Nov 25, 2022

This jupyter notebook project was completed by me and my friend using the dataset from Kaggle

ARM This jupyter notebook project was completed by me and my friend using the dataset from Kaggle. The world Happiness 2017, which ranks 155 countries

1 Jan 23, 2022

machine learning model deployment project of Iris classification model in a minimal UI using flask web framework and deployed it in Azure cloud using Azure app service

This is a machine learning model deployment project of Iris classification model in a minimal UI using flask web framework and deployed it in Azure cloud using Azure app service. We initially made th

73 Dec 01, 2022

2021 Machine Learning Security Evasion Competition

2021 Machine Learning Security Evasion Competition This repository contains code samples for the 2021 Machine Learning Security Evasion Competition. P

8 May 01, 2022

Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models.

Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models. Feature-engine's transformers follow scikit-learn's functionality wit

33 Dec 27, 2022

A simple example of ML classification, cross validation, and visualization of feature importances

Simple-Classifier This is a basic example of how to use several different libraries for classification and ensembling, mostly with sklearn. Example as

2 Aug 25, 2022

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale.

Model Search Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers sp

1 Dec 13, 2021

Lightning ⚡️ fast forecasting with statistical and econometric models.

Nixtla Statistical ⚡️ Forecast Lightning fast forecasting with statistical and econometric models StatsForecast offers a collection of widely used uni

2.1k Dec 29, 2022

Python library for multilinear algebra and tensor factorizations

scikit-tensor is a Python module for multilinear algebra and tensor factorizations

394 Dec 09, 2022

pywFM is a Python wrapper for Steffen Rendle's factorization machines library libFM

pywFM pywFM is a Python wrapper for Steffen Rendle's libFM. libFM is a Factorization Machine library: Factorization machines (FM) are a generic approa

251 Sep 23, 2022

All-in-one web-based development environment for machine learning

All-in-one web-based development environment for machine learning Getting Started • Features & Screenshots • Support • Report a Bug • FAQ • Known Issu

3 Feb 03, 2021

Drug prediction

I have collected data about a set of patients, all of whom suffered from the same illness. During their course of treatment, each patient responded to one of 5 medications, Drug A, Drug B, Drug c, Dr

1 Jan 28, 2022

Climin is a Python package for optimization, heavily biased to machine learning scenarios

climin climin is a Python package for optimization, heavily biased to machine learning scenarios distributed under the BSD 3-clause license. It works

177 Sep 02, 2022

Dragonfly is an open source python library for scalable Bayesian optimisation.

Dragonfly is an open source python library for scalable Bayesian optimisation. Bayesian optimisation is used for optimising black-box functions whose

744 Jan 02, 2023

Microsoft contributing libraries, tools, recipes, sample codes and workshop contents for machine learning & deep learning.

366 Jan 03, 2023