I have the following expression:
log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))
it is giving me the following warning:
RuntimeWarning: invalid value encountered in log
log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))
I don't understand what might be the invalid value or why am I getting it. Any and every help is appreciated.
NOTE: This is a cross-entropy cost function where I added 1e-7 to avoid having zeros inside log. y & a are numpy arrays and numpy is imported as np.
3 Answers
You probably still have negative values inside the log, which gives nan with real numbers.
a and y should represent probability between 0 to 1, So you need to check why do you have smaller/larger values there. Adding 1e-7 shows there is something fishy, because np.log(0) gives -inf, which I think is the value you want.
You can use math.log() replacing numpy.log(), which could raise error
>>> import numpy
>>> numpy.log(0)
-inf
>>> numpy.__version__
'1.3.0'
>>> import math
>>> math.log(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: math domain error
log(x) where x<=0 edge cases
np.log(0)
RuntimeWarning: divide by zero encountered in log
-inf
np.log(-1)
RuntimeWarning: invalid value encountered in log
nan
