Cost function quantifies the error between predicted and expected values. The cost function used in Logistic Regression is called Log Loss or Cross Entropy Function. J(θ) = (1/m) * Σ [ -y * log(hθ(x)) - (1 - y) * log(1 - hθ(x)) ] Where: - J(θ) is the cost that we want t
(h: np.ndarray, y: np.ndarray)
| 64 | |
| 65 | |
| 66 | def cost_function(h: np.ndarray, y: np.ndarray) -> float: |
| 67 | """ |
| 68 | Cost function quantifies the error between predicted and expected values. |
| 69 | The cost function used in Logistic Regression is called Log Loss |
| 70 | or Cross Entropy Function. |
| 71 | |
| 72 | J(θ) = (1/m) * Σ [ -y * log(hθ(x)) - (1 - y) * log(1 - hθ(x)) ] |
| 73 | |
| 74 | Where: |
| 75 | - J(θ) is the cost that we want to minimize during training |
| 76 | - m is the number of training examples |
| 77 | - Σ represents the summation over all training examples |
| 78 | - y is the actual binary label (0 or 1) for a given example |
| 79 | - hθ(x) is the predicted probability that x belongs to the positive class |
| 80 | |
| 81 | @param h: the output of sigmoid function. It is the estimated probability |
| 82 | that the input example 'x' belongs to the positive class |
| 83 | |
| 84 | @param y: the actual binary label associated with input example 'x' |
| 85 | |
| 86 | Examples: |
| 87 | >>> estimations = sigmoid_function(np.array([0.3, -4.3, 8.1])) |
| 88 | >>> cost_function(h=estimations,y=np.array([1, 0, 1])) |
| 89 | 0.18937868932131605 |
| 90 | >>> estimations = sigmoid_function(np.array([4, 3, 1])) |
| 91 | >>> cost_function(h=estimations,y=np.array([1, 0, 0])) |
| 92 | 1.459999655669926 |
| 93 | >>> estimations = sigmoid_function(np.array([4, -3, -1])) |
| 94 | >>> cost_function(h=estimations,y=np.array([1,0,0])) |
| 95 | 0.1266663223365915 |
| 96 | >>> estimations = sigmoid_function(0) |
| 97 | >>> cost_function(h=estimations,y=np.array([1])) |
| 98 | 0.6931471805599453 |
| 99 | |
| 100 | References: |
| 101 | - https://en.wikipedia.org/wiki/Logistic_regression |
| 102 | """ |
| 103 | return float((-y * np.log(h) - (1 - y) * np.log(1 - h)).mean()) |
| 104 | |
| 105 | |
| 106 | def log_likelihood(x, y, weights): |