Skip to content

odak.fit

odak.fit

Provides functions to fit models to a provided data. These functions could be best described as a catalog of machine learning models.

gradient_descent_1d(input_data, ground_truth_data, parameters, function, gradient_function, loss_function, learning_rate=0.1, iteration_number=10)

Vanilla Gradient Descent algorithm for 1D data.

Parameters:

  • input_data – 
                One-dimensional input data.
  • ground_truth_data (array) – 
                One-dimensional ground truth data.
  • parameters – 
                Parameters to be optimized.
  • function – 
                Function to estimate an output using the parameters.
  • gradient_function (function) – 
                Function used in estimating gradient to update parameters at each iteration.
  • learning_rate – 
                Learning rate.
  • iteration_number – 
                Iteration number.

Returns:

  • parameters ( array ) –

    Optimized parameters.

Source code in odak/fit/__init__.py 
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
def gradient_descent_1d(
                        input_data,
                        ground_truth_data,
                        parameters,
                        function,
                        gradient_function,
                        loss_function,
                        learning_rate = 1e-1,
                        iteration_number = 10
                       ):
    """
    Vanilla Gradient Descent algorithm for 1D data.

    Parameters
    ----------
    input_data        : numpy.array
                        One-dimensional input data.
    ground_truth_data : numpy.array
                        One-dimensional ground truth data.
    parameters        : numpy.array
                        Parameters to be optimized.
    function          : function
                        Function to estimate an output using the parameters.
    gradient_function : function
                        Function used in estimating gradient to update parameters at each iteration.
    learning_rate     : float
                        Learning rate.
    iteration_number  : int
                        Iteration number.


    Returns
    -------
    parameters        : numpy.array
                        Optimized parameters.
    """
    t = tqdm(range(iteration_number))
    for i in t:
        gradient = np.zeros(parameters.shape[0])
        for j in range(input_data.shape[0]):
            x = input_data[j]
            y = ground_truth_data[j]
            gradient = gradient + gradient_function(x, y, function, parameters)
        parameters = parameters - learning_rate * gradient / input_data.shape[0]
        loss = loss_function(ground_truth_data, function(input_data, parameters))
        description = 'Iteration number:{}, loss:{:0.4f}, parameters:{}'.format(i, loss, np.round(parameters, 2))
        t.set_description(description)
    return parameters

least_square_1d(x, y)

A function to fit a line to given x and y data (y=mx+n). Inspired from: https://mmas.github.io/least-squares-fitting-numpy-scipy

Parameters:

  • x – 
         1D input data.
  • y – 
         1D output data.

Returns:

  • parameters ( array ) –

    Parameters of m and n in a line (y=mx+n).

Source code in odak/fit/__init__.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
def least_square_1d(x, y):
    """
    A function to fit a line to given x and y data (y=mx+n). Inspired from: https://mmas.github.io/least-squares-fitting-numpy-scipy

    Parameters
    ----------
    x          : numpy.array
                 1D input data.
    y          : numpy.array
                 1D output data.

    Returns
    -------
    parameters : numpy.array
                 Parameters of m and n in a line (y=mx+n).
    """
    w = np.vstack([x, np.ones(x.shape[0])]).T
    parameters = np.dot(np.linalg.inv(np.dot(w.T, w)), np.dot(w.T, y))
    return parameters

perceptron(x, y, learning_rate=0.1, iteration_number=100)

A function to train a perceptron model.

Parameters:

  • x – 
               Input X-Y pairs [m x 2].
  • y – 
               Labels for the input data [m x 1]
  • learning_rate – 
               Learning rate.
  • iteration_number (int, default: 100 ) – 
               Iteration number.

Returns:

  • weights ( array ) –

    Trained weights of our model [3 x 1].

Source code in odak/fit/__init__.py 
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
def perceptron(x, y, learning_rate = 0.1, iteration_number = 100):
    """
    A function to train a perceptron model.

    Parameters
    ----------
    x                : numpy.array
                       Input X-Y pairs [m x 2].
    y                : numpy.array
                       Labels for the input data [m x 1]
    learning_rate    : float
                       Learning rate.
    iteration_number : int
                       Iteration number.

    Returns
    -------
    weights          : numpy.array
                       Trained weights of our model [3 x 1].
    """
    weights = np.zeros((x.shape[1] + 1, 1))
    t = tqdm(range(iteration_number))
    for step in t:
        unsuccessful = 0
        for data_id in range(x.shape[0]):
            x_i = np.insert(x[data_id], 0, 1).reshape(-1, 1)
            y_i = y[data_id]
            y_hat = threshold_linear_model(x_i, weights)
            if y_hat - y_i != 0:
                unsuccessful += 1
                weights = weights + learning_rate * (y_i - y_hat) * x_i 
            description = 'Unsuccessful count: {}/{}'.format(unsuccessful, x.shape[0])
    return weights

threshold_linear_model(x, w, threshold=0)

A function for thresholding a linear model described with a dot product.

Parameters:

  • x – 
               Input data [3 x 1].
  • w – 
               Weights [3 x 1].
  • threshold – 
               Value for thresholding.

Returns:

  • result ( int ) –

    Estimated class of the input data. It could either be one or zero.

Source code in odak/fit/__init__.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def threshold_linear_model(x, w, threshold = 0):
    """
    A function for thresholding a linear model described with a dot product.

    Parameters
    ----------
    x                : numpy.array
                       Input data [3 x 1].
    w                : numpy.array
                       Weights [3 x 1].
    threshold        : float
                       Value for thresholding.

    Returns
    -------
    result           : int
                       Estimated class of the input data. It could either be one or zero.
    """
    value = np.dot(x.T, w)
    result = 0
    if value >= threshold:
       result = 1
    return result