Tutorial Linear Regression Basics

This tutorial details a simple regression analysis based on the "Formaldehyde" dataset.

First, creating the dataset

This is done relying on the DataFrames.jl package.

using DataFrames
df = DataFrame(Carb=[0.1,0.3,0.5,0.6,0.7,0.9], OptDen=[0.086,0.269,0.446,0.538,0.626,0.782])

Second, the model is defined

We want OptDen as the dependent variable (the response) and Carb as the independent variable (the predictor). Our model will have an intercept; however, the package will implicitly add the intercept to the model. We define the model as Optden ~ Carb; the variable's names need to be column names from the DataFrame, which is the second argument to the regress function. The lm object will then present essential information from the regression.

using LinearRegressionKit
using StatsModels # this is requested to use the @formula

lm = regress(@formula(OptDen ~ Carb), df)

Model definition:	OptDen ~ 1 + Carb
Used observations:	6
Model statistics:
  R²: 0.999047			Adjusted R²: 0.998808
  MSE: 7.48e-05			RMSE: 0.0086487
  σ̂²: 7.48e-05
  F Value: 4191.84 with degrees of freedom 1 and 4, Pr > F (p-value): 3.40919e-07
Confidence interval: 95%

Coefficients statistics:
Terms ╲ Stats │      Coefs     Std err           t    Pr(>|t|)        code      low ci     high ci
──────────────┼───────────────────────────────────────────────────────────────────────────────────
(Intercept)   │ 0.00508571  0.00783368    0.649211    0.551595              -0.0166641   0.0268355
Carb          │   0.876286   0.0135345     64.7444  3.40919e-7        ***     0.838708    0.913864

	Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Third, some illustration about the model is created

Here we will only look at the fit-plot. To obtain it, we only need to add a third argument to the regress function. Namely, the name of the plot requested ("fit"). When at least one plot is requested, the regress function will return a pair of objects: the information about the regression (as before), and an object (Dict) to access the requested plot(s).

using VegaLite # this is the package use for plotting

lm, ps = regress(@formula(OptDen ~ Carb), df, "fit")
ps["fit"]

The response is plotted on the y-axis, and the predictor is plotted on the x-axis. The dark orange line represents the regression equation. The dark grey band represents the confidence interval given the α (which defaults to 0.05 and gives a 95% confidence interval). The light grey band represents the individual prediction interval. Finally, the blue circles represent the actual observations from the dataset.

Fourth, generate the predictions from the model

Here we get the predicted values from the model using the same Dataframe.

results = predict_in_sample(lm, df)

Fifth, generate the others statistics about the model

In order to get all the statistics, one can use the "all" keyword as an argument of the req_stats argument.

results = predict_in_sample(lm, df, req_stats="all")

Sixth, generate prediction for new data

We first create a new DataFrame that needs to use the same column names used in the model. In our case, there is only one column: "Carb".

ndf = DataFrame(Carb= [0.11, 0.22, 0.55, 0.77])
predictions = predict_out_of_sample(lm, ndf)