Everything2
Near Matches
Ignore Exact
Full Text
Everything2

Regression

created by Webster 1913

(idea) by dogboy (2 y) (print)   ?   I like it! Mon Oct 16 2000 at 14:46:12

A previous node suggested the following: A statistical method in which the relationship between the mean value of a random variable and the corresponding values of one or more independent variables is observed. I would like to clarify a few things. Only in the most technical sense is this correct, but it fails to note many of the subtleties, some of which I discuss below. 1) The node above is only technically correct for what we call Ordinary Least Squares Regression, or OLS which has a variety of assumptions associated with it. Among these assumptions is that the independent and dependent variables are normally distributed, random variables. However, this assumption can be loosened, thus making the general definition above actually too general. 2) Regression is almost always employed when there are MORE THAN one independent variables. When we have simply one dependent and one independent variable, the Beta coefficient that is generated is functionally equivalent to r, or the correlation coefficient. Furthermore, bivariate ( one independent one dependent variable) almost certainly violates another one of the assumptions of regression, that of specification. An ill specified model (regression of course being a model of reality) leads to estimators ( in the form of Beta coefficients) which are unreliable. Sadly, there is no way of knowing a priori if they are biased upwards or downwards. There are additional considerations to take into account with regression, but these should go into another discussion.

(idea) by pimephalis (4.2 mon) (print)   ?   I like it! Fri Feb 02 2001 at 16:08:41

To clarify the clarification posted by dogboy, the assumptions of simple and multiple linear regression are as follows (these are called the Gauss-Markov conditions):
  1. Linear relationship between the independant variables and the dependant variable; if this is not the case, then one of the following steps is necessary:
    • A polynomial term needs to be applied;
    • The variable should be transformed (logarithm, square root, arcsine square root etc.);
    • The suitability of a regression approach should be reconsidered.
  2. Homoscedasticity of the residuals
  3. Independance of the observations
  4. The independant variables must either be controlled or measured with great precision compared to the dependant variable; if this is not the case, then major axis regression should be applied
If these conditions are not met, not only will the beta coefficients be biased, but the model's predictive capacity will likely be misrepresented.

(definition) by Webster 1913 (print) I like it! Wed Dec 22 1999 at 2:34:02

Re*gres"sion (r?*gr?sh"?n), n. [L. regressio: cf. F. r'egression.]

The act of passing back or returning; retrogression; retrogradation.

Sir T. Browne.

Edge of regression (of a surface) Geom., the line along which a surface turns back upon itself; -- called also a cuspidal edge. -- Regression point Geom., a cusp.

 

© Webster 1913.


printable version
chaos

regression testing The Statistics Project What face did you have before your mother and father were born? Method of least squares
standard deviation homoscedasticity A mathematically fair way to split a taxi ride with multiple stops regress
regression to the mean beta Levenberg-Marquardt Method regression fallacy
Time regression effect while stoned communion leave-one-out cross validation test set cross validation
k-fold cross validation Ego Defense Mechanisms pseudoexperiment Dammit, one of my past lives is more attractive than me!
a priori correlation Methods of Interrogation Duh
Y'know, if you log in, you can write something here, or contact authors directly on the site. Create a New User if you don't already have an account.
  Epicenter
Login
Password

password reminder
register

Everything2 Help

Cool Staff Picks
Look at this mess the Death Borg made!
After a six-month diet of blues and greys I was back to white. I was an empty plate.
Nicholas II
The perfect node
Christian Fundamentalism
Battery
The Skeptical Environmentalist
John Coltrane
Dozenal Society of America
Gettysburg Address
35 mm vs APS
Penny Lane
The Painting of Sounds, Noises, and Smells
Butterfly
New Writeups
Pandeism Fish
How conatus compels divine ketosis through a radical kenosis(essay)
cryforhelp
Major dictionaries of the world(review)
Glowing Fish
The Uncanny X-Men and the New Teen Titans(thing)
WolfKeeper
Launch loop(idea)
TendoKing
Katana(person)
Wuukiee
Highly ornamental cultivars of brambles still have as many thorns as their wild counterparts(idea)
TheDeadGuy
Editor Log: May 2008(log)
everyday j.Lo
pray do not molest them(thing)
ammie
Bands Who Take Their Names from Eighteenth-century English Poetry and Prose(idea)
shaogo
Under My Thumb(review)
ammie
Rock On(person)
The Custodian
The Dresden Files(thing)
Ouzo
PETA becomes you, a proposed future(fiction)
Ereneta
Stone Soup, Part Two(fiction)
jjen
Sorrier than I ever thought I would be(personal)
E2 is a by-product of the existence of The Everything Development Company