Is it good practice to use Linear Least-Squares with SMA?How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms
PTIJ: Who should I vote for? (21st Knesset Edition)
Why do passenger jet manufacturers design their planes with stall prevention systems?
What is the purpose or proof behind chain rule?
How should I state my peer review experience in the CV?
My adviser wants to be the first author
"of which" is correct here?
Custom alignment for GeoMarkers
Is it normal that my co-workers at a fitness company criticize my food choices?
Instead of a Universal Basic Income program, why not implement a "Universal Basic Needs" program?
Violin - Can double stops be played when the strings are not next to each other?
Employee lack of ownership
Is "upgrade" the right word to use in this context?
Do the common programs (for example: "ls", "cat") in Linux and BSD come from the same source code?
How could an airship be repaired midflight?
Describing a chess game in a novel
World War I as a war of liberals against authoritarians?
Fastest way to pop N items from a large dict
A diagram about partial derivatives of f(x,y)
Why does overlay work only on the first tcolorbox?
How to make healing in an exploration game interesting
Brexit - No Deal Rejection
Non-trivial topology where only open sets are closed
Python if-else code style for reduced code for rounding floats
Welcoming 2019 Pi day: How to draw the letter π?
Is it good practice to use Linear Least-Squares with SMA?
How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago
add a comment |
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
regression time-series correlation trend moving-average
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 4 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 4 hours ago
Chef36Chef36
61
asked 4 hours ago
Chef36Chef36
61
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago
add a comment |
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago
1
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
answered 3 hours ago
Gkhan CebsGkhan Cebs
1443
1443
add a comment |
add a comment |
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
3 hours ago