## academia Archives - Student Pulse - Blog ArchivesNovember 6th, 2015 ## How to Use Regression Analysis EffectivelySo, you want to use regression analysis in your paper? While statistical modeling can add great authority to your paper and to the conclusions you draw, it is also easy to use incorrectly. The worst case scenario can occur when you think you’ve done everything right and therefore reach a strong conclusion based on an improperly conceived model. This guide presents a series of suggestions and considerations that you should take into account before you decide to use regression analysis in your paper. The best regression model is based on a strong theoretical foundation that demonstrates not just that A and B are related, but Before you start, ask yourself two important questions: is your research question a good fit for regression analysis? And, do you have access to good data? ## 1. Is Your Research Question a Good Fit for Regression Analysis?This depends on many different factors. Are you trying to explain something that is primarily described by numerical values? This is a key question to ask yourself before you decide to use regression. Although there are various ways to use regression analysis to describe non-numerical outcomes (e.g., dichotomous yes/no or probabilistic outcomes), they become more complicated and you will need to have a much deeper understanding of the underlying principles of regression in order to use them effectively. Before you start, consider whether or not your dependent variable is numerical. Some examples: - Number of years a politician serves in Senate
- Life expectancy
- Lifetime earnings
- Age at birth of first child
At the same time, you need to make sure that there is sufficient For example, you would have a problem if you tried to predict the likelihood of someone being elected as president because ## 2. Do You Have Access to Good Data?Before you can conduct any type of analysis, you need a good data set. Not all data sets are easily suited to regression analysis without considerable manipulation. Some things to consider before you decide to use regression: *Are most of your independent variables numerical in nature?*The best data set for regression will have variables that are primarily described by numbers that vary on a continuous scale. On the other hand, if most of your variables are categorical, you might consider using a different method of analysis (e.g., Chi-squared).
*Are there enough cases (n) in your data set?*Particularly if you think you might use multiple regression, where multiple independent variables are used to predict a single dependent variable, you need to have a sufficient number of cases in your sample to obtain significant results. A general rule of thumb is that you need at*least*20 cases per independent variable in your model. So if your model includes 5 independent variables, you need a minimum of 100 cases.
Keep in mind that your independent variables need to meet the same criteria for Once you decide to proceed with a regression model in your analysis, there are a three key concepts to keep in mind as you design your model to avoid making an easily preventable mistake that could send your conclusions way off track. *Parsimony**Internal Validity**Multicollinearity*
Each is described in more detail below. ## ParsimonyIn statistics, the principle of parsimony is based on the idea that when possible, the simplest model with the fewest independent variables should be used when a model with more variables offers only slightly more explanatory value. In other words, one should not add variables to a model that do not increase the ability of the model to explain something. Only add variables to a model if they significantly increase the ability of the model to explain something. If you add too many variables to your model, you can unwittingly introduce major problems to your analysis. In the extreme case, you must consider that your R There are two good ways to address this problem: use an stepwise regression to analyze the explanatory impact of each variable as it is added to the model.takes into consideration the number of variables used in the model, and only increases when the addition of a new variable explains more than would random chance alone. So although a model with 10 variables might have a very high R*Adjusted R*^{2}^{2 }value, the*Adjusted R*could actually be much lower than a model with fewer variables. Selecting your model based on^{2}*Adjusted R*helps you select a more parsimonious model that is less likely to have other problems (e.g., see^{2}*multicollinearity*below).
is a computational method of assessing the additional explanatory value of each variable as they are added to the model in different orders. It can be used to parse out superfluous variables from a model, however it needs to be used carefully and in concert with theoretical guidance to avoid overfitting your data.**Stepwise Regression**
A good rule of thumb as you consider different models is that ## Internal ValidityInternal validity is the degree to which one factor can be said to *Temporal precedence*, i.e., the “cause” precedes the “effect.”*Covariation*, i.e., the “cause” and “effect” are demonstrably related.*Nonspuriousness, i.e.,*there are no plausible alternative explanations for the observed covariation caused by a confounding variable.
In many cases, internal validity becomes an issue in the form of a “chicken and egg” problem. For example, let’s say you are considering the relationship between obesity and depression (a common example). If you want to include
If you have no clear theoretical guidance to show that, in fact, depression usually precedes obesity (temporal precedence), you could introduce a significant problem to your model if the relationship is in fact the other way around: depression being the Therefore, as you craft your model it is important to have a theoretical basis for the inclusion of each variable. ## MulticollinearityMulticollinearity occurs when the independent variables in a multiple regression model are highly correlated with one another. This can be a problem in several ways: - It reduces the parsimony of your model if the two variables are highly similar (e.g., two different variables that effectively measure the same thing);
- Multicollinearity can lead to erratic changes in the coefficients (measured effect) of predictor variables;
- As a result, it can be difficult to interpret the results of a model with high multicollinearity among predictors. Specifically, it becomes impossible to discern the individual effect of different regressors.
An example of variables that are going to be highly multicollinear are any variables that effectively measure the same thing. One way to show this, for the purposes of an example, is to imagine converting categorical data into a series of binary variables. Any variables that effectively measure the same concept are likely to have high collinearity. For example, let’s say that we have a variable measuring One way to use this data in a regression model would be to convert the data into three dichotomous (yes/no) variables indicating a person’s response. However, if you then include all of these dichotomous variables in your model, you will have a big problem because they will become Another common example can be found in the use of As a result, if both variables are included as predictors in a model, it can be difficult to discern the effect that each variable has Thus, as you build your model, you need to be aware of the potentially confounding impact of using highly similar predictor variables. In an ideal model, all independent variables will have no or very low correlation to each other, but a high correlation with the dependent variable. ## Conclusion: Use Regression Effectively by Keeping it SimpleRegression analysis can be a powerful explanatory tool and a highly persuasive way of demonstrating relationships between complex phenomena, but it is also easy to misuse if you are not an expert statistician. If you decide to use regression analysis, you shouldn’t ask it to do too much: don’t force your data to explain something that you otherwise can’t explain! Moreover, regression should only be used where it is appropriate and when their is sufficient quantity and Lastly, always keep in mind that the best regression model is based on a strong theoretical foundation that demonstrates not just that A and B are related, but If you keep all of these things in mind, you will be on your way to crafting a powerful and persuasive argument. Tags: academia, Modeling, Regression, research, Research Guide, Stats March 10th, 2015 ## Finding Balance in Graduate School In the darkest days of the graduate school “doldrums,” as you wade through readings and midterms and papers, it can be hard to recall How can you balance graduate school with enjoying your personal life? Here are five things you can try: -
Schedule School Like A 9-to-5
Grad school can often feel like a 24/7 job where you need to be thinking about your research, coursework, and teaching all the time in order to compete in the academic job market. If you discipline yourself, you can work a semi-regular “shift” and still make time for dating, relaxing, and hobbies. Figure out when your most productive daytime hours are, and schedule 8-10 working hours during that time. If you stay on task during these hours, you can feel good about shutting it down to enjoy some personal time. Of course there will always be emergencies and last-minute deadlines, but by scheduling working shifts you can actually minimize their occurrence and lead a more ‘normal’ day-to-day life. -
Make Working Time Productive
Procrastination during your scheduled hours will drag work into your personal time, so you need to find strategies to stay productive and on task. Download an app that blocks time-wasting websites; write from a computer with internet disabled; meditate or go for a walk – whatever you have to do to stay on task. Schedule short breaks every 60 to 90-minutes so that you stay energized and give your brain some relief. -
Set Goals & Reward Yourself
If you’ve ever had a pet, you know how effective small rewards can be. But you are trainable, too! Set realistic goals for yourself and then reward yourself when you meet them. Small rewards for finishing tasks or meeting goals can go a long way toward keeping you motivated. Figure out what you respond to – a Starbucks coffee? A homemade cookie? A night out dancing? – and reward yourself when you meet set goals. The more that you give yourself rewards, the more you will be willing to meet your own goals when you set them. -
Schedule Personal Time
Some people dislike the idea of penciling in their partners or setting aside a block of time for pleasure reading – but given how graduate school work tends to expand to fill all of your time, scheduling off chunks of time to take care of your personal needs might be the smartest way to make sure they don’t get constantly sidelined. So schedule yourself some free time, put your school work away, and indulge. You’ll find that the more you allow yourself to refresh your brain, the more you will actually get done when it’s time to work – because your mind will be focused on work, and not how tired of working you are. -
Banish the Guilt
It’s easy to envision yourself working productively around the clock to finish academic obligations or publish one more paper. The flip side is that you often guilt yourself when you aren’t working – you think that any minute you spend relaxing could be spent working! But the truth is, even if you love your research area, it’s easy to get “burned out” in academia. If you work around the clock, you can get disillusioned and discouraged. The more exhausted you are mentally and spiritually with your work, the harder it is over the long-term for you to produce high-quality scholarship. Divest yourself from the guilt that graduate school can bring. Whenever you feel guilty for spending time on non-work things, mentally change the subject and remind yourself that it’s OK to spend time relaxing and recharging – even more, it’s So divest yourself from the guilt that graduate school can bring. Whenever you feel guilty for spending time on non-work things, mentally change the subject and remind yourself that it’s OK to spend time relaxing and recharging – even more, it’s Tags: academia, graduate school, personal life, research, work |