You are currently looking at version 1.1 of this notebook. To download notebooks and datafiles, as well as get help on Jupyter notebooks in the Coursera platform, visit the Jupyter Notebook FAQ course resource.


In [37]:
import pandas as pd
import numpy as np
from scipy.stats import ttest_ind

Assignment 4 - Hypothesis Testing

This assignment requires more individual learning than previous assignments - you are encouraged to check out the pandas documentation to find functions or methods you might not have used yet, or ask questions on Stack Overflow and tag them as pandas and python related. And of course, the discussion forums are open for interaction with your peers and the course staff.

Definitions:

  • A quarter is a specific three month period, Q1 is January through March, Q2 is April through June, Q3 is July through September, Q4 is October through December.
  • A recession is defined as starting with two consecutive quarters of GDP decline, and ending with two consecutive quarters of GDP growth.
  • A recession bottom is the quarter within a recession which had the lowest GDP.
  • A university town is a city which has a high percentage of university students compared to the total population of the city.

Hypothesis: University towns have their mean housing prices less effected by recessions. Run a t-test to compare the ratio of the mean price of houses in university towns the quarter before the recession starts compared to the recession bottom. (price_ratio=quarter_before_recession/recession_bottom)

The following data files are available for this assignment:

  • From the Zillow research data site there is housing data for the United States. In particular the datafile for all homes at a city level, City_Zhvi_AllHomes.csv, has median home sale prices at a fine grained level.
  • From the Wikipedia page on college towns is a list of university towns in the United States which has been copy and pasted into the file university_towns.txt.
  • From Bureau of Economic Analysis, US Department of Commerce, the GDP over time of the United States in current dollars (use the chained value in 2009 dollars), in quarterly intervals, in the file gdplev.xls. For this assignment, only look at GDP data from the first quarter of 2000 onward.

Each function in this assignment below is worth 10%, with the exception of run_ttest(), which is worth 50%.

In [38]:
# Use this dictionary to map state names to two letter acronyms
states = {'OH': 'Ohio', 'KY': 'Kentucky', 'AS': 'American Samoa', 'NV': 'Nevada', 'WY': 'Wyoming', 'NA': 'National', 'AL': 'Alabama', 'MD': 'Maryland', 'AK': 'Alaska', 'UT': 'Utah', 'OR': 'Oregon', 'MT': 'Montana', 'IL': 'Illinois', 'TN': 'Tennessee', 'DC': 'District of Columbia', 'VT': 'Vermont', 'ID': 'Idaho', 'AR': 'Arkansas', 'ME': 'Maine', 'WA': 'Washington', 'HI': 'Hawaii', 'WI': 'Wisconsin', 'MI': 'Michigan', 'IN': 'Indiana', 'NJ': 'New Jersey', 'AZ': 'Arizona', 'GU': 'Guam', 'MS': 'Mississippi', 'PR': 'Puerto Rico', 'NC': 'North Carolina', 'TX': 'Texas', 'SD': 'South Dakota', 'MP': 'Northern Mariana Islands', 'IA': 'Iowa', 'MO': 'Missouri', 'CT': 'Connecticut', 'WV': 'West Virginia', 'SC': 'South Carolina', 'LA': 'Louisiana', 'KS': 'Kansas', 'NY': 'New York', 'NE': 'Nebraska', 'OK': 'Oklahoma', 'FL': 'Florida', 'CA': 'California', 'CO': 'Colorado', 'PA': 'Pennsylvania', 'DE': 'Delaware', 'NM': 'New Mexico', 'RI': 'Rhode Island', 'MN': 'Minnesota', 'VI': 'Virgin Islands', 'NH': 'New Hampshire', 'MA': 'Massachusetts', 'GA': 'Georgia', 'ND': 'North Dakota', 'VA': 'Virginia'}
In [39]:
def get_list_of_university_towns():
    '''Returns a DataFrame of towns and the states they are in from the 
    university_towns.txt list. The format of the DataFrame should be:
    DataFrame( [ ["Michigan", "Ann Arbor"], ["Michigan", "Yipsilanti"] ], 
    columns=["State", "RegionName"]  )
    
    The following cleaning needs to be done:

    1. For "State", removing characters from "[" to the end.
    2. For "RegionName", when applicable, removing every character from " (" to the end.
    3. Depending on how you read the data, you may need to remove newline character '\n'. '''
    uni_towns = pd.read_csv('university_towns.txt',sep='\n', header=None, names=['RegionName'])
    #Entries are seperated by new line and stop the first row entry from becoming a column name
   
    uni_towns['State'] = np.where(uni_towns['RegionName'].str.contains('edit'),uni_towns['RegionName'],np.NaN)
    # Entering name into State column where row RegionName is located and setting all other values to NaN
   
    uni_towns['State'].fillna(method='ffill',inplace=True)
    # Forward filling in State names to replace NaNs
    
    uni_towns = uni_towns[['State','RegionName']]
    # Rearranging order
    
    for col in uni_towns:
        uni_towns[col] = uni_towns[col].str.split('(',expand=True)[0].str.split('[', expand=True)[0].str.rstrip()
    # Splitting string into different columns and using both '(' and '[' instead of only '(' to avoid need for future debugging
    
    uni_towns = uni_towns[uni_towns['State'] != uni_towns['RegionName']] 
    # To remove rows where there are the same entries for State and Region Name
    
    return uni_towns
    
    
get_list_of_university_towns()
Out[39]:
State RegionName
1 Alabama Auburn
2 Alabama Florence
3 Alabama Jacksonville
4 Alabama Livingston
5 Alabama Montevallo
6 Alabama Troy
7 Alabama Tuscaloosa
8 Alabama Tuskegee
10 Alaska Fairbanks
12 Arizona Flagstaff
13 Arizona Tempe
14 Arizona Tucson
16 Arkansas Arkadelphia
17 Arkansas Conway
18 Arkansas Fayetteville
19 Arkansas Jonesboro
20 Arkansas Magnolia
21 Arkansas Monticello
22 Arkansas Russellville
23 Arkansas Searcy
25 California Angwin
26 California Arcata
27 California Berkeley
28 California Chico
29 California Claremont
30 California Cotati
31 California Davis
32 California Irvine
33 California Isla Vista
34 California University Park, Los Angeles
... ... ...
533 Virginia Wise
534 Virginia Chesapeake
536 Washington Bellingham
537 Washington Cheney
538 Washington Ellensburg
539 Washington Pullman
540 Washington University District, Seattle
542 West Virginia Athens
543 West Virginia Buckhannon
544 West Virginia Fairmont
545 West Virginia Glenville
546 West Virginia Huntington
547 West Virginia Montgomery
548 West Virginia Morgantown
549 West Virginia Shepherdstown
550 West Virginia West Liberty
552 Wisconsin Appleton
553 Wisconsin Eau Claire
554 Wisconsin Green Bay
555 Wisconsin La Crosse
556 Wisconsin Madison
557 Wisconsin Menomonie
558 Wisconsin Milwaukee
559 Wisconsin Oshkosh
560 Wisconsin Platteville
561 Wisconsin River Falls
562 Wisconsin Stevens Point
563 Wisconsin Waukesha
564 Wisconsin Whitewater
566 Wyoming Laramie

517 rows × 2 columns

In [40]:
def get_recession_start():
    GDP = pd.read_excel('gdplev.xls', skiprows=4)
    GDP = GDP.drop(GDP.columns[[0,1,2,3,5,7]],axis=1)
    # Remove unnessary rows and data
    
    new_header = GDP.iloc[0]
    GDP = GDP[3:]
    GDP.columns = new_header
    # Set first row header
    
    GDP = GDP.reset_index(drop=True)
    GDP.columns = ['Quarter','GDP']
    
    GDP = GDP.drop(GDP.index[0:212])
    # Remove date before first quarter 2000
    
    GDP = GDP.reset_index(drop=True)
    GDP['GDP Diff'] = GDP['GDP'].diff()
    # Difference between successive row entries for GDP column
    
    
    GDP_dec = GDP.where(GDP['GDP Diff']<0)
    GDP_dec = GDP_dec.dropna()
    # Find all quarters in decline
    
    GDP_dec['Index'] = GDP_dec.index 
    GDP_dec['Index Diff'] = GDP_dec['Index'].diff()
    min_index = GDP_dec['Index Diff'].idxmin()
    # Find first quarter with successive decline
    
    return GDP['Quarter'].iloc[min_index-1]
    # Find first quarter of the two successive quarters in decline

get_recession_start()
Out[40]:
'2008q3'
In [41]:
def get_recession_end():
    GDP = pd.read_excel("gdplev.xls", skiprows = 7)
    GDP = GDP.iloc[: ,[4,6]]
    GDP.rename(columns = {'Unnamed: 4': 'Quarter','Unnamed: 6':'GDP'}, inplace = True)
    GDP = GDP.iloc[212:]
    start_index = GDP[GDP['Quarter'] == get_recession_start()].index[0]
    GDP_new =  GDP.loc[start_index:]
    GDP_new['GDP'].astype(float,inplace = True)
    end = []
    for i in range(len(GDP_new)-2):
        if (GDP_new.iloc[i][1] < GDP_new.iloc[i+1][1]) and (GDP_new.iloc[i+1][1] < GDP_new.iloc[i+2][1]):
            end.append(GDP_new.iloc[i+2][0])
    return end[0]
get_recession_end()
Out[41]:
'2009q4'
In [42]:
def get_recession_bottom():
    GDP = pd.read_excel("gdplev.xls", skiprows = 7)
    GDP = GDP.iloc[: ,[4,6]]
    GDP.rename(columns = {'Unnamed: 4': 'Quarter','Unnamed: 6':'GDP'}, inplace = True)
    GDP = GDP.loc[212:]
    start_index = GDP[GDP['Quarter'] == get_recession_start()].index[0]
    stop_index = GDP[GDP['Quarter'] == get_recession_end()].index[0]
    bottom_index = GDP.loc[start_index:stop_index]['GDP'].argmin()
    return GDP.loc[249][0]
get_recession_bottom()
Out[42]:
'2009q2'
In [43]:
def convert_housing_data_to_quarters():
    house_df = pd.read_csv("City_Zhvi_AllHomes.csv")
    house_df['State'] = house_df['State'].map(states)
    house_df.set_index(['State','RegionName'], inplace =True)
    sel_col = house_df.columns[house_df.columns > '2000'][4:]
    house_df = house_df[sel_col]
    house_df.columns = pd.to_datetime(house_df.columns)
    house_df = house_df.resample('Q', axis = 1).mean()
    house_df.rename(columns = lambda x: str(x.to_period('Q')).lower(), inplace = True)
    return house_df
convert_housing_data_to_quarters()
Out[43]:
2000q1 2000q2 2000q3 2000q4 2001q1 2001q2 2001q3 2001q4 2002q1 2002q2 ... 2014q2 2014q3 2014q4 2015q1 2015q2 2015q3 2015q4 2016q1 2016q2 2016q3
State RegionName
New York New York NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 5.154667e+05 5.228000e+05 5.280667e+05 5.322667e+05 5.408000e+05 5.572000e+05 5.728333e+05 5.828667e+05 5.916333e+05 587200.0
California Los Angeles 2.070667e+05 2.144667e+05 2.209667e+05 2.261667e+05 2.330000e+05 2.391000e+05 2.450667e+05 2.530333e+05 2.619667e+05 2.727000e+05 ... 4.980333e+05 5.090667e+05 5.188667e+05 5.288000e+05 5.381667e+05 5.472667e+05 5.577333e+05 5.660333e+05 5.774667e+05 584050.0
Illinois Chicago 1.384000e+05 1.436333e+05 1.478667e+05 1.521333e+05 1.569333e+05 1.618000e+05 1.664000e+05 1.704333e+05 1.755000e+05 1.775667e+05 ... 1.926333e+05 1.957667e+05 2.012667e+05 2.010667e+05 2.060333e+05 2.083000e+05 2.079000e+05 2.060667e+05 2.082000e+05 212000.0
Pennsylvania Philadelphia 5.300000e+04 5.363333e+04 5.413333e+04 5.470000e+04 5.533333e+04 5.553333e+04 5.626667e+04 5.753333e+04 5.913333e+04 6.073333e+04 ... 1.137333e+05 1.153000e+05 1.156667e+05 1.162000e+05 1.179667e+05 1.212333e+05 1.222000e+05 1.234333e+05 1.269333e+05 128700.0
Arizona Phoenix 1.118333e+05 1.143667e+05 1.160000e+05 1.174000e+05 1.196000e+05 1.215667e+05 1.227000e+05 1.243000e+05 1.265333e+05 1.283667e+05 ... 1.642667e+05 1.653667e+05 1.685000e+05 1.715333e+05 1.741667e+05 1.790667e+05 1.838333e+05 1.879000e+05 1.914333e+05 195200.0
Nevada Las Vegas 1.326000e+05 1.343667e+05 1.354000e+05 1.370000e+05 1.395333e+05 1.417333e+05 1.433667e+05 1.461333e+05 1.493333e+05 1.509333e+05 ... 1.700667e+05 1.734000e+05 1.754667e+05 1.775000e+05 1.816000e+05 1.867667e+05 1.906333e+05 1.946000e+05 1.972000e+05 199950.0
California San Diego 2.229000e+05 2.343667e+05 2.454333e+05 2.560333e+05 2.672000e+05 2.762667e+05 2.845000e+05 2.919333e+05 3.012333e+05 3.128667e+05 ... 4.802000e+05 4.890333e+05 4.964333e+05 5.033667e+05 5.120667e+05 5.197667e+05 5.254667e+05 5.293333e+05 5.362333e+05 539750.0
Texas Dallas 8.446667e+04 8.386667e+04 8.486667e+04 8.783333e+04 8.973333e+04 8.930000e+04 8.906667e+04 9.090000e+04 9.256667e+04 9.380000e+04 ... 1.066333e+05 1.089000e+05 1.115333e+05 1.137000e+05 1.211333e+05 1.285667e+05 1.346000e+05 1.405000e+05 1.446000e+05 149300.0
California San Jose 3.742667e+05 4.065667e+05 4.318667e+05 4.555000e+05 4.706667e+05 4.702000e+05 4.568000e+05 4.455667e+05 4.414333e+05 4.577667e+05 ... 6.794000e+05 6.970333e+05 7.149333e+05 7.314333e+05 7.567333e+05 7.764000e+05 7.891333e+05 8.036000e+05 8.189333e+05 822200.0
Florida Jacksonville 8.860000e+04 8.970000e+04 9.170000e+04 9.310000e+04 9.440000e+04 9.560000e+04 9.706667e+04 9.906667e+04 1.012333e+05 1.034333e+05 ... 1.207667e+05 1.217333e+05 1.231667e+05 1.241667e+05 1.269000e+05 1.301333e+05 1.320000e+05 1.339667e+05 1.372000e+05 139900.0
California San Francisco 4.305000e+05 4.644667e+05 4.835333e+05 4.930000e+05 4.940667e+05 4.961333e+05 5.041000e+05 5.134000e+05 5.204333e+05 5.381667e+05 ... 9.269333e+05 9.545333e+05 9.687667e+05 1.000733e+06 1.060800e+06 1.095100e+06 1.105467e+06 1.121767e+06 1.119267e+06 1106400.0
Texas Austin 1.429667e+05 1.452667e+05 1.494667e+05 1.557333e+05 1.612333e+05 1.607333e+05 1.595333e+05 1.600333e+05 1.589667e+05 1.575000e+05 ... 2.488667e+05 2.528000e+05 2.581333e+05 2.665000e+05 2.750333e+05 2.816333e+05 2.872333e+05 2.935000e+05 3.014333e+05 304450.0
Michigan Detroit 6.616667e+04 6.830000e+04 6.676667e+04 6.703333e+04 6.750000e+04 6.836667e+04 6.926667e+04 6.996667e+04 7.100000e+04 7.233333e+04 ... 3.730000e+04 3.710000e+04 3.713333e+04 3.620000e+04 3.583333e+04 3.706667e+04 3.836667e+04 3.796667e+04 3.746667e+04 37900.0
Ohio Columbus 9.436667e+04 9.583333e+04 9.713333e+04 9.826667e+04 9.940000e+04 1.002667e+05 1.010667e+05 1.022000e+05 1.034000e+05 1.048000e+05 ... 1.031333e+05 1.045000e+05 1.064333e+05 1.078667e+05 1.094333e+05 1.115667e+05 1.150000e+05 1.167000e+05 1.182000e+05 120100.0
Tennessee Memphis 7.250000e+04 7.320000e+04 7.386667e+04 7.400000e+04 7.416667e+04 7.493333e+04 7.550000e+04 7.606667e+04 7.633333e+04 7.676667e+04 ... 6.810000e+04 6.910000e+04 7.116667e+04 7.053333e+04 6.870000e+04 6.866667e+04 6.953333e+04 7.090000e+04 7.416667e+04 75900.0
North Carolina Charlotte 1.269333e+05 1.283667e+05 1.302000e+05 1.315667e+05 1.329333e+05 1.332000e+05 1.328000e+05 1.331000e+05 1.343667e+05 1.353667e+05 ... 1.494667e+05 1.506333e+05 1.527333e+05 1.551667e+05 1.579000e+05 1.601667e+05 1.628667e+05 1.664667e+05 1.694333e+05 172400.0
Texas El Paso 7.626667e+04 7.686667e+04 7.673333e+04 7.730000e+04 7.823333e+04 7.830000e+04 7.743333e+04 7.680000e+04 7.660000e+04 7.640000e+04 ... 1.118000e+05 1.117333e+05 1.117667e+05 1.115000e+05 1.113000e+05 1.110667e+05 1.102667e+05 1.106667e+05 1.114667e+05 112200.0
Massachusetts Boston 2.069333e+05 2.191667e+05 2.331000e+05 2.425000e+05 2.496000e+05 2.570667e+05 2.669333e+05 2.749667e+05 2.825000e+05 2.893000e+05 ... 4.266667e+05 4.314333e+05 4.407333e+05 4.485000e+05 4.553667e+05 4.639667e+05 4.716333e+05 4.826000e+05 4.903667e+05 501700.0
Washington Seattle 2.486000e+05 2.556000e+05 2.625333e+05 2.674000e+05 2.710000e+05 2.724333e+05 2.741667e+05 2.781667e+05 2.805000e+05 2.846000e+05 ... 4.418000e+05 4.515000e+05 4.591667e+05 4.679333e+05 4.933667e+05 5.142667e+05 5.334667e+05 5.517333e+05 5.755333e+05 589700.0
Maryland Baltimore 5.966667e+04 5.950000e+04 5.883333e+04 5.950000e+04 5.956667e+04 6.013333e+04 6.210000e+04 6.340000e+04 6.366667e+04 6.490000e+04 ... 1.092333e+05 1.095333e+05 1.073667e+05 1.080667e+05 1.114333e+05 1.139667e+05 1.139000e+05 1.146667e+05 1.147333e+05 115150.0
Colorado Denver 1.622333e+05 1.678333e+05 1.743333e+05 1.803333e+05 1.865000e+05 1.925333e+05 1.964000e+05 1.991000e+05 2.012333e+05 2.024333e+05 ... 2.708667e+05 2.775000e+05 2.872333e+05 2.976333e+05 3.103667e+05 3.205000e+05 3.301000e+05 3.355667e+05 3.427667e+05 351550.0
District of Columbia Washington 1.377667e+05 1.442000e+05 1.487000e+05 1.477000e+05 1.497667e+05 1.551333e+05 1.646333e+05 1.725333e+05 1.805000e+05 1.933000e+05 ... 4.469333e+05 4.530000e+05 4.603000e+05 4.661667e+05 4.810667e+05 4.934000e+05 5.009000e+05 5.041000e+05 5.058000e+05 516250.0
Tennessee Nashville 1.138333e+05 1.152667e+05 1.158667e+05 1.169333e+05 1.180333e+05 1.191667e+05 1.201000e+05 1.208000e+05 1.215667e+05 1.226333e+05 ... 1.607000e+05 1.623000e+05 1.669000e+05 1.714667e+05 1.762667e+05 1.818000e+05 1.892000e+05 1.950667e+05 2.003667e+05 206100.0
Wisconsin Milwaukee 7.803333e+04 7.906667e+04 8.103333e+04 8.233333e+04 8.403333e+04 8.556667e+04 8.706667e+04 8.840000e+04 8.953333e+04 9.136667e+04 ... 9.216667e+04 9.216667e+04 9.196667e+04 9.333333e+04 9.410000e+04 9.413333e+04 9.456667e+04 9.466667e+04 9.636667e+04 98850.0
Arizona Tucson 1.018333e+05 1.029667e+05 1.044667e+05 1.056667e+05 1.072000e+05 1.087667e+05 1.105667e+05 1.128000e+05 1.150000e+05 1.172000e+05 ... 1.424667e+05 1.434333e+05 1.442333e+05 1.441667e+05 1.451333e+05 1.466000e+05 1.481667e+05 1.495333e+05 1.511667e+05 152700.0
Oregon Portland 1.528000e+05 1.547667e+05 1.565667e+05 1.574667e+05 1.599000e+05 1.618000e+05 1.642667e+05 1.677667e+05 1.707667e+05 1.741333e+05 ... 2.822333e+05 2.872667e+05 2.955333e+05 3.019333e+05 3.119000e+05 3.257333e+05 3.430667e+05 3.560000e+05 3.698000e+05 387050.0
Oklahoma Oklahoma City 7.643333e+04 7.750000e+04 7.856667e+04 7.916667e+04 7.983333e+04 8.040000e+04 8.113333e+04 8.173333e+04 8.260000e+04 8.343333e+04 ... 1.180333e+05 1.189667e+05 1.201000e+05 1.208000e+05 1.223667e+05 1.247000e+05 1.271000e+05 1.279000e+05 1.293000e+05 130300.0
Nebraska Omaha 1.128000e+05 1.141000e+05 1.167333e+05 1.189000e+05 1.208667e+05 1.197667e+05 1.178667e+05 1.174000e+05 1.180667e+05 1.176333e+05 ... 1.301000e+05 1.303000e+05 1.325000e+05 1.330667e+05 1.344667e+05 1.367333e+05 1.400667e+05 1.416333e+05 1.426667e+05 143450.0
New Mexico Albuquerque 1.258667e+05 1.267000e+05 1.264333e+05 1.267333e+05 1.271000e+05 1.277333e+05 1.285667e+05 1.299000e+05 1.310667e+05 1.321000e+05 ... 1.632667e+05 1.640000e+05 1.648000e+05 1.651667e+05 1.659000e+05 1.665333e+05 1.673333e+05 1.691000e+05 1.706333e+05 171900.0
California Fresno 9.410000e+04 9.526667e+04 9.646667e+04 9.823333e+04 1.005667e+05 1.035667e+05 1.072333e+05 1.103000e+05 1.140333e+05 1.185333e+05 ... 1.696333e+05 1.736000e+05 1.781333e+05 1.804667e+05 1.820333e+05 1.857000e+05 1.874667e+05 1.890333e+05 1.927333e+05 196450.0
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
Texas Granite Shoals NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.169667e+05 1.175333e+05 1.175333e+05 1.171667e+05 1.191000e+05 1.216000e+05 1.280000e+05 1.337667e+05 1.400667e+05 146450.0
Maryland Piney Point 1.556667e+05 1.551667e+05 1.584667e+05 1.637000e+05 1.634000e+05 1.648333e+05 1.647000e+05 1.679000e+05 1.782667e+05 1.812000e+05 ... 2.964000e+05 3.090000e+05 3.092333e+05 3.095667e+05 3.017000e+05 3.052333e+05 3.099667e+05 3.195000e+05 3.241667e+05 324600.0
Wisconsin Maribel NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.306000e+05 1.289667e+05 1.296333e+05 1.312667e+05 1.301333e+05 1.297333e+05 1.293000e+05 1.278333e+05 1.292667e+05 134200.0
Idaho Middleton 1.060667e+05 1.043333e+05 1.019000e+05 1.041667e+05 1.061667e+05 1.083667e+05 1.110333e+05 1.112333e+05 1.141000e+05 1.141667e+05 ... 1.443667e+05 1.457000e+05 1.462333e+05 1.461667e+05 1.477333e+05 1.482000e+05 1.511333e+05 1.539000e+05 1.571667e+05 160750.0
Colorado Bennett 1.329000e+05 1.358333e+05 1.398000e+05 1.446667e+05 1.483000e+05 1.521000e+05 1.542333e+05 1.562000e+05 1.587333e+05 1.606333e+05 ... 1.514667e+05 1.620667e+05 1.714000e+05 1.780333e+05 1.844333e+05 1.916667e+05 1.958000e+05 1.997667e+05 2.074667e+05 212600.0
New Hampshire East Hampstead 1.618333e+05 1.691000e+05 1.739667e+05 1.805000e+05 1.909000e+05 1.950667e+05 1.992667e+05 2.074000e+05 2.123000e+05 2.122333e+05 ... 2.495000e+05 2.521000e+05 2.557333e+05 2.587333e+05 2.613667e+05 2.616000e+05 2.688000e+05 2.725333e+05 2.778000e+05 282450.0
Missouri Garden City NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.055000e+05 1.043000e+05 1.047667e+05 1.060333e+05 9.606667e+04 9.930000e+04 1.034333e+05 1.062667e+05 1.116667e+05 113600.0
Arkansas Mountainburg 5.716667e+04 6.433333e+04 6.783333e+04 6.900000e+04 6.866667e+04 6.386667e+04 6.376667e+04 6.546667e+04 6.533333e+04 6.600000e+04 ... 8.160000e+04 8.506667e+04 8.846667e+04 8.903333e+04 8.556667e+04 8.370000e+04 9.043333e+04 9.833333e+04 1.019000e+05 103400.0
Wisconsin Oostburg 1.072667e+05 1.081000e+05 1.124333e+05 1.155000e+05 1.191000e+05 1.204333e+05 1.203667e+05 1.196333e+05 1.198667e+05 1.185667e+05 ... 1.295667e+05 1.279333e+05 1.274333e+05 1.270667e+05 1.274000e+05 1.303333e+05 1.320333e+05 1.327667e+05 1.341000e+05 136350.0
California Twin Peaks 9.736667e+04 1.001667e+05 1.013333e+05 1.017000e+05 1.040000e+05 1.076667e+05 1.098333e+05 1.111333e+05 1.132000e+05 1.166000e+05 ... 1.501000e+05 1.475333e+05 1.460667e+05 1.435000e+05 1.523000e+05 1.552667e+05 1.591667e+05 1.641667e+05 1.679667e+05 173500.0
New York Upper Brookville 1.230967e+06 1.230967e+06 1.237700e+06 1.261567e+06 1.295167e+06 1.340033e+06 1.403667e+06 1.481933e+06 1.536167e+06 1.562033e+06 ... 1.780633e+06 1.749233e+06 1.729467e+06 1.749867e+06 1.789600e+06 1.777267e+06 1.834367e+06 1.904500e+06 1.944067e+06 1968800.0
Hawaii Volcano 9.870000e+04 1.053667e+05 1.146667e+05 1.247667e+05 1.181333e+05 1.194000e+05 1.232667e+05 1.211667e+05 1.233000e+05 1.169000e+05 ... 2.064667e+05 2.276333e+05 2.332000e+05 2.346333e+05 2.323667e+05 2.249667e+05 2.324333e+05 2.420667e+05 2.489667e+05 247850.0
South Carolina Wedgefield NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 7.436667e+04 7.026667e+04 7.206667e+04 7.570000e+04 7.206667e+04 7.033333e+04 6.903333e+04 6.886667e+04 7.426667e+04 80700.0
Michigan Williamston 1.591667e+05 1.613000e+05 1.643000e+05 1.662000e+05 1.664333e+05 1.686333e+05 1.716667e+05 1.750333e+05 1.786667e+05 1.793333e+05 ... 1.657000e+05 1.689333e+05 1.708667e+05 1.744333e+05 1.758667e+05 1.794667e+05 1.823000e+05 1.814667e+05 1.824000e+05 183000.0
Arkansas Decatur 6.360000e+04 6.440000e+04 6.566667e+04 6.673333e+04 6.720000e+04 6.770000e+04 6.650000e+04 6.540000e+04 6.460000e+04 6.490000e+04 ... 8.966667e+04 9.256667e+04 9.470000e+04 9.350000e+04 9.490000e+04 9.543333e+04 9.700000e+04 9.650000e+04 9.663333e+04 96850.0
Tennessee Briceville 4.000000e+04 4.173333e+04 4.366667e+04 4.490000e+04 4.480000e+04 4.530000e+04 4.463333e+04 4.370000e+04 4.446667e+04 4.340000e+04 ... 5.623333e+04 5.423333e+04 5.260000e+04 4.963333e+04 4.590000e+04 4.793333e+04 4.360000e+04 4.080000e+04 4.180000e+04 40850.0
Indiana Edgewood 9.170000e+04 9.186667e+04 9.293333e+04 9.490000e+04 9.893333e+04 1.000667e+05 1.008333e+05 1.010000e+05 1.021667e+05 1.017667e+05 ... 9.213333e+04 9.406667e+04 9.466667e+04 9.586667e+04 9.433333e+04 9.663333e+04 9.996667e+04 9.943333e+04 9.996667e+04 100950.0
Tennessee Palmyra NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.227667e+05 1.269333e+05 1.262333e+05 1.223000e+05 1.204667e+05 1.198000e+05 1.258000e+05 1.276667e+05 1.328667e+05 137750.0
Maryland Saint Inigoes 1.480667e+05 1.476000e+05 1.572333e+05 1.633667e+05 1.642333e+05 1.682000e+05 1.665000e+05 1.653333e+05 1.673000e+05 1.688000e+05 ... 2.822333e+05 2.884333e+05 2.869667e+05 2.847000e+05 2.807667e+05 2.778333e+05 2.768333e+05 2.793333e+05 2.826333e+05 281400.0
Indiana Marysville NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.166000e+05 1.151000e+05 1.165000e+05 1.118667e+05 1.118000e+05 1.156667e+05 1.201667e+05 1.282333e+05 1.232333e+05 124200.0
California Forest Falls 1.135333e+05 1.144000e+05 1.141667e+05 1.111333e+05 1.134333e+05 1.130000e+05 1.130333e+05 1.151667e+05 1.187000e+05 1.250667e+05 ... 1.653667e+05 1.675000e+05 1.771000e+05 1.765333e+05 1.818000e+05 1.911667e+05 1.987333e+05 1.886333e+05 1.898667e+05 186650.0
Missouri Bois D Arc 1.078000e+05 1.069667e+05 1.071000e+05 1.081000e+05 1.107000e+05 1.136667e+05 1.126333e+05 1.127333e+05 1.130667e+05 1.154000e+05 ... 1.375667e+05 1.375667e+05 1.404000e+05 1.450333e+05 1.475667e+05 1.463000e+05 1.494333e+05 1.468667e+05 1.437667e+05 144000.0
Virginia Henrico 1.285667e+05 1.307667e+05 1.322667e+05 1.332667e+05 1.352333e+05 1.367333e+05 1.386000e+05 1.413333e+05 1.435333e+05 1.461333e+05 ... 2.016333e+05 2.040000e+05 2.059000e+05 2.065667e+05 2.104333e+05 2.121000e+05 2.139667e+05 2.160333e+05 2.162000e+05 220150.0
New Jersey Diamond Beach 1.739667e+05 1.831000e+05 1.889667e+05 1.931333e+05 1.944000e+05 2.102667e+05 2.302667e+05 2.486667e+05 2.599333e+05 2.656333e+05 ... 3.818000e+05 3.878667e+05 3.876667e+05 3.931667e+05 3.980000e+05 3.992333e+05 4.004333e+05 4.045333e+05 4.039000e+05 399000.0
Tennessee Gruetli Laager 3.540000e+04 3.546667e+04 3.666667e+04 3.730000e+04 3.773333e+04 3.790000e+04 3.936667e+04 4.040000e+04 4.156667e+04 4.163333e+04 ... 5.556667e+04 5.636667e+04 5.713333e+04 5.890000e+04 6.536667e+04 6.950000e+04 7.170000e+04 7.533333e+04 7.646667e+04 77500.0
Wisconsin Town of Wrightstown 1.017667e+05 1.054000e+05 1.113667e+05 1.148667e+05 1.259667e+05 1.299000e+05 1.299000e+05 1.294333e+05 1.319000e+05 1.342000e+05 ... 1.448667e+05 1.468667e+05 1.492333e+05 1.486667e+05 1.493333e+05 1.498667e+05 1.499333e+05 1.498333e+05 1.512667e+05 155000.0
New York Urbana 7.920000e+04 8.166667e+04 9.170000e+04 9.836667e+04 9.486667e+04 9.853333e+04 1.029667e+05 9.803333e+04 9.396667e+04 9.460000e+04 ... 1.321333e+05 1.370333e+05 1.400667e+05 1.417000e+05 1.378667e+05 1.364667e+05 1.361667e+05 1.389667e+05 1.442000e+05 143000.0
Wisconsin New Denmark 1.145667e+05 1.192667e+05 1.260667e+05 1.319667e+05 1.438000e+05 1.469667e+05 1.483667e+05 1.491667e+05 1.531333e+05 1.567333e+05 ... 1.745667e+05 1.811667e+05 1.861667e+05 1.876000e+05 1.886667e+05 1.884333e+05 1.889333e+05 1.910667e+05 1.928333e+05 197600.0
California Angels 1.510000e+05 1.559000e+05 1.581000e+05 1.674667e+05 1.768333e+05 1.837667e+05 1.902333e+05 1.845667e+05 1.840333e+05 1.861333e+05 ... 2.444667e+05 2.540667e+05 2.599333e+05 2.601000e+05 2.506333e+05 2.635000e+05 2.795000e+05 2.765333e+05 2.716000e+05 269950.0
Wisconsin Holland 1.510333e+05 1.505000e+05 1.532333e+05 1.558333e+05 1.618667e+05 1.657333e+05 1.680333e+05 1.674000e+05 1.657667e+05 1.619667e+05 ... 2.012667e+05 2.015667e+05 2.012667e+05 2.060000e+05 2.076000e+05 2.128667e+05 2.178333e+05 2.219667e+05 2.280333e+05 234950.0

10730 rows × 67 columns

In [44]:
def run_ttest():
    df = convert_housing_data_to_quarters()[['2008q1',get_recession_bottom()]]
    df['Ratio'] = df['2008q1']/df['2009q2']
    df['Ratio'].dropna(inplace = True)
    univ_towns = get_list_of_university_towns()
    df_new  = df['Ratio'].reset_index()
    df_univ = df_new[(df_new['State'].isin(univ_towns['State'].values)) & (df_new['RegionName'].isin(univ_towns['RegionName'].values))]
    df_nonuniv = df_new[~ ((df_new['State'].isin(univ_towns['State'].values)) & (df_new['RegionName'].isin(univ_towns['RegionName'].values)))]
    def better():
        if df_univ['Ratio'].mean() < df_nonuniv['Ratio'].mean():
            x = "university town"
        else:
            x = "non-university town"
        return x
    p = list(ttest_ind(df_univ['Ratio'],df_nonuniv['Ratio']))[1]
    return (True,p,better())
run_ttest()
Out[44]:
(True, 3.9204110809200024e-05, 'university town')
In [ ]: