MCPcopy Index your code
hub / github.com/gto76/python-cheatsheet / scrape_data

Function scrape_data

web/update_plots.py:51–63  ·  view source on GitHub ↗
()

Source from the content-addressed store, hash-verified

49 write_to_png_file('covid_cases.png', f, width=960, height=315)
50
51 def scrape_data():
52 def scrape_covid():
53 url = 'https://covid.ourworldindata.org/data/owid-covid-data.csv'
54 df = pd.read_csv(url, usecols=['location', 'date', 'total_cases'])
55 return df[df.location == 'World'].set_index('date').total_cases
56 def scrape_yahoo(slug):
57 url = f'https://query1.finance.yahoo.com/v7/finance/download/{slug}' + \
58 '?period1=1579651200&period2=9999999999&interval=1d&events=history'
59 df = pd.read_csv(url, usecols=['Date', 'Close'])
60 return df.set_index('Date').Close
61 out = [scrape_covid(), scrape_yahoo('BTC-USD'), scrape_yahoo('GC=F'),
62 scrape_yahoo('^DJI')]
63 return map(pd.Series.rename, out, ['Total Cases', 'Bitcoin', 'Gold', 'Dow Jones'])
64
65 def wrangle_data(covid, bitcoin, gold, dow):
66 df = pd.concat([dow, gold, bitcoin], axis=1) # Joins columns on dates.

Callers 1

mainFunction · 0.85

Calls 2

scrape_covidFunction · 0.85
scrape_yahooFunction · 0.85

Tested by

no test coverage detected