Tamkang

金融科技 University
FinTech: Financial Technology
Python Pandas財務大數據分析
(Finance Big Data Analytics with
Pandas in Python)
1052FinTech07
MIS EMBA (M2263) (8595)
Fri, 12,13,14 (19:20-22:10) (D409)

Min-Yuh Day
戴敏育
Assistant Professor
專任助理教授
Dept. of Information Management, Tamkang University
淡江大學 資訊管理學系
http://mail. tku.edu.tw/myday/
2017-04-28 1
 課程大綱 (Syllabus)
週次 (Week) 日期 (Date) 內容 (Subject/Topics)
1 2017/02/17 Fintech 金融科技課程介紹
(Course Orientation for Fintech: Financial Technology)
2 2017/02/24 Fintech 金融科技的演進：貨幣與金融服務
(Evolution of Fintech: Money and Financial Services)
3 2017/03/03 Fintech 金融科技：金融服務科技創新
(Fintech: Technology Innovation in Financial Services)
4 2017/03/10 Fintech 金融科技與金融服務價值鏈
(Fintech and Financial Services Value Chain)
5 2017/03/17 Fintech 金融科技商業模式創新
(Fintech Business Models Innovation)
6 2017/03/24 Fintech 金融科技個案研究 I
(Case Study on Fintech I)
2
 課程大綱 (Syllabus)
週次 (Week) 日期 (Date) 內容 (Subject/Topics)
7 2017/03/31 金融服務消費者心理與行為
(Consumer Psychology and Behavior on Financial Services)
8 2017/04/07 教學行政觀摩日 (Off-campus study)
9 2017/04/14 區塊鏈技術
(Blockchain Technology)
[Invited Speaker: Dr. Raymund Lin, IBM (林俊叡 博士，IBM)]
10 2017/04/21 期中報告 (Midterm Project Report)
11 2017/04/28 Python Pandas財務大數據分析
(Finance Big Data Analytics with Pandas in Python)
12 2017/05/05 人工智慧與深度學習金融科技
(Artificial Intelligence and Deep Learning for Fintech)

3
 課程大綱 (Syllabus)
週次 (Week) 日期 (Date) 內容 (Subject/Topics)
13 2017/05/12 Fintech 金融科技個案研究 II
(Case Study on Fintech II)
14 2017/05/19 金融科技財富管理：機器人理財顧問
(Robo-Advisors for Wealth Management in Fintech)
15 2017/05/26 投資組合最佳化與程式交易
(Portfolio Optimization and Algorithmic Trading)
16 2017/06/02 金融科技智慧問答系統
(Intelligent Question Answering System for Fintech)
17 2017/06/09 期末報告 I (Final Project Presentation I)
18 2017/06/16 期末報告 II (Final Project Presentation II)

4
Finance Big Data
Analytics
with Pandas in
Python
5
Source: https://www.python.org/community/logos/
http://pandas.pydata.org/ 6
Yahoo Finance

http://finance.yahoo.com/ 7
Apple Inc. (AAPL) -NasdaqGS

http://finance.yahoo.com/quote/AAPL?p=AAPL 8
Yahoo Finance Charts: Apple Inc. (AAPL)

http://finance.yahoo.com/chart/AAPL 9
Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 10
 Wes McKinney (2012),
Python for Data Analysis: Data Wrangling with
Pandas, NumPy, and IPython, O'Reilly Media

Source: http://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1449319793/ 11
 Yves Hilpisch,
Python for Finance: Analyze Big Financial Data,
O'Reilly, 2014

Source: http://www.amazon.com/Python-Finance-Analyze-Financial-Data/dp/1491945281 12
 Yves Hilpisch (2015),
Derivatives Analytics with Python:
Data Analysis, Models, Simulation, Calibration and Hedging, Wiley

Source: http://www.amazon.com/Derivatives-Analytics-Python-Simulation-Calibration/dp/1119037999/ 13
 Michael Heydt ,
Mastering Pandas for Finance,
Packt Publishing, 2015

Source: http://www.amazon.com/Mastering-Pandas-Finance-Michael-Heydt/dp/1783985100 14
Python for Big Data Analytics

Source: http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages 15
Python: Analytics and Data Science Software

Source: http://www.kdnuggets.com/2016/06/r-python-top-analytics-data-mining-data-science-software.html 16
Anaconda

https://www.continuum.io/ 17
Leading Open
Data Science
Platform
Powered by
Python
Source: https://www.continuum.io/ 18
 Python

https://www.python.org/ 19
 Python is an
interpreted,
object-oriented,
high-level
programming language
with
dynamic semantics.
Source: https://www.python.org/doc/essays/blurb/ 20
 Python versions (py2 and py3)
• Python 0.9.0 released in 1991 (first release)
• Python 1.0 released in 1994
• Python 2.0 released in 2000
• Python 2.6 released in 2008
• Python 2.7 released in 2010
• Python 3.0 released in 2008
• Python 3.3 released in 2010
• Python 3.4 released in 2014
• Python 3.5 released in 2015
• Python 3.6 released in 2016
Source: Yves Hilpisch (2014), Python for Finance: Analyze Big Financial Data, O'Reilly 21
 Python (Python 2.7 & Python 3.6)
Standard Syntax
Python 2 Python 3

Source: PyCon Australia (2014), Writing Python 2/3 compatible code by Edward Schofield
https://www.youtube.com/watch?v=KOqk8j11aAI 22
 from __ future __ import ...

Python 2 Python 3

Source: PyCon Australia (2014), Writing Python 2/3 compatible code by Edward Schofield
https://www.youtube.com/watch?v=KOqk8j11aAI 23
 from future.builtins import *

Python 2 Python 3

Source: PyCon Australia (2014), Writing Python 2/3 compatible code by Edward Schofield
https://www.youtube.com/watch?v=KOqk8j11aAI 24
 from past.builtins import *

Python 2 Python 3

Source: PyCon Australia (2014), Writing Python 2/3 compatible code by Edward Schofield
https://www.youtube.com/watch?v=KOqk8j11aAI 25
 NumPy

http://www.numpy.org/ 26
 NumPy
is the
fundamental package
for
scientific computing
with Python.
Source: http://www.numpy.org/ 27
pandas

http://pandas.pydata.org/ 28
 pandas
Python Data Analysis
Library
providing high-performance, easy-to-use
data structures and data analysis tools
for the Python programming language.
Source: http://pandas.pydata.org/ 29
 pandas Ecosystem
• Statistics and Machine Learning • API
– Statsmodels – pandas-datareader
– sklearn-pandas – quandl/Python
• Visualization
– pydatastream
– Bokeh
– pandaSDMX
– yhat/ggplot
– fredapi
– Seaborn
– Vincent • Domain Specific
– IPython Vega – Geopandas
– Plotly – xarray
– Pandas-Qt • Out-of-core
• IDE – Dask
– IPython – Blaze
– quantopian/qgrid – Odo
– Spyder
Source: http://pandas.pydata.org/pandas-docs/stable/index.html 30
pandas: powerful Python data analysis toolkit

http://pandas.pydata.org/pandas-docs/stable/ 31
 pandas:
powerful Python data analysis toolkit
• Tabular data with
heterogeneously-typed columns,
as in an SQL table or Excel spreadsheet
• Ordered and unordered (not necessarily fixed-
frequency) time series data.
• Arbitrary matrix data (homogeneously typed or
heterogeneous) with row and column labels
• Any other form of observational / statistical data
sets. The data actually need not be labeled at all to
be placed into a pandas data structure
Source: http://pandas.pydata.org/pandas-docs/stable/ 32
 Series
DataFrame
• Primary data structures of pandas
–Series (1-dimensional)
–DataFrame (2-dimensional)
• Handle the vast majority of typical use
cases in finance, statistics, social
science, and many areas of
engineering.
Source: http://pandas.pydata.org/pandas-docs/stable/ 33
 pandas DataFrame
• DataFrame provides everything that
R’s data.frame provides and much
more.
• pandas is built on top of NumPy and
is intended to integrate well within a
scientific computing environment
with many other 3rd party libraries.
34
 pandas
Comparison with SAS
pandas SAS
DataFrame data set
column variable
row observation
groupby BY-group
NaN .
Source: http://pandas.pydata.org/pandas-docs/stable/comparison_with_sas.html 35
Python Pandas Cheat Sheet

Source: https://github.com/pandas-dev/pandas/blob/master/doc/cheatsheet/Pandas_Cheat_Sheet.pdf
36
Python
37
Download Anaconda

https://www.continuum.io/downloads 38
Download Anaconda Python 3.6

https://www.continuum.io/downloads 39
 OS X Anaconda Python 3.6
Installation
Command Line Installer
Download the command-line installer

In your terminal window type one of the below

and follow the instructions:

Python 3.6 version

bash Anaconda3-4.3.1-MacOSX-x86_64.sh

Python 2.7 version

bash Anaconda2-4.3.1-MacOSX-x86_64.sh
https://www.continuum.io/downloads 40
 OS X Anaconda 3 - 4.3.1
Python 3.6 Installation
Anaconda3-4.3.1-MacOSX-x86_64.pkg

Installer package 41
Install Anaconda 3

42
Install Anaconda 3

43
Install Anaconda 3

44
Install Anaconda 3

45
Install Anaconda 3

46
Install Anaconda 3

47
Install Anaconda 3

48
Install Anaconda 3

49
Install Anaconda 3

178 python packages

included.
Supported packages:
453

Source: https://docs.continuum.io/anaconda/pkg-docs 50
Install Anaconda 3

178
python
packages
included.
51
Anaconda-Navigator

Launchpad

52
Anaconda-Navigator

53
Jupyter Notebook

54
Jupyter Notebook
New Python 3

55
print(“hello, world”)

56
from platform import python_version
print("Python Version:", python_version())

57
 Create Python Environments
with Anaconda
• Python 3.6
• Python 3.5
–Python 3.5.3
–Python 3.5.2
• Python 2.7

https://conda.io/docs/py2or3.html 58
Anaconda Create New Python 3.5
Environment (py35)

py35
Python 3.5

Source: http://conda.pydata.org/docs/py2or3.html 59
Anaconda Create New Python 2.7
Environment (py27)

py35
Python 3.5

py27
Python 2.7

Source: http://conda.pydata.org/docs/py2or3.html 60
 Verify that conda is installed,
check current conda version

• conda --version
• Update conda to the current version
– conda update conda

http://conda.pydata.org/docs/using/using.html#verify-that-conda-is-installed-check-current-conda-version 61
 Check current conda version
Check current python version
Check conda environments
• conda --version
• python --version
• conda info --envs

http://conda.pydata.org/docs/using/using.html#verify-that-conda-is-installed-check-current-conda-version 62
 Terminal

terminal

63
conda list

http://conda.pydata.org/docs/using/using.html#verify-that-conda-is-installed-check-current-conda-version 64
python --version

65
conda --version
python --version
conda --version
conda info --envs

source activate py35

source deactivate py35

66
conda create -n py352 python=3.5.2 anaconda

Create a
Python 3.5.2
environment
Source: http://conda.pydata.org/docs/py2or3.html 67
conda create -n py352 python=3.5.2 anaconda

source activate py352

Source: http://conda.pydata.org/docs/py2or3.html 68
conda info --envs

69
conda info --envs
source activate py27
python --version
conda install notebook ipykernel

jupyter notebook

70
source activate py27
conda install notebook ipykernel

conda info --envs

source activate py27
python --version
conda install notebook ipykernel
jupyter notebook

71
jupyter notebook

jupyter notebook
ipython notebook

72
jupyter notebook

73
Jupyter Notebook
New Python 2

74
print “hello, world”

75
from platform import python_version
print "Python Version:", python_version()

76
Source: https://www.python.org/community/logos/ 77
Python Fiddle

http://pythonfiddle.com/ 78
 Text input and output
print("Hello World")

print("Hello World\nThis is a message")

x = 3
print(x)

x = 2
y = 3
print(x, ' ', y)

name = input("Enter a name: ")

x = int(input("What is x? "))

x = float(input("Write a number "))

Source: http://pythonprogramminglanguage.com/text-input-and-output/ 79
 Text input and output

Source: http://pythonprogramminglanguage.com/text-input-and-output/ 80
 Variables
x = 2
price = 2.5
word = 'Hello'
word = 'Hello'
word = "Hello"
word = '''Hello'''

x = 2
x = x + 1
x = 5
Source: http://pythonprogramminglanguage.com/ 81
 Python Basic Operators
print('7 + 2 =', 7 + 2)
print('7 - 2 =', 7 - 2)
print('7 * 2 =', 7 * 2)
print('7 / 2 =', 7 / 2)
print('7 // 2 =', 7 // 2)
print('7 % 2 =', 7 % 2)
print('7 ** 2 =', 7 ** 2)

82
 BMI Calculator in Python

height_cm = float(input("Enter your height in cm: "))

weight_kg = float(input("Enter your weight in kg: "))

height_m = height_cm/100
BMI = (weight_kg/(height_m**2))

print("Your BMI is: " + str(round(BMI,1)))

Source: http://code.activestate.com/recipes/580615-bmi-code/ 83
BMI Calculator in Python

Source: http://code.activestate.com/recipes/580615-bmi-code/ 84
 If statements
> greater than
< smaller than
== equals
!= is not

score = 80
if score >=60 :
print("Pass")
else:
print("Fail")
Source: http://pythonprogramminglanguage.com/ 85
 For loops
for i in range(1,11):
print(i)
1
2
3
4
5
6
7
8
9
10
Source: http://pythonprogramminglanguage.com/ 86
 For loops
for i in range(1,10):
for j in range(1,10):
print(i, ' * ' , j , ' = ', i*j)
9 * 1 = 9
9 * 2 = 18
9 * 3 = 27
9 * 4 = 36
9 * 5 = 45
9 * 6 = 54
9 * 7 = 63
9 * 8 = 72
9 * 9 = 81

Source: http://pythonprogramminglanguage.com/ 87
 Functions
def convertCMtoM(xcm):
m = xcm/100
return m

cm = 180
m = convertCMtoM(cm)
print(str(m))
1.8
88
 Lists
x = [60, 70, 80, 90]
print(len(x))
print(x[0])
print(x[1])
print(x[-1])
4
60
70
90
89
 Tuples
A tuple in Python is a collection that
cannot be modified.
A tuple is defined using parenthesis.
x = (10, 20, 30, 40, 50)
print(x[0]) 10
print(x[1]) 20
print(x[2]) 30
print(x[-1]) 50
Source: http://pythonprogramminglanguage.com/tuples/ 90
 Dictionary
k = { 'EN':'English', 'FR':'French' }
print(k['EN'])

English

Source: http://pythonprogramminglanguage.com/dictionary/ 91
 Sets
animals = {'cat', 'dog'}

animals = {'cat', 'dog'}

print('cat' in animals)
print('fish' in animals)
animals.add('fish')
print('fish' in animals)
print(len(animals))
animals.add('cat')
print(len(animals))
animals.remove('cat')
print(len(animals))

Source: http://cs231n.github.io/python-numpy-tutorial/ 92
 Numpy

NumPy
Base
N-dimensional array
package
93
NumPy
NumPy
• NumPy provides a
multidimensional array object
to store homogenous or
heterogeneous data; it also
provides optimized
functions/methods to operate on
this array object.
Source: Yves Hilpisch (2014), Python for Finance: Analyze Big Financial Data, O'Reilly 94
NumPy
NumPy

v = range(1, 6)
print(v)
2 * v
import numpy as np
v = np.arange(1, 6)
v
2 * v
Source: Yves Hilpisch (2014), Python for Finance: Analyze Big Financial Data, O'Reilly 95
NumPy
Base
N-dimensional
array package

96
NumPy
NumPy
import numpy as np
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
c = a * b
c

Source: Yves Hilpisch (2014), Python for Finance: Analyze Big Financial Data, O'Reilly 97
 NumPy
NumPy

Source: http://cs231n.github.io/python-numpy-tutorial/ 98
 Python
Pandas for
Finance 99
Jupyter Notebook New Python 3

100
 Creating pd.DataFrame

a b c
1 4 7 10
2 5 8 11
3 6 9 12

df = pd.DataFrame({"a": [4, 5, 6],

"b": [7, 8, 9],
"c": [10, 11, 12]},
index = [1, 2, 3])

Source: https://github.com/pandas-dev/pandas/blob/master/doc/cheatsheet/Pandas_Cheat_Sheet.pdf
101
 Pandas DataFrame

type(df)

102
conda install pandas-datareader

103
Jupyter Notebook New Python 3

104
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
print('Hello Pandas')

s = pd.Series([1,3,5,np.nan,6,8])
s

dates = pd.date_range('20170301',
periods=6)
dates
Source: http://pandas.pydata.org/pandas-docs/stable/10min.html 105
106
df = pd.DataFrame(np.random.randn(6,4),
index=dates, columns=list('ABCD'))
df

107
df = pd.DataFrame(np.random.randn(4,6),
index=['student1','student2','student3',
'student4'], columns=list('ABCDEF'))
df

108
df2 = pd.DataFrame({ 'A' : 1.,
'B' : pd.Timestamp('20170322'),
'C' : pd.Series(2.5,index=list(range(4)),dtype='float32'),
'D' : np.array([3] * 4,dtype='int32'),
'E' : pd.Categorical(["test","train","test","train"]),
'F' : 'foo' })
df2

109
df2.dtypes

110
Yahoo Finance Symbols: AAPL
Apple Inc. (AAPL)

http://finance.yahoo.com/q?s=AAPL 111
Apple Inc. (AAPL) -NasdaqGS

http://finance.yahoo.com/quote/AAPL?p=AAPL 112
Yahoo Finance Charts: Apple Inc. (AAPL)

http://finance.yahoo.com/chart/AAPL 113
Apple Inc. (AAPL) Historical Data

http://finance.yahoo.com/q/hp?s=AAPL+Historical+Prices 114
Yahoo Finance Historical Prices
Apple Inc. (AAPL)

http://finance.yahoo.com/quote/AAPL/history 115
Yahoo Finance Historical Prices
Apple Inc. (AAPL)

http://finance.yahoo.com/quote/AAPL/history?period1=345398400&period2=1490112000&interval=1d&filter=history&frequency=1d 116
Yahoo Finance Historical Prices
Apple Inc. (AAPL)

http://finance.yahoo.com/quote/AAPL/history?period1=345398400&period2=1490112000&interval=1d&filter=history&frequency=1d 117
 Yahoo Finance Historical Prices
http://ichart.finance.yahoo.com/table.csv?s=AAPL

table.csv Date,Open,High,Low,Close,Volume,Adj Close

2017-03-21,142.110001,142.800003,139.729996,139.839996,39116800,139.839996
2017-03-20,140.399994,141.50,140.229996,141.460007,20213100,141.460007
2017-03-17,141.00,141.00,139.889999,139.990005,43597400,139.990005
2017-03-16,140.720001,141.020004,140.259995,140.690002,19132500,140.690002
2017-03-15,139.410004,140.75,139.029999,140.460007,25566800,140.460007
2017-03-14,139.300003,139.649994,138.839996,138.990005,15189700,138.990005
2017-03-13,138.850006,139.429993,138.820007,139.199997,17042400,139.199997
2017-03-10,139.25,139.360001,138.639999,139.139999,19488000,139.139999
2017-03-09,138.740005,138.789993,137.050003,138.679993,22065200,138.679993
2017-03-08,138.949997,139.800003,138.820007,139.00,18681800,139.00
2017-03-07,139.059998,139.979996,138.789993,139.520004,17267500,139.520004
2017-03-06,139.369995,139.770004,138.600006,139.339996,21155300,139.339996
2017-03-03,138.779999,139.830002,138.589996,139.779999,21108100,139.779999
2017-03-02,140.00,140.279999,138.759995,138.960007,26153300,138.960007
2017-03-01,137.889999,140.149994,137.600006,139.789993,36272400,139.789993
2017-02-28,137.080002,137.440002,136.699997,136.990005,23403500,136.990005
2017-02-27,137.139999,137.440002,136.279999,136.929993,20196400,136.929993
2017-02-24,135.910004,136.660004,135.279999,136.660004,21690900,136.660004
2017-02-23,137.380005,137.479996,136.300003,136.529999,20704100,136.529999
2017-02-22,136.429993,137.119995,136.110001,137.110001,20745300,137.110001

118
 Yahoo Finance Charts
Alphabet Inc. (GOOG)

Alphabet Inc. (GOOG)

http://finance.yahoo.com/echarts?s=GOOG+Interactive#{"showArea":false,"showLine":false,"showCandle":true,"lineType":"candle","range":"5y","allowChartStacking":true} 119
Dow Jones Industrial Average
(^DJI)

http://finance.yahoo.com/chart/^DJI 120
TSEC weighted index (^TWII) -
Taiwan

http://finance.yahoo.com/chart/^DJI 121
Taiwan Semiconductor Manufacturing Company Limited
(2330.TW)

http://finance.yahoo.com/q?s=2330.TW 122
Yahoo Finance Charts
TSMC (2330.TW)

123
import pandas as pd
import pandas_datareader.data as web
df = web.DataReader('AAPL', data_source='yahoo',
start='1/1/2010', end='3/21/2017')
df.to_csv('AAPL.csv')
df.tail()

124
df = web.DataReader('GOOG',
data_source='yahoo', start='1/1/1980',
end='3/21/2017')
df.head(10)

125
df.tail(10)

126
df.count()

127
df.ix['2015-12-31']

128
df.to_csv('2330.TW.Yahoo.Finance.Data.csv')

129
Python Pandas
for Finance
Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 130
 Python Pandas for Finance
import pandas as pd
import pandas_datareader.data as web
import matplotlib.pyplot as plt
import seaborn as sns
import datetime as dt
%matplotlib inline

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 131

 Python Pandas for Finance
#Read Stock Data from Yahoo Finance
end = dt.datetime.now()
#start = dt.datetime(end.year-2, end.month, end.day)
start = dt.datetime(2015, 1, 1)
df = web.DataReader("AAPL", 'yahoo', start, end)
df.to_csv('AAPL.csv')
df.from_csv('AAPL.csv')
df.tail()

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 132

 Python Pandas for Finance
df['Adj Close'].plot(legend=True,
figsize=(12, 8), title='AAPL', label='Adj
Close')

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 133

 Python Pandas for Finance
plt.figure(figsize=(12,9))
top = plt.subplot2grid((12,9), (0, 0),
rowspan=10, colspan=9)
bottom = plt.subplot2grid((12,9), (10,0),
rowspan=2, colspan=9)
top.plot(df.index, df['Adj Close'],
color='blue') #df.index gives the dates
bottom.bar(df.index, df['Volume'])

# set the labels

top.axes.get_xaxis().set_visible(False)
top.set_title('AAPL')
top.set_ylabel('Adj Close')
bottom.set_ylabel('Volume')
Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 134
Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 135

Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 136

 Python Pandas for Finance
plt.figure(figsize=(12,9))
sns.distplot(df['Adj Close'].dropna(),
bins=50, color='purple')

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 137

 Python Pandas for Finance
# simple moving averages
df['MA05'] = df['Adj Close'].rolling(5).mean()
df['MA20'] = df['Adj Close'].rolling(20).mean() #20 days
df['MA60'] = df['Adj Close'].rolling(60).mean() #60 days

df2 = pd.DataFrame({'Adj Close': df['Adj

Close'],'MA05': df['MA05'],'MA20':
df['MA20'], 'MA60': df['MA60']})
df2.plot(figsize=(12, 9), legend=True,
title='AAPL')
df2.to_csv('AAPL_MA.csv')
fig = plt.gcf()
fig.set_size_inches(12, 9)
fig.savefig(’AAPL_plot.png', dpi=300)
plt.show()

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 138

Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 139

Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 140

import pandas as pd
import pandas_datareader.data as web
import matplotlib.pyplot as plt
import seaborn as sns
import datetime as dt
%matplotlib inline

#Read Stock Data from Yahoo Finance

end = dt.datetime.now()
#start = dt.datetime(end.year-2, end.month, end.day)
start = dt.datetime(2015, 1, 1)
df = web.DataReader("AAPL", 'yahoo', start, end)
df.to_csv('AAPL.csv')
df.from_csv('AAPL.csv')
df.tail()

df['Adj Close'].plot(legend=True, figsize=(12, 8), title='AAPL', label='Adj Close')

plt.figure(figsize=(12,9))
top = plt.subplot2grid((12,9), (0, 0), rowspan=10, colspan=9)
bottom = plt.subplot2grid((12,9), (10,0), rowspan=2, colspan=9)
top.plot(df.index, df['Adj Close'], color='blue') #df.index gives the dates
bottom.bar(df.index, df['Volume'])

# set the labels

top.axes.get_xaxis().set_visible(False)
top.set_title('AAPL')
top.set_ylabel('Adj Close')
bottom.set_ylabel('Volume')

plt.figure(figsize=(12,9))
sns.distplot(df['Adj Close'].dropna(), bins=50, color='purple')

# simple moving averages

df['MA05'] = df['Adj Close'].rolling(5).mean() #5 days
df['MA20'] = df['Adj Close'].rolling(20).mean() #20 days
df['MA60'] = df['Adj Close'].rolling(60).mean() #60 days
df2 = pd.DataFrame({'Adj Close': df['Adj Close'],'MA05': df['MA05'],'MA20': df['MA20'], 'MA60': df['MA60']})
df2.plot(figsize=(12, 9), legend=True, title='AAPL')
df2.to_csv('AAPL_MA.csv')
fig = plt.gcf()
fig.set_size_inches(12, 9)
fig.savefig(’AAPL_plot.png', dpi=300)
plt.show()
Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 141
 Examples:
Python Pandas
for Finance
142
143
sSymbol = "AAPL"
#sSymbol = "GOOG"
#sSymbol = "IBM"
#sSymbol = "MSFT"
#sSymbol = "^TWII"
#sSymbol = "000001.SS"
#sSymbol = "2330.TW"
#sSymbol = "2317.TW"

# sURL = "http://ichart.finance.yahoo.com/table.csv?s=AAPL"
# sBaseURL = "http://ichart.finance.yahoo.com/table.csv?s="
sURL = "http://ichart.finance.yahoo.com/table.csv?s=" + sSymbol
#req = requests.get("http://ichart.finance.yahoo.com/table.csv?s=2330.TW")
#req = requests.get("http://ichart.finance.yahoo.com/table.csv?s=AAPL")
req = requests.get(sURL)

sText = req.text
#print(sText)
#df = web.DataReader(sSymbol, 'yahoo', starttime, endtime)
#df = web.DataReader("2330.TW", 'yahoo')

sPath = "data/"
sPathFilename = sPath + sSymbol + ".csv"
print(sPathFilename)

f = open(sPathFilename, 'w')
f.write(sText)
f.close()
sIOdata = io.StringIO(sText)
df = pd.DataFrame.from_csv(sIOdata)
df.head(5)
144
145
df.tail(5)

146
sSymbol = "AAPL”

# sURL = "http://ichart.finance.yahoo.com/table.csv?s=AAPL"
sURL = "http://ichart.finance.yahoo.com/table.csv?s=" + sSymbol
#req = requests.get("http://ichart.finance.yahoo.com/table.csv?s=AAPL")

req = requests.get(sURL)
sText = req.text
#print(sText)

sPath = "data/"
sPathFilename = sPath + sSymbol + ".csv"
print(sPathFilename)

f = open(sPathFilename, 'w')
f.write(sText)
f.close()
sIOdata = io.StringIO(sText)
df = pd.DataFrame.from_csv(sIOdata)
df.head(5)

147
148
149
def getYahooFinanceData(sSymbol, starttime, endtime, sDir):
#GetMarketFinanceData_From_YahooFinance
#"^TWII"
#"000001.SS"
#"AAPL"
#SHA:000016"
#"600000.SS"
#"2330.TW"
#sSymbol = "^TWII"
starttime = datetime.datetime(2000, 1, 1)
endtime = datetime.datetime(2015, 12, 31)
sPath = sDir
#sPath = "data/financedata/"
df_YahooFinance = web.DataReader(sSymbol, 'yahoo', starttime, endtime)
#df_01 = web.DataReader("2330.TW", 'yahoo')
sSymbol = sSymbol.replace(":","_")
sSymbol = sSymbol.replace("^","_")
sPathFilename = sPath + sSymbol + "_Yahoo_Finance.csv"
df_YahooFinance.to_csv(sPathFilename)
#df_YahooFinance.head(5)
return sPathFilename
#End def getYahooFinanceData(sSymbol, starttime, endtime, sDir):

150
151
sSymbol = "AAPL”
starttime = datetime.datetime(2000, 1, 1)
endtime = datetime.datetime(2015, 12, 31)
sDir = "data/financedata/"

sPathFilename = getYahooFinanceData(sSymbol, starttime, endtime,

sDir)
print(sPathFilename)

152
import pandas as pd
import pandas_datareader.data as web
import matplotlib.pyplot as plt
import seaborn as sns
import datetime as dt
%matplotlib inline

#Read Stock Data from Yahoo Finance

end = dt.datetime.now()
#start = dt.datetime(end.year-2, end.month, end.day)
start = dt.datetime(2015, 1, 1)
df = web.DataReader("AAPL", 'yahoo', start, end)
df.to_csv('AAPL.csv')
df.from_csv('AAPL.csv')
df.tail()

df['Adj Close'].plot(legend=True, figsize=(12, 8), title='AAPL', label='Adj Close')

plt.figure(figsize=(12,9))
top = plt.subplot2grid((12,9), (0, 0), rowspan=10, colspan=9)
bottom = plt.subplot2grid((12,9), (10,0), rowspan=2, colspan=9)
top.plot(df.index, df['Adj Close'], color='blue') #df.index gives the dates
bottom.bar(df.index, df['Volume'])

# set the labels

top.axes.get_xaxis().set_visible(False)
top.set_title('AAPL')
top.set_ylabel('Adj Close')
bottom.set_ylabel('Volume')

plt.figure(figsize=(12,9))
sns.distplot(df['Adj Close'].dropna(), bins=50, color='purple')

# simple moving averages

df['MA05'] = df['Adj Close'].rolling(5).mean() #5 days
df['MA20'] = df['Adj Close'].rolling(20).mean() #20 days
df['MA60'] = df['Adj Close'].rolling(60).mean() #60 days
df2 = pd.DataFrame({'Adj Close': df['Adj Close'],'MA05': df['MA05'],'MA20': df['MA20'], 'MA60': df['MA60']})
df2.plot(figsize=(12, 9), legend=True, title='AAPL')
df2.to_csv('AAPL_MA.csv')
fig = plt.gcf()
fig.set_size_inches(12, 9)
fig.savefig(’AAPL_plot.png', dpi=300)
plt.show()
Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 153
Python Pandas for Finance

Source: https://mapattack.wordpress.com/2017/02/12/using-python-for-stocks-1/ 154

 References
• Wes McKinney (2012), Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython,
O'Reilly Media
• Yves Hilpisch (2014), Python for Finance: Analyze Big Financial Data, O'Reilly
• Yves Hilpisch (2015), Derivatives Analytics with Python: Data Analysis, Models, Simulation,
Calibration and Hedging, Wiley
• Michael Heydt (2015) , Mastering Pandas for Finance, Packt Publishing
• Michael Heydt (2015), Learning Pandas - Python Data Discovery and Analysis Made Easy, Packt
Publishing
• James Ma Weiming (2015), Mastering Python for Finance, Packt Publishing
• Fabio Nelli (2015), Python Data Analytics: Data Analysis and Science using PANDAs, matplotlib and
the Python Programming Language, Apress
• Wes McKinney (2013), 10-minute tour of pandas, https://vimeo.com/59324550
• Jason Wirth (2015), A Visual Guide To Pandas, https://www.youtube.com/watch?v=9d5-Ti6onew
• Edward Schofield (2013), Modern scientific computing and big data analytics in Python, PyCon
Australia, https://www.youtube.com/watch?v=hqOsfS3dP9w
• Python Programming, https://pythonprogramming.net/
• Python, https://www.python.org/
• Python Programming Language, http://pythonprogramminglanguage.com/
• Numpy, http://www.numpy.org/
• Pandas, http://pandas.pydata.org/
155