样本 红 大 好苹果

0 1 1 1

1 1 0 1

2 0 1 0

3 0 0 0

1.5 50 thin

1.5 60 fat

1.6 40 thin

1.6 60 fat

1.7 60 thin

1.7 80 fat

1.8 60 thin

1.8 90 fat

1.9 70 thin

1.9 80 fat

​# -*- coding: utf-8 -*-

import numpy as np

import scipy as sp

from sklearn import tree

from sklearn.metrics import precision_recall_curve

from sklearn.metrics import classification_report

from sklearn.cross_validation import train_test_split

""" 数据读入 """

data = []

labels = []

with open("d:\\python\\ml\\data\\1.txt") as ifile:

for line in ifile:

tokens = line.strip().split(" ")

data.append([float(tk) for tk in tokens[:-1]])

labels.append(tokens[-1])

x = np.array(data)

labels = np.array(labels)

y = np.zeros(labels.shape)

""" 标签转换为0/1 """

y[labels=="fat"]=1

""" 拆分训练数据与测试数据 """

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.2)

""" 使用信息熵作为划分标准，对决策树进行训练 """

clf = tree.DecisionTreeClassifier(criterion="entropy")

print(clf)

clf.fit(x_train, y_train)

""" 把决策树结构写入文件 """

with open("tree.dot", "w") as f:

f = tree.export_graphviz(clf, out_file=f)

""" 系数反映每个特征的影响力。越大表示该特征在分类中起到的作用越大 """

print(clf.feature_importances_)

"""测试结果的打印"""

answer = clf.predict(x_train)

print(x_train)

print(answer)

print(y_train)

print(np.mean( answer == y_train))

"""准确率与召回率"""

precision, recall, thresholds = precision_recall_curve(y_train, clf.predict(x_train))

answer = clf.predict_proba(x)[:,1]

print(classification_report(y, answer, target_names = ["thin", "fat"]))​

[ 0.2488562 0.7511438]

array([[ 1.6, 60. ],

[ 1.7, 60. ],

[ 1.9, 80. ],

[ 1.5, 50. ],

[ 1.6, 40. ],

[ 1.7, 80. ],

[ 1.8, 90. ],

[ 1.5, 60. ]])

array([ 1., 0., 1.​, 0., 0., 1., 1., 1.])

array([ 1., 0., 1., 0., 0., 1., 1., 1.])

1.0

precision recall f1-score support

thin 0.83 1.00 0.91 5

fat 1.000.80 0.89 5

avg / total 1.00 1.00 1.00 8

array([ 0., 1., 0., 1., 0., 1., 0., 1.,0., 0.])

array([ 0., 1., 0., 1., 0., 1., 0., 1.,0., 1.])

可以看到，对训练过的数据做测试，准确率是100%。但是最后将所有数据进行测试，会出现1个测试样本分类错误。

说明本例的决策树对训练集的规则吸收的很好，但是预测性稍微差点。​

​digraph Tree {

0 [label="X[1]<= 55.0000\nentropy = 0.954434002925\nsamples = 8", shape="box"] ;

1 [label="entropy = 0.0000\nsamples = 2\nvalue = [ 2. 0.]", shape="box"] ;

0 -> 1 ;

2 [label="X[1]<= 70.0000\nentropy = 0.650022421648\nsamples = 6", shape="box"] ;

0 -> 2 ;

3 [label="X[0]<= 1.6500\nentropy = 0.918295834054\nsamples = 3", shape="box"] ;

2 -> 3 ;

4 [label="entropy = 0.0000\nsamples = 2\nvalue = [ 0. 2.]", shape="box"] ;

3 -> 4 ;

5 [label="entropy = 0.0000\nsamples = 1\nvalue = [ 1. 0.]", shape="box"] ;

3 -> 5 ;

6 [label="entropy = 0.0000\nsamples = 3\nvalue = [ 0. 3.]", shape="box"] ;

2 -> 6 ;

}​