{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 机器学习与社会科学应用"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 第七章 深度学习"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 第一节 神经网络基本原理与前馈神经网络"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"郭峰 \n",
" 教授、博士生导师 \n",
"上海财经大学公共经济与管理学院 \n",
"上海财经大学数实融合与智能治理实验室 \n",
" 邮箱：guofengsfi@163.com "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"本节目录 \n",
"1.1.神经网络用于分类 \n",
"1.2.神经网络回归 \n",
"1.3.手写数字识别 \n",
"1.4.MLPclassifier参数详解 "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经网络分类\n",
"MLPClassifier 类实现了通过 Backpropagation 进行训练的多层感知器（MLP）算法。\n",
"目前， MLPClassifier 只支持交叉熵损失函数，通过运行 predict_proba 方法进行概率估计。\n",
"MLP 算法使用的是反向传播的方式。 更准确地说，它使用了通过反向传播计算得到的梯度和某种形式的梯度下降来进行训练。 对于分类来说，它最小化交叉熵损失函数，为每个样本 x 给出一个向量形式的概率估计 P(y|x) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.1.神经网络用于分类"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#载入数据\n",
"import numpy as np\n",
"from sklearn.neural_network import MLPClassifier\n",
"from sklearn.preprocessing import StandardScaler\n",
"data = [\n",
" [-0.017612, 14.053064, 0],[-1.395634, 4.662541, 1],[-0.752157, 6.53862, 0],[-1.322371, 7.152853, 0],[0.423363, 11.054677, 0],\n",
" [0.406704, 7.067335, 1],[0.667394, 12.741452, 0],[-2.46015, 6.866805, 1],[0.569411, 9.548755, 0],[-0.026632, 10.427743, 0],\n",
" [0.850433, 6.920334, 1],[1.347183, 13.1755, 0],[1.176813, 3.16702, 1],[-1.781871, 9.097953, 0],[-0.566606, 5.749003, 1],\n",
" [0.931635, 1.589505, 1],[-0.024205, 6.151823, 1],[-0.036453, 2.690988, 1],[-0.196949, 0.444165, 1],[1.014459, 5.754399, 1],\n",
" [1.985298, 3.230619, 1],[-1.693453, -0.55754, 1],[-0.576525, 11.778922, 0],[-0.346811, -1.67873, 1],[-2.124484, 2.672471, 1],\n",
" [1.217916, 9.597015, 0],[-0.733928, 9.098687, 0],[1.416614, 9.619232, 0],[1.38861, 9.341997, 0],[0.317029, 14.739025, 0]\n",
" ]\n",
"\n",
"dataMat = np.array(data)\n",
"X=dataMat[:,0:2]\n",
"y = dataMat[:,2]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#数据标准化\n",
"#神经网络对数据尺度敏感，所以最好在训练前标准化，或者归一化，或者缩放到[-1,1]\n",
"scaler = StandardScaler() # 标准化转换\n",
"scaler.fit(X) # 训练标准化对象\n",
"X = scaler.transform(X) # 转换数据集"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#参数\n",
"clf = MLPClassifier(solver='lbfgs', alpha=1e-5,hidden_layer_sizes=(5,2), \n",
" random_state=1) \n",
"#含义\n",
"# 神经网络输入为2，第一隐藏层神经元个数为5，\n",
"#第二隐藏层神经元个数为2，输出结果为2分类。\n",
"#solver='lbfgs', MLP的求解方法：L-BFGS 在小数据上表现较好，Adam 较为鲁棒，\n",
"#SGD在参数调整较优时会有最佳表现（分类效果与迭代次数）；SGD标识随机梯度下降。\n",
"#alpha:L2的参数：MLP是可以支持正则化的，默认为L2，具体参数需要调整\n",
"# hidden_layer_sizes=(5, 2) hidden层2层,第一层5个神经元，第二层2个神经元)，\n",
"#2层隐藏层，也就有3层神经网络"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#数据拟合\n",
"clf.fit(X, y)\n",
"print('每层网络层系数矩阵维度：\\n',[coef.shape for coef in clf.coefs_])\n",
"y_pred = clf.predict([[0.317029, 14.739025]])\n",
"print('预测结果：',y_pred)\n",
"y_pred_pro =clf.predict_proba([[0.317029, 14.739025]])\n",
"print('预测结果概率：\\n',y_pred_pro)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(clf.coefs_)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cengindex = 0\n",
"for wi in clf.coefs_:\n",
" cengindex += 1 # 表示第几层神经网络。\n",
" print('第%d层网络层:' % cengindex)\n",
" print('权重矩阵维度:',wi.shape)\n",
" print('系数矩阵:\\n',wi)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.2.神经网络回归"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#MLPRegressor 类多层感知器（MLP）的实现，在使用反向传播进行训练时的输出层没有使用激活函数\n",
"#也可以看作是使用恒等函数（identity function）作为激活函数。 \n",
"#因此，它使用平方误差作为损失函数，输出是一组连续值。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.neural_network import MLPRegressor # 多层线性回归\n",
"from sklearn.preprocessing import StandardScaler\n",
"data = [\n",
" [ -0.017612,14.053064,14.035452],[ -1.395634, 4.662541, 3.266907],[ -0.752157, 6.53862,5.786463],[ -1.322371, 7.152853, 5.830482],\n",
" [0.423363,11.054677,11.47804 ],[0.406704, 7.067335, 7.474039],[0.667394,12.741452,13.408846],[ -2.46015,6.866805, 4.406655],\n",
" [0.569411, 9.548755,10.118166],[ -0.026632,10.427743,10.401111],[0.850433, 6.920334, 7.770767],[1.347183,13.1755,14.522683],\n",
" [1.176813, 3.16702,4.343833],[ -1.781871, 9.097953, 7.316082],[ -0.566606, 5.749003, 5.182397],[0.931635, 1.589505, 2.52114 ],\n",
" [ -0.024205, 6.151823, 6.127618],[ -0.036453, 2.690988, 2.654535],[ -0.196949, 0.444165, 0.247216],[1.014459, 5.754399, 6.768858],\n",
" [1.985298, 3.230619, 5.215917],[ -1.693453,-0.55754, -2.250993],[ -0.576525,11.778922,11.202397],[ -0.346811,-1.67873, -2.025541],\n",
" [ -2.124484, 2.672471, 0.547987],[1.217916, 9.597015,10.814931],[ -0.733928, 9.098687, 8.364759],[1.416614, 9.619232,11.035846],\n",
" [1.38861,9.341997,10.730607],[0.317029,14.739025,15.056054]\n",
"]\n",
"\n",
"dataMat = np.array(data)\n",
"X=dataMat[:,0:2]\n",
"y = dataMat[:,2]\n",
"scaler = StandardScaler() # 标准化转换\n",
"scaler.fit(X) # 训练标准化对象\n",
"X = scaler.transform(X) # 转换数据集\n",
"\n",
"# solver='lbfgs', MLP的求解方法：L-BFGS 在小数据上表现较好，Adam 较为鲁棒，SGD在参数调整较优时会有最佳表现（分类效果与迭代次数）；SGD标识随机梯度下降。\n",
"# alpha:L2的参数：MLP是可以支持正则化的，默认为L2，具体参数需要调整\n",
"# hidden_layer_sizes=(5, 2) hidden层2层,第一层5个神经元，第二层2个神经元)，2层隐藏层，也就有3层神经网络\n",
"clf = MLPRegressor(solver='lbfgs', alpha=1e-5,hidden_layer_sizes=(5, 2), random_state=1)\n",
"clf.fit(X, y)\n",
"print('预测结果：', clf.predict([[0.317029, 14.739025]])) # 预测某个输入对象\n",
"\n",
"cengindex = 0\n",
"for wi in clf.coefs_:\n",
" cengindex += 1 # 表示底第几层神经网络。\n",
" print('第%d层网络层:' % cengindex)\n",
" print('权重矩阵维度:',wi.shape)\n",
" print('系数矩阵：\\n',wi)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.3.手写数字识别"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.datasets import load_digits\n",
"from sklearn.preprocessing import MinMaxScaler\n",
"from sklearn.neural_network import MLPClassifier\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn import metrics\n",
"\n",
"data = load_digits()\n",
"x = data.data\n",
"y = data.target\n",
"print(x.shape,y.shape)\n",
"stander = MinMaxScaler()\n",
"x = stander.fit_transform(x)\n",
"x_train,x_test,y_train,y_test = train_test_split(x,y,test_size=0.3)\n",
"\n",
"clf = MLPClassifier(hidden_layer_sizes=(100,100))\n",
"clf.fit(x_train,y_train)\n",
"y_pred = clf.predict((x_test))\n",
"print(\"训练集评分：\", clf.score(x_train,y_train))\n",
"print(\"测试集评分：\", clf.score(x_test,y_test))\n",
"print(\"混淆矩阵：\")\n",
"print(metrics.confusion_matrix(y_test,y_pred))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"分类报告：\", metrics.classification_report(y_test,y_pred))\n",
"print(\"W权重：\",clf.coefs_[:1])\n",
"print(\"损失值：\",clf.loss_)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.4.MLPclassifier参数详解"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 参数"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#主要参数：\n",
"#hidden_layer_sizes: 隐藏层单元数(tuple)，如(100,100,100,50)\n",
"#activation : 激活函数，{‘identity’, ‘logistic’, ‘tanh’, ‘relu’}, 缺省 ‘relu‘; [f(x) = x, 1/(1+exp(-x)), tanh(x), max(0, x)]\n",
"#solver : 解决器, {‘lbfgs’, ‘sgd’, ‘adam’}, 缺省 ‘adam’; [牛顿法,随机梯度下降, 自适应momemtum] \n",
"#alpha : L2正则化参数，float, 可选，缺省0.0001\n",
"#batch_size : 批次，可选, 不适用于’lbfgs’, 缺省 ‘auto‘, 此时，batch_size=min(200, n_samples)`\n",
"#learning_rate : 学习率, {‘constant’, ‘invscaling’, ‘adaptive’}, 缺省 ‘constant’, 只适用于梯度下降sgd\n",
"#learning_rate_init : 学习率初始值, 可选, 缺省 0.001, 仅适用于sgd或adam\n",
"#power_t : 下降指数, 可选, 缺省 0.5, 适用于’invscaling’,learning_rate_init/pow(t,power_t), 仅适用于sgd\n",
"#max_iter : 最大迭代数, 可选, 缺省200, 迭代器收敛迭代次数，对于sgd/adam, 代表的是epochs数目，不是下降步数\n",
"#shuffle : 每次迭代, 是否洗牌, 可选, 缺省True,仅适用于sgd或adam\n",
"#random_state: 缺省None; 若int, 随机数产生器seed, 若RandomStates实例, 随机数产生器, 若None, np.random\n",
"#tol : 容忍度, 可选, 缺省le-4, 连续两次迭代loss达不到此值，除非设置成’adaptive’,否则，就停止迭代，\n",
"#beta_1 : adam指数衰减参数1，可选, 缺省0.9\n",
"#beta_2 : adam指数衰减参数2，可选, 缺省0.999\n",
"#epsilon : adam数值稳定值，可选，缺省1e-8"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#更完整参数\n",
"#https://blog.csdn.net/weixin_38278334/article/details/83023958"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 方法"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"fit（X，y）：使模型适合数据矩阵X和目标y。 \n",
"get_params（[deep]）：获取此估算器的参数。 \n",
"predict（X）：使用多层感知器分类器进行预测 \n",
"predict_log_proba（X）：返回概率估计的对数。 \n",
"predict_proba（X）：概率估计。 \n",
"score（X，y [，sample_weight]）：返回给定测试数据和标签的平均准确度。 \n",
"set_params（** params）：设置此估算器的参数。 "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
}