{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 机器学习与社会科学应用" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 第七章 深度学习" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 番外 Keras入门" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "郭峰 \n", " 教授、博士生导师 \n", "上海财经大学公共经济与管理学院 \n", "上海财经大学数实融合与智能治理实验室 \n", " 邮箱：guofengsfi@163.com " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "本节目录： \n", "1.keras简介 \n", "2.二分类演示性案例 \n", "3.手写数字识别 \n", "4.keras参数详解 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1.keras简介" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Keras是Python中以CNTK、Tensorflow或者Theano为计算后台的一个深度学习建模环境。\n", "相对于其他深度学习的框架，如Tensorflow、Theano、Caffe等\n", "Keras在实际应用中有一些显著的优点\n", "其中最主要的优点就是Keras已经高度模块化了，支持现有的常见模型（CNN、RNN) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "安装tensorflow和keras，不同版本可能会导致安装错误，错误了很多次，后来不知道怎么弄的，又成功了 \n", "pip install tensorflow==2.7.0 \n", "pip install keras==2.7.0 " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#查看软件版本\n", "import tensorflow\n", "print(tensorflow.__version__)\n", "import keras\n", "print(keras.__version__)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import tensorflow as tf\n", "tf.__path__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Keras训练模型的过程非常直观，简单来说就三步： \n", "第一步，搭建模型 \n", "第二步，训练模型 \n", "第三步，验证模型 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Keras中设定了两类深度学习的模型，一类是序列模型（Sequential类）；一类是通用模型（Model类） " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "序列模型 " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#第1步：导入所需库，构造数据\n", "import numpy as np\n", "from keras.models import Sequential\n", "from keras.layers import Dense\n", "from pyecharts.charts import Scatter\n", "import pyecharts.options as opts\n", "\n", "#构造数据\n", "X = np.linspace(-2 , 2 , 200)\n", "np.random.shuffle(X) #打乱原来数据的顺序\n", "\n", "#添加一些噪音数据\n", "Y = 0.5 * X + 2 + np.random.normal(0 , 0.05 , (200 , ))\n", "\n", "#显示输入数据\n", "scatter = Scatter()\n", "scatter.add_xaxis(X)\n", "scatter.add_yaxis(\"散点图\",Y)\n", "scatter.set_series_opts(label_opts=opts.LabelOpts(is_show=False))\n", "scatter.render_notebook() " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#把200份数据划分为训练数据、测试数据\n", "X_train,Y_train = X[:160] , Y[:160] #前160个数据点作为训练数据\n", "X_test , Y_test = X[160:] , Y[160:] #后40个数据点作为测试数据" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#第2步：构造模型\n", "#创建序列实例\n", "model = Sequential()\n", "model.add(Dense(units=1 , activation='relu' , input_dim=1))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#第3步：编译模型\n", "#选择代价函数及优化器\n", "model.compile(loss='mse' , optimizer='sgd')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#第4步：训练模型\n", "model.fit(X_train , Y_train , epochs=100 , verbose=0 ,batch_size=64)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#第5步：测试模型\n", "#test\n", "print('\\nTesting..............')\n", "cost = model.evaluate(X_test , Y_test , batch_size= 40)\n", "print('test cost:',cost)\n", "W , b =model.layers[0].get_weights()\n", "print('Weight = ' , W , '\\nbiases = ' , b)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#可视化结果\n", "Y_pred = model.predict(X_test)\n", "scatter = Scatter()\n", "scatter.add_xaxis(X_test)\n", "scatter.add_yaxis(\"散点图\",Y_test)\n", "scatter.set_series_opts(label_opts=opts.LabelOpts(is_show=False))\n", "scatter.render_notebook() " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2.二分类演示性案例" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from keras.models import Sequential\n", "from keras.layers import Dense, Dropout\n", "\n", "# Generate dummy data\n", "x_train = np.random.random((1000, 20))\n", "y_train = np.random.randint(2, size=(1000, 1))\n", "x_test = np.random.random((100, 20))\n", "y_test = np.random.randint(2, size=(100, 1))\n", "\n", "model = Sequential()\n", "model.add(Dense(64, input_dim=20, activation='relu'))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(64, activation='relu'))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(1, activation='sigmoid'))\n", "\n", "model.compile(loss='binary_crossentropy',\n", " optimizer='rmsprop',\n", " metrics=['accuracy'])\n", "\n", "model.fit(x_train, y_train,\n", " epochs=20,\n", " batch_size=128)\n", "score = model.evaluate(x_test, y_test, batch_size=128)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.手写数字识别" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\n", "from tensorflow.keras.datasets import mnist\n", "# load (downloaded if needed) the MNIST dataset\n", "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n", "import numpy #导入数据库\n", "from tensorflow.keras.datasets import mnist\n", "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense\n", "from tensorflow.keras.layers import Dropout\n", "from keras.utils import np_utils\n", "\n", "seed = 7 #设置随机种子\n", "numpy.random.seed(seed)\n", " \n", "(X_train, y_train), (X_test, y_test) = mnist.load_data() #加载数据\n", " \n", "num_pixels = X_train.shape[1] * X_train.shape[2]\n", "X_train = X_train.reshape(X_train.shape[0], num_pixels).astype('float32')\n", "X_test = X_test.reshape(X_test.shape[0], num_pixels).astype('float32')\n", "#数据集是3维的向量（instance length,width,height).对于多层感知机，模型的输入是二维的向量，因此这\n", "#里需要将数据集reshape，即将28*28的向量转成784长度的数组。可以用numpy的reshape函数轻松实现这个过\n", "#程。\n", " \n", "#给定的像素的灰度值在0-255，为了使模型的训练效果更好，通常将数值归一化映射到0-1。\n", "X_train = X_train / 255\n", "X_test = X_test / 255\n", " \n", "#最后，模型的输出是对每个类别的打分预测，对于分类结果从0-9的每个类别都有一个预测分值，表示将模型\n", "#输入预测为该类的概率大小，概率越大可信度越高。由于原始的数据标签是0-9的整数值，通常将其表示成#0ne-hot向量。\n", "#如第一个训练数据的标签为5，one-hot表示为[0,0,0,0,0,1,0,0,0,0]。\n", "y_train = np_utils.to_categorical(y_train)\n", "y_test = np_utils.to_categorical(y_test)\n", "num_classes = y_test.shape[1]\n", " \n", "#现在需要做得就是搭建神经网络模型了，创建一个函数，建立含有一个隐层的神经网络。\n", "# define baseline model\n", "def baseline_model():\n", " # create model\n", " model = Sequential()\n", " model.add(Dense(num_pixels, input_dim=num_pixels, kernel_initializer='normal', activation='relu'))\n", " model.add(Dense(num_classes, kernel_initializer='normal', activation='softmax'))\n", " # Compile model\n", " model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n", " return model\n", " \n", "#型的隐含层含有784个节点，接受的输入长度也是784（28*28），最后用softmax函数将预测结果转换为标签\n", "#的概率值。 \n", "#将训练数据fit到模型，设置了迭代轮数，每轮200个训练样本，将测试集作为验证集，并查看训练的效果。\n", " \n", "# build the model\n", "model = baseline_model()\n", "# Fit the model\n", "model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)\n", "# Final evaluation of the model\n", "scores = model.evaluate(X_test, y_test, verbose=0)\n", "print(\"Baseline Error: %.2f%%\" % (100-scores[1]*100))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.Keras 参数详细说明" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更详细参数介绍可查阅Keras文档） " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（1)全连接层：神经网络中最常用到的，实现对神经网络里的神经元激活。\n", "Dense（units, activation=’relu’, use_bias=True） \n", "参数说明： \n", "units: 全连接层输出的维度，即下一层神经元的个数。 \n", "activation：激活函数，默认使用Relu。 \n", "use_bias：是否使用bias偏置项(即计量经济学中的常数项) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（2)激活层：对上一层的输出应用激活函数。\n", "Activation(activation) \n", "参数说明： \n", "Activation：想要使用的激活函数，如：’relu’、’tanh’、‘sigmoid’等。 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（3）Dropout层：对上一层的神经元随机选取一定比例的失活，不更新，但是权重仍然保留，防止过拟合。\n", "神将网络“失活”原理：https://blog.csdn.net/program_developer/article/details/80737724\n", "Dropout(rate) \n", "参数说明: \n", "rate：失活的比例，0-1的浮点数。 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(4)Flatten层：将一个维度大于或等于3的高维矩阵，“压扁”为一个二维矩阵。即保留第一个维度（如：batch的个数），然后将剩下维度的值相乘作为“压扁”矩阵的第二个维度。\n", "Flatten()\n", "作用：Flatten层用来将输入“压平”，即把多维的输入一维化，常用在从卷积层到全连接层的过渡。Flatten不影响batch的大小。 \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（5）Reshape层：该层的作用和reshape一样，就是将输入的维度重构成特定的shape。\n", "Reshape(target_shape) \n", "参数说明： \n", "target_shape：目标矩阵的维度，不包含batch样本数。 \n", "如我们想要一个9个元素的输入向量重构成一个(None, 3, 3)的二维矩阵：\n", "Reshape((3,3), input_length=(16, )) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "卷积层：卷积操作分为一维、二维、三维，分别为Conv1D、Conv2D、Conv3D。\n", "一维卷积主要应用于以时间序列数据或文本数据，二维卷积通常应用于图像数据。\n", "由于这三种的使用和参数都基本相同，所以主要以处理图像数据的Conv2D进行说明。\n", "Conv2D(filters, kernel_size, strides=(1, 1), padding=’valid’)\n", "参数说明： \n", "filters：卷积核的个数。 \n", "kernel_size：卷积核的大小。 \n", "strdes：步长，二维中默认为(1, 1)，一维默认为1。 \n", "Padding：补“0”策略，’valid‘指卷积后的大小与原来的大小可以不同，’same‘则卷积后大小与原来大小一致。 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（7）池化层：与卷积层一样，最大统计量池化和平均统计量池化也有三种，分别为MaxPooling1D、MaxPooling2D、MaxPooling3D和AveragePooling1D、AveragePooling2D、AveragePooling3D，\n", "由于使用和参数基本相同，所以主要以MaxPooling2D进行说明。 \n", "MaxPooling(pool_size=(2,2), strides=None, padding=’valid’) \n", "参数说明： \n", "pool_size：长度为2的整数tuple，表示在横向和纵向的下采样样子，一维则为纵向的下采样因子。\n", "padding：和卷积层的padding一样。 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（8）循环层：循环神经网络中的RNN、LSTM和GRU都继承本层，所以该父类的参数同样使用于对应的子类SimpleRNN、LSTM和GRU。\n", "Recurrent(return_sequences=False) \n", "return_sequences：控制返回的类型，“False”返回输出序列的最后一个输出，“True”则返回整个序列。 \n", "当我们要搭建多层神经网络（如深层LSTM）时，若不是最后一层，则需要将该参数设为True。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（9）嵌入层：该层只能用在模型的第一层，是将所有索引标号的稀疏矩阵映射到致密的低维矩阵。\n", "如我们对文本数据进行处理时，我们对每个词编号后，我们希望将词编号变成词向量就可以使用嵌入层。 \n", "Embedding(input_dim, output_dim, input_length) \n", "参数说明： \n", "Input_dim：大于或等于0的整数，字典的长度即输入数据的个数。 \n", "output_dim：输出的维度，如词向量的维度。 \n", "input_length：当输入序列的长度为固定时为该长度，然后要在该层后加上Flatten层，然后再加上Dense层，则必须指定该参数，否则Dense层无法自动推断输出的维度。\n", "该层可能有点费解，举个例子，当我们有一个文本，该文本有100句话，我们已经通过一系列操作，使得文本变成一个(100,32)矩阵，每行代表一句话，每个元素代表一个词，我们希望将该词变为64维的词向量：\n", "Embedding(100, 64, input_length=32) \n", "则输出的矩阵的shape变为(100, 32, 64)：即每个词已经变成一个64维的词向量。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（10）模型优化和训练 \n", "compile(optimizer, loss, metrics=None) \n", "参数说明： \n", "optimizer：优化器，如：’SGD‘，’Adam‘等。 \n", "loss：定义模型的损失函数，如：’mse’，’mae‘等。 \n", "metric：模型的评价指标，如：’accuracy‘等。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "（11）模型拟合 \n", "fit(x=None, y=None, batch_size=None, epochs=1, verbose=1, validation_split=0.0) \n", "参数说明： \n", "x：输入数据。 \n", "y：标签。 \n", "batch_size：梯度下降时每个batch包含的样本数。 \n", "epochs：整数，所有样本的训练次数。 \n", "verbose：日志显示，0为不显示，1为显示进度条记录，2为每个epochs输出一行记录。 \n", "validation_split：0-1的浮点数，切割输入数据的一定比例作为验证集。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" } }, "nbformat": 4, "nbformat_minor": 2 }