abupy_lecture/19-数据源(ABU量化使用文档).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# ABU量化系统使用文档 \n",
    "\n",
    "<center>\n",
    "        <img src=\"./image/abu_logo.png\" alt=\"\" style=\"vertical-align:middle;padding:10px 20px;\"><font size=\"6\" color=\"black\"><b>第19节 数据源</b></font>\n",
    "</center>\n",
    "\n",
    "----------------- \n",
    "\n",
    "\n",
    "作者: 阿布\n",
    "\n",
    "阿布量化版权所有 未经允许 禁止转载\n",
    "\n",
    "[abu量化系统github地址](https://github.com/bbfamily/abu) (欢迎+star)\n",
    "\n",
    "[本节ipython notebook](https://github.com/bbfamily/abu/tree/master/abupy_lecture)\n",
    "\n",
    "abupy除了之前章节讲解示例时使用的内置沙盒数据外，也内置了几个数据源，支持实时获取美股，A股，港股，期货和比特币等类型数据，内置的数据源仅供学习使用，abupy提供了接入外部数据源的接口和规范，本节的示例将讲解内置数据源的切换，以及接入用户外部的数据源的接口规范。\n",
    "\n",
    "首先导入abupy中本节使用的模块："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "enable example env will only read RomDataBu/df_kl.h5\n"
     ]
    }
   ],
   "source": [
    "# 基础库导入\n",
    "\n",
    "from __future__ import print_function\n",
    "from __future__ import division\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "warnings.simplefilter('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import os\n",
    "import sys\n",
    "# 使用insert 0即只使用github，避免交叉使用了pip安装的abupy，导致的版本不一致问题\n",
    "sys.path.insert(0, os.path.abspath('../'))\n",
    "import abupy\n",
    "\n",
    "# 使用沙盒数据，目的是和书中一样的数据环境\n",
    "abupy.env.enable_example_env_ipython()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from abupy import abu, EMarketTargetType, AbuMetricsBase, ABuMarketDrawing, ABuProgress, ABuSymbolPd, get_price, ABuIndustries\n",
    "from abupy import EMarketDataFetchMode, EDataCacheType, EMarketSourceType, FuturesBaseMarket, TCBaseMarket, ABuDateUtil\n",
    "from abupy import AbuDataParseWrap, StockBaseMarket, SupportMixin, ABuNetWork, Symbol, code_to_symbol"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和之前的章节不一样，关闭沙盒数据接口，使用实时数据源提供数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "disable example env\n"
     ]
    }
   ],
   "source": [
    "abupy.env.disable_example_env_ipython()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 数据模式的切换\n",
    "\n",
    "使用g_data_fetch_mode可以查看当前的数据模式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<EMarketDataFetchMode.E_DATA_FETCH_NORMAL: 0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_data_fetch_mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "默认使用模式为E_DATA_FETCH_NORMAL，NORMAL的意义是默认优先从缓存中获取，如果缓存中不存在，再访问网络，尝试从网络获取，除此之外还有一些优化，比如虽然缓存中的数据也无法满足要求，但是缓存索引纪录今天已经尝试从网络获取，这种情况下也不再访问网络。\n",
    "\n",
    "更多详情请阅读ABuDataSource代码\n",
    "\n",
    "E_DATA_FETCH_FORCE_NET为强制使用网络进行数据更新，一般不推荐使用，如果切换了数据源，或者缓存中的数据存在问题的情况下会使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_data_fetch_mode = EMarketDataFetchMode.E_DATA_FETCH_FORCE_NET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>atr14</th>\n",
       "      <th>atr21</th>\n",
       "      <th>close</th>\n",
       "      <th>date</th>\n",
       "      <th>date_week</th>\n",
       "      <th>high</th>\n",
       "      <th>key</th>\n",
       "      <th>low</th>\n",
       "      <th>open</th>\n",
       "      <th>p_change</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>9.7977</td>\n",
       "      <td>8.4559</td>\n",
       "      <td>221.96</td>\n",
       "      <td>20170802</td>\n",
       "      <td>2</td>\n",
       "      <td>224.64</td>\n",
       "      <td>499</td>\n",
       "      <td>218.58</td>\n",
       "      <td>224.12</td>\n",
       "      <td>-1.613</td>\n",
       "      <td>225.60</td>\n",
       "      <td>3688218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>9.4807</td>\n",
       "      <td>8.3617</td>\n",
       "      <td>227.39</td>\n",
       "      <td>20170803</td>\n",
       "      <td>3</td>\n",
       "      <td>227.97</td>\n",
       "      <td>500</td>\n",
       "      <td>220.55</td>\n",
       "      <td>221.00</td>\n",
       "      <td>2.446</td>\n",
       "      <td>221.96</td>\n",
       "      <td>3433083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>9.3339</td>\n",
       "      <td>8.3634</td>\n",
       "      <td>226.05</td>\n",
       "      <td>20170804</td>\n",
       "      <td>4</td>\n",
       "      <td>230.34</td>\n",
       "      <td>501</td>\n",
       "      <td>225.70</td>\n",
       "      <td>227.28</td>\n",
       "      <td>-0.589</td>\n",
       "      <td>227.39</td>\n",
       "      <td>3520335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>8.6520</td>\n",
       "      <td>7.9867</td>\n",
       "      <td>227.16</td>\n",
       "      <td>20170807</td>\n",
       "      <td>0</td>\n",
       "      <td>228.99</td>\n",
       "      <td>502</td>\n",
       "      <td>224.77</td>\n",
       "      <td>225.60</td>\n",
       "      <td>0.491</td>\n",
       "      <td>226.05</td>\n",
       "      <td>2998194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>8.4744</td>\n",
       "      <td>7.9261</td>\n",
       "      <td>229.45</td>\n",
       "      <td>20170808</td>\n",
       "      <td>1</td>\n",
       "      <td>233.37</td>\n",
       "      <td>503</td>\n",
       "      <td>226.54</td>\n",
       "      <td>226.60</td>\n",
       "      <td>1.008</td>\n",
       "      <td>227.16</td>\n",
       "      <td>4021199</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             atr14   atr21   close      date  date_week    high  key     low  \\\n",
       "2017-08-02  9.7977  8.4559  221.96  20170802          2  224.64  499  218.58   \n",
       "2017-08-03  9.4807  8.3617  227.39  20170803          3  227.97  500  220.55   \n",
       "2017-08-04  9.3339  8.3634  226.05  20170804          4  230.34  501  225.70   \n",
       "2017-08-07  8.6520  7.9867  227.16  20170807          0  228.99  502  224.77   \n",
       "2017-08-08  8.4744  7.9261  229.45  20170808          1  233.37  503  226.54   \n",
       "\n",
       "              open  p_change  pre_close   volume  \n",
       "2017-08-02  224.12    -1.613     225.60  3688218  \n",
       "2017-08-03  221.00     2.446     221.96  3433083  \n",
       "2017-08-04  227.28    -0.589     227.39  3520335  \n",
       "2017-08-07  225.60     0.491     226.05  2998194  \n",
       "2017-08-08  226.60     1.008     227.16  4021199  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('usBIDU').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "E_DATA_FETCH_FORCE_LOCAL为强制从缓存获取，实际上在做回测的时候，使用的一般都是这种模式，因为比如编写了一个策略进行回测结果度量，通常情况下需要反复的修改策略，重新进行回测，强制使用缓存的好处是：\n",
    "\n",
    "1. 保证使用的数据集没有发生变化，度量结果有可比性\n",
    "2. 提高回测运行效率，特别是针对全市场回测\n",
    "3. 分类数据获取和回测，方便问题排除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_data_fetch_mode = EMarketDataFetchMode.E_DATA_FETCH_FORCE_LOCAL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>atr14</th>\n",
       "      <th>atr21</th>\n",
       "      <th>close</th>\n",
       "      <th>date</th>\n",
       "      <th>date_week</th>\n",
       "      <th>high</th>\n",
       "      <th>key</th>\n",
       "      <th>low</th>\n",
       "      <th>open</th>\n",
       "      <th>p_change</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>8.4744</td>\n",
       "      <td>7.9261</td>\n",
       "      <td>229.45</td>\n",
       "      <td>20170808</td>\n",
       "      <td>1</td>\n",
       "      <td>233.37</td>\n",
       "      <td>503</td>\n",
       "      <td>226.54</td>\n",
       "      <td>226.6</td>\n",
       "      <td>1.008</td>\n",
       "      <td>227.16</td>\n",
       "      <td>4021199</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             atr14   atr21   close      date  date_week    high  key     low  \\\n",
       "2017-08-08  8.4744  7.9261  229.45  20170808          1  233.37  503  226.54   \n",
       "\n",
       "             open  p_change  pre_close   volume  \n",
       "2017-08-08  226.6     1.008     227.16  4021199  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('usBIDU').tail(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面把数据获取模式恢复为默认的E_DATA_FETCH_NORMAL："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_data_fetch_mode = EMarketDataFetchMode.E_DATA_FETCH_NORMAL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 数据存储的切换\n",
    "\n",
    "默认的缓存数据存储模式为CSV，如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<EDataCacheType.E_DATA_CACHE_CSV: 1>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_data_cache_type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "缓存的csv数据存贮文件路径在~/abu/data/csv/，可使运行下面命令直接打开目录："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if abupy.env.g_is_mac_os:\n",
    "    !open $abupy.env.g_project_kl_df_data_csv\n",
    "else:\n",
    "    !echo $abupy.env.g_project_kl_df_data_csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以通过g_data_cache_type切换其它存贮模式，如下使用HDF5进行数据存贮："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_HDF5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "缓存的hdf5文件路径在~/abu/data/df_kl.h5，可使运行下面命令直接打开目录, 或者显示完整路径："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if abupy.env.g_is_mac_os:\n",
    "    !open $abupy.env.g_project_data_dir\n",
    "else:\n",
    "    !echo $abupy.env.g_project_data_dir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>p_change</th>\n",
       "      <th>open</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>date_week</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "      <th>key</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>315.96</td>\n",
       "      <td>319.22</td>\n",
       "      <td>311.41</td>\n",
       "      <td>-1.130</td>\n",
       "      <td>319.00</td>\n",
       "      <td>319.57</td>\n",
       "      <td>119040</td>\n",
       "      <td>20170802</td>\n",
       "      <td>2</td>\n",
       "      <td>14.8108</td>\n",
       "      <td>14.7172</td>\n",
       "      <td>497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>347.09</td>\n",
       "      <td>350.00</td>\n",
       "      <td>343.15</td>\n",
       "      <td>9.853</td>\n",
       "      <td>345.33</td>\n",
       "      <td>315.96</td>\n",
       "      <td>13535033</td>\n",
       "      <td>20170803</td>\n",
       "      <td>3</td>\n",
       "      <td>16.2307</td>\n",
       "      <td>16.8122</td>\n",
       "      <td>498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>356.91</td>\n",
       "      <td>357.27</td>\n",
       "      <td>343.30</td>\n",
       "      <td>2.829</td>\n",
       "      <td>347.00</td>\n",
       "      <td>347.09</td>\n",
       "      <td>9268909</td>\n",
       "      <td>20170804</td>\n",
       "      <td>4</td>\n",
       "      <td>18.5106</td>\n",
       "      <td>20.0786</td>\n",
       "      <td>499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>355.17</td>\n",
       "      <td>359.48</td>\n",
       "      <td>352.75</td>\n",
       "      <td>-0.488</td>\n",
       "      <td>357.35</td>\n",
       "      <td>356.91</td>\n",
       "      <td>6324480</td>\n",
       "      <td>20170807</td>\n",
       "      <td>0</td>\n",
       "      <td>17.9542</td>\n",
       "      <td>19.0535</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>365.22</td>\n",
       "      <td>368.58</td>\n",
       "      <td>357.40</td>\n",
       "      <td>2.830</td>\n",
       "      <td>357.53</td>\n",
       "      <td>355.17</td>\n",
       "      <td>7449837</td>\n",
       "      <td>20170808</td>\n",
       "      <td>1</td>\n",
       "      <td>17.3829</td>\n",
       "      <td>18.0690</td>\n",
       "      <td>501</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             close    high     low  p_change    open  pre_close    volume  \\\n",
       "2017-08-02  315.96  319.22  311.41    -1.130  319.00     319.57    119040   \n",
       "2017-08-03  347.09  350.00  343.15     9.853  345.33     315.96  13535033   \n",
       "2017-08-04  356.91  357.27  343.30     2.829  347.00     347.09   9268909   \n",
       "2017-08-07  355.17  359.48  352.75    -0.488  357.35     356.91   6324480   \n",
       "2017-08-08  365.22  368.58  357.40     2.830  357.53     355.17   7449837   \n",
       "\n",
       "                date  date_week    atr21    atr14  key  \n",
       "2017-08-02  20170802          2  14.8108  14.7172  497  \n",
       "2017-08-03  20170803          3  16.2307  16.8122  498  \n",
       "2017-08-04  20170804          4  18.5106  20.0786  499  \n",
       "2017-08-07  20170807          0  17.9542  19.0535  500  \n",
       "2017-08-08  20170808          1  17.3829  18.0690  501  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('usTSLA').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面仍然切换回默认的csv模式，csv的优点是存贮空间需要小，可以并行读写，且针对不同平台兼容性好："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 数据源的切换\n",
    "\n",
    "如下显示当前的数据源g_market_source为百度数据源："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<EMarketSourceType.E_MARKET_SOURCE_bd: 0>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_market_source"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下想要获取比特币的数据，但是输出显示百度的数据源不支持比特币："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception kline_pd symbol:btc e:BDApi don't support tc_COIN:btc!\n"
     ]
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('btc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "切换数据源为火币数据源，即可正常获取数据，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "      <th>key</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-05</th>\n",
       "      <td>19310.05</td>\n",
       "      <td>21191.00</td>\n",
       "      <td>21540.00</td>\n",
       "      <td>19113.03</td>\n",
       "      <td>23110</td>\n",
       "      <td>20170805</td>\n",
       "      <td>19339.00</td>\n",
       "      <td>5</td>\n",
       "      <td>9.577</td>\n",
       "      <td>499</td>\n",
       "      <td>1652.5121</td>\n",
       "      <td>1667.0629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-06</th>\n",
       "      <td>21198.00</td>\n",
       "      <td>21959.73</td>\n",
       "      <td>22490.00</td>\n",
       "      <td>21126.45</td>\n",
       "      <td>13821</td>\n",
       "      <td>20170806</td>\n",
       "      <td>21191.00</td>\n",
       "      <td>6</td>\n",
       "      <td>3.628</td>\n",
       "      <td>500</td>\n",
       "      <td>1788.7382</td>\n",
       "      <td>1864.9212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>21959.00</td>\n",
       "      <td>22200.00</td>\n",
       "      <td>22833.00</td>\n",
       "      <td>21258.00</td>\n",
       "      <td>12381</td>\n",
       "      <td>20170807</td>\n",
       "      <td>21959.73</td>\n",
       "      <td>0</td>\n",
       "      <td>1.094</td>\n",
       "      <td>501</td>\n",
       "      <td>1775.3984</td>\n",
       "      <td>1835.1984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>22200.00</td>\n",
       "      <td>22588.08</td>\n",
       "      <td>23398.96</td>\n",
       "      <td>21940.00</td>\n",
       "      <td>15933</td>\n",
       "      <td>20170808</td>\n",
       "      <td>22200.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.748</td>\n",
       "      <td>502</td>\n",
       "      <td>1746.6313</td>\n",
       "      <td>1785.0332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-09</th>\n",
       "      <td>22588.08</td>\n",
       "      <td>22380.01</td>\n",
       "      <td>23149.99</td>\n",
       "      <td>22360.00</td>\n",
       "      <td>4679</td>\n",
       "      <td>20170809</td>\n",
       "      <td>22588.08</td>\n",
       "      <td>2</td>\n",
       "      <td>-0.921</td>\n",
       "      <td>503</td>\n",
       "      <td>1674.2093</td>\n",
       "      <td>1673.6941</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                open     close      high       low  volume      date  \\\n",
       "2017-08-05  19310.05  21191.00  21540.00  19113.03   23110  20170805   \n",
       "2017-08-06  21198.00  21959.73  22490.00  21126.45   13821  20170806   \n",
       "2017-08-07  21959.00  22200.00  22833.00  21258.00   12381  20170807   \n",
       "2017-08-08  22200.00  22588.08  23398.96  21940.00   15933  20170808   \n",
       "2017-08-09  22588.08  22380.01  23149.99  22360.00    4679  20170809   \n",
       "\n",
       "            pre_close  date_week  p_change  key      atr21      atr14  \n",
       "2017-08-05   19339.00          5     9.577  499  1652.5121  1667.0629  \n",
       "2017-08-06   21191.00          6     3.628  500  1788.7382  1864.9212  \n",
       "2017-08-07   21959.73          0     1.094  501  1775.3984  1835.1984  \n",
       "2017-08-08   22200.00          1     1.748  502  1746.6313  1785.0332  \n",
       "2017-08-09   22588.08          2    -0.921  503  1674.2093  1673.6941  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_hb_tc\n",
    "ABuSymbolPd.make_kl_df('btc').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类似，下面想要获取期货鸡蛋的数据，但是输出显示火币的数据源不支持期货市场："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception kline_pd symbol:jd0 e:HBApi don't support futures_cn_DCE:JD0!\n"
     ]
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('jd0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "切换数据源为新浪期货数据源，即可正常获取数据，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_sn_futures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "      <th>key</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>4060.0</td>\n",
       "      <td>4107.0</td>\n",
       "      <td>4122.0</td>\n",
       "      <td>4052.0</td>\n",
       "      <td>69136.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>3982.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.139</td>\n",
       "      <td>485</td>\n",
       "      <td>110.7679</td>\n",
       "      <td>109.0249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>4096.0</td>\n",
       "      <td>4088.0</td>\n",
       "      <td>4111.0</td>\n",
       "      <td>4080.0</td>\n",
       "      <td>55790.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4107.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>-0.463</td>\n",
       "      <td>486</td>\n",
       "      <td>112.4254</td>\n",
       "      <td>111.6882</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>4120.0</td>\n",
       "      <td>4260.0</td>\n",
       "      <td>4290.0</td>\n",
       "      <td>4117.0</td>\n",
       "      <td>248006.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4088.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.207</td>\n",
       "      <td>487</td>\n",
       "      <td>118.8413</td>\n",
       "      <td>121.1965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>4298.0</td>\n",
       "      <td>4295.0</td>\n",
       "      <td>4410.0</td>\n",
       "      <td>4272.0</td>\n",
       "      <td>249786.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4260.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.822</td>\n",
       "      <td>488</td>\n",
       "      <td>137.3103</td>\n",
       "      <td>147.9703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>4295.0</td>\n",
       "      <td>4345.0</td>\n",
       "      <td>4358.0</td>\n",
       "      <td>4281.0</td>\n",
       "      <td>175570.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4295.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.164</td>\n",
       "      <td>489</td>\n",
       "      <td>133.7366</td>\n",
       "      <td>141.3076</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              open   close    high     low    volume        date  pre_close  \\\n",
       "2017-08-02  4060.0  4107.0  4122.0  4052.0   69136.0  2.0171e+07     3982.0   \n",
       "2017-08-03  4096.0  4088.0  4111.0  4080.0   55790.0  2.0171e+07     4107.0   \n",
       "2017-08-04  4120.0  4260.0  4290.0  4117.0  248006.0  2.0171e+07     4088.0   \n",
       "2017-08-07  4298.0  4295.0  4410.0  4272.0  249786.0  2.0171e+07     4260.0   \n",
       "2017-08-08  4295.0  4345.0  4358.0  4281.0  175570.0  2.0171e+07     4295.0   \n",
       "\n",
       "            date_week  p_change  key     atr21     atr14  \n",
       "2017-08-02        2.0     3.139  485  110.7679  109.0249  \n",
       "2017-08-03        3.0    -0.463  486  112.4254  111.6882  \n",
       "2017-08-04        4.0     4.207  487  118.8413  121.1965  \n",
       "2017-08-07        0.0     0.822  488  137.3103  147.9703  \n",
       "2017-08-08        1.0     1.164  489  133.7366  141.3076  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('jd0').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. 全市场数据的更新\n",
    "\n",
    "在之前的章节使用abu.run_loop_back进行交易回测，使用的都是沙盒数据，如果需要使用实时数据，特别是回测交易多的情况下，比如全市场测试，推荐在使用abu.run_loop_back进行回测前先使用abu.run_kl_update将数据进行更新。\n",
    "\n",
    "在run_kl_update中会首先强制使用网络数据进行全市场数据更新，在更新完毕后会将全市场的交易数据都写入缓存，再abu.run_loop_back运行回测的时候使用本地数据模式，即实现数据更新与策略回测分离，运行效率提高。\n",
    "\n",
    "下面的代码将分别获取美股，A股，港股，期货，比特币，莱特币6年的交易数据，在后面的章节将分别使用这些数据做回测示例，读者可只获取自己关心的市场的交易数据，不必全部运行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所有获取的数据已经存放在百度云盘上，后面的章节使用的数据都是本节更新的数据，建议直接从云盘下载入库完毕的数据库，不需要从各个数据源再一个一个的下载数据进行入库，百度云地址如下：\n",
    "\n",
    "[csv格式美股，A股，港股，币类，期货6年日k数据](https://pan.baidu.com/s/1geNZgqf) 密码: gvtr\n",
    "\n",
    "下面数据存贮格式为hdf5数据，由于hdf5文件解压后非常大，还需要区分python版本，所以如果没有足够的存贮空间\n",
    "特别是python2下，建议使用csv格式的缓存文件：\n",
    "\n",
    "[mac系统python3 美股，A股，港股，币类，期货6年日k数据](https://pan.baidu.com/s/1o8sldNk) 密码: ecyp\n",
    "\n",
    "[mac系统python2 A股6年日k数据: ](https://pan.baidu.com/s/1bptn25h) 密码: sid8\n",
    "\n",
    "[mac系统python2 美股6年日k数据: ](https://pan.baidu.com/s/1geHR9Lp) 密码: uaww\n",
    "\n",
    "[windows python3 美股，A股，港股，币类，期货6年日k数据](https://pan.baidu.com/s/1bzeVHO) 密码: 3cwe\n",
    "\n",
    "[windows python2 A股6年日k数据: ](https://pan.baidu.com/s/1skZOe7N) 密码: 78mb\n",
    "\n",
    "[windows python2 美股6年日k数据: ](https://pan.baidu.com/s/1i5MNzKx) 密码: 63r3\n",
    "\n",
    "下载完毕上述数据后，hdf5解压得到df_kl.h5文件，csv解压得到csv文件夹，解压后放到下面路径下即可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if abupy.env.g_is_mac_os:\n",
    "    !open $abupy.env.g_project_data_dir\n",
    "else:\n",
    "    !echo $abupy.env.g_project_data_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果不想通过直接下载数据文件的方式，也可下面通过切换至腾讯数据源，然后进行美股数据全市场更新：\n",
    "\n",
    "备注：耗时操作，大概需要运行15分钟左右，可以在做其它事情的时候运行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_tx\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV\n",
    "abu.run_kl_update(start='2011-08-08', end='2017-08-08', market=EMarketTargetType.E_MARKET_TARGET_US, n_jobs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果不想通过直接下载数据文件的方式，也可切换至百度数据源，然后进行A股数据全市场更新：\n",
    "\n",
    "备注：耗时操作，大概需要运行20分钟左右，可以在做其它事情的时候运行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 19min 21s, sys: 2min 48s, total: 22min 9s\n",
      "Wall time: 21min 11s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_bd\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV\n",
    "abu.run_kl_update(start='2011-08-08', end='2017-08-08', market=EMarketTargetType.E_MARKET_TARGET_CN, n_jobs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果不想通过直接下载数据文件的方式，也可切换至网易数据源，然后进行港股数据全市场更新：\n",
    "\n",
    "备注：耗时操作，大概需要运行5分钟左右，可以在做其它事情的时候运行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5min 9s, sys: 50.9 s, total: 6min\n",
      "Wall time: 5min 41s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_nt\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV\n",
    "abu.run_kl_update(start='2011-08-08', end='2017-08-08', market=EMarketTargetType.E_MARKET_TARGET_HK, n_jobs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "切换至新浪期货数据源，然后进行期货数据全市场更新：\n",
    "\n",
    "备注：非耗时操作，大概30秒"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12.9 s, sys: 2.04 s, total: 14.9 s\n",
      "Wall time: 19.5 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_sn_futures\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV\n",
    "abu.run_kl_update(start='2011-08-08', end='2017-08-08', market=EMarketTargetType.E_MARKET_TARGET_FUTURES_CN, n_jobs=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "切换至火币数据源，然后进行比特币，莱特币数据全市场更新：\n",
    "\n",
    "备注：非耗时操作，大概需要5秒"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 718 ms, sys: 418 ms, total: 1.14 s\n",
      "Wall time: 3.19 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_hb_tc\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_CSV\n",
    "abu.run_kl_update(start='2011-08-08', end='2017-08-08', market=EMarketTargetType.E_MARKET_TARGET_TC, n_jobs=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. 接入外部数据源，股票数据源\n",
    "\n",
    "abupy中内置的数据源：\n",
    "\n",
    "* 美股市场：腾讯数据源，百度数据源，网易数据源，新浪数据源\n",
    "* 港股市场：腾讯数据源，百度数据源，网易数据源\n",
    "* A股市场： 腾讯数据源，百度数据源，网易数据源\n",
    "* 期货市场：新浪期货数据源，新浪国际期货数据源\n",
    "* 比特币，莱特币：火币网数据源\n",
    "\n",
    "这些数据源都只是为用户学习使用，并不能保证数据一直通畅，而且如果用户很在乎数据质量，比如有些数据源会有前复权数据错误问题，有些数据源成交量不准确等问题，那么就需要接入用户自己的数据源。\n",
    "\n",
    "下面首先示例接入股票类型的数据源，首先实现一个数据源返回数据解析类，如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@AbuDataParseWrap()\n",
    "class SNUSParser(object):\n",
    "    \"\"\"snus数据源解析类，被类装饰器AbuDataParseWrap装饰\"\"\"\n",
    "    def __init__(self, symbol, json_dict):\n",
    "        \"\"\"\n",
    "        :param symbol: 请求的symbol str对象\n",
    "        :param json_dict: 请求返回的json数据\n",
    "        \"\"\"\n",
    "        data = json_dict\n",
    "        # 为AbuDataParseWrap准备类必须的属性序列\n",
    "        if len(data) > 0:\n",
    "            # 时间日期序列\n",
    "            self.date = [item['d'] for item in data]\n",
    "            # 开盘价格序列\n",
    "            self.open = [item['o'] for item in data]\n",
    "            # 收盘价格序列\n",
    "            self.close = [item['c'] for item in data]\n",
    "            # 最高价格序列\n",
    "            self.high = [item['h'] for item in data]\n",
    "            # 最低价格序列\n",
    "            self.low = [item['l'] for item in data]\n",
    "            # 成交量序列\n",
    "            self.volume = [item['v'] for item in data]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的SNUSParser即为一个数据源返回数据解析类：\n",
    "\n",
    "1. 数据源解析类需要被类装饰器AbuDataParseWrap装饰\n",
    "2. 完成`__init__`函数，根据这里的数据json_dict来拆分成self.date，self.open，self.close，self.high，self.low，self.volume\n",
    "\n",
    "如本例中每一条json数据的格式为：\n",
    "\n",
    "    {'d': '2017-08-08', 'o': '102.29', 'h': '102.35', 'l': '99.16', 'c': '100.07', 'v': '1834706'}\n",
    "\n",
    "init函数目的就是通过拆解网络原始数据形成上述的五个基本序列，之后在类装饰器AbuDataParseWrap中会进行数据的再次加工以及规范标准化处理\n",
    "\n",
    "更多详情请阅读源代码ABuDataParser\n",
    "\n",
    "备注：这里拆解的过程没有在乎效率，只为好理解过程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在编写数据解析类后就需要编写个数据源类，如下所示，以新浪美股数据源为例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class SNUSApi(StockBaseMarket, SupportMixin):\n",
    "    \"\"\"snus数据源，支持美股\"\"\"\n",
    "    K_NET_BASE = \"http://stock.finance.sina.com.cn/usstock/api/json_v2.php/US_MinKService.getDailyK?\" \\\n",
    "                 \"symbol=%s&___qn=3n\"\n",
    "\n",
    "    def __init__(self, symbol):\n",
    "        \"\"\"\n",
    "        :param symbol: Symbol类型对象\n",
    "        \"\"\"\n",
    "        super(SNUSApi, self).__init__(symbol)\n",
    "        # 设置数据源解析对象类\n",
    "        self.data_parser_cls = SNUSParser\n",
    "\n",
    "    def _support_market(self):\n",
    "        \"\"\"声明数据源支持美股\"\"\"\n",
    "        return [EMarketTargetType.E_MARKET_TARGET_US]\n",
    "\n",
    "    def kline(self, n_folds=2, start=None, end=None):\n",
    "        \"\"\"日k线接口\"\"\"\n",
    "        url = SNUSApi.K_NET_BASE % self._symbol.symbol_code\n",
    "        data = ABuNetWork.get(url=url, timeout=(10, 60)).json()\n",
    "        kl_df = self.data_parser_cls(self._symbol, data).df\n",
    "        if kl_df is None:\n",
    "            return None\n",
    "        return StockBaseMarket._fix_kline_pd(kl_df, n_folds, start, end)\n",
    "\n",
    "    def minute(self, n_fold=5, *args, **kwargs):\n",
    "        \"\"\"分钟k线接口\"\"\"\n",
    "        raise NotImplementedError('SNUSApi minute NotImplementedError!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的SNUSApi即为一个股票数据源类：\n",
    "\n",
    "1. 股票类型数据源类需要继承StockBaseMarket\n",
    "2. `__init__`函数中指定数据源解析类\n",
    "3. 数据源类需要混入SupportMixin类，实现_support_market方法，声明支持的市场，本例只支持美股市场\n",
    "4. 数据源类需要实现kline接口，完成获取指定symbol的日线数据，将日线数据交给数据源解析类进行处理\n",
    "5. 数据源类需要实现分钟k线接口，也可以直接raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面示例使用，如下通过check_support检测是否支持A股，结果显示False:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SNUSApi(code_to_symbol('sh601766')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "如下通过check_support检测是否支持美股，结果显示True:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SNUSApi(code_to_symbol('usSINA')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下通过kline接口获取数据，如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>95.91</td>\n",
       "      <td>93.97</td>\n",
       "      <td>96.61</td>\n",
       "      <td>92.37</td>\n",
       "      <td>895105</td>\n",
       "      <td>20170802</td>\n",
       "      <td>95.90</td>\n",
       "      <td>2</td>\n",
       "      <td>-2.013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>94.37</td>\n",
       "      <td>96.26</td>\n",
       "      <td>97.04</td>\n",
       "      <td>93.52</td>\n",
       "      <td>671606</td>\n",
       "      <td>20170803</td>\n",
       "      <td>93.97</td>\n",
       "      <td>3</td>\n",
       "      <td>2.437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>96.87</td>\n",
       "      <td>96.12</td>\n",
       "      <td>98.00</td>\n",
       "      <td>95.54</td>\n",
       "      <td>1052727</td>\n",
       "      <td>20170804</td>\n",
       "      <td>96.26</td>\n",
       "      <td>4</td>\n",
       "      <td>-0.145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>97.25</td>\n",
       "      <td>100.33</td>\n",
       "      <td>100.79</td>\n",
       "      <td>97.16</td>\n",
       "      <td>2330438</td>\n",
       "      <td>20170807</td>\n",
       "      <td>96.12</td>\n",
       "      <td>0</td>\n",
       "      <td>4.380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>102.29</td>\n",
       "      <td>100.07</td>\n",
       "      <td>102.35</td>\n",
       "      <td>99.16</td>\n",
       "      <td>1834706</td>\n",
       "      <td>20170808</td>\n",
       "      <td>100.33</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.259</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              open   close    high    low   volume      date  pre_close  \\\n",
       "2017-08-02   95.91   93.97   96.61  92.37   895105  20170802      95.90   \n",
       "2017-08-03   94.37   96.26   97.04  93.52   671606  20170803      93.97   \n",
       "2017-08-04   96.87   96.12   98.00  95.54  1052727  20170804      96.26   \n",
       "2017-08-07   97.25  100.33  100.79  97.16  2330438  20170807      96.12   \n",
       "2017-08-08  102.29  100.07  102.35  99.16  1834706  20170808     100.33   \n",
       "\n",
       "            date_week  p_change  \n",
       "2017-08-02          2    -2.013  \n",
       "2017-08-03          3     2.437  \n",
       "2017-08-04          4    -0.145  \n",
       "2017-08-07          0     4.380  \n",
       "2017-08-08          1    -0.259  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SNUSApi(code_to_symbol('usSINA')).kline().tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接入SNUSApi至abupy系统中，只需要将数据源类名称直接赋予abupy.env.g_private_data_source："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "abupy.env.g_private_data_source = SNUSApi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面使用make_kl_df接口获取sh601766数据，显示SNUSApi不支持："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception kline_pd symbol:sh601766 e:SNUSApi don't support hs_sh:601766!\n"
     ]
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('sh601766')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面使用make_kl_df接口获取美股数据, 返回的数据即是通过上面实现的SNUSApi返回的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "      <th>key</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>95.91</td>\n",
       "      <td>93.97</td>\n",
       "      <td>96.61</td>\n",
       "      <td>92.37</td>\n",
       "      <td>895105</td>\n",
       "      <td>20170802</td>\n",
       "      <td>95.90</td>\n",
       "      <td>2</td>\n",
       "      <td>-2.013</td>\n",
       "      <td>3.3232</td>\n",
       "      <td>3.2947</td>\n",
       "      <td>499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>94.37</td>\n",
       "      <td>96.26</td>\n",
       "      <td>97.04</td>\n",
       "      <td>93.52</td>\n",
       "      <td>671606</td>\n",
       "      <td>20170803</td>\n",
       "      <td>93.97</td>\n",
       "      <td>3</td>\n",
       "      <td>2.437</td>\n",
       "      <td>3.3411</td>\n",
       "      <td>3.3248</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>96.87</td>\n",
       "      <td>96.12</td>\n",
       "      <td>98.00</td>\n",
       "      <td>95.54</td>\n",
       "      <td>1052727</td>\n",
       "      <td>20170804</td>\n",
       "      <td>96.26</td>\n",
       "      <td>4</td>\n",
       "      <td>-0.145</td>\n",
       "      <td>3.4037</td>\n",
       "      <td>3.4188</td>\n",
       "      <td>501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>97.25</td>\n",
       "      <td>100.33</td>\n",
       "      <td>100.79</td>\n",
       "      <td>97.16</td>\n",
       "      <td>2330438</td>\n",
       "      <td>20170807</td>\n",
       "      <td>96.12</td>\n",
       "      <td>0</td>\n",
       "      <td>4.380</td>\n",
       "      <td>3.5061</td>\n",
       "      <td>3.5670</td>\n",
       "      <td>502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>102.29</td>\n",
       "      <td>100.07</td>\n",
       "      <td>102.35</td>\n",
       "      <td>99.16</td>\n",
       "      <td>1834706</td>\n",
       "      <td>20170808</td>\n",
       "      <td>100.33</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.259</td>\n",
       "      <td>3.7537</td>\n",
       "      <td>3.9220</td>\n",
       "      <td>503</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              open   close    high    low   volume      date  pre_close  \\\n",
       "2017-08-02   95.91   93.97   96.61  92.37   895105  20170802      95.90   \n",
       "2017-08-03   94.37   96.26   97.04  93.52   671606  20170803      93.97   \n",
       "2017-08-04   96.87   96.12   98.00  95.54  1052727  20170804      96.26   \n",
       "2017-08-07   97.25  100.33  100.79  97.16  2330438  20170807      96.12   \n",
       "2017-08-08  102.29  100.07  102.35  99.16  1834706  20170808     100.33   \n",
       "\n",
       "            date_week  p_change   atr21   atr14  key  \n",
       "2017-08-02          2    -2.013  3.3232  3.2947  499  \n",
       "2017-08-03          3     2.437  3.3411  3.3248  500  \n",
       "2017-08-04          4    -0.145  3.4037  3.4188  501  \n",
       "2017-08-07          0     4.380  3.5061  3.5670  502  \n",
       "2017-08-08          1    -0.259  3.7537  3.9220  503  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABuSymbolPd.make_kl_df('usSINA').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. 接入外部数据源，期货数据源\n",
    "\n",
    "下面示例接入期货类型的数据源，首先实现一个数据源返回数据解析类，如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@AbuDataParseWrap()\n",
    "class SNFuturesParser(object):\n",
    "    \"\"\"示例期货数据源解析类，被类装饰器AbuDataParseWrap装饰\"\"\"\n",
    "    # noinspection PyUnusedLocal\n",
    "    def __init__(self, symbol, json_dict):\n",
    "        \"\"\"\n",
    "        :param symbol: 请求的symbol str对象\n",
    "        :param json_dict: 请求返回的json数据\n",
    "        \"\"\"\n",
    "        data = json_dict\n",
    "        # 为AbuDataParseWrap准备类必须的属性序列\n",
    "        if len(data) > 0:\n",
    "            # 时间日期序列\n",
    "            self.date = [item[0] for item in data]\n",
    "            # 开盘价格序列\n",
    "            self.open = [item[1] for item in data]\n",
    "            # 最高价格序列\n",
    "            self.high = [item[2] for item in data]\n",
    "            # 最低价格序列\n",
    "            self.low = [item[3] for item in data]\n",
    "            # 收盘价格序列\n",
    "            self.close = [item[4] for item in data]\n",
    "            # 成交量序列\n",
    "            self.volume = [item[5] for item in data]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的SNFuturesParser与SNUSParser基本相同：被类装饰器AbuDataParseWrap装饰，实现`__init__`函数\n",
    "\n",
    "如本例中每一条json数据的格式为, 所以通过序号解析对应的值：\n",
    "\n",
    "    ['2017-08-08', '4295.000', '4358.000', '4281.000', '4345.000', '175570']\n",
    "\n",
    "在编写数据解析类后就需要编写个数据源类，如下所示，以新浪期货数据源为例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class SNFuturesApi(FuturesBaseMarket, SupportMixin):\n",
    "    \"\"\"sn futures数据源，支持国内期货\"\"\"\n",
    "\n",
    "    K_NET_BASE = \"http://stock.finance.sina.com.cn/futures/api/json_v2.php/\" \\\n",
    "                 \"IndexService.getInnerFuturesDailyKLine?symbol=%s\"\n",
    "\n",
    "    def __init__(self, symbol):\n",
    "        \"\"\"\n",
    "        :param symbol: Symbol类型对象\n",
    "        \"\"\"\n",
    "        super(SNFuturesApi, self).__init__(symbol)\n",
    "        # 设置数据源解析对象类\n",
    "        self.data_parser_cls = SNFuturesParser\n",
    "\n",
    "    def _support_market(self):\n",
    "        \"\"\"声明数据源支持期货数据\"\"\"\n",
    "        return [EMarketTargetType.E_MARKET_TARGET_FUTURES_CN]\n",
    "\n",
    "    def kline(self, n_folds=2, start=None, end=None):\n",
    "        \"\"\"日k线接口\"\"\"\n",
    "        url = SNFuturesApi.K_NET_BASE % self._symbol.symbol_code\n",
    "        data = ABuNetWork.get(url=url, timeout=(10, 60)).json()\n",
    "        kl_df = self.data_parser_cls(self._symbol, data).df\n",
    "        if kl_df is None:\n",
    "            return None\n",
    "        return FuturesBaseMarket._fix_kline_pd(kl_df, n_folds, start, end)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的SNFuturesApi即为一个期货数据源类：\n",
    "\n",
    "1. 期货类型数据源类需要继承FuturesBaseMarket\n",
    "2. `__init__`函数中指定数据源解析类\n",
    "3. 数据源类需要混入SupportMixin类，实现_support_market方法，声明支持的市场，本例只支持期货市场\n",
    "4. 数据源类需要实现kline接口，完成获取指定symbol的日线数据，将日线数据交给数据源解析类进行处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面示例使用，如下通过check_support检测是否支持美股，结果显示False:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SNFuturesApi(code_to_symbol('usSINA')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下通过check_support检测是否支持期货，结果显示True："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SNFuturesApi(code_to_symbol('jd0')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接入SNFuturesApi至abupy系统中, 使用make_kl_df接口获取期货鸡蛋连续数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "      <th>key</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-02</th>\n",
       "      <td>4060.0</td>\n",
       "      <td>4107.0</td>\n",
       "      <td>4122.0</td>\n",
       "      <td>4052.0</td>\n",
       "      <td>69136.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>3982.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.139</td>\n",
       "      <td>485</td>\n",
       "      <td>110.7679</td>\n",
       "      <td>109.0249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-03</th>\n",
       "      <td>4096.0</td>\n",
       "      <td>4088.0</td>\n",
       "      <td>4111.0</td>\n",
       "      <td>4080.0</td>\n",
       "      <td>55790.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4107.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>-0.463</td>\n",
       "      <td>486</td>\n",
       "      <td>112.4254</td>\n",
       "      <td>111.6882</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-04</th>\n",
       "      <td>4120.0</td>\n",
       "      <td>4260.0</td>\n",
       "      <td>4290.0</td>\n",
       "      <td>4117.0</td>\n",
       "      <td>248006.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4088.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.207</td>\n",
       "      <td>487</td>\n",
       "      <td>118.8413</td>\n",
       "      <td>121.1965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>4298.0</td>\n",
       "      <td>4295.0</td>\n",
       "      <td>4410.0</td>\n",
       "      <td>4272.0</td>\n",
       "      <td>249786.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4260.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.822</td>\n",
       "      <td>488</td>\n",
       "      <td>137.3103</td>\n",
       "      <td>147.9703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>4295.0</td>\n",
       "      <td>4345.0</td>\n",
       "      <td>4358.0</td>\n",
       "      <td>4281.0</td>\n",
       "      <td>175570.0</td>\n",
       "      <td>2.0171e+07</td>\n",
       "      <td>4295.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.164</td>\n",
       "      <td>489</td>\n",
       "      <td>133.7366</td>\n",
       "      <td>141.3076</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              open   close    high     low    volume        date  pre_close  \\\n",
       "2017-08-02  4060.0  4107.0  4122.0  4052.0   69136.0  2.0171e+07     3982.0   \n",
       "2017-08-03  4096.0  4088.0  4111.0  4080.0   55790.0  2.0171e+07     4107.0   \n",
       "2017-08-04  4120.0  4260.0  4290.0  4117.0  248006.0  2.0171e+07     4088.0   \n",
       "2017-08-07  4298.0  4295.0  4410.0  4272.0  249786.0  2.0171e+07     4260.0   \n",
       "2017-08-08  4295.0  4345.0  4358.0  4281.0  175570.0  2.0171e+07     4295.0   \n",
       "\n",
       "            date_week  p_change  key     atr21     atr14  \n",
       "2017-08-02        2.0     3.139  485  110.7679  109.0249  \n",
       "2017-08-03        3.0    -0.463  486  112.4254  111.6882  \n",
       "2017-08-04        4.0     4.207  487  118.8413  121.1965  \n",
       "2017-08-07        0.0     0.822  488  137.3103  147.9703  \n",
       "2017-08-08        1.0     1.164  489  133.7366  141.3076  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_private_data_source = SNFuturesApi\n",
    "ABuSymbolPd.make_kl_df('jd0').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与期货市场类似的是美股期权市场，abupy同样支持美股期权市场的回测分析等操作，但由于暂时没有合适的可对外的数据源提供，所以暂时无示例，用户也可以在abupy中接入自己的美股期权数据源。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. 接入外部数据源，比特币，莱特币数据源\n",
    "\n",
    "下面示例接入币类市场数据源，首先实现一个数据源返回数据解析类，如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@AbuDataParseWrap()\n",
    "class HBTCParser(object):\n",
    "    \"\"\"示例币类市场数据源解析类，被类装饰器AbuDataParseWrap装饰\"\"\"\n",
    "    def __init__(self, symbol, json_dict):\n",
    "        \"\"\"\n",
    "        :param symbol: 请求的symbol str对象\n",
    "        :param json_dict: 请求返回的json数据\n",
    "        \"\"\"\n",
    "        data = json_dict\n",
    "        # 为AbuDataParseWrap准备类必须的属性序列\n",
    "        if len(data) > 0:\n",
    "            # 时间日期序列\n",
    "            self.date = [item[0] for item in data]\n",
    "            # 开盘价格序列\n",
    "            self.open = [item[1] for item in data]\n",
    "            # 最高价格序列\n",
    "            self.high = [item[2] for item in data]\n",
    "            # 最低价格序列\n",
    "            self.low = [item[3] for item in data]\n",
    "            # 收盘价格序列\n",
    "            self.close = [item[4] for item in data]\n",
    "            # 成交量序列\n",
    "            self.volume = [item[5] for item in data]\n",
    "\n",
    "            # 时间日期进行格式转化，转化为如2017-07-26格式字符串\n",
    "            self.date = list(map(lambda date: ABuDateUtil.fmt_date(date), self.date))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的HBTCParser与上面的数据解析类基本相同：被类装饰器AbuDataParseWrap装饰，实现`__init__`函数\n",
    "\n",
    "如本例中每一条json数据的格式为, 所以通过序号解析对应的值：\n",
    "\n",
    "    ['20170809000000000', 22588.08, 23149.99, 22250.0, 22730.0, 7425.5134]\n",
    "    \n",
    "所以需要使用ABuDateUtil.fmt_date将时间进行格式转化，下面编写对应的数据源类，如下所示，以火币数据源为例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class HBApi(TCBaseMarket, SupportMixin):\n",
    "    \"\"\"hb数据源，支持币类，比特币，莱特币\"\"\"\n",
    "\n",
    "    K_NET_BASE = 'https://www.huobi.com/qt/staticmarket/%s_kline_100_json.js?length=%d'\n",
    "\n",
    "    def __init__(self, symbol):\n",
    "        \"\"\"\n",
    "        :param symbol: Symbol类型对象\n",
    "        \"\"\"\n",
    "        super(HBApi, self).__init__(symbol)\n",
    "        # 设置数据源解析对象类\n",
    "        self.data_parser_cls = HBTCParser\n",
    "\n",
    "    def _support_market(self):\n",
    "        \"\"\"只支持币类市场\"\"\"\n",
    "        return [EMarketTargetType.E_MARKET_TARGET_TC]\n",
    "\n",
    "    def kline(self, n_folds=2, start=None, end=None):\n",
    "        \"\"\"日k线接口\"\"\"\n",
    "        req_cnt = n_folds * 365\n",
    "        if start is not None and end is not None:\n",
    "            # 向上取整数，下面使用_fix_kline_pd再次进行剪裁, 要使用current_str_date不能是end\n",
    "            folds = math.ceil(ABuDateUtil.diff(ABuDateUtil.date_str_to_int(start),\n",
    "                                               ABuDateUtil.current_str_date()) / 365)\n",
    "            req_cnt = folds * 365\n",
    "\n",
    "        url = HBApi.K_NET_BASE % (self._symbol.symbol_code, req_cnt)\n",
    "        data = ABuNetWork.get(url=url, timeout=(10, 60)).json()\n",
    "        kl_df = self.data_parser_cls(self._symbol, data).df\n",
    "        if kl_df is None:\n",
    "            return None\n",
    "        return TCBaseMarket._fix_kline_pd(kl_df, n_folds, start, end)\n",
    "\n",
    "    def minute(self, *args, **kwargs):\n",
    "        \"\"\"分钟k线接口\"\"\"\n",
    "        raise NotImplementedError('HBApi minute NotImplementedError!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面编写的HBApi即为一个支持比特币，莱特币数据源类：\n",
    "\n",
    "1. 期货类型数据源类需要继承TCBaseMarket\n",
    "2. `__init__`函数中指定数据源解析类\n",
    "3. 数据源类需要混入SupportMixin类，实现_support_market方法，声明支持的市场，本例只支持币类市场\n",
    "4. 数据源类需要实现kline接口，完成获取指定symbol的日线数据，将日线数据交给数据源解析类进行处理\n",
    "4. 数据源类需要实现分钟k线接口minute，也可raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面示例使用，如下通过check_support检测是否支持美股，结果显示False:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HBApi(code_to_symbol('usSINA')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下通过check_support检测是否支持比特币，结果显示True："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HBApi(code_to_symbol('btc')).check_support(rs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接入HBApi至abupy系统中, 使用make_kl_df接口获取期货比特币数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>open</th>\n",
       "      <th>close</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "      <th>date</th>\n",
       "      <th>pre_close</th>\n",
       "      <th>date_week</th>\n",
       "      <th>p_change</th>\n",
       "      <th>key</th>\n",
       "      <th>atr21</th>\n",
       "      <th>atr14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-08-05</th>\n",
       "      <td>19310.05</td>\n",
       "      <td>21191.00</td>\n",
       "      <td>21540.00</td>\n",
       "      <td>19113.03</td>\n",
       "      <td>23110</td>\n",
       "      <td>20170805</td>\n",
       "      <td>19339.00</td>\n",
       "      <td>5</td>\n",
       "      <td>9.577</td>\n",
       "      <td>499</td>\n",
       "      <td>1652.5121</td>\n",
       "      <td>1667.0629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-06</th>\n",
       "      <td>21198.00</td>\n",
       "      <td>21959.73</td>\n",
       "      <td>22490.00</td>\n",
       "      <td>21126.45</td>\n",
       "      <td>13821</td>\n",
       "      <td>20170806</td>\n",
       "      <td>21191.00</td>\n",
       "      <td>6</td>\n",
       "      <td>3.628</td>\n",
       "      <td>500</td>\n",
       "      <td>1788.7382</td>\n",
       "      <td>1864.9212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-07</th>\n",
       "      <td>21959.00</td>\n",
       "      <td>22200.00</td>\n",
       "      <td>22833.00</td>\n",
       "      <td>21258.00</td>\n",
       "      <td>12381</td>\n",
       "      <td>20170807</td>\n",
       "      <td>21959.73</td>\n",
       "      <td>0</td>\n",
       "      <td>1.094</td>\n",
       "      <td>501</td>\n",
       "      <td>1775.3984</td>\n",
       "      <td>1835.1984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-08</th>\n",
       "      <td>22200.00</td>\n",
       "      <td>22588.08</td>\n",
       "      <td>23398.96</td>\n",
       "      <td>21940.00</td>\n",
       "      <td>15933</td>\n",
       "      <td>20170808</td>\n",
       "      <td>22200.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.748</td>\n",
       "      <td>502</td>\n",
       "      <td>1746.6313</td>\n",
       "      <td>1785.0332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-08-09</th>\n",
       "      <td>22588.08</td>\n",
       "      <td>22380.01</td>\n",
       "      <td>23149.99</td>\n",
       "      <td>22360.00</td>\n",
       "      <td>4679</td>\n",
       "      <td>20170809</td>\n",
       "      <td>22588.08</td>\n",
       "      <td>2</td>\n",
       "      <td>-0.921</td>\n",
       "      <td>503</td>\n",
       "      <td>1674.2093</td>\n",
       "      <td>1673.6941</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                open     close      high       low  volume      date  \\\n",
       "2017-08-05  19310.05  21191.00  21540.00  19113.03   23110  20170805   \n",
       "2017-08-06  21198.00  21959.73  22490.00  21126.45   13821  20170806   \n",
       "2017-08-07  21959.00  22200.00  22833.00  21258.00   12381  20170807   \n",
       "2017-08-08  22200.00  22588.08  23398.96  21940.00   15933  20170808   \n",
       "2017-08-09  22588.08  22380.01  23149.99  22360.00    4679  20170809   \n",
       "\n",
       "            pre_close  date_week  p_change  key      atr21      atr14  \n",
       "2017-08-05   19339.00          5     9.577  499  1652.5121  1667.0629  \n",
       "2017-08-06   21191.00          6     3.628  500  1788.7382  1864.9212  \n",
       "2017-08-07   21959.73          0     1.094  501  1775.3984  1835.1984  \n",
       "2017-08-08   22200.00          1     1.748  502  1746.6313  1785.0332  \n",
       "2017-08-09   22588.08          2    -0.921  503  1674.2093  1673.6941  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abupy.env.g_private_data_source = HBApi\n",
    "ABuSymbolPd.make_kl_df('btc').tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结：\n",
    "\n",
    "abupy内置的数据源仅供学习使用，abupy提供了接入外部数据源的接口和规范，推荐购买数据源接入使用，特别是在实盘中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### abu量化文档目录章节\n",
    "\n",
    "1. [择时策略的开发](http://www.abuquant.com/lecture/lecture_1.html)\n",
    "2. [择时策略的优化](http://www.abuquant.com/lecture/lecture_2.html)\n",
    "3. [滑点策略与交易手续费](http://www.abuquant.com/lecture/lecture_3.html)\n",
    "4. [多支股票择时回测与仓位管理](http://www.abuquant.com/lecture/lecture_4.html)\n",
    "5. [选股策略的开发](http://www.abuquant.com/lecture/lecture_5.html)\n",
    "6. [回测结果的度量](http://www.abuquant.com/lecture/lecture_6.html)\n",
    "7. [寻找策略最优参数和评分](http://www.abuquant.com/lecture/lecture_7.html)\n",
    "8. [A股市场的回测](http://www.abuquant.com/lecture/lecture_8.html)\n",
    "9. [港股市场的回测](http://www.abuquant.com/lecture/lecture_9.html)\n",
    "10. [比特币，莱特币的回测](http://www.abuquant.com/lecture/lecture_10.html)\n",
    "11. [期货市场的回测](http://www.abuquant.com/lecture/lecture_11.html)\n",
    "12. [机器学习与比特币示例](http://www.abuquant.com/lecture/lecture_12.html)\n",
    "13. [量化技术分析应用](http://www.abuquant.com/lecture/lecture_13.html)\n",
    "14. [量化相关性分析应用](http://www.abuquant.com/lecture/lecture_14.html)\n",
    "15. [量化交易和搜索引擎](http://www.abuquant.com/lecture/lecture_15.html)\n",
    "16. [UMP主裁交易决策](http://www.abuquant.com/lecture/lecture_16.html)\n",
    "17. [UMP边裁交易决策](http://www.abuquant.com/lecture/lecture_17.html)\n",
    "18. [自定义裁判决策交易](http://www.abuquant.com/lecture/lecture_18.html)\n",
    "19. [数据源](http://www.abuquant.com/lecture/lecture_19.html)\n",
    "20. [A股全市场回测](http://www.abuquant.com/lecture/lecture_20.html)\n",
    "21. [A股UMP决策](http://www.abuquant.com/lecture/lecture_21.html)\n",
    "22. [美股全市场回测](http://www.abuquant.com/lecture/lecture_22.html)\n",
    "23. [美股UMP决策](http://www.abuquant.com/lecture/lecture_23.html)\n",
    "\n",
    "abu量化系统文档教程持续更新中，请关注公众号中的更新提醒。\n",
    "\n",
    "#### 《量化交易之路》目录章节及随书代码地址\n",
    "\n",
    "1. [第二章 量化语言——Python](https://github.com/bbfamily/abu/tree/master/ipython/第二章-量化语言——Python.ipynb)\n",
    "2. [第三章 量化工具——NumPy](https://github.com/bbfamily/abu/tree/master/ipython/第三章-量化工具——NumPy.ipynb)\n",
    "3. [第四章 量化工具——pandas](https://github.com/bbfamily/abu/tree/master/ipython/第四章-量化工具——pandas.ipynb)\n",
    "4. [第五章 量化工具——可视化](https://github.com/bbfamily/abu/tree/master/ipython/第五章-量化工具——可视化.ipynb)\n",
    "5. [第六章 量化工具——数学：你一生的追求到底能带来多少幸福](https://github.com/bbfamily/abu/tree/master/ipython/第六章-量化工具——数学.ipynb)\n",
    "6. [第七章 量化系统——入门：三只小猪股票投资的故事](https://github.com/bbfamily/abu/tree/master/ipython/第七章-量化系统——入门.ipynb)\n",
    "7. [第八章 量化系统——开发](https://github.com/bbfamily/abu/tree/master/ipython/第八章-量化系统——开发.ipynb)\n",
    "8. [第九章 量化系统——度量与优化](https://github.com/bbfamily/abu/tree/master/ipython/第九章-量化系统——度量与优化.ipynb)\n",
    "9. [第十章 量化系统——机器学习•猪老三](https://github.com/bbfamily/abu/tree/master/ipython/第十章-量化系统——机器学习•猪老三.ipynb)\n",
    "10. [第十一章 量化系统——机器学习•ABU](https://github.com/bbfamily/abu/tree/master/ipython/第十一章-量化系统——机器学习•ABU.ipynb)\n",
    "11. [附录A 量化环境部署](https://github.com/bbfamily/abu/tree/master/ipython/附录A-量化环境部署.ipynb)\n",
    "12. [附录B 量化相关性分析](https://github.com/bbfamily/abu/tree/master/ipython/附录B-量化相关性分析.ipynb)\n",
    "13. [附录C 量化统计分析及指标应用](https://github.com/bbfamily/abu/tree/master/ipython/附录C-量化统计分析及指标应用.ipynb)\n",
    "\n",
    "[更多阿布量化量化技术文章](http://www.abuquant.com/article)\n",
    "\n",
    "\n",
    "更多关于量化交易相关请阅读[《量化交易之路》](http://www.abuquant.com/books/quantify-trading-road.html)\n",
    "\n",
    "更多关于量化交易与机器学习相关请阅读[《机器学习之路》](http://www.abuquant.com/books/machine-learning-road.html)\n",
    "\n",
    "更多关于abu量化系统请关注微信公众号: abu_quant\n",
    "\n",
    "![](./image/qrcode.jpg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}