博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
python构建对等网络_用Python构建网络流量监控器
阅读量:2510 次
发布时间:2019-05-11

本文共 23971 字,大约阅读时间需要 79 分钟。

python构建对等网络

In today’s tech world, it is important to know where the traffic on your website comes from because it helps content creators and website owners understand their visitors better. Website owners can further use this information to improve their website, render better advertisements or generally improve their content based on this data.

在当今的技术世界中,重要的是要知道您网站的流量来自何处,因为它可以帮助内容创建者和网站所有者更好地了解其访问者。 网站所有者可以根据此数据进一步使用此信息来改进其网站,提供更好的广告或总体上改进其内容。

In this tutorial, we will build a simple website and a traffic monitor, the traffic monitor will display details about visitors on the website. For every visitor, there will multiple columns such as time of visit, continent, country, city, operating system, browser, and pages visited. Here is a display of what the final application will look like:

在本教程中,我们将构建一个简单的网站和一个流量监视器,该流量监视器将在网站上显示有关访问者的详细信息。 对于每个访问者,都会有多个列,例如访问时间,大洲,国家/地区,城市,操作系统,浏览器和访问页面。 这是最终应用程序的显示:

python-web-traffic-demo

From the image above, we can see an incognito window to the left and the traffic monitor dashboard window to the right. In the incognito window, we visit our website on the local address and it updates in realtime on the traffic monitor dashboard. It registers that a new user has visited the website and displays the associated data. The user data is also saved to the database so that whenever we open the traffic monitor dashboard, the data persists.

在上图中,我们可以在左侧看到隐身窗口,在右侧可以看到流量监控器仪表板窗口。 在隐身窗口中,我们访问本地地址为网站,并在流量监控仪表板上实时更新。 它注册一个新用户已访问该网站并显示相关数据。 用户数据也保存到数据库中,以便每当我们打开流量监控器仪表板时,数据就会保留。

We will build the backend server for this application using the Python framework called The source code for this tutorial is available here on .

我们将使用称为的Python框架为此应用程序构建后端服务器 本教程的源代码可在上 。

( )

To follow along with this tutorial, a basic knowledge of Python, Flask, and JavaScript (ES6 syntax) is required. You will also need the following installed:

为了跟随本教程,需要具备Python,Flask和JavaScript(ES6语法)的基础知识。 您还需要安装以下软件:

Virtualenv is great for creating isolated Python environments, so we can install dependencies in an isolated environment, and not pollute our global packages directory.

Virtualenv非常适合创建隔离的Python环境,因此我们可以在隔离的环境中安装依赖项,而不会污染我们的全局包目录。

Let’s install virtualenv with this command:

让我们使用以下命令安装virtualenv

$ pip install virtualenv

( )

Let’s create our project folder, and activate a virtual environment in it. Run the commands below:

让我们创建项目文件夹,并在其中激活虚拟环境。 运行以下命令:

$ mkdir python-pusher-traffic-monitor    $ cd python-pusher-traffic-monitor    $ virtualenv .venv    $ source .venv/bin/activate # Linux based systems    $ \path\to\env\Scripts\activate # Windows users

Now that we have the virtual environment setup, we can install Flask within it with this command:

现在我们有了虚拟环境设置,我们可以使用以下命令在其中安装Flask:

$ pip install flask

Let’s also install httpagentparser:

我们还要安装httpagentparser

$ pip install httpagentparser

Before we do anything else, we need to install the Pusher library as we will need that for realtime updates.

在执行其他任何操作之前,我们需要安装Pusher库,因为我们需要它来进行实时更新。

( )

The first step will be to get a Pusher Channels application. We will need the application credentials for our realtime features to work.

第一步将是获得Pusher Channels应用程序。 我们将需要应用程序凭据才能使我们的实时功能正常工作。

Go to the and create an account. After creating an account, you should create a new application. Follow the application creation wizard and then you should be given your application credentials, we will use this later in the article.

转到并创建一个帐户。 创建帐户后,您应该创建一个新的应用程序。 遵循应用程序创建向导,然后应为您提供应用程序凭据,我们将在本文后面使用它。

python-realtime-poll-keys

We also need to install the Pusher Python Library to send events to Pusher. Install this using the command below:

我们还需要安装Pusher Python库以将事件发送到Pusher。 使用以下命令安装它:

$ pip install pusher

( )

We don’t need to create so many files and folders for this application since it’s a simple one. Here’s the file/folder structure:

我们不需要为此应用程序创建那么多文件和文件夹,因为它很简单。 这是文件/文件夹结构:

├── python-pusher-traffic-monitor           ├── app.py           ├── dbsetup.py           ├── static           └── templates

The static folder will contain the static files to be used as is defined by . The templates folder will contain the HTML templates. In our application, app.py is the main entry point and will contain our server-side code. To keep things modular, we will write all the code that we need to interact with the database in dbsetup.py.

static文件夹将包含定义的要使用的静态文件。 templates文件夹将包含HTML模板。 在我们的应用程序中, app.py是主要入口点,将包含我们的服务器端代码。 为了使事情保持模块化,我们将在dbsetup.pydbsetup.py与数据库交互所需的所有代码。

Create the app.py and dbsetup.py files, and then the static and templates folders.

创建app.pydbsetup.py文件,然后创建statictemplates文件夹。

( )

Let’s start by working on the file that will handle the database operations. In the dbsetup.py file, we will write all the code that is needed for creating a database and interacting with it. Open the dbsetup.py file and paste the following:

让我们从处理数据库操作的文件开始。 在dbsetup.py文件中,我们将编写创建数据库并与之交互所需的所有代码。 打开dbsetup.py文件并粘贴以下内容:

import sqlite3    from sqlite3 import Error    def create_connection(database):        try:            conn = sqlite3.connect(database, isolation_level=None, check_same_thread = False)            conn.row_factory = lambda c, r: dict(zip([col[0] for col in c.description], r))            return conn        except Error as e:            print(e)    def create_table(c,sql):        c.execute(sql)    def update_or_create_page(c,data):        sql = "SELECT * FROM pages where name=? and session=?"        c.execute(sql,data[:-1])        result = c.fetchone()        if result == None:            create_pages(c,data)        else:            print(result)            update_pages(c, result['id'])    def create_pages(c, data):        print(data)        sql = ''' INSERT INTO pages(name,session,first_visited)                  VALUES (?,?,?) '''        c.execute(sql, data)    def update_pages(c, pageId):        print(pageId)        sql = ''' UPDATE pages                  SET visits = visits+1                   WHERE id = ?'''        c.execute(sql, [pageId])    def create_session(c, data):        sql = ''' INSERT INTO sessions(ip, continent, country, city, os, browser, session, created_at)                  VALUES (?,?,?,?,?,?,?,?) '''        c.execute(sql, data)    def select_all_sessions(c):        sql = "SELECT * FROM sessions"        c.execute(sql)        rows = c.fetchall()        return rows    def select_all_pages(c):        sql = "SELECT * FROM pages"        c.execute(sql)        rows = c.fetchall()        return rows    def select_all_user_visits(c, session_id):        sql = "SELECT * FROM pages where session =?"        c.execute(sql,[session_id])        rows = c.fetchall()        return rows    def main():        database = "./pythonsqlite.db"        sql_create_pages = """             CREATE TABLE IF NOT EXISTS pages (                id integer PRIMARY KEY,                name varchar(225) NOT NULL,                session varchar(255) NOT NULL,                first_visited datetime NOT NULL,                visits integer NOT NULL Default 1            );         """        sql_create_session = """             CREATE TABLE IF NOT EXISTS sessions (                id integer PRIMARY KEY,                ip varchar(225) NOT NULL,                continent varchar(225) NOT NULL,                 country varchar(225) NOT NULL,                city varchar(225) NOT NULL,                 os varchar(225) NOT NULL,                 browser varchar(225) NOT NULL,                 session varchar(225) NOT NULL,                created_at datetime NOT NULL            );         """        # create a database connection        conn = create_connection(database)        if conn is not None:            # create tables            create_table(conn, sql_create_pages)            create_table(conn, sql_create_session)            print("Connection established!")        else:            print("Could not establish connection")    if __name__ == '__main__':        main()

Next, run the dbsetup.py file so that it creates a new SQLite database for us. We can run it with this command:

接下来,运行dbsetup.py文件,以便它为我们创建一个新SQLite数据库。 我们可以使用以下命令运行它:

$ python dbsetup.py

We should see this text logged to the terminal — ‘Connection established!’ — and there should be a new file — pythonsqlite.db — added to the project’s root directory.

我们应该看到此文本记录到终端-“连接已建立!” —并且应该在项目的根目录中添加一个新文件pythonsqlite.db

Next, let’s open the app.py file and start writing the backend code that will handle incoming requests. We are going to register multiple routes here. Four of these routes will load a webpage each while the other routes will process submitted data and return a JSON response.

接下来,让我们打开app.py文件,并开始编写将处理传入请求的后端代码。 我们将在这里注册多条路线。 其中四个路由将分别加载一个网页,而其他路由将处理提交的数据并返回JSON响应。

We will also create a Pusher instance and use it to broadcast data through two channels that we will shortly define — pageview and numbers — in the application. After that, we will import the database handler methods we defined in dbsetup.py so that we can use them in the app.py file. Open the app.py file and paste the following:

我们还将创建一个Pusher实例,并使用它通过两个通道(我们即将在应用程序中定义的页面浏览量和数字)广播数据。 之后,我们将导入在dbsetup.py定义的数据库处理程序方法,以便可以在app.py文件中使用它们。 打开app.py文件并粘贴以下内容:

from flask import Flask, render_template, request, session, jsonify    import urllib.request    from pusher import Pusher    from datetime import datetime    import httpagentparser    import json    import os    import hashlib    from dbsetup import create_connection, create_session, update_or_create_page, select_all_sessions, select_all_user_visits, select_all_pages    app = Flask(__name__)    app.secret_key = os.urandom(24)    # configure pusher object    pusher = Pusher(    app_id='PUSHER_APP_ID',    key='PUSHER_APP_KEY',    secret='PUSHER_APP_SECRET',    cluster='PUSHER_APP_CLUSTER',    ssl=True)    database = "./pythonsqlite.db"    conn = create_connection(database)    c = conn.cursor()    userOS = None    userIP = None    userCity = None    userBrowser = None    userCountry = None    userContinent = None    sessionID = None    def main():        global conn, c    def parseVisitor(data):        update_or_create_page(c,data)        pusher.trigger(u'pageview', u'new', {
u'page': data[0], u'session': sessionID, u'ip': userIP }) pusher.trigger(u'numbers', u'update', {
u'page': data[0], u'session': sessionID, u'ip': userIP }) @app.before_request def getAnalyticsData(): global userOS, userBrowser, userIP, userContinent, userCity, userCountry,sessionID userInfo = httpagentparser.detect(request.headers.get('User-Agent')) userOS = userInfo['platform']['name'] userBrowser = userInfo['browser']['name'] userIP = "72.229.28.185" if request.remote_addr == '127.0.0.1' else request.remote_addr api = "https://www.iplocate.io/api/lookup/" + userIP try: resp = urllib.request.urlopen(api) result = resp.read() result = json.loads(result.decode("utf-8")) userCountry = result["country"] userContinent = result["continent"] userCity = result["city"] except: print("Could not find: ", userIP) getSession() def getSession(): global sessionID time = datetime.now().replace(microsecond=0) if 'user' not in session: lines = (str(time)+userIP).encode('utf-8') session['user'] = hashlib.md5(lines).hexdigest() sessionID = session['user'] pusher.trigger(u'session', u'new', {
u'ip': userIP, u'continent': userContinent, u'country': userCountry, u'city': userCity, u'os': userOS, u'browser': userBrowser, u'session': sessionID, u'time': str(time), }) data = [userIP, userContinent, userCountry, userCity, userOS, userBrowser, sessionID, time] create_session(c,data) else: sessionID = session['user'] @app.route('/') def index(): data = ['home', sessionID, str(datetime.now().replace(microsecond=0))] parseVisitor(data) return render_template('index.html') @app.route('/about') def about(): data = ['about',sessionID, str(datetime.now().replace(microsecond=0))] parseVisitor(data) return render_template('about.html') @app.route('/dashboard') def dashboard(): return render_template('dashboard.html') @app.route('/dashboard/
', methods=['GET']) def sessionPages(session_id): result = select_all_user_visits(c,session_id) return render_template("dashboard-single.html",data=result) @app.route('/get-all-sessions') def get_all_sessions(): data = [] dbRows = select_all_sessions(c) for row in dbRows: data.append({
'ip' : row['ip'], 'continent' : row['continent'], 'country' : row['country'], 'city' : row['city'], 'os' : row['os'], 'browser' : row['browser'], 'session' : row['session'], 'time' : row['created_at'] }) return jsonify(data) if __name__ == '__main__': main() app.run(debug=True)

In the code above, we imported the required modules and objects, then we initialized a new Flask application. We also initialized and configured Pusher before defining all the routes and their handler functions.

在上面的代码中,我们导入了所需的模块和对象,然后初始化了一个新的Flask应用程序。 在定义所有路由及其处理函数之前,我们还初始化并配置了Pusher。

Replace the PUSHER_APP_* keys with the values on your Pusher dashboard.

将PUSHER_APP_ *键替换为Pusher仪表板上的值。

With the Pusher object instantiated, we can trigger events on whatever channels we define. The trigger method has the following syntax:

实例化Pusher对象后,我们可以在定义的任何通道上触发事件。 触发方法具有以下语法:

pusher.trigger("a_channel", "an_event", {key: "data to pass with event"})

You can find the docs for the Pusher Python library , to get more information on configuring and using Pusher in Python.

您可以在找到Pusher Python库的文档,以获取有关在Python中配置和使用Pusher的更多信息。

The getAnalyticsData function gets the visitor’s IP address and retrieves data using the to read information from the API.

getAnalyticsData函数获取访问者的IP地址,并使用检索数据以从 API中读取信息。

In the code above, we have defined four routes that will render web pages when visited, let’s create these web pages in the next section and use the frontend to communicate with our Python backend.

在上面的代码中,我们定义了四种访问时将呈现网页的路由,让我们在下一部分中创建这些网页,并使用前端与我们的Python后端进行通信。

( )

To begin, we need to create four files in the templates directory. These files will be named:

首先,我们需要在templates目录中创建四个文件。 这些文件将被命名为:

  • index.html — this is the home page for the website

    index.html —这是网站的主页
  • about.html — this is the about page for the website

    about.html这是网站的关于页面
  • dashboard.html — this is the traffic monitor dashboard

    dashboard.html —这是流量监控器的dashboard
  • dashboard-single.html — this is the dashboard page that displays more info (i.e. the types of pages a user has visited) on a single visitor.

    dashboard-single.html —这是在单个访问者上显示更多信息(即用户访问过的页面类型)的仪表盘页面。

Open the ./templates/index.html file and paste the following:

打开./templates/index.html文件并粘贴以下内容:

                Pusher Python Realtime Traffic Monitoring                

This is the homepage

Open the ./templates/about.html file and paste the following:

打开./templates/about.html文件并粘贴以下内容:

                Pusher Python Realtime Traffic Monitoring                

This is the about page

Before we write the code for dashboard-single.html and dashboard files, we will pull in some CSS and JS from . Go to the URL and click Download. Unzip the file and copy the css and js directories into the static directory of our project.

在为dashboard-single.htmldashboard文件编写代码之前,我们将从提取一些CSS和JS。 转到URL,然后单击“ 下载”。 解压缩文件,然后将cssjs目录复制到我们项目的static目录中。

Now, let’s continue building the frontend of our application, so open the ./templates/dashboard.html file and paste the following:

现在,让我们继续构建应用程序的前端,因此打开./templates/dashboard.html文件并粘贴以下内容:

              
SB Admin - Start Bootstrap Template
Total session
26
View Details
Total visitors
26
View Details
Total Page Views
123
View Details
Unique Page Views
123
View Details
User Sessions
Time IP Address Continent Country City Opertating System Browser
Time IP Address Continent Country City Opertating System Browser
Copyright © Your Website 2018

Finally, open the ./templates/dashboard-single.html file and paste the following:

最后,打开./templates/dashboard-single.html文件并粘贴以下内容:

              
SB Admin - Start Bootstrap Template
User Sessions
{% for row in data %}
{% endfor %}
Time Page Session_id No of Visits
Time Page Session_id No of Visits
{ { row.first_visited }} { { row.name }} { { row.session }} { { row.visits }}
Copyright © Your Website 2018

The markup we have written in all four files define the structure of the elements on the DOM when each page is rendered. We have also loaded the JavaScript Pusher package in the dashboard.html and dashboard-single.html files.

我们在所有四个文件中编写的标记定义了呈现每个页面时DOM上元素的结构。 我们还将JavaScript Pusher程序包加载到了dashboard.htmldashboard-single.html文件中。

Next, we will listen for Pusher updates on the frontend of our application using the Pusher magic.

接下来,我们将使用Pusher魔术监听应用程序前端的Pusher更新。

( )

In this step, we will listen for Pusher events from the Backend API and use the data to update the traffic monitor dashboard pages in realtime. First, let’s create a new file — app.js— in the js folder that is located in the static directory.

在此步骤中,我们将侦听来自后端API的Pusher事件,并使用该数据实时更新流量监控器仪表板页面。 首先,让我们在static目录中的js文件夹中创建一个新文件app.js

Next, open the ./static/js/app.js file and paste the following:

接下来,打开./static/js/app.js文件并粘贴以下内容:

// Configure Pusher instance    const pusher = new Pusher('PUSHER_APP_KEY', {
cluster: 'PUSHER_APP_CLUSTER', encrypted: true }); var months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; $(document).ready(function(){
var dataTable = $("#dataTable").DataTable() // var userSessions = $("#userSessions").DataTable() var pages = $("#pages").DataTable() axios.get('/get-all-sessions') .then(response => {
response.data.forEach((data) => {
insertDatatable(data) }) var d = new Date(); var updatedAt = `${
d.getFullYear()}/${
months[d.getMonth()]}/${
d.getDay()} ${
d.getHours()}:${
d.getMinutes()}:${
d.getSeconds()}` document.getElementById('session-update-time').innerText = updatedAt }) var sessionChannel = pusher.subscribe('session'); sessionChannel.bind('new', function(data) {
insertDatatable(data) }); var d = new Date(); var updatedAt = `${
d.getFullYear()}/${
months[d.getMonth()]}/${
d.getDay()} ${
d.getHours()}:${
d.getMinutes()}:${
d.getSeconds()}` document.getElementById('session-update-time').innerText = updatedAt }); function insertDatatable(data){
var dataTable = $("#dataTable").DataTable() dataTable.row.add([ data.time, data.ip, data.continent, data.country, data.city, data.os, data.browser, `View pages visited` ]); dataTable.order([0, 'desc']).draw(); }

Replace the PUSHER_APP_* keys with the keys on your Pusher dashboard.

PUSHER_APP_*键替换为Pusher仪表板上的键。

In the code above, first, we register a new instance of Pusher then we use to get all the sessions that have been saved to the database when the document is ready. We also created a new Pusher channel called sessionChannel and subscribed to a session event.

在上面的代码中,首先,我们注册一个Pusher的新实例,然后在文档准备好时使用获取所有已保存到数据库的会话。 我们还创建了一个名为sessionChannel的新Pusher通道,并订阅了一个会话事件。

We defined a function insertDatatable and it inserts a new user record whenever Pusher sends an update.

我们定义了一个函数insertDatatable ,每当Pusher发送更新时,它都会插入一个新的用户记录。

That is everything! We can run the application now using this command:

那就是一切! 我们现在可以使用以下命令运行该应用程序:

$ flask run

Now we can visit to see the website and to see the traffic monitor dashboard, here’s a display of what both screens look like when viewed side by side.

现在,我们可以访问看网站和看交通监控仪表板,这里是什么都看屏幕并排从侧面看时的显示器等。

python-web-traffic-demo

( )

In this tutorial, we have learned how to integrate a realtime traffic monitor with our website using Python’s Flask, Pusher and JavaScript.

在本教程中,我们学习了如何使用Python的Flask,Pusher和JavaScript将实时流量监控器与我们的网站集成。

The source code for this tutorial is available on

本教程的源代码可在上

翻译自:

python构建对等网络

转载地址:http://qiuwd.baihongyu.com/

你可能感兴趣的文章
Java魔法堂:找外援的利器——Runtime.exec详解
查看>>
mysql数据库存放路径
查看>>
TestNG(五)常用元素的操作
查看>>
解决 Visual Studio 点击添加引用无反应的问题
查看>>
通过镜像下载Android系统源码
查看>>
python字符串格式化 %操作符 {}操作符---总结
查看>>
windows 不能在 本地计算机 启动 Apache
查看>>
iOS开发报duplicate symbols for architecture x86_64错误的问题
查看>>
Chap-6 6.4.2 堆和栈
查看>>
【Java学习笔记之九】java二维数组及其多维数组的内存应用拓展延伸
查看>>
C# MySql 连接
查看>>
div+css教程网站建设门户网站和电子商务网站CSS样式表
查看>>
sk_buff Structure
查看>>
oracle的级联更新、删除
查看>>
多浏览器开发需要注意的问题之一
查看>>
Maven配置
查看>>
HttpServletRequest /HttpServletResponse
查看>>
SAM4E单片机之旅——24、使用DSP库求向量数量积
查看>>
从远程库克隆库
查看>>
codeforces Unusual Product
查看>>