基Python+爬虫的个性化书籍推荐系统

作者主页：编程千纸鹤

作者简介：Java、前端、Python开发多年，做过高程，项目经理，架构师

主要内容：Java项目开发、Python项目开发、大学数据和AI项目开发、单片机项目设计、面试技术整理、最新技术分享

收藏点赞不迷路关注作者有好处

文末获得源码

项目编号：BS-Python-064

一，环境介绍

语言环境：Python3.8

数据库：Mysql: mysql5.7

WEB框架：Django

爬虫框架：Scrapy

开发工具：IDEA或PyCharm

二，项目简介

近年来，随着互联网的蓬勃发展，企事业单位对信息的管理提出了更高的要求。以传统的管理方式已无法满足现代人们的需求。为了迎合时代需求，优化管理效率，各种各样的管理系统应运而生，随着各行业的不断发展，个性化书籍推荐系统建设也逐渐进入了信息化的进程。

这个系统的设计主要包括系统页面的设计和方便用户互动的后端数据库，而前端软件的开发则需要良好的数据处理能力、友好的界面和易用的功能。

数据要被工作人员通过界面操作传输至数据库中。通过研究，以MySQL为后端数据库，以PYTHON为前端技术，以IDEA为开发平台，采用DJANGO架构，建立一个提供个人中心、用户管理、书籍分类管理、热门书籍管理、公告栏管理、小说信息管理、在线反馈、系统管理等必要功能的、稳定的个性化书籍推荐系统。

随着个性化书籍推荐规模的扩大,个性化书籍推荐信息呈现多样化、复杂化趋势,从而给用户管理个性化书籍推荐信息时带来了许多新的问题。随着用户人数的增多,个性化书籍推荐信息越来越多，而传统工作方式则是获得资料缓慢，查找难度大，准确性低，难以满足高水平作业要求。所以，如何为用户提供更方便快捷查询个性化书籍推荐信息，以及如何利用现代技术更科学、高效地管理个性化书籍推荐信息，是开发此系统的主要目标。

采用传统的人力资源管理方式，有很多不足之处，例如：效率低、保密程度低、时间久了会生成很多资料，而且不方便查询、升级、维修等。我们可以利用计算机技术来取代传统的管理模式，实现个性化书籍推荐信息的现代化管理。只有这样，才能让个性化书籍推荐管理的进程中获得竞争的有利地位，才能真正地走向科学、规范化、现代化。实现了管理的科学化和统一，有效地防止了人为的差错，使工作的工作质量得到了极大的提升。

为了提高个性化书籍推荐信息管理的效率；充分利用现有资源；减少不必要的人力、物力和财政支出来实现管理人员更充分掌握个性化书籍推荐信息的管理；开发设计专用系统--基于爬虫的个性化书籍推荐系统来进行管理个性化书籍推荐信息，它将实现检索迅速和查找方便；信息的录入，修改和删除功能。

管理员用例：

用户：

三，系统展示

前端用户功能模块

后台管理

可视化图形报表展示：

四，核心代码展示

# coding:utf-8
__author__ = "ila"

import logging, os, xlrd, json, datetime
from flask import request, jsonify,session
from sqlalchemy.sql import func,and_,or_
from api.models.brush_model import *
from . import main_bp
from utils.codes import *
from utils.jwt_auth import Auth
from configs import configs
from utils.helper import *
import random
import smtplib
from email.mime.text import MIMEText
from email.header import Header
from utils.baidubce_api import BaiDuBce
from api.models.config_model import config





# 注册接口
@main_bp.route("/python7bf5u/discussremenshuji/register", methods=['POST'])
def python7bf5u_discussremenshuji_register():
    if request.method == 'POST':
        msg = {'code': normal_code, 'message': 'success', 'data': [{}]}
        req_dict = session.get("req_dict")


        error = discussremenshuji.createbyreq(discussremenshuji, discussremenshuji, req_dict)
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = "注册用户已存在"
        return jsonify(msg)

# 登录接口
@main_bp.route("/python7bf5u/discussremenshuji/login", methods=['GET','POST'])
def python7bf5u_discussremenshuji_login():
    if request.method == 'GET' or request.method == 'POST':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        req_model = session.get("req_dict")
        try:
            del req_model['role']
        except:
            pass


        datas = discussremenshuji.getbyparams(discussremenshuji, discussremenshuji, req_model)
        if not datas:
            msg['code'] = password_error_code
            msg['msg']='密码错误或用户不存在'
            return jsonify(msg)


        req_dict['id'] = datas[0].get('id')
        return Auth.authenticate(Auth, discussremenshuji, req_dict)


# 登出接口
@main_bp.route("/python7bf5u/discussremenshuji/logout", methods=['POST'])
def python7bf5u_discussremenshuji_logout():
    if request.method == 'POST':
        msg = {
            "msg": "退出成功",
            "code": 0
        }
        req_dict = session.get("req_dict")

        return jsonify(msg)

# 重置密码接口
@main_bp.route("/python7bf5u/discussremenshuji/resetPass", methods=['POST'])
def python7bf5u_discussremenshuji_resetpass():
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success"}

        req_dict = session.get("req_dict")

        if req_dict.get('mima') != None:
            req_dict['mima'] = '123456'

        error = discussremenshuji.updatebyparams(discussremenshuji, discussremenshuji, req_dict)

        if error != None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        else:
            msg['msg'] = '密码已重置为：123456'
        return jsonify(msg)

# 获取会话信息接口
@main_bp.route("/python7bf5u/discussremenshuji/session", methods=['GET'])
def python7bf5u_discussremenshuji_session():
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "data": {}}
        req_dict={"id":session.get('params').get("id")}
        msg['data']  = discussremenshuji.getbyparams(discussremenshuji, discussremenshuji, req_dict)[0]

        return jsonify(msg)

# 分类接口（后端）
@main_bp.route("/python7bf5u/discussremenshuji/page", methods=['GET'])
def python7bf5u_discussremenshuji_page():
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success",  "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}}
        req_dict = session.get("req_dict")
        userinfo = session.get("params")

        try:
            __hasMessage__=discussremenshuji.__hasMessage__
        except:
            __hasMessage__=None
        if __hasMessage__ and __hasMessage__!="否":
            tablename=session.get("tablename")
            if tablename!="users" and session.get("params")!=None and discussremenshuji!='chat':
                req_dict["userid"]=session.get("params").get("id")

        tablename=session.get("tablename")
        if tablename=="users" :
            try:
                pass
            except:
                pass
        else:
            mapping_str_to_object = {}
            for model in Base_model._decl_class_registry.values():
                if hasattr(model, '__tablename__'):
                    mapping_str_to_object[model.__tablename__] = model

            try:
                __isAdmin__=mapping_str_to_object[tablename].__isAdmin__
            except:
                __isAdmin__=None

            if __isAdmin__!="是" and session.get("params")!=None:
                req_dict["userid"]=session.get("params").get("id")
            else:
                try:
                    del req_dict["userid"]
                except:
                    pass


        clause_args = []
        or_clauses = or_(*clause_args)

        msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \
        msg['data']['pageSize']  = discussremenshuji.page(discussremenshuji, discussremenshuji, req_dict, or_clauses)

        return jsonify(msg)

# 排序接口
@main_bp.route("/python7bf5u/discussremenshuji/autoSort", methods=['GET'])
def python7bf5u_discussremenshuji_autosort():
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success",  "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}}
        req_dict = session.get("req_dict")
        req_dict['sort']='clicktime'
        req_dict['order']='desc'

        try:
            __browseClick__= discussremenshuji.__browseClick__
        except:
            __browseClick__=None

        if __browseClick__ =='是':
            req_dict['sort']='clicknum'
        elif __browseClick__ =='时长':
            req_dict['sort']='browseduration'
        else:
            req_dict['sort']='clicktime'
        msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \
        msg['data']['pageSize']  = discussremenshuji.page(discussremenshuji, discussremenshuji, req_dict)

        return jsonify(msg)

# 分页接口（前端）
@main_bp.route("/python7bf5u/discussremenshuji/list", methods=['GET'])
def python7bf5u_discussremenshuji_list():
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success",  "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}}
        req_dict = session.get("req_dict")
        if req_dict.__contains__('vipread'):
            del req_dict['vipread']
            
        userinfo = session.get("params")

        try:
            __foreEndList__=discussremenshuji.__foreEndList__
        except:
            __foreEndList__=None

        if __foreEndList__ and __foreEndList__!="否":
            tablename=session.get("tablename")
            if tablename!="users" and session.get("params")!=None:
                req_dict['userid']=session.get("params").get("id")

        try:
            __foreEndListAuth__=discussremenshuji.__foreEndListAuth__
        except:
            __foreEndListAuth__=None

        if __foreEndListAuth__ and __foreEndListAuth__!="否":
            tablename=session.get("tablename")
            if tablename!="users" and session.get("params")!=None:
                req_dict['userid']=session.get("params").get("id")

        tablename=session.get("tablename")
        if tablename=="users" :
            try:
                del req_dict["userid"]
            except:
                pass
        else:
            mapping_str_to_object = {}
            for model in Base_model._decl_class_registry.values():
                if hasattr(model, '__tablename__'):
                    mapping_str_to_object[model.__tablename__] = model

            try:
                __isAdmin__=mapping_str_to_object[tablename].__isAdmin__
            except:
                __isAdmin__=None

            if __isAdmin__!="是" and session.get("params")!=None:
                req_dict["userid"]=session.get("params").get("id")
            else:
                try:
                    del req_dict["userid"]
                except:
                    pass

        if 'luntan' in 'discussremenshuji':
            if 'userid' in req_dict.keys():
                del req_dict["userid"]



        if 'discuss' in 'discussremenshuji':
            if 'userid' in req_dict.keys():
                del req_dict["userid"]

        msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \
        msg['data']['pageSize']  = discussremenshuji.page(discussremenshuji, discussremenshuji, req_dict)

        return jsonify(msg)

# 保存接口（后端）
@main_bp.route("/python7bf5u/discussremenshuji/save", methods=['POST'])
def python7bf5u_discussremenshuji_save():
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        for key in req_dict:
            if req_dict[key] == '':
                req_dict[key] = None
        error= discussremenshuji.createbyreq(discussremenshuji, discussremenshuji, req_dict)
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return jsonify(msg)

# 添加接口（前端）
@main_bp.route("/python7bf5u/discussremenshuji/add", methods=['POST'])
def python7bf5u_discussremenshuji_add():
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        try:
            __foreEndListAuth__=discussremenshuji.__foreEndListAuth__
        except:
            __foreEndListAuth__=None

        if __foreEndListAuth__ and __foreEndListAuth__!="否":
            tablename=session.get("tablename")
            if tablename!="users":
                req_dict['userid']=session.get("params").get("id")
            
        error= discussremenshuji.createbyreq(discussremenshuji, discussremenshuji, req_dict)
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return jsonify(msg)

# 踩、赞接口
@main_bp.route("/python7bf5u/discussremenshuji/thumbsup/<id_>", methods=['GET'])
def python7bf5u_discussremenshuji_thumbsup(id_):
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        id_=int(id_)
        type_=int(req_dict.get("type",0))
        rets=discussremenshuji.getbyid(discussremenshuji, discussremenshuji,id_)

        update_dict={
        "id":id_,
        }
        if type_==1:#赞
            update_dict["thumbsupnum"]=int(rets[0].get('thumbsupnum'))+1
        elif type_==2:#踩
            update_dict["crazilynum"]=int(rets[0].get('crazilynum'))+1
        error = discussremenshuji.updatebyparams(discussremenshuji, discussremenshuji, update_dict)
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return jsonify(msg)

# 获取详情信息（后端）
@main_bp.route("/python7bf5u/discussremenshuji/info/<id_>", methods=['GET'])
def python7bf5u_discussremenshuji_info(id_):
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}

        data = discussremenshuji.getbyid(discussremenshuji, discussremenshuji, int(id_))
        if len(data)>0:
            msg['data']=data[0]
        #浏览点击次数
        try:
            __browseClick__= discussremenshuji.__browseClick__
        except:
            __browseClick__=None

        if __browseClick__  and  "clicknum"  in discussremenshuji.__table__.columns:
            click_dict={"id":int(id_),"clicknum":str(int(data[0].get("clicknum") or 0)+1)}
            ret=discussremenshuji.updatebyparams(discussremenshuji,discussremenshuji,click_dict)
            if ret!=None:
                msg['code'] = crud_error_code
                msg['msg'] = ret
        return jsonify(msg)

# 获取详情信息（前端）
@main_bp.route("/python7bf5u/discussremenshuji/detail/<id_>", methods=['GET'])
def python7bf5u_discussremenshuji_detail(id_):
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}

        data = discussremenshuji.getbyid(discussremenshuji, discussremenshuji, int(id_))
        if len(data)>0:
            msg['data']=data[0]

        #浏览点击次数
        try:
            __browseClick__= discussremenshuji.__browseClick__
        except:
            __browseClick__=None

        if __browseClick__   and  "clicknum"  in discussremenshuji.__table__.columns:
            click_dict={"id":int(id_),"clicknum":str(int(data[0].get("clicknum") or 0)+1)}
            ret=discussremenshuji.updatebyparams(discussremenshuji,discussremenshuji,click_dict)
            if ret!=None:
                msg['code'] = crud_error_code
                msg['msg'] = ret
        return jsonify(msg)

# 更新接口
@main_bp.route("/python7bf5u/discussremenshuji/update", methods=['POST'])
def python7bf5u_discussremenshuji_update():
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        if req_dict.get("mima") and "mima" not in discussremenshuji.__table__.columns :
            del req_dict["mima"]
        if req_dict.get("password") and "password" not in discussremenshuji.__table__.columns :
            del req_dict["password"]
        try:
            del req_dict["clicknum"]
        except:
            pass


        error = discussremenshuji.updatebyparams(discussremenshuji, discussremenshuji, req_dict)
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return jsonify(msg)

# 删除接口
@main_bp.route("/python7bf5u/discussremenshuji/delete", methods=['POST'])
def python7bf5u_discussremenshuji_delete():
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")

        error=discussremenshuji.delete(
            discussremenshuji,
            req_dict
        )
        if error!=None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return jsonify(msg)

# 投票接口
@main_bp.route("/python7bf5u/discussremenshuji/vote/<int:id_>", methods=['POST'])
def python7bf5u_discussremenshuji_vote(id_):
    '''
    '''
    if request.method == 'POST':
        msg = {"code": normal_code, "msg": "success"}


        data= discussremenshuji.getbyid(discussremenshuji, discussremenshuji, int(id_))
        for i in data:
            votenum=i.get('votenum')
            if votenum!=None:
                params={"id":int(id_),"votenum":votenum+1}
                error=discussremenshuji.updatebyparams(discussremenshuji,discussremenshuji,params)
                if error!=None:
                    msg['code'] = crud_error_code
                    msg['msg'] = error
        return jsonify(msg)





# 分组统计接口
@main_bp.route("/python7bf5u/discussremenshuji/group/<columnName>", methods=['GET'])
def python7bf5u_discussremenshuji_group(columnName):
    '''
    分组统计接口
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        userinfo = session.get("params")


        msg['data'] = discussremenshuji.groupbycolumnname(discussremenshuji,discussremenshuji,columnName,req_dict)
        msg['data'] = msg['data'][:10]
        return jsonify(msg)

# 按值统计接口
@main_bp.route("/python7bf5u/discussremenshuji/value/<xColumnName>/<yColumnName>", methods=['GET'])
def python7bf5u_discussremenshuji_value(xColumnName, yColumnName):
    '''
    按值统计接口,
    {
        "code": 0,
        "data": [
            {
                "total": 10.0,
                "shangpinleibie": "aa"
            },
            {
                "total": 20.0,
                "shangpinleibie": "bb"
            },
            {
                "total": 15.0,
                "shangpinleibie": "cc"
            }
        ]
    }
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}
        req_dict = session.get("req_dict")
        userinfo = session.get("params")


        msg['data'] = discussremenshuji.getvaluebyxycolumnname(discussremenshuji,discussremenshuji,xColumnName,yColumnName,req_dict)
        msg['data'] = msg['data'][:10]
        return jsonify(msg)

# 按日期统计接口
@main_bp.route("/python7bf5u/discussremenshuji/value/<xColumnName>/<yColumnName>/<timeStatType>", methods=['GET'])
def python7bf5u_discussremenshuji_value_riqi(xColumnName, yColumnName, timeStatType):
    '''
    按日期统计接口
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, "msg": "success", "data": {}}

        where = ' where 1 = 1 '

        sql = ''
        if timeStatType == '日':
            sql = "SELECT DATE_FORMAT({0}, '%Y-%m-%d') {0}, sum({1}) total FROM discussremenshuji {2} GROUP BY DATE_FORMAT({0}, '%Y-%m-%d')".format(xColumnName, yColumnName, where, '%Y-%m-%d')

        if timeStatType == '月':
            sql = "SELECT DATE_FORMAT({0}, '%Y-%m') {0}, sum({1}) total FROM discussremenshuji {2} GROUP BY DATE_FORMAT({0}, '%Y-%m')".format(xColumnName, yColumnName, where, '%Y-%m')

        if timeStatType == '年':
            sql = "SELECT DATE_FORMAT({0}, '%Y') {0}, sum({1}) total FROM discussremenshuji {2} GROUP BY DATE_FORMAT({0}, '%Y')".format(xColumnName, yColumnName, where, '%Y')

        data = db.session.execute(sql)
        data = data.fetchall()
        results = []
        for i in range(len(data)):
            result = {
                xColumnName: decimalEncoder(data[i][0]),
                'total': decimalEncoder(data[i][1])
            }
            results.append(result)
            
        msg['data'] = results
        return jsonify(msg)

# 推荐算法接口
@main_bp.route("/python7bf5u/discussremenshuji/autoSort2", methods=['GET'])
def python7bf5u_discussremenshuji_autoSort2():
    if request.method == 'GET':

        leixing = set()
        req_dict = session.get("req_dict")
        userinfo = session.get("params")
        sql = "select inteltype from storeup where userid = "+userinfo.get("id")+" and tablename = 'discussremenshuji' order by addtime desc"
        try:
            data = db.session.execute(sql)
            rows = data.fetchall()
            for row in rows:
                for item in row:
                    if item != None:
                        leixing.add(item)
        except:
            leixing = set()

        L = []
        sql ="select * from discussremenshuji where $intelRecomColumn in ('%s"%("','").join(leixing)+"') union all select * from discussremenshuji where $intelRecomColumn not in('%s"%("','").join(leixing)+"')"
        data = db.session.execute(sql)
        data_dict = [dict(zip(result.keys(), result)) for result in data.fetchall()]
        for online_dict in data_dict:
            for key in online_dict:
                if 'datetime.datetime' in str(type(online_dict[key])):
                    online_dict[key] = online_dict[key].strftime(
                        "%Y-%m-%d %H:%M:%S")
                else:
                    pass
            L.append(online_dict)

        return jsonify({"code": 0, "msg": '',  "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":5,"list": L[0:int(req_dict['limit'])]}})







# 统计接口
@main_bp.route("/python7bf5u/discussremenshuji/remind/<columnName>/<type>", methods=['GET'])  #
def python7bf5u_discussremenshuji_remind(columnName,type):
    '''
    '''
    if request.method == 'GET':
        msg = {"code": normal_code, 'count': 0}
        # 组合查询参数
        params = session.get("req_dict")
        if int(type)==1:#数字
            if params.get('remindstart') == None and params.get('remindend') != None:
                remindstart = 0
                remindend = int(params['remindend'])
            elif params.get('remindstart') != None and params.get('remindend') == None:
                remindstart = int(params['remindstart'])
                remindend = 999999
            elif params.get('remindstart') == None and params.get('remindend') == None:
                remindstart = 0
                remindend = 999999
        elif int(type)==2:#日期
            current_time=int(time.time())
            if params.get('remindstart') == None and params.get('remindend') != None:
                starttime=current_time-60*60*24*365*2
                params['remindstart'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime))
                endtime=current_time+60*60*24*params.get('remindend')
                params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime))

            elif params.get('remindstart') != None and params.get('remindend') == None:
                starttime= current_time - 60 * 60 * 24 * params.get('remindstart')
                params['remindstart']=time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime))
                endtime=current_time+60*60*24*365*2
                params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime))
            elif params.get('remindstart') == None and params.get('remindend') == None:
                starttime = current_time - 60 * 60 * 24 * 365 * 2
                params['remindstart'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime))
                endtime = current_time + 60 * 60 * 24 * 365 * 2
                params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime))

        data = discussremenshuji.getbetweenparams(
            discussremenshuji,
            discussremenshuji,
            columnName,
            {
                "remindStart": remindstart,
                "remindEnd": remindend
            }
        )

        msg['count'] = len(data)
        return jsonify(msg)