精华内容
下载资源
问答
  • 用Python通过用户cookies访问微博首页.pdf
  • 通过用户cookies访问微博首页 1.登录微博 2.获取cooikes,保存json文件 3..读取json文件,获得cooikes,访问主页 4.详细代码 输出结果

    通过用户cookies访问微博首页

    1.登录微博

     

    2.获取cooikes,保存json文件

     

    3..读取json文件,获得cooikes,访问主页

    4.详细代码

     

    输出结果

     

    展开全文
  • 通过用户cookies访问微博首页 1.登录微博 self.driver.delete_all_cookies() # 删除cookies self.driver.get(self.url) time.sleep(2) user = self.wait.until(EC.presence_of_element_loca...

    通过用户cookies访问微博首页

    1.登录微博

            self.driver.delete_all_cookies()  # 删除cookies
            self.driver.get(self.url)
            time.sleep(2)
            user = self.wait.until(EC.presence_of_element_located((By.ID,'loginName')))
            pwd = self.wait.until(EC.presence_of_element_located((By.ID, 'loginPassword')))
            submit = self.wait.until(EC.presence_of_element_located((By.ID, 'loginAction')))
            user.send_keys(self.username)
            time.sleep(1)
            pwd.send_keys(self.password)
            time.sleep(1)
            submit.click()
            time.sleep(5)

    2.获取cooikes,保存json文件

    cookies = self.driver.get_cookies()
    cookies_dict = {}
    for item in cookies:
        cookies_dict[item.get('name')] = item.get('value')
    with open('sina_cookies.TXT','w',encoding='utf-8') as f:
        f.write(json.dumps(cookies_dict, ensure_ascii='False',indent=4))

    3..读取json文件,获得cooikes,访问主页

    with open('sina_cookies.TXT','r',encoding='utf-8') as f:
                cooikes_dict = json.loads(f.read())
    response = requests.get('https://weibo.cn/', cookies = cookies_dict, timeout=5, allow_redirects=False)
            if response.status_code == 200:
                print('用户cookies有效')

    4.详细代码

    #!/usr/local/bin/python3
    # -*- coding: utf-8 -*-
    # @Time    : 2018/4/26 0:24
    # @Author  : hyang
    # @File    : WeiboCooikes.py
    # @Software: 
    
    
    import time
    import json
    from selenium import webdriver
    from selenium.webdriver.support.ui import WebDriverWait  # 等待元素加载的
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.common.exceptions import TimeoutException, NoSuchElementException
    from selenium.webdriver.common.by import By
    import requests
    
    class WeiboCooikesLogin(object):
        """
        通过cookies访问微博
        """
        def __init__(self,username, password):
            self.url = 'https://passport.weibo.cn/signin/login'
            self.driver = webdriver.Chrome()
            self.driver.maximize_window()  # 最大化窗口
            self.wait = WebDriverWait(self.driver, 6)
            self.username = username
            self.password = password
    
        def __del__(self):
            """
             关闭浏览器
            :return: 
            """
            print('close browser')
            self.driver.close()
    
        def open_url(self):
            """
            打开url登录微博
            :return: 
            """
            self.driver.delete_all_cookies()  # 删除cookies
            self.driver.get(self.url)
            time.sleep(2)
            user = self.wait.until(EC.presence_of_element_located((By.ID,'loginName')))
            pwd = self.wait.until(EC.presence_of_element_located((By.ID, 'loginPassword')))
            submit = self.wait.until(EC.presence_of_element_located((By.ID, 'loginAction')))
            user.send_keys(self.username)
            time.sleep(1)
            pwd.send_keys(self.password)
            time.sleep(1)
            submit.click()
            time.sleep(5)
    
        def password_error(self):
            """
            判断用户名密码错误
            :return: 
            """
            try:
                return self.wait.until(EC.text_to_be_present_in_element((By.ID,'errorMsg'),'用户名或密码错误'))
            except TimeoutException as e:
                return False
    
        def login_successful(self):
            """
             获得登录成功标志
            :return: 
            """
            try:
                return self.wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'drop-title')))
    
            except TimeoutException as e:
                return False
    
        def process_cookies(self,cookies):
            """
            处理cookies
            :param cookies: 
            :return: 
            """
            cookies_dict = {}
            for item in cookies:
                cookies_dict[item.get('name')] = item.get('value')
            return cookies_dict
    
        def save_cookies(self,cookies_dict):
            """
             保存cookies
            :param cookies_dict: 
            :return: 
            """
            with open('sina_cookies.TXT','w',encoding='utf-8') as f:
                f.write(json.dumps(cookies_dict, ensure_ascii='False',indent=4))
    
        def get_cookies_main(self):
            self.open_url()
            if self.password_error():
                print('用户名或密码错误')
            if self.login_successful():
                print('用户登录成功')
                cookies = self.driver.get_cookies()
                d = self.process_cookies(cookies)
                self.save_cookies(d)
                print('保存用户cookies成功')
    
        def get_cooikes(self):
            """
            从文件中读取cookies
            :return: 
            """
            with open('sina_cookies.TXT','r',encoding='utf-8') as f:
                cooikes_dict = json.loads(f.read())
    
            return cooikes_dict
    
        def login_with_cookies(self, cookies_dict):
            """
            通过cookies访问主页读取信息
            :param cookies_dict: 
            :return: 
            """
            time.sleep(2)
            response = requests.get('https://weibo.cn/', cookies = cookies_dict, timeout=5, allow_redirects=False)
            if response.status_code == 200:
                print('用户cookies有效')
                time.sleep(1)
                if '我的首页' in response.text:
                    print('通过cookies登录成功')
    
    
        def login_cookies_main(self):
            print('用户开始刷新主页!!')
            d = self.get_cooikes()
            print('读取用户cookies!!')
            self.login_with_cookies(d)
            print('通过cookies访问主页!!')
    
    
    
    if __name__ == '__main__':
        username = '908099665' # 新浪微博用户
        pwd = '111111' # 新浪微博用户密码
        wb = WeiboCooikesLogin(username, pwd)
        wb.get_cookies_main()   # 得到cookies
        wb.login_cookies_main()  # 用cookies访问主页

    输出结果

     

    转载于:https://www.cnblogs.com/xiao-apple36/p/8948817.html

    展开全文
  • scrapy 设置代理ip和cookies微博

    千次阅读 2018-07-09 21:38:59
    request.cookies = eval ( self .get_cookies()) return None settings.py 代码: DOWNLOADER_MIDDLEWARES = { 'Weibo.middlewares.WeiboDownloaderMiddleware' : None , 'Weibo.middlewares....



    pipelines.py代码
    from fake_useragent import UserAgent
    import requests
    
    
    class UserAgentMiddlewares(object):
        """
        自定义一个UserAgent的下载中间件。
        """
        def __init__(self, user_agent_type):
            self.ua = UserAgent()
            self.user_agent_type = user_agent_type
    
        @classmethod
        def from_crawler(cls, crawler):
            obj = cls(
                user_agent_type=crawler.settings.get('USER_AGENT_TYPE', 'random')
            )
            return obj
    
        def get_user_agent(self):
            # getattr():通过self.ua调用self.user_agent_type
            user_agent = getattr(self.ua, self.user_agent_type)
            return user_agent
    
        def get_cookies(self):
            return requests.get('http://localhost:5000/weibo/random').text
    
        def get_proxy(self):
            return requests.get('http://localhost:5010/get/').text
    
        def process_request(self, request, spider):
            # 设置随机的User-Agent
            request.headers.setdefault(b'User-Agent', self.get_user_agent())
            # 设置代理IP
            # request.meta['proxy'] = 'http://' + self.get_proxy()
            request.cookies = eval(self.get_cookies())
            return None
    settings.py 代码:

    DOWNLOADER_MIDDLEWARES = {
       'Weibo.middlewares.WeiboDownloaderMiddleware': None,
       'Weibo.middlewares.UserAgentMiddlewares': 543,
    
    }
    
    USER_AGENT_TYPE = 'random'
    展开全文
  • java 通过模拟cookies登陆新浪微博

    千次阅读 2013-08-07 11:49:30
    基于这点,我们可以在后台通过模拟cookies信息,然后将cookies信息设置到httpclient里,从而达到免除登陆的目的。以下是实现代码: httpclient是org.apache.commom包 private static BasicClientCookie ...

    众所周知,浏览器通过cookies记录用户登陆的一些信息,这样用户下次可以自动登陆等。基于这点,我们可以在后台通过模拟cookies信息,然后将cookies信息设置到httpclient里,从而达到免除登陆的目的。以下是实现代码:

    httpclient是org.apache.commom包

    private static BasicClientCookie setWeiboCookies(String name,String value,String date){
            BasicClientCookie2 cookie = new BasicClientCookie2(name,value);
            cookie.setDomain(".weibo.com");
            cookie.setPath("/");
            if (StringUtils.isNotBlank(date)) {
                cookie.setExpiryDate(new Date(date));
            }else{
                cookie.setExpiryDate(null);
            }
            return cookie;
        }
    
        public static void test(){
            DefaultHttpClient httpclient = new DefaultHttpClient();
            httpclient.getParams().setParameter("http.protocol.cookie-policy",
                    CookiePolicy.BROWSER_COMPATIBILITY);
            HttpParams params = httpclient.getParams();
            HttpConnectionParams.setConnectionTimeout(params, 5000);
            HttpConnectionParams.setSoTimeout(params, 1000*60*10);
            DefaultHttpRequestRetryHandler dhr = new DefaultHttpRequestRetryHandler(3,true);
            HttpContext localContext = new BasicHttpContext();
            HttpRequest request2 = (HttpRequest) localContext.getAttribute(
                    ExecutionContext.HTTP_REQUEST);
            httpclient.setHttpRequestRetryHandler(dhr);
            BasicCookieStore cookieStore = new BasicCookieStore();
    
            /**
             *  weibo.com
             */
            cookieStore.addCookie(setWeiboCookies("SUS", "查看浏览器的SUS值", null)); //ok
            cookieStore.addCookie(setWeiboCookies("SUP", "查看浏览器的SUP值", null) ); //ok
            cookieStore.addCookie(setWeiboCookies("SUE","查看浏览器SUE值",null)); //ok
    
            localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
            HttpGet request = new HttpGet();
            request.setURI(URI.create("http://weibo.com"));
            HttpResponse response = null;
            try {
                response = httpclient.execute(request,localContext);
                System.out.println(EntityUtils.toString(response.getEntity(), "utf-8"));
            } catch (IOException e) {
                System.out.println(e);
            }


    展开全文
  • cookies = cook).content ##################################### # # 两种方法二选一 # # content返回的是一个byte型的数据 # # text返回的是一个Unicode字符串,不支持显示 # html = requests.get(url,...
  • 登录新浪微博和豆瓣时把相应的cookie 替换即可 import requests cookie = 'login_sid_t=42b3d0009212fc42be1479203c253428;'\ 'cross_origin_proto=SSL;'\ 'WBStorage=8daec78e6a891122|undefined; '\ '_s_tentry=...
  • 搭建微博Cookies池(一)

    千次阅读 2019-06-18 20:41:22
    搭建之前需要一些微博账号,这里使用接码平台自行注册。 使用Redis数据库进行账号信息的存储与读取 配置好webdriver Chrome的浏览器插件 开始搭建 1.存储模块的搭建: 需要存储的内容无非就是账号,密码,cookies这...
  • = "E:\\学习\\微博爬虫\\cookie\\chromedriver.exe" ; GetMultiCookieMSinaTimely.cookieSavePath = "E:\\学习\\微博爬虫\\cookie\\" ; if ( new File(GetMultiCookieMSinaTimely.cookieFile).isFile()) { ...
  • Sina微博模拟登录获取Cookies

    千次阅读 2017-05-26 18:21:01
    很多豆友反馈有模拟登录新浪微博抓取数据的需求,其实对于一般的微博数据获取,如用户信息、微博内容等,使用微博开放平台API是更明智的选择:速度更快,而且节省许多网页处理的功夫。对于API没有开放的数据,我们再...
  • python实现扫码登入微博网页版
  • # @File : 微博cookies.py # @Software: PyCharm from selenium import webdriver from time import sleep import json if __name__ == '__main__': driver = webdriver.Chrome(executable_path='./chromedriver....
  • 用flask框架搭建cookies池 1.首先安装flask模块 pip install flask 2.代码实现 在这里插入代码片 from flask import Flask,g from cunchu import Redisclient from config import * app=Flask(name) @app...
  • 在完成存储、cookies获取、检测等模块后,我们需要一个调度模块将不同模块在不同进程上运行起来。 这里导入python自带多进程库 multiprocessing里的Process,注意这里Process要大写,有些IDE里自动补全为小写会报错。...
  • 模拟登陆就是获取新浪服务器返回的登陆参数(cookies等),然后添加到爬虫的post请求中来伪装用户提交给新浪访客系统 首先感谢 Bgods(https://blog.csdn.net/songzhilian22/article/details/48396545) 敲代码的...
  • 一直想用selenium实现个小功能,比如发微博之类的,但是有的网站在登录会有验证码,没想到太好的方法解决,于是想到利用cookies来登录网站 第一步:获取一个可用的cookies,获取的cookies会保存为一个json格式的...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 4,334
精华内容 1,733
关键字:

cookies微博