ChatPaperFree Project

README.md

@@ -1,13 +1,185 @@
----
-title: ChatPaperFree
-emoji: 📉
-colorFrom: purple
-colorTo: indigo
-sdk: gradio
-sdk_version: 4.10.0
-app_file: app.py
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# ChatPaperFree GeminiPro
+
+实际上，这个项目是在ChatPaper的基础上进行的更新，采用了最近由Google开源的Gemini Pro大模型。这样一来，我们可以免费使用ChatPaper，并且未来我还计划加入对论文图片的读取以提取摘要（测试结果是OK的）。
+
+目前,我们能够对用户输入的论文进行自动总结。未来,我还计划加入对论文图片/表格/公式的识别 extraction,从而生成更全面而易读的总结。如果在对话中chatbot能提供更优质的服务,我还会尝试进行更深层次的模型fine-tuning。我会在后续尝试进行更新，大家也可以提出自己的意见，也欢迎提PR。这是ChatPaper的[GitHub链接](https://github.com/kaixindelele/ChatPaper.git)，多多关注。
+
+另外,为了跟上快速发展的人工智能领域和庞大的arxiv论文,我们从arXiv爬取了大量论文,并制作了网站[进步屋](https://ipaper.today/)使研究人员能够便捷获取当前热点。通过Gemini Pro对论文进行自动归纳,我们可以用极少的文本来展示论文要点,帮助用户快速评估哪篇值得深入研读。
+
+在这个不断变化的时代,ChatPaperFree GeminiPro通过利用强大技术,有效提升我们学习效率。同时,我们也欢迎你提供关键词,共同完善这个平台,让每个人都能在技术演进中取得新的进步。（可在https://github.com/wmpscc/ipaper.today提出关键词，欢迎）
+
+希望未来,我们可以搭建更友好的人机合作模式。
+
+## Usage
+
+```bash
+pip install -r requriements.txt
+python app.py
+```
+
+成功后，进入http://127.0.0.1:7860即可，接着就可以输入自己的API key。
+
+Google的Gemini Pro的API key是免费的，所以大家都是可以申请的，具体申请可以在[https://makersuite.google.com/](https://makersuite.google.com/)进行获取，每个人都可以获取多个，如果大家有自己的一些想法，也可以看谷歌的API使用文档[https://ai.google.dev/tutorials/python_quickstart](https://ai.google.dev/tutorials/python_quickstart)。
+
+![](apikey.png)
+
+
+
+## 获取最新文章（进步屋）
+
+实际上，这个操作与ChatPaper的爬取论文类似，我们通过爬取最新arxiv论文，根据设定的关键词，每天对论文进行爬取，以最快的速度跟踪最新的文章。实验室已经搭建了一个网站，大家可以多多支持，提出自己的关键词，我们会不定时进行更新，让你每天都能看到最新的文章。
+
+这是进步屋的网站：[进步屋](https://ipaper.today/) https://ipaper.today/ ，大家如果有什么需要学习的内容，可以给出关键词，在[GitHub](https://github.com/wmpscc/ipaper.today)提issue。
+
+后续，我还计划加入自动生成摘要和分析的功能，包括尝试让数字人播报等等，敬请期待。
+
+![image-20231217160221886](ipaper.png)
+
+
+
+最近，我还会尝试生成文章的简单摘要，丰富内容，这是即将上线的由GPT生成简单摘要的部分，后续还可以加入更多功能，比如ChatPaper的prompt，我猜可以得到更好的效果。
+
+![image-20231217160410161](example.png)
+
+
+
+## 使用技巧：
+
+- 找到好的文章后，可以精读这篇文章；
+- 推荐其他两个精读论文的AI辅助网站：[Typeset.io](https://typeset.io/) 和 [chatpdf](https://chatpdf)。
+
+
+
+## Example1(进步屋)
+
+比如最近`3D Gassian Splatting`很火，所以我也有关注这一方面，这是我最近爬取最新的几篇文章
+
+
+
+### Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D   Reconstruction with Transformers
+
+**Authors:Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, Song-Hai Zhang**
+
+Recent advancements in 3D reconstruction from single images have been driven by the evolution of generative models. Prominent among these are methods based on Score Distillation Sampling (SDS) and the adaptation of diffusion models in the 3D domain. Despite their progress, these techniques often face limitations due to slow optimization or rendering processes, leading to extensive training and optimization times. In this paper, we introduce a novel approach for single-view reconstruction that efficiently generates a 3D model from a single image via feed-forward inference. Our method utilizes two transformer-based networks, namely a point decoder and a triplane decoder, to reconstruct 3D objects using a hybrid Triplane-Gaussian intermediate representation. This hybrid representation strikes a balance, achieving a faster rendering speed compared to implicit representations while simultaneously delivering superior rendering quality than explicit representations. The point decoder is designed for generating point clouds from single images, offering an explicit representation which is then utilized by the triplane decoder to query Gaussian features for each point. This design choice addresses the challenges associated with directly regressing explicit 3D Gaussian attributes characterized by their non-structural nature. Subsequently, the 3D Gaussians are decoded by an MLP to enable rapid rendering through splatting. Both decoders are built upon a scalable, transformer-based architecture and have been efficiently trained on large-scale 3D datasets. The evaluations conducted on both synthetic datasets and real-world images demonstrate that our method not only achieves higher quality but also ensures a faster runtime in comparison to previous state-of-the-art techniques. Please see our project page at https://zouzx.github.io/TriplaneGaussian/. 
+
+[PDF](http://arxiv.org/abs/2312.09147v1) 
+
+**Summary**
+基于变分自编码器和扩散模型的3D单视图重建方法取得了长足的进步。
+
+**Key Takeaways**
+
+- 利用变压器网络和点解码器实现单图像的3D对象重建
+- 使用混合的Triplane-Gaussian中间表示实现更快的渲染速度和更高的渲染质量
+- 通过MLP解码3D高斯以实现快速渲染
+- 在合成数据集和真实图像上进行了有效的评估
+- 相比于先前的技术，方法达到了更高的质量和更快的运行时间
+- 方法的详细信息可在https://zouzx.github.io/TriplaneGaussian/上找到。
+
+
+<details>
+  <summary>点此查看论文截图</summary>
+<img src="./crop_Gaussian Splatting/2312.09147v1/page_0_0.jpg" align="middle">
+<img src="./crop_Gaussian Splatting/2312.09147v1/page_2_0.jpg" align="middle">
+<img src="./crop_Gaussian Splatting/2312.09147v1/page_5_0.jpg" align="middle">
+</details>
+
+​    
+
+### 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting
+
+**Authors:Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang**
+
+We introduce an approach that creates animatable human avatars from monocular videos using 3D Gaussian Splatting (3DGS). Existing methods based on neural radiance fields (NeRFs) achieve high-quality novel-view/novel-pose image synthesis but often require days of training, and are extremely slow at inference time. Recently, the community has explored fast grid structures for efficient training of clothed avatars. Albeit being extremely fast at training, these methods can barely achieve an interactive rendering frame rate with around 15 FPS. In this paper, we use 3D Gaussian Splatting and learn a non-rigid deformation network to reconstruct animatable clothed human avatars that can be trained within 30 minutes and rendered at real-time frame rates (50+ FPS). Given the explicit nature of our representation, we further introduce as-isometric-as-possible regularizations on both the Gaussian mean vectors and the covariance matrices, enhancing the generalization of our model on highly articulated unseen poses. Experimental results show that our method achieves comparable and even better performance compared to state-of-the-art approaches on animatable avatar creation from a monocular input, while being 400x and 250x faster in training and inference, respectively. 
+
+[PDF](http://arxiv.org/abs/2312.09228v1) 
+
+**Summary**
+使用三维高斯点阵和非刚性变形网络创建可动人体化身，训练时间短，渲染速度快。
+
+**Key Takeaways**
+
+- 3D高斯点阵创建可动人体化身，训练时间30分钟，渲染速度50+ FPS
+- 引入正则化提高模型在高度复杂的姿势上的泛化能力
+- 实验结果表明方法在从单眼输入创建可动化身方面性能优越
+- 与现有方法相比，训练速度快400倍，推理速度快250倍
+
+
+<details>
+  <summary>点此查看论文截图</summary>
+<img src="./crop_Gaussian Splatting/2312.09228v1/page_0_0.jpg" align="middle">
+<img src="./crop_Gaussian Splatting/2312.09228v1/page_1_0.jpg" align="middle">
+<img src="./crop_Gaussian Splatting/2312.09228v1/page_3_0.jpg" align="middle">
+<img src="./crop_Gaussian Splatting/2312.09228v1/page_4_0.jpg" align="middle">
+</details>
+
+​    
+
+## Example2(ChatPaperFree)
+
+比如这里使用上述的 `3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting` 进行测试，大家都可以尝试一下
+
+![](example2.png)
+
+1. 论文标题：3DGS-Avatar：可变形3D高斯点云生成的可动画身
+
+2. 作者：
+
+   - Changil Kim
+   - Jinwoo Kim
+   - Tae-Hyun Oh
+   - Joon-Young Lee
+   - In So Kweon
+
+3. 第一作者单位：韩国科学技术院（KAIST）
+
+4. 关键词：
+
+   - 可动画身
+   - 3D高斯点云生成
+   - 可变形神经场
+   - 单目视频
+
+5. 论文链接：https://arxiv.org/abs/2312.09228 Github 链接：None
+
+6. 摘要：
+
+    
+
+   (1)：研究背景：
+
+   - 重建可动画身具有广泛的应用前景，但从图像输入中重建衣着的人体模型是一项具有挑战性的任务。
+   - 基于NeRF的神经辐射场方法在几何和外观重建方面取得了很好的效果，但训练和推理过程非常耗时。
+   - 最近提出的3D高斯点���生成方法在静态场景重建中取得了很好的效果，具有快速训练和推理的优点。
+
+   (2)：过去方法和问题：
+
+   - 现有方法主要依赖于NeRF或其变体，但这些方法通常需要大型的多层感知机来建模神经辐射场，导致训练和推理过程非常耗时。
+   - 一些方法尝试使用更快的训练和推理方法，但它们通常会牺牲渲染质量或无法对姿势相关的非刚性变形进行建模。
+
+   (3)：研究方法：
+
+   - 本文提出了一种基于3D高斯点云生成的可动画身重建方法。
+   - 该方法将刚性人体关节与非刚性变形场有效地集成到3D高斯点云生成框架中。
+   - 使用了一个小型多层感知机来解码颜色，该感知机能够对局部非刚性变形和动态光照条件做出响应。
+   - 应用了“尽可能等距”正则化项，以保持几何一致性和逼真的变形，尤其是在动态和变化的姿势中。
+
+   (4)：实验结果：
+
+   - 在单目视频输入的情况下，该方法在可动画身创建方面与最先进的方法相当或更好，训练速度提高了400倍，推理速度提高了250倍。
+   - 与专注于快速训练的方法相比，该方法虽然训练速度较慢，但能够对姿势相关的非刚性变形进行建模，并产生明显更好的渲染质量，同时渲染速度提高了3倍。
+
+7. 方法：
+
+（1）：提出了一种基于3D高斯点云生成的可动画身重建方法，将刚性人体关节与非刚性变形场有效地集成到3D高斯点云生成框架中；
+
+（2）：使用了一个小型多层感知机来解码颜色，该感知机能够对局部非刚性变形和动态光照条件做出响应；
+
+（3）：应用了“尽可能等距”正则化项，以保持几何一致性和逼真的变形，尤其是在动态和变化的姿势中。
+
+8. 结论：
+
+（1）：本文提出了一种基于 3D 高斯点云生成的可动画身重建方法，该方法在单目视频输入的情况下，在可动画身创建方面与最先进的方法相当或更好，训练速度提高了 400 倍，推理速度提高了 250 倍。
+
+（2）：创新点：提出了一种基于 3D 高斯点云生成的可动画身重建方法，将刚性人体关节与非刚性变形场有效地集成到 3D 高斯点云生成框架中；使用了一个小型多层感知机来解码颜色，该感知机能够对局部非刚性变形和动态光照条件做出响应；应用了“尽可能等距”正则化项，以保持几何一致性和逼真的变形，尤其是在动态和变化的姿势中。性能：在单目视频输入的情况下，该方法在可动画身创建方面与最先进的方法相当或更好，训练速度提高了 400 倍，推理速度提高了 250 倍；与专注于快速训练的方法相比，该方法虽然训练速度较慢，但能够对姿势相关的非刚性变形进行建模，并产生明显更好的渲染质量，同时渲染速度提高了 3 倍。工作量：该方法的训练和推理速度都非常快，训练一个模型只需要几分钟，推理一个模型只需要几毫秒。
+

app.py

@@ -0,0 +1,892 @@
+import numpy as np
+import os
+import re
+import datetime
+import arxiv
+import openai, tenacity
+import base64, requests
+import argparse
+import configparser
+import fitz, io, os
+from PIL import Image
+import gradio
+import markdown
+import mistune
+import json
+import tiktoken
+import concurrent.futures
+from optimizeGoogle import chatPaper
+
+proxy = {'http':'http://127.0.0.1:7890', 
+         'https':'http://127.0.0.1:7890'}
+
+def parse_text(text):
+    lines = text.split("\n")
+    for i, line in enumerate(lines):
+        if "```" in line:
+            items = line.split('`')
+            if items[-1]:
+                lines[i] = f'<pre><code class="{items[-1]}">'
+            else:
+                lines[i] = f'</code></pre>'
+        else:
+            if i > 0:
+                line = line.replace("<", "&lt;")
+                line = line.replace(">", "&gt;")
+                lines[i] = '<br/>' + line.replace(" ", "&nbsp;")
+    return "".join(lines)
+
+
+# def get_response(system, context, myKey, raw = False):
+#     openai.api_key = myKey
+#     response = openai.ChatCompletion.create(
+#         model="gpt-3.5-turbo",
+#         messages=[system, *context],
+#     )
+#     openai.api_key = ""
+#     if raw:
+#         return response
+#     else:
+#         message = response["choices"][0]["message"]["content"]
+#         message_with_stats = f'{message}'
+#         return message, parse_text(message_with_stats)
+
+valid_api_keys = []
+
+
+def api_key_check(api_key):
+    try:
+        chat = chatPaper([api_key], proxy=proxy)
+        if chat.check_api_available():
+            return api_key
+        else:
+            return None
+    except Exception as e:
+        # print(e)
+        return None
+
+
+def valid_apikey(api_keys):
+    api_keys = api_keys.replace(' ', '')
+    api_key_list = api_keys.split(',')
+    print(api_key_list)
+    global valid_api_keys
+    with concurrent.futures.ThreadPoolExecutor() as executor:
+        future_results = {
+            executor.submit(api_key_check, api_key): api_key
+            for api_key in api_key_list
+        }
+        for future in concurrent.futures.as_completed(future_results):
+            result = future.result()
+            if result:
+                valid_api_keys.append(result)
+    if len(valid_api_keys) > 0:
+        return "有效的api-key一共有{}个，分别是：{}, 现在可以提交你的paper".format(
+            len(valid_api_keys), valid_api_keys)
+    return "无效的api-key"
+
+
+class Paper:
+
+    def __init__(self, path, title='', url='', abs='', authers=[], sl=[]):
+        # 初始化函数，根据pdf路径初始化Paper对象
+        self.url = url  # 文章链接
+        self.path = path  # pdf路径
+        self.sl = sl
+        self.section_names = []  # 段落标题
+        self.section_texts = {}  # 段落内容
+        self.abs = abs
+        self.title_page = 0
+        if title == '':
+            self.pdf = fitz.open(self.path)  # pdf文档
+            self.title = self.get_title()
+            self.parse_pdf()
+        else:
+            self.title = title
+        self.authers = authers
+        self.roman_num = [
+            "I", "II", 'III', "IV", "V", "VI", "VII", "VIII", "IIX", "IX", "X"
+        ]
+        self.digit_num = [str(d + 1) for d in range(10)]
+        self.first_image = ''
+
+    def parse_pdf(self):
+        self.pdf = fitz.open(self.path)  # pdf文档
+        self.text_list = [page.get_text() for page in self.pdf]
+        self.all_text = ' '.join(self.text_list)
+        self.section_page_dict = self._get_all_page_index()  # 段落与页码的对应字典
+        print("section_page_dict", self.section_page_dict)
+        self.section_text_dict = self._get_all_page()  # 段落与内容的对应字典
+        self.section_text_dict.update({"title": self.title})
+        self.section_text_dict.update({"paper_info": self.get_paper_info()})
+        self.pdf.close()
+
+    def get_paper_info(self):
+        first_page_text = self.pdf[self.title_page].get_text()
+        if "Abstract" in self.section_text_dict.keys():
+            abstract_text = self.section_text_dict['Abstract']
+        else:
+            abstract_text = self.abs
+        introduction_text = self.section_text_dict['Introduction']
+        first_page_text = first_page_text.replace(abstract_text, "").replace(
+            introduction_text, "")
+        return first_page_text
+
+    def get_image_path(self, image_path=''):
+        """
+        将PDF中的第一张图保存到image.png里面，存到本地目录，返回文件名称，供gitee读取
+        :param filename: 图片所在路径，"C:\\Users\\Administrator\\Desktop\\nwd.pdf"
+        :param image_path: 图片提取后的保存路径
+        :return:
+        """
+        # open file
+        max_size = 0
+        image_list = []
+        with fitz.Document(self.path) as my_pdf_file:
+            # 遍历所有页面
+            for page_number in range(1, len(my_pdf_file) + 1):
+                # 查看独立页面
+                page = my_pdf_file[page_number - 1]
+                # 查看当前页所有图片
+                images = page.get_images()
+                # 遍历当前页面所有图片
+                for image_number, image in enumerate(page.get_images(),
+                                                     start=1):
+                    # 访问图片xref
+                    xref_value = image[0]
+                    # 提取图片信息
+                    base_image = my_pdf_file.extract_image(xref_value)
+                    # 访问图片
+                    image_bytes = base_image["image"]
+                    # 获取图片扩展名
+                    ext = base_image["ext"]
+                    # 加载图片
+                    image = Image.open(io.BytesIO(image_bytes))
+                    image_size = image.size[0] * image.size[1]
+                    if image_size > max_size:
+                        max_size = image_size
+                    image_list.append(image)
+        for image in image_list:
+            image_size = image.size[0] * image.size[1]
+            if image_size == max_size:
+                image_name = f"image.{ext}"
+                im_path = os.path.join(image_path, image_name)
+                print("im_path:", im_path)
+
+                max_pix = 480
+                origin_min_pix = min(image.size[0], image.size[1])
+
+                if image.size[0] > image.size[1]:
+                    min_pix = int(image.size[1] * (max_pix / image.size[0]))
+                    newsize = (max_pix, min_pix)
+                else:
+                    min_pix = int(image.size[0] * (max_pix / image.size[1]))
+                    newsize = (min_pix, max_pix)
+                image = image.resize(newsize)
+
+                image.save(open(im_path, "wb"))
+                return im_path, ext
+        return None, None
+
+    # 定义一个函数，根据字体的大小，识别每个章节名称，并返回一个列表
+    def get_chapter_names(self, ):
+        # # 打开一个pdf文件
+        doc = fitz.open(self.path)  # pdf文档
+        text_list = [page.get_text() for page in doc]
+        all_text = ''
+        for text in text_list:
+            all_text += text
+        # # 创建一个空列表，用于存储章节名称
+        chapter_names = []
+        for line in all_text.split('\n'):
+            line_list = line.split(' ')
+            if '.' in line:
+                point_split_list = line.split('.')
+                space_split_list = line.split(' ')
+                if 1 < len(space_split_list) < 5:
+                    if 1 < len(point_split_list) < 5 and (
+                            point_split_list[0] in self.roman_num
+                            or point_split_list[0] in self.digit_num):
+                        print("line:", line)
+                        chapter_names.append(line)
+
+        return chapter_names
+
+    def get_title(self):
+        doc = self.pdf  # 打开pdf文件
+        max_font_size = 0  # 初始化最大字体大小为0
+        max_string = ""  # 初始化最大字体大小对应的字符串为空
+        max_font_sizes = [0]
+        for page_index, page in enumerate(doc):  # 遍历每一页
+            text = page.get_text("dict")  # 获取页面上的文本信息
+            blocks = text["blocks"]  # 获取文本块列表
+            for block in blocks:  # 遍历每个文本块
+                if block["type"] == 0 and len(block['lines']):  # 如果是文字类型
+                    if len(block["lines"][0]["spans"]):
+                        font_size = block["lines"][0]["spans"][0][
+                            "size"]  # 获取第一行第一段文字的字体大小
+                        max_font_sizes.append(font_size)
+                        if font_size > max_font_size:  # 如果字体大小大于当前最大值
+                            max_font_size = font_size  # 更新最大值
+                            max_string = block["lines"][0]["spans"][0][
+                                "text"]  # 更新最大值对应的字符串
+        max_font_sizes.sort()
+        print("max_font_sizes", max_font_sizes[-10:])
+        cur_title = ''
+        for page_index, page in enumerate(doc):  # 遍历每一页
+            text = page.get_text("dict")  # 获取页面上的文本信息
+            blocks = text["blocks"]  # 获取文本块列表
+            for block in blocks:  # 遍历每个文本块
+                if block["type"] == 0 and len(block['lines']):  # 如果是文字类型
+                    if len(block["lines"][0]["spans"]):
+                        cur_string = block["lines"][0]["spans"][0][
+                            "text"]  # 更新最大值对应的字符串
+                        font_flags = block["lines"][0]["spans"][0][
+                            "flags"]  # 获取第一行第一段文字的字体特征
+                        font_size = block["lines"][0]["spans"][0][
+                            "size"]  # 获取第一行第一段文字的字体大小
+                        # print(font_size)
+                        if abs(font_size - max_font_sizes[-1]) < 0.3 or abs(
+                                font_size - max_font_sizes[-2]) < 0.3:
+                            # print("The string is bold.", max_string, "font_size:", font_size, "font_flags:", font_flags)
+                            if len(cur_string
+                                   ) > 4 and "arXiv" not in cur_string:
+                                # print("The string is bold.", max_string, "font_size:", font_size, "font_flags:", font_flags)
+                                if cur_title == '':
+                                    cur_title += cur_string
+                                else:
+                                    cur_title += ' ' + cur_string
+                            self.title_page = page_index
+
+        title = cur_title.replace('\n', ' ')
+        return title
+
+    def _get_all_page_index(self):
+        # 定义需要寻找的章节名称列表
+        section_list = self.sl
+        # 初始化一个字典来存储找到的章节和它们在文档中出现的页码
+        section_page_dict = {}
+        # 遍历每一页文档
+        for page_index, page in enumerate(self.pdf):
+            # 获取当前页面的文本内容
+            cur_text = page.get_text()
+            # 遍历需要寻找的章节名称列表
+            for section_name in section_list:
+                # 将章节名称转换成大写形式
+                section_name_upper = section_name.upper()
+                # 如果当前页面包含"Abstract"这个关键词
+                if "Abstract" == section_name and section_name in cur_text:
+                    # 将"Abstract"和它所在的页码加入字典中
+                    section_page_dict[section_name] = page_index
+                # 如果当前页面包含章节名称，则将章节名称和它所在的页码加入字典中
+                else:
+                    if section_name + '\n' in cur_text:
+                        section_page_dict[section_name] = page_index
+                    elif section_name_upper + '\n' in cur_text:
+                        section_page_dict[section_name] = page_index
+        # 返回所有找到的章节名称及它们在文档中出现的页码
+        return section_page_dict
+
+    def _get_all_page(self):
+        """
+        获取PDF文件中每个页面的文本信息，并将文本信息按照章节组织成字典返回。
+        Returns:
+            section_dict (dict): 每个章节的文本信息字典，key为章节名，value为章节文本。
+        """
+        text = ''
+        text_list = []
+        section_dict = {}
+
+        # 再处理其他章节：
+        text_list = [page.get_text() for page in self.pdf]
+        for sec_index, sec_name in enumerate(self.section_page_dict):
+            print(sec_index, sec_name, self.section_page_dict[sec_name])
+            if sec_index <= 0 and self.abs:
+                continue
+            else:
+                # 直接考虑后面的内容：
+                start_page = self.section_page_dict[sec_name]
+                if sec_index < len(list(self.section_page_dict.keys())) - 1:
+                    end_page = self.section_page_dict[list(
+                        self.section_page_dict.keys())[sec_index + 1]]
+                else:
+                    end_page = len(text_list)
+                print("start_page, end_page:", start_page, end_page)
+                cur_sec_text = ''
+                if end_page - start_page == 0:
+                    if sec_index < len(list(
+                            self.section_page_dict.keys())) - 1:
+                        next_sec = list(
+                            self.section_page_dict.keys())[sec_index + 1]
+                        if text_list[start_page].find(sec_name) == -1:
+                            start_i = text_list[start_page].find(
+                                sec_name.upper())
+                        else:
+                            start_i = text_list[start_page].find(sec_name)
+                        if text_list[start_page].find(next_sec) == -1:
+                            end_i = text_list[start_page].find(
+                                next_sec.upper())
+                        else:
+                            end_i = text_list[start_page].find(next_sec)
+                        cur_sec_text += text_list[start_page][start_i:end_i]
+                else:
+                    for page_i in range(start_page, end_page):
+                        #                         print("page_i:", page_i)
+                        if page_i == start_page:
+                            if text_list[start_page].find(sec_name) == -1:
+                                start_i = text_list[start_page].find(
+                                    sec_name.upper())
+                            else:
+                                start_i = text_list[start_page].find(sec_name)
+                            cur_sec_text += text_list[page_i][start_i:]
+                        elif page_i < end_page:
+                            cur_sec_text += text_list[page_i]
+                        elif page_i == end_page:
+                            if sec_index < len(
+                                    list(self.section_page_dict.keys())) - 1:
+                                next_sec = list(
+                                    self.section_page_dict.keys())[sec_index +
+                                                                   1]
+                                if text_list[start_page].find(next_sec) == -1:
+                                    end_i = text_list[start_page].find(
+                                        next_sec.upper())
+                                else:
+                                    end_i = text_list[start_page].find(
+                                        next_sec)
+                                cur_sec_text += text_list[page_i][:end_i]
+                section_dict[sec_name] = cur_sec_text.replace('-\n',
+                                                              '').replace(
+                                                                  '\n', ' ')
+        return section_dict
+
+
+# 定义Reader类
+class Reader:
+    # 初始化方法，设置属性
+    def __init__(self,
+                 key_word='',
+                 query='',
+                 filter_keys='',
+                 root_path='./',
+                 gitee_key='',
+                 sort=arxiv.SortCriterion.SubmittedDate,
+                 user_name='defualt',
+                 language='cn',
+                 api_keys: list = [],
+                 model_name="gemini-Pro",
+                 p=1.0,
+                 temperature=1.0):
+        self.api_keys = api_keys
+        self.chatPaper = chatPaper(api_keys=self.api_keys,
+                                   apiTimeInterval=10,
+                                   temperature=temperature,
+                                   top_p=p,
+                                   model_name=model_name,
+                                   proxy=proxy)  #openAI api封装
+        self.user_name = user_name  # 读者姓名
+        self.key_word = key_word  # 读者感兴趣的关键词
+        self.query = query  # 读者输入的搜索查询
+        self.sort = sort  # 读者选择的排序方式
+        self.language = language  # 读者选择的语言
+        self.filter_keys = filter_keys  # 用于在摘要中筛选的关键词
+        self.root_path = root_path
+        self.file_format = 'md'  # or 'txt'，如果为图片，则必须为'md'
+        self.save_image = False
+        if self.save_image:
+            self.gitee_key = self.config.get('Gitee', 'api')
+        else:
+            self.gitee_key = ''
+        self.max_token_num = 4096
+        self.encoding = tiktoken.get_encoding("gpt2")
+
+    def get_arxiv(self, max_results=30):
+        search = arxiv.Search(
+            query=self.query,
+            max_results=max_results,
+            sort_by=self.sort,
+            sort_order=arxiv.SortOrder.Descending,
+        )
+        return search
+
+    def filter_arxiv(self, max_results=30):
+        search = self.get_arxiv(max_results=max_results)
+        print("all search:")
+        for index, result in enumerate(search.results()):
+            print(index, result.title, result.updated)
+
+        filter_results = []
+        filter_keys = self.filter_keys
+
+        print("filter_keys:", self.filter_keys)
+        # 确保每个关键词都能在摘要中找到，才算是目标论文
+        for index, result in enumerate(search.results()):
+            abs_text = result.summary.replace('-\n', '-').replace('\n', ' ')
+            meet_num = 0
+            for f_key in filter_keys.split(" "):
+                if f_key.lower() in abs_text.lower():
+                    meet_num += 1
+            if meet_num == len(filter_keys.split(" ")):
+                filter_results.append(result)
+                # break
+        print("filter_results:", len(filter_results))
+        print("filter_papers:")
+        for index, result in enumerate(filter_results):
+            print(index, result.title, result.updated)
+        return filter_results
+
+    def validateTitle(self, title):
+        # 将论文的乱七八糟的路径格式修正
+        rstr = r"[\/\\\:\*\?\"\<\>\|]"  # '/ \ : * ? " < > |'
+        new_title = re.sub(rstr, "_", title)  # 替换为下划线
+        return new_title
+
+    def download_pdf(self, filter_results):
+        # 先创建文件夹
+        date_str = str(datetime.datetime.now())[:13].replace(' ', '-')
+        key_word = str(self.key_word.replace(':', ' '))
+        path = self.root_path + 'pdf_files/' + self.query.replace(
+            'au: ', '').replace('title: ', '').replace('ti: ', '').replace(
+                ':', ' ')[:25] + '-' + date_str
+        try:
+            os.makedirs(path)
+        except:
+            pass
+        print("All_paper:", len(filter_results))
+        # 开始下载：
+        paper_list = []
+        for r_index, result in enumerate(filter_results):
+            try:
+                title_str = self.validateTitle(result.title)
+                pdf_name = title_str + '.pdf'
+                # result.download_pdf(path, filename=pdf_name)
+                self.try_download_pdf(result, path, pdf_name)
+                paper_path = os.path.join(path, pdf_name)
+                print("paper_path:", paper_path)
+                paper = Paper(
+                    path=paper_path,
+                    url=result.entry_id,
+                    title=result.title,
+                    abs=result.summary.replace('-\n', '-').replace('\n', ' '),
+                    authers=[str(aut) for aut in result.authors],
+                )
+                # 下载完毕，开始解析：
+                paper.parse_pdf()
+                paper_list.append(paper)
+            except Exception as e:
+                print("download_error:", e)
+                pass
+        return paper_list
+
+    @tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, min=4,
+                                                   max=10),
+                    stop=tenacity.stop_after_attempt(5),
+                    reraise=True)
+    def try_download_pdf(self, result, path, pdf_name):
+        result.download_pdf(path, filename=pdf_name)
+
+    @tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, min=4,
+                                                   max=10),
+                    stop=tenacity.stop_after_attempt(5),
+                    reraise=True)
+    def upload_gitee(self, image_path, image_name='', ext='png'):
+        """
+        上传到码云
+        :return:
+        """
+        with open(image_path, 'rb') as f:
+            base64_data = base64.b64encode(f.read())
+            base64_content = base64_data.decode()
+
+        date_str = str(datetime.datetime.now())[:19].replace(':', '-').replace(
+            ' ', '-') + '.' + ext
+        path = image_name + '-' + date_str
+
+        payload = {
+            "access_token": self.gitee_key,
+            "owner": self.config.get('Gitee', 'owner'),
+            "repo": self.config.get('Gitee', 'repo'),
+            "path": self.config.get('Gitee', 'path'),
+            "content": base64_content,
+            "message": "upload image"
+        }
+        # 这里需要修改成你的gitee的账户和仓库名，以及文件夹的名字：
+        url = f'https://gitee.com/api/v5/repos/' + self.config.get(
+            'Gitee', 'owner') + '/' + self.config.get(
+                'Gitee', 'repo') + '/contents/' + self.config.get(
+                    'Gitee', 'path') + '/' + path
+        rep = requests.post(url, json=payload).json()
+        print("rep:", rep)
+        if 'content' in rep.keys():
+            image_url = rep['content']['download_url']
+        else:
+            image_url = r"https://gitee.com/api/v5/repos/" + self.config.get(
+                'Gitee', 'owner') + '/' + self.config.get(
+                    'Gitee', 'repo') + '/contents/' + self.config.get(
+                        'Gitee', 'path') + '/' + path
+
+        return image_url
+
+
+    def summary_with_chat(self, paper_list):
+        htmls = []
+        utoken = 0
+        ctoken = 0
+        ttoken = 0
+        for paper_index, paper in enumerate(paper_list):
+            # 第一步先用title，abs，和introduction进行总结。
+            text = ''
+            text += 'Title:' + paper.title
+            text += 'Url:' + paper.url
+            text += 'Abstrat:' + paper.abs
+            text += 'Paper_info:' + paper.section_text_dict['paper_info']
+            # intro
+            text += list(paper.section_text_dict.values())[0]
+            #max_token = 2500 * 4
+            #text = text[:max_token]
+            chat_summary_text, utoken1, ctoken1, ttoken1 = self.chat_summary(
+                text=text)
+            htmls.append(chat_summary_text)
+
+            # TODO 往md文档中插入论文里的像素最大的一张图片，这个方案可以弄的更加智能一些：
+            method_key = ''
+            for parse_key in paper.section_text_dict.keys():
+                if 'method' in parse_key.lower(
+                ) or 'approach' in parse_key.lower():
+                    method_key = parse_key
+                    break
+
+            if method_key != '':
+                text = ''
+                method_text = ''
+                summary_text = ''
+                summary_text += "<summary>" + chat_summary_text
+                # methods
+                method_text += paper.section_text_dict[method_key]
+                text = summary_text + "\n<Methods>:\n" + method_text
+                chat_method_text, utoken2, ctoken2, ttoken2 = self.chat_method(
+                    text=text)
+            else:
+                chat_method_text = ''
+            htmls.append(chat_method_text)
+            htmls.append("\n")
+
+            # 第三步总结全文，并打分：
+            conclusion_key = ''
+            for parse_key in paper.section_text_dict.keys():
+                if 'conclu' in parse_key.lower():
+                    conclusion_key = parse_key
+                    break
+
+            text = ''
+            conclusion_text = ''
+            summary_text = ''
+            summary_text += "<summary>" + chat_summary_text + "\n <Method summary>:\n" + chat_method_text
+            if conclusion_key != '':
+                # conclusion
+                conclusion_text += paper.section_text_dict[conclusion_key]
+                text = summary_text + "\n <Conclusion>:\n" + conclusion_text
+            else:
+                text = summary_text
+            chat_conclusion_text, utoken3, ctoken3, ttoken3 = self.chat_conclusion(
+                text=text)
+            htmls.append(chat_conclusion_text)
+            htmls.append("\n")
+            # token统计
+            utoken = utoken + utoken1 + utoken2 + utoken3
+            ctoken = ctoken + ctoken1 + ctoken2 + ctoken3
+            ttoken = ttoken + ttoken1 + ttoken2 + ttoken3
+            cost = (ttoken / 1000) * 0.002
+            pos_count = {
+                "usage_token_used": str(utoken),
+                "completion_token_used": str(ctoken),
+                "total_token_used": str(ttoken),
+                "cost": str(cost),
+            }
+            md_text = "\n\n".join(htmls)
+
+            with open(os.path.join('./', 'output.md'), "w", encoding="utf8") as f:
+               f.write(md_text)
+            return mistune.markdown(md_text), pos_count
+            return markdown.markdown(md_text), pos_count # , os.path.join('./', 'output.md')
+
+    @tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, min=4,
+                                                   max=10),
+                    stop=tenacity.stop_after_attempt(5),
+                    reraise=True)
+    def chat_conclusion(self, text):
+        conclusion_prompt_token = 650
+        text_token = len(self.encoding.encode(text))
+        clip_text_index = int(
+            len(text) * (self.max_token_num - conclusion_prompt_token) /
+            text_token)
+        clip_text = text[:clip_text_index]
+        self.chatPaper.reset(
+            convo_id="chatConclusion",
+            system_prompt="You are a reviewer in the field of [" +
+            self.key_word + "] and you need to critically review this article")
+        self.chatPaper.add_to_conversation_last(
+            convo_id="chatConclusion",
+            role="user",
+            message=
+            "This is the <summary> and <conclusion> part of an English literature, where <summary> you have already summarized, but <conclusion> part, I need your help to summarize the following questions:"
+            + clip_text)  # 背景知识，可以参考OpenReview的审稿流程
+        content = """                 
+                 8. Make the following summary.Be sure to use Chinese answers (proper nouns need to be marked in English).
+                    (1):What is the significance of this piece of work?
+                    (2):Summarize the strengths and weaknesses of this article in three dimensions: innovation point, performance, and workload.                   
+                    .......
+                 Follow the format of the output later: 
+                 8. Conclusion: \n\n
+                    (1):xxx;\n                     
+                    (2):Innovation point: xxx; Performance: xxx; Workload: xxx;\n                      
+                 
+                 Be sure to use Chinese answers (proper nouns need to be marked in English), statements as concise and academic as possible, do not repeat the content of the previous <summary>, the value of the use of the original numbers, be sure to strictly follow the format, the corresponding content output to xxx, in accordance with \n line feed, ....... means fill in according to the actual requirements, if not, you can not write.                 
+                 """
+        result = self.chatPaper.ask(
+            prompt=content,
+            role="user",
+            convo_id="chatConclusion",
+        )
+        print(result)
+        return result[0], result[1], result[2], result[3]
+
+    @tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, min=4,
+                                                   max=10),
+                    stop=tenacity.stop_after_attempt(5),
+                    reraise=True)
+    def chat_method(self, text):
+        method_prompt_token = 650
+        text_token = len(self.encoding.encode(text))
+        clip_text_index = int(
+            len(text) * (self.max_token_num - method_prompt_token) /
+            text_token)
+        clip_text = text[:clip_text_index]
+        self.chatPaper.reset(
+            convo_id="chatMethod",
+            system_prompt="You are a researcher in the field of [" +
+            self.key_word +
+            "] who is good at summarizing papers using concise statements"
+        )  # chatgpt 角色
+        self.chatPaper.add_to_conversation_last(
+            convo_id="chatMethod",
+            role="user",
+            message=str(
+                "This is the <summary> and <Method> part of an English document, where <summary> you have summarized, but the <Methods> part, I need your help to read and summarize the following questions."
+                + clip_text))
+        content = """                 
+                 7. Describe in detail the methodological idea of this article. Be sure to use Chinese answers (proper nouns need to be marked in English). For example, its steps are. 
+                    (1):...
+                    (2):...
+                    (3):...
+                    .......
+                 Follow the format of the output that follows: 
+                 7. Methods: \n\n
+                    (1):xxx;\n 
+                    (2):xxx;\n 
+                    (3):xxx;\n  
+                    ....... \n\n     
+                 \n\n
+                 Be sure to use Chinese answers (proper nouns need to be marked in English), statements as concise and academic as possible, do not repeat the content of the previous <summary>, the value of the use of the original numbers, be sure to strictly follow the format, the corresponding content output to xxx, in accordance with \n line feed, ....... means fill in according to the actual requirements, if not, you can not write.                 
+                 """
+        result = self.chatPaper.ask(
+            prompt=content,
+            role="user",
+            convo_id="chatMethod",
+        )
+        print(result)
+        return result[0], result[1], result[2], result[3]
+
+    @tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, min=4,
+                                                   max=10),
+                    stop=tenacity.stop_after_attempt(5),
+                    reraise=True)
+    def chat_summary(self, text):
+        summary_prompt_token = 1000
+        text_token = len(self.encoding.encode(text))
+        clip_text_index = int(
+            len(text) * (self.max_token_num - summary_prompt_token) /
+            text_token)
+        clip_text = text[:clip_text_index]
+        # 重新设定GPT 角色
+        self.chatPaper.reset(
+            convo_id="chatSummary",
+            system_prompt="You are a researcher in the field of [" +
+            self.key_word +
+            "] who is good at summarizing papers using concise statements")
+        # 提交summary prompt
+        self.chatPaper.add_to_conversation_last(
+            convo_id="chatSummary",
+            role="user",
+            message=str(
+                "This is the title, author, link, abstract and introduction of an English document. I need your help to read and summarize the following questions: "
+                + clip_text))
+        content = """                 
+                 1. Mark the title of the paper (with Chinese translation)
+                 2. list all the authors' names (use English)
+                 3. mark the first author's affiliation (output Chinese translation only)                 
+                 4. mark the keywords of this article (use English)
+                 5. link to the paper, Github code link (if available, fill in Github:None if not)
+                 6. summarize according to the following four points.Be sure to use Chinese answers (proper nouns need to be marked in English)
+                    (1):What is the research background of this article?
+                    (2):What are the past methods? What are the problems with them? Is the approach well motivated?
+                    (3):What is the research methodology proposed in this paper?
+                    (4):On what task and what performance is achieved by the methods in this paper? Can the performance support their goals?
+                 Follow the format of the output that follows:                  
+                 1. Title: xxx\n\n
+                 2. Authors: xxx\n\n
+                 3. Affiliation: xxx\n\n                 
+                 4. Keywords: xxx\n\n   
+                 5. Urls: xxx or xxx , xxx \n\n      
+                 6. Summary: \n\n
+                    (1):xxx;\n 
+                    (2):xxx;\n 
+                    (3):xxx;\n  
+                    (4):xxx.\n\n     
+                 
+                 Be sure to use Chinese answers (proper nouns need to be marked in English), statements as concise and academic as possible, do not have too much repetitive information, numerical values using the original numbers, be sure to strictly follow the format, the corresponding content output to xxx, in accordance with \n line feed.                 
+                 """
+        # 提出对应的问题
+        result = self.chatPaper.ask(
+            prompt=content,
+            role="user",
+            convo_id="chatSummary",
+        )
+        print(result)
+        return result[0], result[1], result[2], result[3]
+
+    def export_to_markdown(self, text, file_name, mode='w'):
+        # 使用markdown模块的convert方法，将文本转换为html格式
+        # html = markdown.markdown(text)
+        # 打开一个文件，以写入模式
+        with open(file_name, mode, encoding="utf-8") as f:
+            # 将html格式的内容写入文件
+            f.write(text)
+
+    # 定义一个方法，打印出读者信息
+    def show_info(self):
+        print(f"Key word: {self.key_word}")
+        print(f"Query: {self.query}")
+        print(f"Sort: {self.sort}")
+
+
+def upload_pdf(api_keys, text, model_name, p, temperature, file):
+    # 检查两个输入都不为空
+    api_key_list = None
+    if api_keys:
+        api_key_list = api_keys.split(',')
+    elif not api_keys and valid_api_keys != []:
+        api_key_list = valid_api_keys
+    if not text or not file or not api_key_list:
+        return "两个输入都不能为空，请输入字符并上传 PDF 文件！"
+
+    # 判断PDF文件
+    #if file and file.name.split(".")[-1].lower() != "pdf":
+    #    return '请勿上传非 PDF 文件！'
+    else:
+        section_list = text.split(',')
+        paper_list = [Paper(path=file, sl=section_list)]
+        # 创建一个Reader对象
+        print(api_key_list)
+        reader = Reader(api_keys=api_key_list,
+                        model_name=model_name,
+                        p=p,
+                        temperature=temperature)
+        sum_info, cost = reader.summary_with_chat(
+            paper_list=paper_list)  # type: ignore
+        return cost, sum_info
+
+
+api_title = "api-key可用验证(Gemini Pro)"
+api_description = '''<div align='left'>
+
+使用 Google 的 GeminiPro 进行论文摘要生成，灵感来源于 ChatPaper 项目。
+
+💥💥💥<strong>面向全球，服务万千科研人的ChatPaper在线版正式上线：<a href="https://chatpaper.org/">https://chatpaper.org/</a> </strong>💥💥💥
+
+Use ChatGPT to summary the papers.Star our Github [🌟ChatPaper](https://github.com/kaixindelele/ChatPaper) .
+
+基于此，修改部分为Gemini Pro来进行论文摘要，并且这一部分是免费的，只要获取API key即可，也请多多关注
+
+Use Gemini Pro to summary the papers.Star our Github [🌟ChatPaperFree](https://github.com/Kedreamix/ChatPaperFree) .
+
+另外，请多多关注 [进步屋](https://ipaper.today/)，我们会根据关键词，每天更新最新的arixv论文，follow最新的研究方向，并且使用GPT进行总结，帮助你更好的读论文
+
+💗如果您觉得我们的项目对您有帮助，还请您给我们一些鼓励！💗
+
+🔴请注意：千万不要用于严肃的学术场景，只能用于论文阅读前的初筛！
+
+
+</div>
+'''
+
+api_input = [
+    gradio.inputs.Textbox(label="请输入你的API-key(必填, 多个API-key请用英文逗号隔开)",
+                          default="",
+                          type='password')
+]
+api_gui = gradio.Interface(fn=valid_apikey,
+                           inputs=api_input,
+                           outputs="text",
+                           title=api_title,
+                           description=api_description)
+
+# 标题
+title = "ChatPaper(Gemini Pro)"
+# 描述
+description = api_description = '''<div align='left'>
+
+使用 Google 的 GeminiPro 进行论文摘要生成，灵感来源于 ChatPaper 项目。
+
+💥💥💥<strong>面向全球，服务万千科研人的ChatPaper在线版正式上线：<a href="https://chatpaper.org/">https://chatpaper.org/</a> </strong>💥💥💥
+
+Use ChatGPT to summary the papers.Star our Github [🌟ChatPaper](https://github.com/kaixindelele/ChatPaper) .
+
+基于此，修改部分为Gemini Pro来进行论文摘要，并且这一部分是免费的，只要获取API key即可，也请多多关注
+
+Use Gemini Pro to summary the papers.Star our Github [🌟ChatPaperFree](https://github.com/Kedreamix/ChatPaperFree) .
+
+另外，请多多关注 [进步屋](https://ipaper.today/)，我们会根据关键词，每天更新最新的arixv论文，follow最新的研究方向，并且使用GPT进行总结，帮助你更好的读论文
+
+💗如果您觉得我们的项目对您有帮助，还请您给我们一些鼓励！💗
+
+🔴请注意：千万不要用于严肃的学术场景，只能用于论文阅读前的初筛！
+
+
+</div>
+'''
+# 创建Gradio界面
+ip = [
+    gradio.inputs.Textbox(label="请输入你的API-key(必填, 多个API-key请用英文逗号隔开),不需要空格",
+                          default="",
+                          type='password'),
+    gradio.inputs.Textbox(
+        label="请输入论文大标题索引(用英文逗号隔开,必填)",
+        default=
+        "'Abstract,Introduction,Related Work,Background,Preliminary,Problem Formulation,Methods,Methodology,Method,Approach,Approaches,Materials and Methods,Experiment Settings,Experiment,Experimental Results,Evaluation,Experiments,Results,Findings,Data Analysis,Discussion,Results and Discussion,Conclusion,References'"
+    ),
+    gradio.inputs.Radio(choices=["gemini-Pro", "gemini-Pro-vision(comming Soon)"],
+                        default="gemini-Pro",
+                        label="Select model"),
+    gradio.inputs.Slider(minimum=-0,
+                         maximum=1.0,
+                         default=1.0,
+                         step=0.05,
+                         label="Top-p (nucleus sampling)"),
+    gradio.inputs.Slider(minimum=-0,
+                         maximum=5.0,
+                         default=0.5,
+                         step=0.5,
+                         label="Temperature"),
+    gradio.inputs.File(label="请上传论文PDF(必填)")
+]
+
+chatpaper_gui = gradio.Interface(fn=upload_pdf,
+                                 inputs=ip,
+                                 outputs=["json", "html"],
+                                 title=title,
+                                 description=description)
+
+# Start server
+gui = gradio.TabbedInterface(interface_list=[api_gui, chatpaper_gui],
+                             tab_names=["API-key", "ChatPaper"])
+gui.launch(quiet=True, show_api=False, debug=True)

optimizeGoogle.py

@@ -0,0 +1,228 @@
+"""
+A simple wrapper for the official Google GeminiPro API
+"""
+import json
+import os
+import threading
+import time
+import requests
+import tiktoken
+from typing import Generator
+from queue import PriorityQueue as PQ
+import json
+import os
+import time
+import google.generativeai as genai
+ENCODER = tiktoken.get_encoding("gpt2")
+class chatPaper:
+    """
+    Official Google API
+    """
+    def __init__(
+        self,
+        api_keys: list,
+        proxy = None,
+        api_proxy = None,
+        max_tokens: int = 4000,
+        temperature: float = 0.5,
+        top_p: float = 1.0,
+        model_name: str = "gemini-Pro",
+        reply_count: int = 1,
+        system_prompt = "You are ChatPaper, A paper reading bot",
+        lastAPICallTime = time.time()-100,
+        apiTimeInterval = 20,
+    ) -> None:
+        self.model_name = model_name
+        self.system_prompt = system_prompt
+        self.apiTimeInterval = apiTimeInterval
+        self.session = requests.Session()
+        self.api_keys = PQ()
+        for key in api_keys:
+            self.api_keys.put((lastAPICallTime,key))
+        self.proxy = proxy
+        if self.proxy:
+            proxies = {
+                "http": self.proxy['http'],
+                "https": self.proxy['https'],
+            }
+            os.environ["http_proxy"] = self.proxy['http']
+            os.environ["https_proxy"] = self.proxy['https']
+            self.session.proxies = proxies
+        self.max_tokens = max_tokens
+        self.temperature = temperature
+        self.top_p = top_p
+        self.reply_count = reply_count
+        self.decrease_step = 250
+        self.conversation = {}
+        if self.token_str(self.system_prompt) > self.max_tokens:
+            raise Exception("System prompt is too long")
+        self.lock = threading.Lock()
+        
+    def get_api_key(self):
+        with self.lock:
+            apiKey = self.api_keys.get()
+            delay = self._calculate_delay(apiKey)
+            time.sleep(delay)
+            self.api_keys.put((time.time(), apiKey[1]))
+            return apiKey[1]
+
+    def _calculate_delay(self, apiKey):
+        elapsed_time = time.time() - apiKey[0]
+        if elapsed_time < self.apiTimeInterval:
+            return self.apiTimeInterval - elapsed_time
+        else:
+            return 0
+
+    def add_to_conversation(self, message: str, role: str, convo_id: str = "default"):
+        if(convo_id not in self.conversation):
+            self.reset(convo_id)
+        self.conversation[convo_id].append({"role": role, "parts": [message]})
+
+    def add_to_conversation_last(self, message: str, role: str, convo_id: str = "default"):
+        if(convo_id not in self.conversation):
+            self.reset(convo_id)
+        self.conversation[convo_id][-1]["parts"] += message
+
+    def __truncate_conversation(self, convo_id: str = "default"):
+        """
+        Truncate the conversation
+        """
+        last_dialog = self.conversation[convo_id][-1]
+        query = str(last_dialog['parts'][0])
+        if(len(ENCODER.encode(str(query)))>self.max_tokens):
+            query = query[:int(1.5*self.max_tokens)]
+        while(len(ENCODER.encode(str(query)))>self.max_tokens):
+            query = query[:self.decrease_step]
+        self.conversation[convo_id] = self.conversation[convo_id][:-1]
+        # print(self.conversation)
+        full_conversation = "\n".join([str(x["parts"][0]) for x in self.conversation[convo_id]],)
+        if len(ENCODER.encode(full_conversation)) > self.max_tokens:
+            self.conversation_summary(convo_id=convo_id)
+        full_conversation = ""
+        for x in self.conversation[convo_id]:
+            full_conversation = str(x["parts"][0]) + "\n" + full_conversation
+        while True:
+            if (len(ENCODER.encode(full_conversation+query)) > self.max_tokens):
+                query = query[:self.decrease_step]
+            else:
+                break
+        # print(last_dialog)
+        last_dialog['parts'][0] = str(query)
+        self.conversation[convo_id].append(last_dialog)
+
+    def ask_stream(
+        self,
+        prompt: str,
+        role: str = "user",
+        convo_id: str = "default",
+        **kwargs,
+    ) -> Generator:
+        if convo_id not in self.conversation:
+            self.reset(convo_id=convo_id)
+        self.add_to_conversation_last(prompt, "user", convo_id=convo_id)
+        self.__truncate_conversation(convo_id=convo_id)
+        
+        genai.configure(api_key=self.get_api_key())
+
+        model = genai.GenerativeModel('gemini-pro')
+
+        response = model.generate_content(self.conversation[convo_id],
+                                          generation_config=genai.types.GenerationConfig(
+                                            # Only one candidate for now.
+                                            candidate_count=kwargs.get("n", self.reply_count),
+                                            # stop_sequences=['x'],
+                                            max_output_tokens=self.max_tokens,
+                                            temperature=kwargs.get("temperature", self.temperature)),
+                                          stream = True)
+        # try:
+        #     response.text
+        # except Exception as e:
+        #     print(f"Exception: {e}")
+        #     raise Exception(
+        #         f"Gemini Error",
+        #     )
+        
+        for line in response:    
+            yield line.text
+
+    def ask(self, prompt: str, role: str = "user", convo_id: str = "default", **kwargs):
+        """
+        Non-streaming ask
+        """
+        response = self.ask_stream(
+            prompt=prompt,
+            role=role,
+            convo_id=convo_id,
+            **kwargs,
+        )
+        full_response: str = "".join(response)
+        self.add_to_conversation(full_response, role, convo_id=convo_id)
+        usage_token = self.token_str(prompt)
+        com_token = self.token_str(full_response)
+        total_token = self.token_cost(convo_id=convo_id)
+        return full_response, usage_token, com_token, total_token
+
+    def check_api_available(self):
+        url = f"https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key={self.get_api_key()}"
+
+        headers = {
+            'Content-Type': 'application/json',
+        }
+
+        data = {
+            'prompt': {
+                'text': "hello",
+            },
+        }
+
+        response = requests.post(url, headers=headers, json=data)
+
+        if response.status_code == 200:
+            print(response)
+        else:
+            print(False)
+        if response.status_code == 200:
+            return True
+        else:
+            return False
+       
+    def reset(self, convo_id: str = "default", system_prompt = None):
+        """
+        Reset the conversation
+        """
+        self.conversation[convo_id] = [
+            {"role": "user", "parts": [str(system_prompt or self.system_prompt)]},
+        ]
+        
+    def conversation_summary(self, convo_id: str = "default"):
+        input = ""
+        role = ""
+        for conv in self.conversation[convo_id]:
+            if (conv["role"]=='user'):
+                role = 'User'
+            else:
+                role = 'Gemini'
+            input+=role+' : '+conv['content']+'\n'
+        prompt = "Your goal is to summarize the provided conversation in English. Your summary should be concise and focus on the key information to facilitate better dialogue for the large language model.Ensure that you include all necessary details and relevant information while still reducing the length of the conversation as much as possible. Your summary should be clear and easily understandable for the ChatGpt model providing a comprehensive and concise summary of the conversation."
+        if(self.token_str(str(input)+prompt)>self.max_tokens):
+            input = input[self.token_str(str(input))-self.max_tokens:]
+        while self.token_str(str(input)+prompt)>self.max_tokens:
+            input = input[self.decrease_step:]
+        prompt = prompt.replace("{conversation}", input)
+        self.reset(convo_id='conversationSummary')
+        response = self.ask(prompt,convo_id='conversationSummary')
+        while self.token_str(str(response))>self.max_tokens:
+            response = response[:-self.decrease_step]
+        self.reset(convo_id='conversationSummary',system_prompt='Summariaze our diaglog')
+        self.conversation[convo_id] = [
+            {"role": "user", "parts": [self.system_prompt]},
+            {"role": "user", "parts": ["Summariaze our diaglog"]},
+            {"role": 'model', "parts": [response]},
+        ]
+        return self.conversation[convo_id]
+    def token_cost(self,convo_id: str = "default"):
+        return len(ENCODER.encode("\n".join([x["parts"][0] for x in self.conversation[convo_id]])))
+    def token_str(self,content:str):
+        return len(ENCODER.encode(content))
+def main():
+    return

requirements.txt

@@ -0,0 +1,11 @@
+arxiv==1.4.3
+PyMuPDF==1.21.1
+requests==2.26.0
+tiktoken==0.2.0
+tenacity==8.2.2
+pybase64==1.2.3
+Pillow==9.4.0
+openai==0.27.0
+markdown
+gradio==3.20.1
+#tiktoken